1、yarn那点事 讲说人 朱钰锋 Client ApplicationManager ResourceScheduler NodeManager NodeManager MapTask MRAppMstr Container Container ReduceTask MapTask Container Container ResourceManager 1 1 client向yarn提交job 首先找ResourceManager分配资源 Client ApplicationManager ResourceScheduler NodeManager NodeManager MapTask MRA
2、ppMstr Container Container ReduceTask MapTask Container Container ResourceManager 2 2 ResourceManager开启一个Container 在Container中运行一个Applicationmanager 1 Client ApplicationManager ResourceScheduler NodeManager NodeManager MapTask MRAppMstr Container Container ReduceTask MapTask Container Container Reso
3、urceManager 3 3 Applicationmanager找一台nodemanager启动Applicationmaster 计算任务所需的计算 1 2 Client ApplicationManager ResourceScheduler NodeManager NodeManager MapTask MRAppMstr Container Container ReduceTask MapTask Container Container ResourceManager 4 4 Applicationmaster向Applicationmanager Yarn 申请运行任务所需的资源
4、 1 2 3 Client ApplicationManager ResourceScheduler NodeManager NodeManager MapTask MRAppMstr Container Container ReduceTask MapTask Container Container ResourceManager 5 5 Resourcescheduler将资源封装发给Applicationmaster 1 4 2 3 Client ApplicationManager ResourceScheduler NodeManager NodeManager MapTask MR
5、AppMstr Container Container ReduceTask MapTask Container Container ResourceManager 6 Applicationmaster将获取到的资源分配给各个nodemanager 6 6 1 4 2 3 5 Client ApplicationManager ResourceScheduler NodeManager NodeManager MapTask MRAppMstr Container Container ReduceTask MapTask Container Container ResourceManager
6、 7 7 各个nodemanager得到任务和资源开始执行maptask 7 1 3 6 4 2 5 6 Client ApplicationManager ResourceScheduler NodeManager NodeManager MapTask MRAppMstr Container Container ReduceTask MapTask Container Container ResourceManager 8 8 maptask执行结束后 开始执行reducetask 8 1 3 6 4 7 5 7 6 Client ApplicationManager ResourceSc
7、heduler NodeManager NodeManager MapTask MRAppMstr Container Container ReduceTask MapTask Container Container ResourceManager 8 9 maptask和reducetask将执行结果反馈给Applicationmaster 8 1 3 6 4 7 5 7 9 6 Client ApplicationManager ResourceScheduler NodeManager NodeManager MapTask MRAppMstr Container Container ReduceTask MapTask Container Container ResourceManager 8 10 Applicationmaster将任务执行的结果反馈Applicationmanager 8 1 3 6 4 7 5 7 10 2 9 6 这就是yarn的大致流程那点事 ThankYou
