Robust Elastic Scaling of Deep Learning on Kubernates

Junfeng Liu

2019-09-12 16:30 #machinelearning

With the increasing popularity of deep learning workloads in academia, government and industry, several organizations have developed cloud-based deep learning platforms whose goals are to enable organizations utilize expensive resources effectively, and to share said resources among multiple teams in a fair and effective manner. We present the elastic scaling in the context of such large scale deep learning platforms. The traditional deep learning workload is the parallel job where can not fully utilize the cluster gpu resources. We begin by analyzing deep learning training workloads, and how they differ from conventional workloads as far as elastic scaling is concerned. The scheduler driven scaling to optimize resource utilization, support priorities, maintenance, at the same time promising the accuracy and user can define the strategy for scale out. The presentation will describe the design and implementation of a DL workload aware scheduling supporting flexible scheduling-driven scaling on the top of kubernetes cluster. Compare with kub-batch, kub-flow, the new scheduler offer much better job through put.