Towards Application-Oriented Scheduling in Next-Generation Data Center Networks

[摘要] Today, data centers are important facilities that power a variety of distributed applications to support large-scale computation. To meet the surging traffic demand from these applications, a range of unconventional network designs have been proposed. Scheduling application traffic towards the objective of high communication efficiency at the application level, or application-oriented network scheduling, is the crucial component to bridge the gap between complex application traffic and the underlying network architectures. There are four equally important challenges towards application-oriented scheduling in data center networks. First, given application traffic requirements, how to design application-oriented scheduling algorithms on novel network architectures. Second, how data center network architectures evolve and how to adapt application-oriented scheduling algorithms to the new architecture. Third, how to exploit endpoint flexibility in distributed applications to improve the traffic scheduling efficiency at the application level. Fourth, how to characterize the communication patterns of emerging distributed applications to design application-aware network schedulers. My thesis studied how to address these challenges with a range of work. First, Sunflow is the first scheduling algorithm to support application-oriented scheduling in the emerging optical circuit switched network. Second, Weaver enables application-oriented scheduling in Heterogeneous Parallel Networks, a network architecture that supports incremental network evolutions. Third, 2D-Placement is a traffic placement algorithm that improves the application-oriented scheduling efficiency by exploiting the endpoint flexibility of application traffic. Fourth, TensorLights is an end-host traffic scheduler to provide application-oriented scheduling for distributed deep learning, an emerging branch of distributed applications. These efforts have demonstrated the advantages of application-oriented scheduling on a range of novel network architectures for data centers.

[发布机构] Rice University

[效力级别] Center