Notice

A paper have been accepted for publication at ATC 2020. This paper is co-authored by Jay H. Park, Gyeongchan Yun, Chang M. Yi, Nguyen T. Nguyen, Seungmin Lee, Jaesik Choi (KAIST), Sam H. Noh, and Young-ri Choi.

A research paper entitled “HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism” proposes novel DNN training system, HetPipe, that integrates pipelined model parallelism with data parallelism. Leveraging multiple virtual workers, each of which consists of multiple, possibly whimpy, heterogeneous GPUs, HetPipe makes it possible to efficiently train large DNN models.

USENIX ATC (USENIX Annual Technical Conference) is one of the top conferences in the field of computer systems.