Students

Past Interns and Advised Students

• Chengming Zhang (co-advised with Leon Song), summer Ph.D. intern from Indiana University Bloomington, April--September 2023
  • DeepSpeed-Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models, published at PODC 2024
  • DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies, published at NeurIPS AI4Science Workshop 2023
• Xinyue Ma (co-advised with Myeongjae Jeon from Ulsan National Institute of Science and Technology), April 2022-June 2023
• Cost-effective On-device Continual Learning over Memory Hierarchy with Miro, published at MobiCom 2023
• Jiangfei Duan and Ziang Song (co-advised with Zhihao Jia from CMU), June 2022-December 2022
• Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances, published at NSDI 2024
• Yongye Su (co-advised with Jianguo Wang from Purdue University), July 2022-September 2023
• Serverless vector search database, published at SIGMOD 2024
• Shuangyan Yang (co-advised with Dong Li from UC Merced), 2021 June--Present
• Betty: Enabling Large-Scale GNN Training with Batch-Level Graph Partitioning, published at ASPLOS 2023
• Yucheng Lu, summer Ph.D. intern from Cornell University, June 2021-Feburary 2022
• Maximizing the communication efficiency of DNN training with 0/1 Adam, published at ICML 2023
• Soobee Lee (co-advised with Myeongjae Jeon from Ulsan National Institute of Science and Technology), September 2020-November 2021
• CarM: hierarchical episodic memory for continual learning, published at DAC 2022
• Connor Holmes (co-advised with Bo Wu from Colorado School of Mines), June 2020--September 2022
• NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM, published at NeurIPS 2021
• Compressing Pre-trained Transformers via Low-Bit NxM Sparsity for Natural Language Understanding
• Hongyi Wang, summer Ph.D. intern from University of Wisconsin Madison, June 2020-September 2020
• Efficient DNN training via elastic tranining and low-rank decomposition
• Zhen Peng (co-advised with Bin Ren from College of Walliam and Mary and Ruoming Jin from Kent University), Sep 2019-June 2021
• iQAN: Fast and Accurate Vector Search with Efficient Intra-Query Parallelism on Multi-Core Architectures, published at PPoPP 2023
• Jie Ren (co-advised with Dong Li from University of California at Merced), June 2019-December 2020
• Sentinel: Efficient tensor migration and allocation on heterogeneous memory systems for deep learning, published at HPCA 20213
• Hm-ann: Efficient billion-point nearest neighbor search on heterogeneous memory, published at NeurIPS 2020
• ZeRO-Offload: Democratizing Billion-Scale Model Training, published at USENIX ATC 2021
• Dantong Zhu, visiting scholar, Janurary 2020-June 2020
• Understanding and Generalizing Monotonic Proximity Graphs for Approximate Nearest Neighbor Search
• Zehua Hu, intern student from Peking University, July 2019-March 2021
• DUET: A Compiler-Runtime Subgraph Scheduling Approach for Tensor Programs on a Coupled CPU-GPU Architecture, published at IPDPS 2021
• Menghao Li, intern student from Peking University, February 2020-March 2021
• AdaTune: Adaptive Tensor Program Compilation Made Efficient, published at NeurIPS 2020
• DynaTune: Dynamic Tensor Program Optimization in Deep Neural Network Compilation, published at NeurIPS 2021
• Conglong Li (co-advised with Dave Andersen from CMU), May 2019-August 2019
• Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination, published at SIGMOD 2020
• Stephen Zhou, summer PhD intern from MIT, June 2018-August 2018
• Automatic model optimization via deep learning compiler