At the 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2026), held in Seattle, USA, starting July 13th, a team led by Prof. LI Cheng from the University of Science and Technology of China (USTC), in collaboration with the ByteDance Seed Team, won the Jay Lepreau Best Paper Award for their work titled “Teaching The Old Dog New Tricks: Building Efficient Data Pipelines for Large-Scale LLM Pre-training.”

Prof. LI’s team won the Best Paper Award at OSDI 2026. (Image by USTC)

OSDI, alongside SOSP, is widely regarded as the "Oscars" of the systems field. As a flagship international conference in operating systems, distributed systems, cloud computing, and AI infrastructure, OSDI not only defines the academic frontier but also continues to shape the evolution of the global computing industry, standing as a bellwether for systems research.

For the first time, based on production data from over 30,000 training tasks, the paper systematically revealed three core bottlenecks facing pre-training data pipelines. These included cross-datacenter traffic overhead triggered by companion evaluation, initialization I/O storms during startup phases, and CPU-intensive bottlenecks resulting from multimodal data transformation. To address these challenges, the research team proposed three optimizations: predictive checkpoint replication, proactive hot-file replication, and storage-side data transformation offloading. Experimental results demonstrated that these methods significantly enhanced the overall efficiency of large-scale LLM training systems.