Dr. Jia (Kevin) Liu

The growing computational abilities of modern GPU-powered data centers have enabled industrial giants to train and deploy large language models (LLMs) on massive amounts of data. Yet the enormous scale of both models and training data makes centralized development increasingly difficult to sustain, while simultaneously excluding smaller institutions that possess relevant private data but lack equivalent infrastructure to participate in LLM training. In this research, we consider distributed LLM training in both pretraining and fine-tuning paradigms, covering federated learning, split learning, and decentralized approaches, and assess their readiness for geo-distributed settings with heterogeneous hardware and unreliable connectivity. We then present a unified system architecture built around a cloud-based parameter server that coordinates heterogeneous clients, using the same communication, compression, and fault-tolerance infrastructure for both pretraining and a family of fine-tuning methods. We validate the design with real geo-distributed experiments on GPT-2 Medium and Llama3-1B pretraining across multiple university sites, showing stable convergence under realistic conditions. 

Biography:

Jia (Kevin) Liu is an Associate Professor in the Dept. of Electrical and Computer Engineering at The Ohio State University (OSU) and an Amazon Scholar with Amazon.com. He received his Ph.D. degree from the Dept. of Electrical and Computer Engineering at Virginia Tech in 2010. From Aug. 2017 to Aug. 2020, he was an Assistant Professor in the Dept. of Computer Science at Iowa State University (ISU). He currently serves as the Managing Director of the NSF AI Institute for Future Edge Networks and Distributed Intelligence (AI-EDGE) at OSU. Dr. Liu's research areas include theoretical machine learning, stochastic network optimization and control, and performance analysis for data analytics infrastructure and cyber-physical systems. Dr. Liu is a senior member of IEEE and a member of ACM. He has received numerous best paper awards at top venues, including IEEE INFOCOM'19 Best Paper Award, IEEE INFOCOM'16 Best Paper Award, IEEE INFOCOM'13 Best Paper Runner-up Award, IEEE INFOCOM'11 Best Paper Runner-up Award, and IEEE ICC'08 Best Paper Award. He has also received multiple honors of long/spotlight presentations at top machine learning conferences, including ICML, NeurIPS, and ICLR. His joint work with IBM Research was selected to receive the IBM Pat Goldberg Memorial Best Paper Award Distinction of Honorable Mention in 2024. Dr. Liu is an NSF CAREER Award recipient in 2020, a winner of the DARPA Young Faculty Award (YFA) in 2024, and a winner of the Google Faculty Research Award in 2020. He received the LAS Award for Early Achievement in Research at Iowa State University in 2020, and the Bell Labs President Gold Award. Dr. Liu is the Lead Editor of the Special Issue on AI and Networking of IEEE/ACM Transactions on Networking in 2025. He is an Associate Editor for IEEE Transactions on Cognitive Communications and Networking. He has served the TPC for numerous top conferences, including ICML, NeurIPS, ICLR, ACM SIGMETRICS, IEEE INFOCOM, and ACM MobiHoc. His research is supported by NSF, DARPA, AFOSR, AFRL, ONR, Google, Meta, and Cisco.