Dr. Yuejie Chi

Abstract. Communication efficiency has been widely recognized as the bottleneck for large-scale decentralized machine learning applications in multi-agent or federated environments. To tackle this bottleneck, this talk will highlight trade-offs and strategies to improve communication efficiency in decentralized nonconvex optimization, where the clients are only allowed to communicate with their neighbors over a predefined graph topology. Two communication-efficient algorithms for finding first-order stationary points will be introduced: (1) DESTRESS, which matches the optimal incremental first-order oracle complexity of centralized algorithms, while maintaining communication efficiency by performing stochastic variance-reduced local gradient updates; and (2) BEER, which is the first algorithm with communication compression to match the convergence rate without compression under arbitrary data heterogeneity. We will highlight algorithmic ideas that are crucial in the algorithm developments, including but not limited to gradient tracking, compression with error feedbacks, and variance reduction.

Biography:

Dr. Yuejie Chi is a Professor in the department of Electrical and Computer Engineering, and a faculty affiliate with the Machine Learning department and CyLab at Carnegie Mellon University. She received her Ph.D. and M.A. from Princeton University, and B. Eng. (Hon.) from Tsinghua University, all in Electrical Engineering. Her research interests lie in the theoretical and algorithmic foundations of data science, signal processing, machine learning and inverse problems, with applications in sensing and societal systems, broadly defined. Among others, Dr. Chi received the Presidential Early Career Award for Scientists and Engineers (PECASE), the inaugural IEEE Signal Processing Society Early Career Technical Achievement Award for contributions to high-dimensional structured signal processing, and held the inaugural Robert E. Doherty Early Career Development Professorship. She was named a Goldsmith Lecturer by IEEE Information Theory Society and a Distinguished Lecturer by IEEE Signal Processing Society.

Email:

Address:United States