Analytic Continual Learning with a Fast and Non-forgetting Closed-form Solution by Prof. Zhiping Lin
Continual learning is a vital area of deep learning that enables trained models to acquire new tasks incrementally, without retraining from scratch for each task. This talk introduces Analytic Continual Learning (ACL), a novel branch of continual learning that leverages classical recursive least squares methods. By employing simple linear algebra, ACL brings closed-form recursive algorithms into deep continual learning, achieving an equivalence between continual learning and joint training. As a result, deep models can be trained in a single epoch while maintaining high training speed and near-zero forgetting. This approach has been successfully applied to conventional continual learning settings as well as highly challenging scenarios involving large models.
Date and Time
Location
Hosts
Registration
-
Add Event to Calendar
- Binary Coffee, Qingshuihe Campus, University of Electronic Science and Technology of China
- Chengdu, Sichuan
- China
Speakers
Analytic Continual Learning with a Fast and Non-forgetting Closed-form Solution
Continual learning is a vital area of deep learning that enables trained models to acquire new tasks incrementally, without retraining from scratch for each task. This talk introduces Analytic Continual Learning (ACL), a novel branch of continual learning that leverages classical recursive least squares methods. By employing simple linear algebra, ACL brings closed-form recursive algorithms into deep continual learning, achieving an equivalence between continual learning and joint training. As a result, deep models can be trained in a single epoch while maintaining high training speed and near-zero forgetting. This approach has been successfully applied to conventional continual learning settings as well as highly challenging scenarios involving large models.
Biography:
Zhiping Lin (S’86-M’88-SM’00) received B.Eng degree in control and automation from South China Institute of Technology, Guangzhou, China, Ph.D. degree in information engineering from the University of Cambridge, UK, in 1987. Since 1999, he has been an associate professor at the School of EEE, Nanyang Technological University (NTU), Singapore, where he is now serving as Programme Director of Master of Sciences Programme, the School of EEE, NTU. Prior to that, he worked at DSO National Labs, Singapore and Shantou University, China. Dr. Lin was the Editor-in-Chief of Multidimensional Systems and Signal Processing from 2011 to 2015, and a subject editor of the Journal of the Franklin Institute from 2015 to 2019. He also served as an associate editor for several other international journals, including IEEE Trans. on Circuits and Systems II. He was the coauthor of the 2007 Young Author Best Paper Award from the IEEE Signal Processing Society (SPS), and a Distinguished Lecturer of the IEEE Circuits and Systems Society (CASS) during 2007-2008. He served as IEEE CASS Singapore Chapter Chair in 2007-2008 and 2019, and is now serving as IEEE SPS Singapore Chapter Chair. His research interests include multidimensional systems, statistical signal processing and image/video processing, robotics, and machine learning. He has published more than 200 journal papers and over 200 conference papers.