From Sensing to Understanding: World Models for Semantic-Aware Collaborative Perception
Autonomous mobility systems increasingly rely on collaborative perception to overcome occlusion, limited field of view, and social navigation challenges in dynamic environments. However, effective collaboration is not simply about sharing more sensing data; it requires identifying information that is semantically valuable for a mobility agent’s task, decision-making, and evolving situational awareness. This talk explores how collaborative perception can move from extensive sensing to comprehensive understanding through world models. We begin with recent advances in vision-language models for semantic-aware perception, while highlighting key limitations: insufficient sensing data for reliable reasoning and the time-varying nature of perception evidence. To address these challenges, we introduce world models for evaluating collaboration policies that maintain reliable situational awareness as sensing coverage, mobility patterns, and communication conditions evolve. By predicting whether and how collaboration can improve semantic confidence under evolving sensing, mobility, and communication conditions, this approach transforms collaboration from reactive raw-data sharing into predictive, semantic-aware communication and policy reasoning, enabling autonomous systems to proactively identify efficient collaboration patterns.
Date and Time
Location
Hosts
Registration
-
Add Event to Calendar
- University of Waterloo
- Waterloo, Ontario
- Canada
- Building: Centre for Environmental & Information Technology (EIT)
- Room Number: 4152
- Starts 25 May 2026 04:00 AM UTC
- Ends 02 June 2026 04:00 AM UTC
- No Admission Charge
Speakers
Mushu Li of Lehigh University
From Sensing to Understanding: World Models for Semantic-Aware Collaborative Perception
Autonomous mobility systems increasingly rely on collaborative perception to overcome occlusion, limited field of view, and social navigation challenges in dynamic environments. However, effective collaboration is not simply about sharing more sensing data; it requires identifying information that is semantically valuable for a mobility agent’s task, decision-making, and evolving situational awareness. This talk explores how collaborative perception can move from extensive sensing to comprehensive understanding through world models. We begin with recent advances in vision-language models for semantic-aware perception, while highlighting key limitations: insufficient sensing data for reliable reasoning and the time-varying nature of perception evidence. To address these challenges, we introduce world models for evaluating collaboration policies that maintain reliable situational awareness as sensing coverage, mobility patterns, and communication conditions evolve. By predicting whether and how collaboration can improve semantic confidence under evolving sensing, mobility, and communication conditions, this approach transforms collaboration from reactive raw-data sharing into predictive, semantic-aware communication and policy reasoning, enabling autonomous systems to proactively identify efficient collaboration patterns.
Biography:
Dr. Mushu Li received the Ph.D. degree in Electrical and Computer Engineering from the University of Waterloo, ON, Canada, in 2021. She is an Assistant Professor in the Department of Computer Science and Engineering at Lehigh University, USA. Prior to this, she was a Postdoctoral Fellow at Toronto Metropolitan University (from 2022 to 2024) and the University of Waterloo (from 2021 to 2022), Canada. Her research interests include machine learning for communications and networking, cloud and multi-access edge computing, Intelligent Internet of Things (IoT), and next-generation (6G) wireless networks. She has served as an associate editor for Peer-to-Peer Networking and Applications since 2023 and IEEE Wireless Communications Letters since 2025. Dr. Li received the Best Land Transportation Paper Award from the IEEE Vehicular Technology Society in 2024, the NSERC Postdoctoral Fellowship in 2022, and the NSERC Canada Graduate Scholarship in 2018.
Email:
Address:Lehigh University, , Bethlehem, Pennsylvania, United States