Safe Robot Autonomy in Interactive Open-World Environments

#IEEE #IEEE_Toronto #WIE #robotics #predictive-control #control #autonomy #digital-twin #modeling

Join the IEEE Toronto Instrumentation & Measurement – Robotics & Automation Joint Chapter for a technical talk presented by Dr. Roya Fallah Firoozi from University of Waterloo.

Monday, August 11, 2025 @ 1:00 – 2:00 PM (EST)

Abstract: As a robot manipulates 3D objects and navigates within 3D scenes, it requires spatial reasoning to ensure safe planning. Recent advances in 3D scene representation, such as Neural Radiance Fields (NeRFs) and Gaussian Splatting, provide high-fidelity digital twins of arbitrary real-world environments from multi-view images. In the first part of the talk, Dr. Firoozi will discuss employing these 3D visual fields augmented to 3D vision-language fields using internet-scale semantic representations from Vision-Language Models (VLMs) for open-vocabulary robot planning.

 As the robot interacts with other dynamic agents in the scene (multi-agent settings), it also requires temporal reasoning to ensure safe interactive planning. In the second part of the talk, Dr. Firoozi will discuss safe and fault-resilient planning techniques across two categories of interactive planning: (i) Model Predictive Control (MPC), where the prediction and planning steps are decoupled, and (ii) more abstract approaches such as game-theoretic planning, where these steps are tightly coupled. While MPC offers computational efficiency, game-theoretic planning enables more complex modeling of agents' preferences and their mutual influences.