Robust depth estimation for robots with stereo cameras using bespoke DNNs
Santa Clara Valley Robotics and Automation Society Chapter
Autonomous robots depend on their perception systems to understand the world around them. These machines often leverage a host of sensors including cameras, lidars, radars, and ultrasonic sensors to create this environmental understanding. Stereo cameras play a big role in providing depth perception to robotic systems. This depth information can be estimated using classical computer vision techniques, like semi-global matching (SGM) or leverage deep neural networks (DNNs). Each individual algorithm may struggle in a particular set of operating conditions. But when multiple depth estimation algorithms are leveraged simultaneously, It is possible that more robust depth information can be calculated.
In this talk, we'll cover work at NVIDIA to train the ESS DNN model for determining stereo disparity using both synthetic and real-world data to perform well where SGM may not. We'll also introduce the Bi3D model which is trained on the simplified question of "is X closer than M meters?" rather than "how far away is X?", yielding improvements in both accuracy and speed. As every approach has deficiencies on its own, we'll touch upon how ensembling the responses of ESS and Bi3D, DNNs developed specifically for robotic perception with SGM could lead to robust obstacle detection. Finally, we'll discuss how we've tuned the performance of these models to run on embedded compute for the responsive stopping behavior required in autonomous mobile robots (AMRs).
Date and Time
Location
Hosts
Registration
- Date: 07 Dec 2022
- Time: 06:30 PM to 08:00 PM
- All times are (UTC-08:00) Pacific Time (US & Canada)
-
Add Event to Calendar
- Starts 17 November 2022 09:16 AM
- Ends 05 December 2022 10:00 PM
- All times are (UTC-08:00) Pacific Time (US & Canada)
- No Admission Charge
Speakers
Hemal Shah of NVIDIA
Biography:
Hemal Shah, Director of Robotics Solutions Engineering at NVIDIA: Hemal leads Isaac ROS at NVIDIA, delivering high-performance computing to the ROS2 ecosystem to power intelligent robotics. Prior to joining NVIDIA, Hemal has led engineering teams at Google, X, and Boston Dynamics, working on logistics automation, fleet management, perception, and machine learning at scale. He holds a BS in computer science from Georgia Institute of Technology and a MS in computer science from Stanford University focusing on ML and robotics.
Gerard Andrews of NVIDIA
Biography:
Gerard Andrews, Sr Product Manager - Robotics at NVIDIA: Gerard Andrews is focused on revolutionizing the way intelligent robots are developed, trained, tested and deployed using the NVIDIA Isaac robotics platform. Prior to joining NVIDIA, Gerard was at Cadence where he was Product Marketing Director, responsible for product planning, marketing, and business development for licensable processor IP. He holds a BS in electrical engineering from Southern Methodist University and an MS in electrical engineering from Georgia Institute of Technology.
Agenda
6:30 PM Introduction (Tom Coughlin)
6:45 PM Talk
7:30 PM Q&A
8:00 PM End