Near-Sensor Computing and Model–Hardware Co-Design for Energy-Efficient Vision, Generative AI, and Healthcare
Abstract: The rapid emergence of generative AI and vision-language models (VLMs) is transforming multimodal perception and reasoning, but deployment in real-world edge systems remains constrained by bandwidth, memory movement, and energy. I will present a near-sensor software–hardware co-design framework that moves early feature extraction and compression closer to where data are acquired. Instead of treating the sensor, memory, and processor as separate stages, we co-optimize models and hardware to form compact, task-specific representations at the sensor boundary. This reduces off-chip transfer and DRAM energy while preserving downstream accuracy, enabling more scalable edge intelligence for vision and multimodal AI. I will then show how this broader principle extends beyond visual sensing to AI for healthcare. In wearable health AI, efficient inference requires models that can adaptively process multimodal physiological signals, activate sensing only when needed, and remain reliable under incomplete observations. Looking forward, this points toward closed-loop medical systems that not only sense and infer efficiently, but also guide timely actions or interventions. Finally, I will discuss trustworthy clinical AI, where generative models for medical reports must be evaluated not only for fluency, but also for consistency, factual grounding, and verifiable reasoning. Together, these directions outline a path toward efficient, adaptive, and trustworthy AI systems across edge and healthcare applications.
Date and Time
Location
Hosts
Registration
-
Add Event to Calendar
- Rice University
- 6100 Main Street
- Houston, Texas
- United States 77005
- Building: Duncan Hall
- Room Number: 3076
- Contact Event Hosts
-
Prof. Joseph Cavallaro, cavallar@rice.edu
- Co-sponsored by Rice University ECE and Rice IEEE HKN and Student Branch
Speakers
Gourav Datta of Case Western Reserve University
Near-Sensor Computing and Model–Hardware Co-Design for Energy-Efficient Vision, Generative AI, and Healthcare
Abstract: The rapid emergence of generative AI and vision-language models (VLMs) is transforming multimodal perception and reasoning, but deployment in real-world edge systems remains constrained by bandwidth, memory movement, and energy. I will present a near-sensor software–hardware co-design framework that moves early feature extraction and compression closer to where data are acquired. Instead of treating the sensor, memory, and processor as separate stages, we co-optimize models and hardware to form compact, task-specific representations at the sensor boundary. This reduces off-chip transfer and DRAM energy while preserving downstream accuracy, enabling more scalable edge intelligence for vision and multimodal AI. I will then show how this broader principle extends beyond visual sensing to AI for healthcare. In wearable health AI, efficient inference requires models that can adaptively process multimodal physiological signals, activate sensing only when needed, and remain reliable under incomplete observations. Looking forward, this points toward closed-loop medical systems that not only sense and infer efficiently, but also guide timely actions or interventions. Finally, I will discuss trustworthy clinical AI, where generative models for medical reports must be evaluated not only for fluency, but also for consistency, factual grounding, and verifiable reasoning. Together, these directions outline a path toward efficient, adaptive, and trustworthy AI systems across edge and healthcare applications.
Biography:
Bio: Dr. Gourav Datta is an Assistant Professor in the Department of Electrical, Computer, and Systems Engineering (ECSE) at Case Western Reserve University (CWRU), with a joint appointment in the Department of Computer and Data Sciences (CDS). Before that, he was an Applied Scientist at Amazon AGI, where he contributed to enhancing the video understanding capabilities of Amazon Nova, a next-generation foundation model. He received his Ph.D. in Electrical Engineering from the University of Southern California (USC) in 2023 and his B.Tech. from IIT Kharagpur in 2018. His research interests include efficient algorithm-hardware co-design for edge deep learning and AI for healthcare. His major honors include the USC Viterbi School of Engineering’s Best Ph.D. Dissertation Award, the 2022 IEEE Graduate Fellowship in Applied Superconductivity, the Best Research Assistant Award from USC ECE, and recognition as the academically best outgoing student in his class at IIT Kharagpur. He has published more than 60 peer-reviewed papers in top-tier venues including ICLR, ICML, ECCV, TMLR, TCAS-I/II, DATE, DAC, ICCAD, npj Unconventional Computing, among others, and received a best paper award nomination at VLSI-SoC 2022. His research on in-sensor computing has been highlighted multiple times by Edge Impulse, a leading semiconductor IP company for TinyML, as well as by the USC Viterbi School of Engineering and the Case School of Engineering. He is a Senior Member of the IEEE.
Email:
Address:Case Western Reserve University, , Cleveland, Ohio, United States
Agenda
Presentation at 4:00 to 5:00 pm CDT at Duncan Hall, Room 3076 at Rice University