MedProb: Probing Internal Representations of Vision-Language Models for Medical Question Answering
Erfan Nourbakhsh will present on his PhD work.
Title: MedProb: Probing Internal Representations of Vision-Language Models for Medical Question Answering
See Preprint:
https://arxiv.org/abs/2609.04336
This paper reveals that standard AI models already contain a wealth of hidden medical knowledge that traditional testing methods miss. By using a simpler, direct evaluation technique called MedProb, researchers extracted remarkably accurate medical insights from smaller, general-purpose models without needing expensive specialized training.
Registration details:
Seating goes to those who register via this vtools event in the order received; plan ahead to REGISTER.
Registration Fees:
At the door, there is a $20.00 fee for Members, $30.00 for Guests and $5.00 for Students. We accept Cash or Check, no credit cards. Exact amounts preferable. Remainder of the tab will be picked up by the EMB.
Date and Time
Location
Hosts
Registration
-
Add Event to Calendar
- Alamo Cafe
- 14250 San Pedro Ave, San Antonio, TX 78232
- San Antonio , Texas
- United States 78232
- Room Number: Mission room
- Starts 10 October 2026 05:00 AM UTC
- Ends 23 October 2026 04:59 AM UTC
- No Admission Charge
Speakers
Erfan Nourbakhsh
MedProb: Probing Internal Representations of Vision-Language Models for Medical Question
Medical visual question answering (Med-VQA) is often assumed to require medical fine-tuning, large models, or complex multi-agent pipelines. We revisit this assumption with MedProb, a lightweight probing framework that predicts multiple-choice Med-VQA answers from frozen VLM representations without free-text generation. Across PATH-VQA, SLAKE, and VQA-RAD, MedProb recovers substantially more answer-relevant signal than prompting and performs stronger than medical VLMs and agentic systems. Probing also reduces the apparent gap between small and large models compared to prompting, suggesting that smaller VLMs contain more recoverable Med-VQA signal than generation-based evaluation reveals. Across 14 matched general-purpose and medical VLM pairs, medical adaptation does not consistently improve this linear decodability. Finally, free-text generation exhibits an answer-position bias of up to 10 percentage points, whereas MedProb also has positional bias, however, it is impacted differently than prompting. Our main results target the multiple-choice/multiclass Med-VQA setting; we additionally show the probe can be extended to open-ended generation via a rejection-sampling scoring procedure.
Preprint is available: https://arxiv.org/abs/2609.04336
Biography:
Erfan Nourbakhsh is a Ph.D. student in Computer Science at the University of Texas at San Antonio (UTSA) under the supervision of Drs. Ke Yang and Anthony Rios. His research focuses on Medical AI, Natural Language Processing (NLP), Biomedical Informatics, Graph Neural Networks, and Multimodal AI, with an emphasis on developing intelligent systems for healthcare and related interdisciplinary applications. His research has been published or accepted at venues such as EMNLP, BioNLP, and SIGDIAL. In 2026, he received the Outstanding and Innovative Research Award at UTSA’s Graduate Student Appreciation Week (GSAW) Symposium, recognizing his research poster and its impact.
Agenda
11:00 - Welcome
11:30 - Introduction and presentation
12:00 - Lunch
12:30 - Q and A
13:00 - Adjourn