The Self-Correcting RAG: Improving Retrieval via User Feedback Loops
Improving Retrieval via User Feedback Loops
Standard Retrieval-Augmented Generation works well out of the box, but in specialized technical domains, general-purpose embeddings struggle with internal terminology, jargon, and nuanced context. The conventional fix — fine-tuning models or rebuilding embeddings — is expensive, slow, and breaks down the moment your knowledge base changes.
There's a better way. In this workshop, we'll introduce FLAIR (Feedback Learning for Adaptive Information Retrieval), a lightweight framework that continuously adapts your retrieval strategy using domain-expert feedback — no model re-training required. You'll learn how FLAIR:
- Gathers real and LLM-synthesized query indicators offline to understand what experts actually need
- Deploys a two-track online ranking system that dynamically promotes relevant documents and filters out past retrieval errors — in real time
- Evolves with your knowledge base so retrieval quality improves continuously, not just after the next expensive rebuild
Due to Security, no walk-ins will be accepted. All attendees must register and log-in.
SPONSORED BY:
IEEE Computational Intelligence Society Boston Chapter and the IEEE Boston AI Local Group
Date and Time
Location
Hosts
Registration
-
Add Event to Calendar
- Microsoft NERD Center, Kendall Square
- 1 Memorial Drive
- Cambridge, Massachusetts
- United States 02142
- Room Number: NERD/1250, Abigail Adams
- Contact Event Hosts
-
For co-sponsor inquiries with the IEEE-Boston Local Group, reach out below:
- Co-sponsored by IEEE-Boston AI Local Group
Speakers
Nutan Sahoo of Microsoft Corp.
Feedback Learning for Adaptive Information Retrieval (FLAIR)
Nutan Sahoo, Applied AI Scientist at Microsoft, will present on Feedback Learning for Adaptive Retrieval.
In this workshop, she'll introduce FLAIR (Feedback Learning for Adaptive Information Retrieval), a lightweight framework that continuously adapts your retrieval strategy using domain-expert feedback — no model re-training required. You'll learn how FLAIR:
- Gathers real and LLM-synthesized query indicators offline to understand what experts actually need
- Deploys a two-track online ranking system that dynamically promotes relevant documents and filters out past retrieval errors — in real time
- Evolves with your knowledge base so retrieval quality improves continuously, not just after the next expensive rebuild
Who is this for? Anyone interested in RAG, and those building or maintaining RAG systems.
PAPER REFERENCED:
📄 FLAIR: Feedback Learning for Adaptive Information Retrieval
Biography:
Nutan Sahoo is an Applied AI Scientist at Microsoft, where she develops and deploys production-scale machine learning and generative AI solutions for enterprise applications. Her current work focuses on building RAG and agent-based AI systems to help on-call engineers resolve incidents faster, while also developing scalable evaluation pipelines to improve the performance and reliability of these generative AI systems.
She holds a master’s degree in Health Data Science from Harvard University. And her experience spans multiple industries, where she has applied machine learning to real-world problems at organizations such as Brigham and Women's Hospital, Children's Hospital of Philadelphia, Meta, and Microsoft.
Beyond her industry work, Nutan is actively involved in the AI research community, serving as a technical reviewer for leading AI conferences and participating in technical talks and community initiatives focused on generative AI and applied machine learning. Her recent research on high-fidelity synthetic ECG generation—conducted in collaboration with Mass General Brigham—was presented at and received the Best Paper Award at the Generative AI for Health Workshop at NeurIPS 2025.
Keywords: Artificial Intelligence, Machine Learning, Natural Language Processing, Information Retrieval, Adaptive Information Retrieval, Feedback, Workshop.