Engineering Enterprise AI Agents: From Orchestration to Grounding

#ieeeday #ai #application #data-science #prototypes #architecture

Abstract:
Moving AI agents from prototypes to production requires more than a powerful language model. Enterprise systems must orchestrate agents and tools, manage context, enforce security and tenant isolation, integrate with existing services, and ground responses in complex organizational knowledge.

In this session, we explore these challenges through two complementary perspectives. The first part presents a working model inspired by an enterprise customer service scenario to demonstrate architectural patterns for building enterprise Copilot systems, including multi-tenant design, multi-agent and tool orchestration, security, graphical workflow design, and direct MCP-based tool invocation. The model illustrates how these capabilities can come together to enable flexible and governed agent workflows.

The second part focuses on keeping agents grounded so they answer from real data instead of hallucinating. Following one customer question end to end, it covers identity files that set an agent’s rules and boundaries, MCP-connected knowledge bases, and the search layer underneath. Topics include why vector search alone misses exact names and terms, how hybrid retrieval fixes that, how multimodal embeddings reach knowledge in tables and images, and the compute, memory and cost walls retrieval hits as knowledge bases grow to billions of documents, along with the techniques that get past them.

Together, the talks provide an end-to-end technical perspective on building production-oriented agentic AI systems, connecting orchestration, tools, security, context, and grounding into a cohesive architecture.

Speakers:

Anuj Kapoor is a Senior Software Engineer at Microsoft with over 16 years of experience building enterprise software and distributed systems. His current interests include agentic AI, enterprise Copilot architectures, multi-agent orchestration, context engineering, and secure AI systems. He enjoys exploring practical approaches for taking AI agents beyond prototypes and integrating them with complex enterprise applications.

Aaditya Chauhan is a Senior Software Development Engineer at Amazon and a specialist in large-scale search and retrieval systems, with over 12 years of experience in distributed systems and enterprise search. He leads a team building vector search, hybrid retrieval, and RAG infrastructure for large-scale consumer applications. His retrieval research has been published in IEEE Access, at ICEIS, and at EMNLP 2026 workshops, and he has written on hybrid search for InfoQ. He reviews for IEEE Access and for workshops at NeurIPS, KDD, and EMNLP and is an IEEE Senior Member.