Generative AI is transforming the landscape of scientific discovery across three distinct operational pathways: combinatorial space search, data-driven hypothesis generation, and paradigmatic reframing. However, each pathway introduces unique evaluation challenges that require a formal discipline of AI metrology—a systematic framework for quantifying system reliability, operational scale, and human oversight requirements.

This talk presents a structured metrological roadmap mapping established cognitive theories (such as the Cattell-Horn-Carroll hierarchy) and stages of AI capability—Artificial Limited Intelligence (ALI), Artificial General Intelligence (AGI), and Artificial Superintelligence (ASI)—to these three modes of discovery:

• ALI & Combinatorial Search: Focuses on optimizing search bounds, parameter scaling, and uncertainty quantification, with humans acting as ground-truth oracles to measure correction rates and time-to-override.
• AGI & Hypothesis Generation: Addresses explore-exploit cycles via dual-layer neuro-symbolic governance, formal ontology mapping, and the Garland Test (evaluating perceived problem awareness), while humans serve as symbolic arbiters to calibrate appropriate reliance.
• ASI & Paradigmatic Reframing: Reconceptualizes fundamental laws through asynchronous multi-agent scaffolding ("A-Teams"), placing humans in an on-the-loop auditing role measured by spot-audit coverage and escalation latency.

Finally, the talk outlines actionable tracks for institutions to standardize evaluation protocols across this spectrum, emphasizing that AI metrology must evolve qualitatively—transitioning from output verification to trust calibration and system-level auditing—to ensure scientific claims remain trustworthy and reproducible.

Biography:

Ram D. Sriram is a Senior Science Advisor, at the National Institute of Standards and Technology. Prior to joining NIST, he was on the engineering faculty (1986-1994) at the Massachusetts Institute of Technology (MIT) and was instrumental in setting up the Intelligent Engineering Systems Laboratory. Sriram has co-authored or authored more than 300 publications, including several books.   Sriram was a founding co-editor of the International Journal for AI in Engineering. Ram received many awards, including five life -ime achievement/pioneer awards and a distinguished alumnus award from IIT Madras, India. He has also been made a fellow of prominent engineering (ASME, IET, INCOSE, SME, SMA), computer science (ACM, IEEE), medical (AIMBE), and science (AAAS, WAS) societies. In 2023, Sriram  was elected as an honorary member of the Institute of Industrial and Systems Engineers – the highest honor IISE grants to an individual of acknowledged professional eminence who is not a member of IISE. Sriram has a B.Tech. from IIT, Madras, India, and an M.S. and a Ph.D. from Carnegie Mellon University, Pittsburgh, USA.