#robots #uncertainty #safety #humans

Abstract: Ensuring that AI systems do what we, as humans, actually want them to do, is one of the biggest open research challenges in AI alignment and safety. Dr. Brown's research seeks to directly address this challenge by enabling AI systems to interact with humans to learn aligned and robust behaviors. The way robots and other AI systems behave is often the result of optimizing a reward function. However, manually designing good reward functions is highly challenging and error prone, even for domain experts. Although reward functions for complex tasks are difficult to manually specify, human feedback in the form of demonstrations or preferences is often much easier to obtain. However, human data is often difficult to interpret due to ambiguity and noise. Thus, it is critical that AI systems take into account uncertainty over the human's true intent. Dr. Brown's talk will give an overview of my lab's progress along the following fundamental research areas: (1) efficiently maintaining uncertainty over human intent, (2) directly optimizing behavior to be robust to uncertainty, and (3) actively querying for additional human input to reduce uncertainty.