Minimax Problems in Reinforcement Learning
We will discuss progress on two minimax problems in reinforcement learning. The first is a standard Markov game, where two agents are competing for rewards in an environment described by a Markov decision process. Given a description of this environment, our goal is to compute a Nash equilibrium, a pair of policies that neither agent can improve on in response to the other. We show that a simple gradient descent/ascent algorithm converges linearly for a regularized version of this problem, and we can acheive sublinear convergence to the true equilibrium through careful manipulation of the regularization parameter.
In the second part of the talk, we will discuss a standard form of robust reinforcement learning where a (single) agent searches for a policy that has the best worst-case performance over a convex set of reward functions. Although this problem is convex in one variable but non-concave in the other, we show that there is minimax equality. Key to this result is showing that the superlevel sets of the long-term reward for the agent are connected. We also show that the minimax result extends to policies that are parameterized with neural networks.
Dr. Romberg is also presenting another talk at 6pm Dimensionality Reduction For Sensor Arrays
Date and Time
Location
Hosts
Registration
- Date: 13 Dec 2023
- Time: 01:00 PM to 02:00 PM
- All times are (UTC-05:00) Eastern Time (US & Canada)
-
Add Event to Calendar
- Syracuse University
- 111 College Pl
- Syracuse, New York
- United States 13210
- Building: Center of Science & Technology
- Room Number: CST 4-201
- Click here for Map
- Contact Event Host
-
Contact Sam Stone stone@ieee.org
- Starts 26 November 2023 11:04 AM
- Ends 13 December 2023 11:00 AM
- All times are (UTC-05:00) Eastern Time (US & Canada)
- No Admission Charge
Speakers
Justin Romberg of Georgia Institute of Technology
Biography:
Dr. Justin Romberg is the Schlumberger Professor in the School of Electrical and Computer Engineering and Associate Director of the Center for Machine Learning at the Georgia Institute of Technology, where he has been on the faculty since 2006. Dr. Romberg received the B.S.E.E. (1997), M.S. (1999) and Ph.D. (2004) degrees from Rice University in Houston, Texas. From Fall 2003 until Fall 2006, he was a Postdoctoral Scholar in Applied and Computational Mathematics at the California Institute of Technology. In 2008 he received an ONR Young Investigator Award, in 2009 he received a PECASE award and a Packard Fellowship, and in 2010 he was named a Rice University Outstanding Young Engineering Alumnus, and in 2021 he received the IEEE Jack S. Kilby Signal Processing Medal. He is a Fellow of the IEEE.
Broadly speaking, Dr. Romberg’s research interests are in the intersection of signal processing, optimization, and machine learning. One of his current interests is how online and distributed algorithms developed for statistical learning and inference can be used for next-generation sensor arrays.