Скачать или смотреть Safe Reinforcement Learning in the Presence of Non-stationarity: Theory and Algorithms

Abstract: Despite the successes of reinforcement learning (RL) in simulation-based systems such as video games and Go, the existing RL techniques are not yet applicable or are too risky to employ in real-world autonomous systems. Those applications often require safety assurance, and the underlying environment may undergo changes and be nonstationary. While both aspects have been tackled separately in the literature to some limited extent, there remains a substantial gap when these issues arise simultaneously, imposing challenges for the deployment of concurrent methods in real-world systems. To overcome these challenges and realize the full potential of RL for adaptability and performance gains, we develop a new mathematical foundation and a set of computational tools for the design of safe RL algorithms that can be deployed in environments that undergo changes. Along this line, we will present the following three objectives: (1) non-stationary constrained Markov decision processes, (2) non-stationary risk-sensitive RL, and (3) meta-safe RL.

Bio: Yuhao Ding is currently a fifth-year Ph.D. student at UC Berkeley - Operations Research department. His research interests include reinforcement learning, control theory, optimization, and statistical learning. During his Ph.D., he focuses on non-stationary sequential decision-making problems such as time-varying optimization, the global convergence of policy gradient methods, and non-stationary reinforcement learning.