Proper Value Equivalence: Simplifying Model-based RL

Which aspects of a reinforcement learning environment should be modelled? In this paper we explore this question and more by analysing Proper Value Equivalence: Simplifying Model-based RL.

Proper Value Equivalence Paper (University of Michigan / DeepMind): https://arxiv.org/abs/2106.10316

We start by generalising the concept of VE to order-k counterparts defined with respect to k applications of the Bellman operator. This leads to a family of VE classes that increase in size as k → ∞. In the limit, all functions become value functions, and we have a special instantiation of VE which we call proper VE or simply PVE.

Unlike VE, the PVE class may contain multiple models even in the limit when all possible value functions and policies are considered. Crucially, all these models are sufficient for planning, meaning that they will yield an optimal policy despite the fact that they may ignore many aspects of the environment. Follow along as we explore this paper more in depth.

What are your thoughts about this research paper? Do you have any burning questions or ideas? Share below in the comments.👇

Please find the presented material here: https://dry-peak.cloudvent.net/Additi...

NiklasOPF
Do you want rapid access to the most fundamental aspects of a research paper? SUBSCRIBE 🔔
to explore more trusted content and cutting edge research, such as Computer Science Research Papers and more! ►    / @niklasopf  

Check out these other Research Papers! 👇👇

• Human-level control through deep reinforcement learning - Intro || NiklasOPF:    • Human-level control through deep rein...  
• Deterministic Policy Gradient Algorithms:    • Плейлист  
• DPGA - Theory 1:    • DPGA - Theory 1  


#propervalueequivalence #research #machinelearning #NiklasOPF #NiklasOPF #Phd #Scientists #Theory #Research #ResearchPapers #ScholarlyArticles #ComputerScience #deepmind #cornell