Скачать или смотреть Tea Time Talks 2024: Mahshid Rahmani Hanzaki, Tile-coding for Count-based Exploration

Tea Time Talks are back for another year. This summer lecture series, presented by Amii and the RLAI Lab at the University of Alberta, give researchers the chance to discuss early-stage ideas and prospective research. Join us for another series of informal 20-minute talks where AI leaders discuss the future of machine learning research.

Abstract:
Exploration-exploitation tradeoff is one of the challenges in reinforcement learning where the agent must tradeoff between choosing actions that have previously been effective in producing rewards or trying actions it has not yet explored. Despite recent advances in reinforcement learning, most complex agents still rely on randomness to explore the environment because of its simplicity.

An alternative to random exploration is count-based methods, where actions with fewer visitation counts are preferred over those that have been visited more frequently. Despite their theoretical guarantees, count-based exploration methods have not been widely used with function approximation in practice.

In this talk, I will explain how tile-coding can be used as a simple method to generalize counts over states. I will highlight two experiments to demonstrate how tile-coding for count-based methods can lead to better exploration compared to randomness in certain environments.