Скачать или смотреть AI Seminar Series 2023: Zichen (Vincent) Zhang - A Simple Decentralized Cross-Entropy Method

The AI Seminar is a weekly meeting at the University of Alberta where researchers interested in artificial intelligence (AI) can share their research. Presenters include both local speakers from the University of Alberta and visitors from other institutions. Topics can be related in any way to artificial intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems.

Abstract:
In this talk, I will present a simple extension to the Cross-Entropy Method (CEM), a gradient-free optimization method frequently used for planning in model-based reinforcement learning (MBRL).

The classical CEM employs a centralized approach to update the sampling distribution based on a global top-k operation’s results on samples. However, we demonstrate that this approach can make CEM prone to local optima, thus impairing its sample efficiency. To address this issue, we propose Decentralized CEM (DecentCEM), a simple yet effective improvement over classical CEM, by using an ensemble of CEM instances running independently from one another, and each performing a local improvement of its own sampling distribution. We show in an optimization task that our DecentCEM finds the global optimum more consistently. than CEM that uses either a single or even a mixture of Gaussian distributions. Notably, this improvement does not compromise CEM’s convergence guarantee. When applied to MBRL planning problems in continuous control environments, DecentCEM shows an improved sample efficiency, with only a reasonable increase in computational cost.

For those interested in exploring our work further, please check out our paper at: https://arxiv.org/abs/2212.08235

And the code is available at https://github.com/vincentzhang/decen...

Presenter Bio:
Zichen (Vincent) Zhang is a Ph.D. student in Computing Science at University of Alberta, working with Prof. Dale Schuurmans and Prof. Martin Jagersand. He is interested in machine learning and its applications on robotics perception and control.

Currently he’s working toward developing efficient algorithms for continuous control.