Скачать или смотреть Andrei Lupu and Brandon Cui: Adversarial Diversity in Multi Agent Coordination

Abstract: Many Dec-POMDPs admit a qualitatively diverse set of ''reasonable'' joint policies, where reasonableness is indicated by symmetry equivariance, non-sabotaging behaviour and the graceful degradation of performance when paired with ad-hoc partners. Some of the work in diversity literature is concerned with generating these policies. Unfortunately, existing methods fail to produce teams of agents that are simultaneously diverse, high performing, and reasonable. In this work, we propose a novel approach, adversarial diversity (ADVERSITY), which is designed for turn-based Dec-POMDPs with public actions. ADVERSITY relies on off-belief learning to encourage reasonableness and skill, and on ''repulsive'' fictitious transitions to encourage diversity. We use this approach to generate new agents with distinct but reasonable play styles for the card game Hanabi and open-source our agents to be used for future research on (ad-hoc) coordination.

Andrei Lupu: Andrei is a first year PhD student at the University of Oxford and Meta AI London, supervised by Prof. Jakob Foerster and Roberta Raileanu. Prior to that he completed a M.Sc. at Mila with Prof. Foerster and Prof. Doina Precup, where he studied the importance of policy diversity in zero-shot coordination. Following TrajeDi and ADVERSITY, his current focus is on explainable reinforcement learning and few-shot partner adaptation. He also firmly believes that research on cooperative AI should not shy away from the wonderful challenges posed by stochasticity and partial observability.

Brandon Cui: Brandon is a Research Engineer at MosaicML. Prior to that, he spent 3 years at Meta AI (FAIR) working in deep reinforcement learning, with a particular focus in cooperative multi-agent reinforcement learning. Currently, he is focused on efficient ML and incorporating feedback in large scale generative AI.