Jonas Buchli & Federico Felici: Magnetic control of tokamak plasmas with deep reinforcement learning

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A key challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires regulating the plasma position and shape via magnetic fields generated by a set of control coils. In this work, EPFL and DeepMind introduce a new architecture for designing a tokamak magnetic controller based on deep reinforcement learning. The controller is entirely trained on a physics-based simulator and then deployed on the tokamak hardware. They successfully produced and controlled a diverse set of plasma configurations on the Tokamak à Configuration Variable (TCV) device, including a new configuration featuring two plasmas in the vessel simultaneously. The control architecture replaces separate controllers used in traditional architectures with a single control policy and allows focus on ‘what’ to control rather than ‘how’. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied.