Скачать или смотреть Machine Learning Security Seminar Series - Xin Chen (ETH Zurich)

Title: Learning Safety Constraints for Large Language Models
Speaker: Xin Chen, Cynthia

Abstract: Large language models (LLMs) have emerged as powerful tools but pose significant safety risks through harmful outputs and vulnerability to adversarial attacks. We propose SaP, short for Safety Polytope, a geometric approach to LLM safety that learns and enforces multiple safety constraints directly in the model's representation space. We develop a framework that identifies safe and unsafe regions via the polytope's facets, enabling both detection and correction of unsafe outputs through geometric steering. Unlike existing approaches that modify model weights, SaP operates post-hoc in the representation space, preserving model capabilities while enforcing safety constraints. Experiments across multiple LLMs demonstrate that our method can effectively detect unethical inputs, reduce adversarial attack success rates while maintaining performance on standard tasks, thus highlighting the importance of having an explicit geometric model for safety. Analysis of the learned polytope facets reveals emergence of specialization in detecting different semantic notions of safety, providing interpretable insights into how safety is captured in LLMs' representation space.

Short bio: Xin Chen, Cynthia is a PhD student at ETH Zurich, supervised by Profs. Andreas Krause and Florian Tramer. Her research focuses on Large Language Model safety and alignment, combining principled methods with empirical findings to make LLM safety mechanisms more trustworthy. Cynthia is a fellow at the Open Philanthropy AI Fellowship and the Vitalik Buterin PhD Fellowship.