Скачать или смотреть AI + Math Chat #2 - Formalizing Proofs & Breaking Codes

In this second installment of AI + Math Chat, Epoch AI's lead mathematician gathers four experts at the intersection of formal mathematics, cryptography, and AI, to explore a compelling challenge: Can large-language models go beyond pattern-matching to produce fully formalized mathematical proofs and novel cryptographic attacks?

Elliot Glazer from Epoch welcomes Kevin Buzzard, Yang-Hui He, Alex Kontorovich, and Kristin Lauter to discuss the potential and obstructions for AI formalizing mathematics in Lean, and how LLMs learning the rules of arithmetic through pattern-matching might pave the way to discovering novel attacks in cryptography.

-- Timestamps --
00:00 - Intro
01:13 - Kevin Buzzard intro
01:52 - Yang-Hui He intro
04:55 - Alex Kontorovich intro
05:45 - How Alex and Kevin got into Lean
14:09 - Kristin Lauter intro
17:31 - Implications of AI finding a cryptographic attack
24:15 - When/how will AI jump from pattern-matching to rigorous mathematical reasoning?
27:26 - The math/chess analogy: can math capabilities be RL'd through synthetic data?
35:15 - Does the hierarchical nature of math make it feasible to search for reasonable next steps in a deduction?
40:31 - On keeping FrontierMath's problems private
45:27 - How to benchmark a model's competence in producing Lean proofs
48:34 - Testing LLMs' arithmetic abilities in the absence of tools.
53:11 - Studying how transformers' learn to pattern match increasingly complex mathematical phenomena, towards discovering new algorithms
56:45 - Epoch would be excited to use Lean as a way to design new math benchmarks without the shortcomings of FrontierMath

Originally recorded in March, 2025.

Supported by OpenAI
Epoch AI gratefully acknowledges OpenAI for financially supporting FrontierMath. OpenAI has access to the FrontierMath datasets for evaluations, except for a set of held-out problems to allow for Epoch AI’s independent evaluations. Epoch AI runs frequent public evaluations on a wide range of frontier models from across the AI ecosystem to determine the state of the art of mathematical capabilities.

Learn more about FrontierMath and explore our benchmark: https://epoch.ai/frontiermath