Скачать или смотреть loss of plasticity

Ever wondered why AI models like ChatGPT sometimes hit a wall and stop learning new tricks? In this video, we break down the "loss of plasticity" in Large Language Models (LLMs) – think of it as an old dog struggling to learn new commands after too much routine training. We'll explore what causes it (dead neurons, shifting data, and more), why it's a big deal for continual learning, and simple fixes like regularization, noise injection, and reinitialization techniques.
Using easy analogies, real-world examples from datasets like Permuted MNIST and CIFAR-100, and insights from recent research (including a 2025 paper on weight vs. unit reinitialization), we'll make this complex AI topic fun and accessible. Whether you're an AI enthusiast, developer, or just curious about how machines "think," this is your guide to keeping models adaptable!
Key Topics Covered:

What is loss of plasticity? (0:45)
Causes: Dead neurons, non-stationary data, and more (2:30)
Fixes: Algorithms like L2 Regularization, Shrink and Perturb, and Selective Weight Reinitialization (5:15)
Pros, cons, and biological inspirations (10:20)
Why it matters for future AI (12:45)

Inspired by the paper: Hernandez-Garcia et al. (2025) - Check it out: https://arxiv.org/pdf/2508.00212
If you enjoyed this, like, subscribe, and hit the bell for more AI deep dives! Drop a comment: What's your biggest AI challenge? #AILearning #LLMs #NeuralNetworks #ContinualLearning #TruVerAck