Скачать или смотреть Nested Learning: The Illusion of Deep Learning

Paper titile:
Nested Learning: The Illusion of Deep Learning
Architectures of Goole

The academic paper introduces Nested Learning (NL), a novel paradigm representing machine learning models and their optimization algorithms as a multi-level hierarchy of optimization problems, providing a mathematically transparent view of internal processes. This framework addresses critical limitations in existing systems, arguing that current Deep Learning models are static and suffer from inadequate mechanisms for continual learning beyond in-context adaptation. NL reveals that established optimizers like Adam and SGD with Momentum are effectively associative memory modules designed to compress gradients. Building on this core insight, the authors present a generalized memory view called the Continuum Memory System (CMS) and propose the design of more expressive Deep Optimizers. These components are integrated into the new HOPE architecture, a self-referential model that shows promising results across language modeling and long-context reasoning tasks, often outperforming traditional recurrent and Transformer baselines.