Скачать или смотреть Classical linguistics vs. digital linguistics: multi-layeredness.

The hierarchical structures in LLMs correspond remarkably precisely to the theoretical considerations on the multi-layered nature of language in Volume 2 of Serebrennikov's "General Linguistics." These parallels go far beyond superficial similarities and reveal fundamental organizational principles that shape both natural and artificial language systems.
Spontaneous Layering in Neural Networks.
Multiple Timescale Recurrent Neural Networks (MTRNNs):
Multi-timescale neural networks spontaneously develop a hierarchical temporal structure corresponding to the linguistic levels. These networks organize themselves into three distinct timescales:
Short timescales (0.17 words): Correspond to phonological processes and letter-to-sound mappings;
Medium timescales (1-10 words): Represent morphological and syntactic structures;
Long timescales (360,000+ words): Capture lexical and semantic relationships;
This spontaneous differentiation arises without explicit programming of the linguistic levels and reflects Serebrennikov's concept of natural layering.
Hierarchical Emergence Without Explicit Structure
Functional hierarchies arise in neural networks through different activation rates. "Fast" neurons respond to short-term patterns (phonemes, morphemes), while "slow" neurons encode long-term structures (syntax, semantics). This self-organizing hierarchy reproduces the layered structure of natural languages ​​without a predefined architecture.
Layer-Specific Linguistic Representations.
Transformer Layers as Linguistic Levels:
BERT and related transformers exhibit a systematic distribution of linguistic information across their layers:
Early Layers (1-3):
Phonological Processing: Sound-to-Letter Mappings.
Morphological Segmentation: Recognition of Prefixes, Suffixes, and Word Roots.
Basic Word Boundaries: Tokenization and Word Formation
Middle Layers :
Syntactic Structures: Phrase Structure, Grammatical Relations
Morphosyntax: Inflectional Markers, Agreement
Hierarchical Dependencies: Subject-Verb Agreement over Distance
Late Layers: Semantic Composition: Meaning Composition
Lexical Semantics: Word Meanings in Context
Discursive Coherence: Textual Level and Pragmatic Inferences
Morphological Awareness and Structure Recognition
Emergent Morphological Processing
Morphological awareness develops in LLMs at two levels:
Implicit Morphological Knowledge: Automatic Segmentation into Meaningful Units;
Explicit Morphological Analysis: Conscious Manipulation of Morphological Structures;
Studies show that derivational morphology (word formation through affixation) produces more robust representations than compositional morphology (composition). This corresponds to Serebrennikov's observation of the differential productivity of morphological processes.
Hierarchical Word Structure Representations.
Tree-LSTM architectures process logographic structures (Chinese characters) hierarchically. These models develop recursive representations that: Encode phonological information into subcomponents
Convey semantic categories through structural position
Generate compositional meaning through hierarchical combination
Serebrennikov's Validation: The empirical confirmation of Serebrennikov's multi-layeredness principle by modern AI systems demonstrates the universality of this organizational structure. Parsimony leads to hierarchy, which in turn enables complexity—a principle that permeates both biological and artificial language systems.
The remarkable convergence between theoretical linguistics and computational modeling points to fundamental principles of information organization that structure linguistic systems of all kinds.