Скачать или смотреть Understanding Self-Attention⚡️

The goal of the embeddings:

We want to simulate meaning in a very simple, numerical way.
Each word (cat, eats, fish) is represented by a vector of features, like:
[animal, action, food, subject]
Each number (0 → 1) represents how much that word expresses that feature.
So it’s not a real embedding (like from Word2Vec or BERT), but a toy, interpretable version for teaching.
How each word’s numbers were chosen
Word animal action food subject Logic
eats 0.1 0.9 0.1 0.2 Mostly an action word (high action), not an animal or food
cat 0.9 0.1 0.1 0.8 A living animal and a likely subject of a sentence
fish 0.1 0.1 0.9 0.7 Mostly food, sometimes subject (it can do things too)

Why these particular numbers (e.g. 0.9, 0.1, 0.8)?
They are chosen by intuition, to make attention behave meaningfully.
When we compute the dot products between these embeddings:
scores = emb @ emb.T

words that have similar features get higher attention scores.
For example:
cat and fish both have moderately high subject scores (0.8, 0.7) → they’re related.
cat and eats both have some overlap (subject/action) → cat attends a bit to eats.
eats and fish connect through food/action features → also related.
This gives a natural triangle of attention where:
→ cat attends to eats,
→ eats attends to fish,
→ fish attends back to cat — forming a coherent sentence context.

These numbers aren’t arbitrary, but chosen to:
Be small enough for clarity (4 features only)
Produce interpretable attention scores
Reflect real linguistic relationships between the words
You can think of them as handcrafted mini-embeddings that mimic the behavior of real learned embeddings — but simplified so we can actually see the logic.

python tutorial, python tutorial for beginners,python,python programming tutorial,learn python,python programming,python course,python for beginners,python tutorials,tutorial,python full course,python basics,learn python programming,python tutorial 2021,python 3,python language,python from scratch,python programming language,python tutorial,python django tutorial,python malayalam tutorial,best python tutorial
python classes,python,python class,python tutorial,python programming,python objects,classes in python,python classes and objects,classes,classes and objects in python,classes python,python 3 classes,python classes tutorial,python tutorial for beginners,python basics,python classes for beginners,python oop,python course,class,python classes and objects tutorial,python for beginners,python class tutorial,learn python,python 3