How do plain numbers gain meaning?
Sentence so far: "The human drives the ___". The model must predict the next word. Watch the vector for the last token the travel through one attention step, then flip between untrained and trained weights to see what training changes.
Embeddings: the starting point
Each token enters as a vector of 3 numbers. d1, d2, d3 are not height, colour or age. They have no label. Notice that the last the looks identical to the first The: on its own, it carries no clue about what comes next.
The learned weights (this is what training changes)
Three 3×3 grids of numbers. Embeddings never change between the two modes. Only these grids do.
Attention scores: Q of "the" vs K of every token
The last the makes a query Q ("what do I need?"). Every token offers a key K ("what do I contain?"). Score = Q·K ÷ √3. Softmax turns scores into attention weights that add up to 100%.
Contextualized vector for "the"
Result = weighted sum of every token's V (the information it shares). The old context-free vector is replaced by one that has absorbed human and drives, in proportion to the attention weights.
before embedding
after contextualized
Next-word prediction
The contextualized vector is scored against each candidate word's vector (dot product, then softmax).
Simplified on purpose: one attention head, one layer, 3 dimensions, and weights that I picked to illustrate the idea rather than ones produced by a real training run. Real models use hundreds of dimensions, many heads, dozens of layers, and weights found by gradient descent over huge text corpora. The pipeline (embed → Q/K/V → attention → blend → predict) is the same.