AI JOURNEY / INTERACTIVE SCIENTIFIC SIMULATION

Inside the Transformer

Trace hidden-state vectors through attention, residual connections, and the feed-forward network. The motion follows the signal; the numbers update when you change the inputs.

01 / TRANSFORMER BLOCK Β· LIVE SYSTEM VIEWFOCUS Β· INPUT
HIDDEN-STATE FLOW Β· SELECT ANY MODULE Input vectorsX Β· token states3 token rows Normalizestabilize scale Self-attentionQ Β· K Β· Vcontext mixing +residual add MLP / FFNexpand β†’ activatefeature transform +residual add OUTPUT CAUSAL ATTENTION MAP Β· ROW = QUERY TOKEN● CYAN = SIGNAL● VIOLET = ATTENTION● GREEN = RESIDUAL● AMBER = MLP
STEP 1 OF 70.0s
Go deeper into Transformer

You have just watched one Transformer block work. Each subtopic below zooms into one part of it: why the design exists, what each sublayer does, and how the blocks stack into the full model.

12 subtopics β€” click to study
Why Transformer? πŸ”΄ Self-Attention πŸ”΄ Feed-Forward Network πŸ”΄ Residual Connections πŸ”΄ Layer Normalization πŸ”΄ Encoder Block 🟑 Encoder 🟑 Masked Attention πŸ”΄ Decoder Block πŸ”΄ Cross-Attention 🟑 Decoder πŸ”΄ Complete Transformer Flow πŸ”΄
🌐 Elite Interactive Visualizer Spotlight

Georgia Tech's Transformer Explainer

This is the absolute gold standard in interactive education. It visualizes the entire neural pathway of a Transformer in real-time, down to individual float values, Softmax charts, Key/Query/Value math, and the Feed-Forward neural expansion. Highly recommended to use alongside these lessons!

πŸš€ Open Transformer Explainer β†’
πŸ§ͺ
RASL Pull Β· Sprint S-05 β€” Parallel Learning
You're reading this because RASL is blocked on: how do residuals and layer norm keep training stable β€” why does my train.py diverge without them?
Residual connections and layer normalization are what make deep networks trainable. Understanding these lets you build transformer.py correctly and debug when loss explodes. Return to mini-llm-lab/tiny_llm/transformer.py and continue building. Read only what unblocks you.
πŸ”¨ Lab: M-05 complete β†’ tiny_llm/transformer.py πŸ” Residual pattern β†’ RASL execution state checkpointing πŸ“‹ View full parallel plan β†’
← Back to Roadmap
β–Ά LIVE DEMO Transformer Β· Live β†’
This page 0:00
This session 0:00
Total (all pages) 0:00
Today 0:00
β–Ά Running
πŸ“Š View all study stats β†’
0:00 studying