MiniLLM Lab · Interactive Visual Story

From a sentence to an AI assistant

Follow the full journey: tokens become vectors, attention gathers context, Transformer blocks refine representations, and a trained model generates text — one token at a time.

🧠
Speed:
Examples:
01 · Tokens STAGE 1 / 19

Text → representations

STAGE 1 / 19
A sentence becomes tokens
Select any token to inspect its vectors and attention connections.
1 · Tokenized sentence
2 · Embedding / representation
3 · Query, Key & Value
QUERY
KEY
VALUE
What this stage means
The "If You…" narration

Animated attention network

Line brightness ≈ attention weight
SVG · LIVE
Token connections
Selected (query) Other tokens Attention flow Moving dots = information flow

Attention matrix

Rows = query tokens · Columns = key tokens
SOFTMAX

Context vector

WEIGHTED SUM

Pipeline checkpoint

REPRESENT TEXT

Text is being converted into a form the model can process.

Keep the mental model

MINILLM LAB

Tokens are pieces; IDs identify them; embeddings are learned vectors; attention mixes information across positions; Transformer blocks refine representations; the output head predicts a next token. All displayed numbers are illustrative, not outputs from a pretrained model.

Educational simulation · local calculations · not a real pretrained LLM