← Resources #charan#dimension#vector
MINI VISUAL LESSON · EMBEDDINGS

Rows. Dimensions. Vectors.

Each token gets a row of numbers. That row is its vector. How many numbers it holds is the dimension.

Row = one token's vectorColumn = one dimensionMatrix = all rows together
What does Dimension (1,536) mean? Visual overview with person analogy, 1,536D vectors, and matrix stacking

Infographic: What does Dimension (1,536) mean? Click image to open full size.

1 · WHAT IS A DIMENSION?

One value = one dimension

Describe a person with four measurements:

DIM 1 · HEIGHT175cm
DIM 2 · AGE30years
DIM 3 · WEIGHT70kg
DIM 4 · SHOE9size
[175, 30, 70, 9] has four values → a 4-dimensional vector. A token vector works the same way, but its values are learned, not measured.
2 · THE EMBEDDING MATRIX

A spreadsheet with one row per token

Tiny vocabulary: 3 tokens × 4 dimensions. Illustrative values.

Token (row)Dim 1Dim 2Dim 3Dim 4
cat0.20.7−0.10.4
dog0.30.6−0.20.5
car−0.50.10.8−0.3
Shape 3 × 4: 3 rows (tokens), 4 columns (dimensions). The cat row is one vector: [0.2, 0.7, −0.1, 0.4].
3 · FOLLOW ONE QUESTION

From text to Transformer input

Change the dimension size D and watch the vectors and the matrix shape change.

STEP 1

Your question

The cat sleeps

↓
STEP 2

Tokenizer

Converts text into tokens and token IDs

[ID₁, ID₂, ID₃]

IDs are shown symbolically — they are only row numbers.

↓
STEP 3

Embedding layer

Each token ID selects its row from the learned embedding matrix.

Vector 1 · The
Vector 2 · cat
Vector 3 · sleeps

Each vector has 4 values = 4 dimensions. Values are illustrative.

↓
STEP 4

Input representations

Vectors are stacked in sequence order. Positional information is added by the model's architecture.

3 × 4 matrix

3 token positions × D dimensions

↓
STEP 5

Transformer

Attention and neural-network layers process the vectors.

Remember the distinction
  • Tokenizer: text → token IDs.
  • Embedding layer: token IDs → vectors.
  • Transformer: processes those vectors using context.
4 · SCALE IT UP

Vocabulary size ≠ dimension

ROWS · VOCABULARY
50,000

One row per token the model knows.

COLUMNS · DIMENSION
1,536

Numbers in each token's vector.

Embedding matrix: 50,000 × 1,536. A 20-token input selects 20 rows → a 20 × 1,536 input matrix. Example sizes only; real models differ.

Token ID = row index · dimension = one value · vector = all the values together.
Illustrative numbers, real concept.