📍 Similarity in Vector Space

1. What is vector space?

If every token has a vector, then each token can be thought of as a point in a high-dimensional space.

Example:

cat  → point A
dog  → point B
car  → point C

If cat and dog are close, that suggests they behave similarly in the learned representation.

If cat and car are far apart, that suggests they are less related.

2. What does “close” mean?

In practice, closeness is measured mathematically.

Common measures:

• cosine similarity
• dot product
• Euclidean distance

For intuition, you only need this idea:

closer vectors → more similar behavior or meaning
farther vectors → less similar behavior or meaning

3. Why similarity emerges

Similarity is not manually programmed.

It emerges because training pushes tokens used in similar contexts toward useful related representations.

Example:

The cat slept on the sofa
The dog slept on the rug

Because cat and dog appear in similar contexts, their embeddings may become similar.

4. Why this is powerful

This lets models go beyond exact keyword matching.

Example:

Search query:

affordable smartphone

Relevant result:

best budget phone

The words are not identical, but the vectors may be close enough for the system to recognize the relationship.

That is why embeddings are powerful for semantic systems.

This is also the basic idea behind:

• semantic search
• recommendation systems
• similarity search
• clustering related content

5. Important caution

Similarity in vector space is useful, but not perfect.

A close vector does not always mean:

• exact synonym
• same meaning in every context
• human-level conceptual equivalence

Embeddings capture useful statistical structure, not perfect truth.

6. Analogy intuition

Older embedding systems became famous for examples like:

king - man + woman ≈ queen

This showed that some relationships can appear as geometric patterns in vector space.

Important note:

• this is a useful intuition
• but not every embedding model behaves this cleanly
• and modern contextual models are more complex than simple analogy demos

So treat this as a helpful mental model, not a universal law.

If you want to explore this practically later, it connects well to:

• similarity search demos
• nearest-neighbor retrieval
• vector databases in later topics

7. Why this matters later

Attention and transformers do not work on raw words.

They work on vector representations.

So if embeddings already place useful relationships into vector space, later layers can build on that structure.

This is one reason vectorization is such an important bridge topic.