← 00 · Math Foundations Hub
Geometry & Distance for AI
How semantic meaning turns into spatial geometry: Cosine Similarity, Euclidean Distance ($L_2$), Dot Product ($MIPS$), and high-dimensional vector spaces.
💡 The Geometric Principle
In deep learning, similarity is distance. If two concepts are synonymous (like "kitten" and "cat"), their 1536-dimensional embedding vectors point in nearly identical directions ($\theta \approx 0^\circ, \cos\theta \approx 0.98$).
01
The 3 Fundamental Vector Distance Metrics
Cosine vs. Dot Product vs. EuclideanDepending on whether vector magnitude (document length) matters, different geometric metrics are used:
- Cosine Similarity: $\cos\theta = \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\| \|\mathbf{v}\|}$ — Measures angle only, ignoring vector length. Best for comparing long vs. short texts with identical meaning.
- Dot Product (Inner Product): $\mathbf{u} \cdot \mathbf{v} = \sum u_i v_i$ — Fast GPU computation combining angle and length. Standard in Self-Attention.
- Euclidean Distance ($L_2$): $d(\mathbf{u}, \mathbf{v}) = \sqrt{\sum (u_i - v_i)^2}$ — Straight-line distance in physical space.
🎯 Where This Directly Applies:
Module 05: Vector Databases Indexing and querying millions of document embeddings via HNSW.
Module 05: RAG Pipelines Finding the most relevant knowledge chunks to inject into prompts.
02