3D Vector Space & Coordinate Simulator
Explore how discrete token symbols (like “plant”, “Charan”, or “thought”) turn into continuous coordinates in space, how vectors move along axes, and how cosine similarity measures semantic alignment.
Vector a Magnitude ||a||
0.83
Vector b Magnitude ||b||
0.80
Angle θ
28°
Cosine Similarity ($\cos \theta$)
+0.88
🧭 Geometry is Meaning: Cosine Similarity #charan
In neural networks, direction equals semantic meaning. When two words share similar contexts (e.g. “thought” and “idea”), their vector arrows point in nearly the same direction, yielding a Cosine Similarity near +1.0.
Orthogonal vectors ($\cos \theta = 0$) represent unrelated concepts, while opposite vectors ($\cos \theta = -1$) represent diametrically opposed concepts.
🔄 Static Vector $\to$ Attention Context Vector #charan
1. Static Lookup: Word “plant” begins with coordinates pulled from Embedding Matrix $W_E$.
2. Contextual Rotation: When placed into “I plant an idea into Rahul’s mind”, Multi-Head Self-Attention calculates dot-product scores against “idea” and “mind”, rotating and steering the coordinates to encode its verb/action meaning!