🗝️ Query, Key, Value (QKV)

QKV is the mechanism that makes attention computable.

Big intuition

For the current token:

Query (Q) = what am I looking for?
Key (K) = what clue does each token offer?
Value (V) = what information should each token contribute?

Simple analogy

Think of a search engine or a library:

your search text / question = Query (e.g. "Looking for books on the capital of India")
book spine labels / index tags = Keys (e.g. "India")
actual page content / answers = Values (e.g. "New Delhi is the capital of India")

You compare your Query with all available Keys via dot product. The highest matches receive top attention weights through Softmax, which blends their Values into a new contextualized vector.

🖼️ Master QKV Architecture Infographic

Understanding Query, Key, and Value (Q, K, V)

7-step visual journey: Input token vectors $X \to$ Learned projection matrices ($W_Q, W_K, W_V$) $\to$ Real-world library analogy $\to$ Dot-product score matrix ($Q \cdot K^T$) $\to$ Softmax attention probabilities $\to$ Value-weighted sum $\to$ Multi-head layer repetition.

Language example

Sentence:

"The report reached the manager after several revisions, and it changed the release plan."

When processing "it":

the model forms a Query
every earlier token has a Key
every token also has a Value

If the Query for "it" matches the Key of "report" strongly, then the Value of "report" contributes more to the final context.

Why vectorization matters here

Q, K, and V are not words. They are vectors.

That is why Vectorization is important.

The model learns:

how to represent tokens as vectors
how to compare those vectors
how to combine them meaningfully

Before real QKV: intuition demos

If you run the current learning demos:

they help explain the behavioral idea of attention, but they are still pre-QKV demos.

That means:

no real Query vectors
no real Key vectors
no real Value vectors
no dot-product similarity
no softmax normalization over learned scores

Instead, those scripts use handcrafted rules and random score ranges to simulate:

"what matters now?"
"which tokens should get more weight?"
"how stronger clues affect the final interpretation"

So use them for intuition first, then use this file to understand how Transformers turn that intuition into real math.

What happens mathematically

Very roughly:

compare Query with all Keys
get similarity scores
convert them into weights
use those weights to combine the Values

So the output is a weighted combination of useful information.

Why all three are needed

You might ask: why not just use one representation?

Because each token has to play three different roles:

ask for information
advertise what it contains
provide information if selected

Q, K, and V separate those roles.

Bridge to next topic

Once you understand QKV, the next big idea is:

why use multiple heads instead of just one?

3.c-Multi-Head