🎯 Multi-Head Attention

Multi-head attention means:

instead of using one single attention pattern, the model uses multiple attention heads in parallel.

Each head can learn a different kind of relationship.

Why one head is not enough

A sentence contains many kinds of patterns at once:

• grammar
• long-distance references
• semantic similarity
• structure
• emphasis
• positional relationships

If you use only one attention mechanism, it has to do all of that with one pattern.

That is limiting.

What multiple heads allow

Different heads can specialize.

For example:

• one head may focus on subject-verb agreement
• one head may track entity relationships
• one head may focus on nearby syntax
• one head may focus on long-range meaning

So the model gets multiple perspectives on the same sequence.

Analogy

Imagine analyzing a cricket match with multiple experts:

• one watches batting technique
• one watches bowling strategy
• one watches field placement
• one watches scoreboard pressure

Each expert sees something different.

Together, they give a richer understanding.

That is what multi-head attention does.

How it works conceptually

1. create multiple versions of Q, K, and V
2. let each head compute attention separately
3. combine all head outputs
4. project them into one final representation

So the model does not rely on just one notion of relevance.

Why this matters for transformers

Multi-head attention is one of the reasons transformers are so expressive.

It helps them:

• capture several relationships at once
• build richer token representations
• scale better to complex language patterns

Bridge to next topic

Attention itself does not automatically know token order.

So after this, the next question is:

how does a transformer know which token came first?

Next:

• 3.d-Positional-Encoding