Self-Attention, Formalized: Query, Key & Value

nlp1-8 introduced attention conceptually — a score, a softmax weight, a blended context vector. That was enough to explain why attention works. This chapter shows how it's actually computed, with real matrices.

Recap: nlp1-8's Own Conceptual Version

# nlp1-8's own conceptual attention score
scores = [dot(decoder_state, encoder_state_i) for encoder_state_i in encoder_states]
weights = softmax(scores)
context = sum(w * s for w, s in zip(weights, encoder_states))

That version compared one decoder state against a set of encoder states. Self-attention needs something more general: every token comparing itself against every other token in the same sequence, including itself. This is where Query, Key, and Value come in.

Query, Key, Value — What Each One Actually Is

For every input vector (from llm1-3 — a token embedding plus its positional encoding), the model computes three separate vectors by multiplying it against three separate, learned weight matrices:

Q = X @ W_Q    # Query — "what am I looking for?"
K = X @ W_K    # Key    — "what do I contain, for others to match against?"
V = X @ W_V    # Value  — "what do I actually offer, if attended to?"

Every single token produces all three — its own Query, its own Key, its own Value — using the exact same three weight matrices (W_Q, W_K, W_V), shared across the whole sequence and learned during training like every other parameter.

Scaled Dot-Product Attention

scores = (Q @ K.T) / sqrt(d_k)    # every Query compared against every Key
weights = softmax(scores, axis=-1)  # normalized per row — one weight distribution per token
output = weights @ V                 # weighted blend of every token's Value

Q @ K.T computes a similarity score between every token's Query and every other token's Key, in one matrix multiplication — nlp1-8's own single dot-product score, generalized to every pair at once. Dividing by sqrt(d_k) (the square root of the Key dimension) keeps the scores from growing too large as dimensionality increases, which would otherwise push softmax into regions with vanishingly small gradients. Softmax turns each token's own row of scores into a proper probability distribution — exactly nlp1-8's own attention weights, generalized from one decoder step to every token simultaneously. The final multiplication by V blends every token's Value according to those weights, producing one output vector per token.

Why Three Separate Matrices, Not One

Self-Attention, Specifically

This is called self-attention because Q, K, and V are all derived from the same sequence — every token attends to every token in its own input, itself included. This is the exact mechanism nlp1-8 introduced conceptually as "applying attention within a single sequence," now written out as real matrix operations rather than described in prose.

Hands-On Exercises