Inside the Transformer

We've turned text into tokens and tokens into embedding vectors. Now those vectors enter the model itself. For nearly every important LLM today, that model is a transformer — the architecture introduced in 2017 that quietly reset the entire field.

You don't need the math to understand the idea, and the idea is genuinely beautiful. Let's build it up in layers.

The problem transformers solved

Language is full of long-range dependencies. Consider:

"The trophy didn't fit in the suitcase because it was too big."

What does "it" refer to — the trophy or the suitcase? You know instantly: the trophy. Now flip one word:

"The trophy didn't fit in the suitcase because it was too small."

Now "it" is the suitcase. Resolving "it" requires connecting it to a word several positions back and reasoning about the rest of the sentence.

Older architectures processed text strictly left to right, one word at a time, trying to cram everything they'd seen into a single running summary. They tended to forget distant context and couldn't be parallelized well. Transformers threw that out and asked a different question:

What if every word could look directly at every other word and decide which ones matter?

That mechanism is called attention, and it's the whole ballgame.

Attention, intuitively

Imagine each token gets to ask a question and broadcast an answer:

• Every token forms a query: "what am I looking for?"
• Every token also offers a key: "here's what I'm about."
• And a value: "here's the information I'll hand over if you attend to me."

To process the token "it," the model takes its query and compares it against the keys of every other token. Tokens whose keys match the query well get high attention scores; the model then pulls in a blend of their values, weighted by those scores.

That's attention: a learned, content-based lookup where every token decides, for itself, which other tokens to listen to. Crucially, the words "trophy" and "it" can be far apart and it costs the model nothing extra to connect them directly — no information has to survive a long relay.

And because every token does this comparison against every other token at the same time, the whole thing parallelizes onto modern hardware. That combination — long-range connections and parallelism — is why transformers won.

Stacking it up: depth creates abstraction

A single attention step is useful but limited. So transformers stack many layers, dozens to over a hundred in big models. Each layer has two parts:

1. An attention block — tokens exchange information, as above.
2. A feed-forward block — each token is individually run through a small network that transforms its representation, where much of the model's stored "knowledge" is thought to live.

The intuition for depth: early layers handle surface patterns (this token is a verb, this phrase is a date), middle layers assemble meaning (who did what to whom), and later layers work with abstract, task-relevant concepts. Each layer refines the representation a little more, the way understanding a sentence happens in stages in your own head. Nobody programmed these roles; they emerge from training.

Multiple heads: paying attention several ways at once

In practice, attention isn't done once per layer but in parallel "heads." One head might track grammatical subject–verb agreement, another might link pronouns to their referents, another might follow quotation marks. This is multi-head attention: several independent attention patterns running side by side, then combined. It lets a single layer attend to different kinds of relationships simultaneously.

From final vectors to the next token

After the last layer, each token position has a rich vector. To actually predict the next token, the model takes the vector at the final position and projects it out to a score for every token in the vocabulary. Run those scores through a step that turns them into probabilities, and you get exactly the distribution from post #2:

   final vector  ──►  scores over the whole vocabulary  ──►  probabilities
                       "Paris"   8.2  ───┐
                       "Lyon"    3.1     ├──►  Paris  91%
                       "London"  2.7     │     Lyon    4%
                       ...              ─┘     ...

Then we pick a token (post #6 is all about how we pick), append it, and run the whole stack again for the next one. Yes — the entire transformer runs once per token generated. That fact is the reason inference engines (post #7) exist.

A few things worth knowing

• Position matters, and it's added deliberately. Attention by itself treats input as an unordered set — it has no inherent sense of word order. So models inject positional information so the model knows "the cat sat" differs from "sat the cat." Clever schemes here are a big part of how models handle long contexts.
• The context window comes from here. Attention compares every token to every other token, so naïvely the cost grows with the square of the sequence length. Double the context, quadruple the work. That quadratic cost is why long context is expensive and why so much research goes into making attention cheaper.
• "Decoder-only" is the dominant design. Most chat and coding models are "decoder-only" transformers: they only ever predict the next token given previous ones, never peeking ahead. Simple, and it scales.

The takeaway

But a fresh transformer with random weights predicts gibberish. Everything it "knows" has to be learned. How that learning happens — and how a raw text predictor becomes a helpful, instruction-following assistant — is next.