Machinery
Reading path

Machinery · November 2026 edition: Next token, and what comes after

How AI like ChatGPT actually works, explained as it changes

An interactive guide for people who use AI tools every day and want to know what is really going on inside them. Every diagram is live. Every fast-moving fact shows when it was last checked, and the guide is updated each month as the field moves.

Pick a prompt and watch one generation step:

Your text, as tokens

Each token then becomes a long list of numbers (24 of thousands shown):

Through the layers

Next-token probabilities

Illustrative numbers. A real model scores all ~200k tokens in its vocabulary at once.

Part 1: Language models

How today's AI reads, predicts, is trained, and where it breaks. Ten short chapters, each with a hands-on demo and a checkpoint question. Start at zero if you're new; jump around if you're not.

0updated

What an LLM is, in five minutes

The whole picture before the details. What these systems are, how they were built, and what they are not.

5 min
1updated

Text becomes tokens, tokens become numbers

Models never see letters or words. They see numbered pieces of text, and then points on a map of meaning.

8 min
2updated

Attention: how words borrow meaning

The one mechanism that made modern AI work. Each word looks back at earlier words and pulls in what it needs.

9 min
3updated

Inside the stack

Look around, then think: one step repeated dozens of times. Plus the trick that lets giant models run cheaply.

8 min
4updated

Generating text, one token at a time

A weighted dice roll, repeated. Why answers differ each time, what “temperature” does, and why thinking takes time.

9 min
5updated

Training: from autocomplete to assistant to agent

Five stages that each change what the model is. Knowing them explains a lot of everyday behaviour.

10 min
6updated

Is it really just predicting the next word?

Yes, and that sentence hides almost everything interesting. Why a coding agent feels like so much more than autocomplete.

14 min
7updated

What they're good at, and where it breaks

Strengths that hold for good reasons, and weaknesses built into the design. Including the security problem nobody has solved.

11 min
8updated

Around the model: small models, decision models, harnesses

Real AI products are teams, not one big brain. Who does what, and where the new decision models fit.

11 min
9updated

Where research is heading next

What researchers are trying in order to fix today's limits. The most fast-moving chapter, revised every month.

10 min