Machinery · November 2026 edition: Next token, and what comes after
How AI like ChatGPT actually works, explained as it changes
An interactive guide for people who use AI tools every day and want to know what is really going on inside them. Every diagram is live. Every fast-moving fact shows when it was last checked, and the guide is updated each month as the field moves.
Welcome back: since your last visit, highlighted in yellow.
Your text, as tokens
Each token then becomes a long list of numbers (24 of thousands shown):
Through the layers
Next-token probabilities
Illustrative numbers. A real model scores all ~200k tokens in its vocabulary at once.
Part 1: Language models
How today's AI reads, predicts, is trained, and where it breaks. Ten short chapters, each with a hands-on demo and a checkpoint question. Start at zero if you're new; jump around if you're not.
What an LLM is, in five minutes
The whole picture before the details. What these systems are, how they were built, and what they are not.
Text becomes tokens, tokens become numbers
Models never see letters or words. They see numbered pieces of text, and then points on a map of meaning.
Attention: how words borrow meaning
The one mechanism that made modern AI work. Each word looks back at earlier words and pulls in what it needs.
Inside the stack
Look around, then think: one step repeated dozens of times. Plus the trick that lets giant models run cheaply.
Generating text, one token at a time
A weighted dice roll, repeated. Why answers differ each time, what “temperature” does, and why thinking takes time.
Training: from autocomplete to assistant to agent
Five stages that each change what the model is. Knowing them explains a lot of everyday behaviour.
Is it really just predicting the next word?
Yes, and that sentence hides almost everything interesting. Why a coding agent feels like so much more than autocomplete.
What they're good at, and where it breaks
Strengths that hold for good reasons, and weaknesses built into the design. Including the security problem nobody has solved.
Around the model: small models, decision models, harnesses
Real AI products are teams, not one big brain. Who does what, and where the new decision models fit.
Where research is heading next
What researchers are trying in order to fix today's limits. The most fast-moving chapter, revised every month.
This month in the field
Decision models arrive, and the guide goes live
A new kind of model that answers instead of writing, why its benchmarks already disagree, and the first frontier radar.
Frontier radar · November 202619 directions tracked
10 shipping, 5 emerging, 4 still in research. See what moved this month and why it matters.