Glossary
Every dotted term in the guide, in plain language. 70 terms.
- Agent / tool use
- A loop where the model emits structured tool calls (search, shell, API), a harness executes them, and the results go back into the context.
- Agent harness / SDK
- The code loop around a model: holds the system prompt and tools, executes tool calls, manages context and retries. Examples include Strands Agents, the Claude Agent SDK and LangGraph.
- ARC-AGI
- A benchmark of small visual puzzles that test learning a new rule from a few examples. Easy for humans, historically hard for LLMs.
- Attention
- The layer where tokens exchange information. Each token's query is matched against earlier tokens' keys, and the result is a weighted mix of their values: a differentiable, fuzzy key-value lookup.
- Attention head
- One independent attention computation. Models run dozens in parallel per layer (multi-head attention), each free to track a different relation: syntax, coreference, position.
- Autoregressive
- Generating one token at a time and feeding each output back in as input for the next step.
- Base model
- The raw output of pretraining. It continues text rather than answering: ask it a question and it may write five more questions.
- Benchmark
- A standard test set for comparing models. Useful, but prone to contamination (test data leaking into training) and gaming.
- Cascade / router
- A pattern where a cheap model handles every request, acts on the confident answers, and escalates the uncertain ones to a stronger model or a human.
- Causal mask
- The rule that a token may only attend to itself and earlier tokens, never later ones. It's what makes the model a left-to-right generator.
- Chain of thought
- Intermediate reasoning the model writes out before answering. Reasoning models are trained to produce long, self-correcting chains.
- Constrained decoding
- Masking out tokens that would violate a grammar or JSON schema before sampling, so structured outputs and tool calls always parse.
- Context rot
- The decline in how well a model uses its context as the context gets long, including weaker recall of information in the middle.
- Context window
- Everything the model sees in one call: system prompt, conversation, documents, tool results. Working memory: exact, but gone after the call.
- Decision model
- A pretrained transformer with its text-generating head swapped for one that scores a fixed set of options. It answers typed questions (yes/no, choice, score) with a probability per option in one forward pass. Examples: Jev (TypeSafe AI), Strands Decider (AWS, open weights).
- Diffusion language model
- Generates by starting from a fully masked sequence and filling in or refining many tokens in parallel over several passes, instead of strictly left to right.
- Distillation
- Training a small model to imitate a large one's outputs. How most small, fast models get good.
- DPO
- Direct preference optimisation: trains directly on chosen-vs-rejected answer pairs, with no separate reward model. A cheaper alternative to RLHF.
- Embedding
- A learned vector (thousands of floats) per token, looked up by token ID. Similar meanings end up near each other in this high-dimensional space.
- GQA / MLA
- Grouped-query attention and multi-head latent attention: tricks that share or compress keys and values to shrink the KV cache.
- Hallucination
- Fluent, confident output that is false. Partly structural: the model always produces a plausible continuation, whether or not it 'knows'.
- Induction head
- A two-head attention circuit that finds an earlier occurrence of the current token and predicts what followed it. A key mechanism behind in-context learning and pattern copying.
- Inference
- Running a trained model to produce output, as opposed to training it. Most of an LLM's lifetime cost.
- Jagged frontier
- Uneven capability: superhuman on some tasks, failing on neighbouring ones that look easier to a human.
- KV cache
- The stored keys and values of all earlier tokens, so each new token only computes its own. Saves compute, but memory grows linearly with context length, which is often the real bottleneck.
- Latent reasoning
- Reasoning in hidden vectors instead of written-out tokens. Potentially more efficient, but harder to inspect.
- Layer / block
- One attention + MLP unit. Frontier models stack roughly 30–120+ of them.
- Linear attention
- Attention variants that avoid the n² score matrix, e.g. Gated DeltaNet. Usually mixed with a few full-attention layers in hybrid models.
- Logits
- The model's raw, unnormalised score for every vocabulary token at the next position.
- Loss / cross-entropy
- The training signal: how surprised the model was by the actual next token (the negative log of the probability it assigned). Training nudges the weights to lower it, using gradient descent.
- MCP (Model Context Protocol)
- An open protocol for connecting models to tools and data sources through standard servers, so a tool can be written once and used by any compatible harness.
- Mixture of Experts (MoE)
- The MLP is split into many experts, and a small router picks a few per token. Huge total capacity, small per-token compute.
- MLP / feed-forward layer
- The per-token half of a block: two large matrix multiplications with a non-linearity in between. Holds much of the stored knowledge and roughly two-thirds of the parameters.
- Open-weight
- A model whose trained weights can be downloaded (Llama, Qwen, DeepSeek, Mistral and others). Not necessarily open source: training data and code often aren't published.
- Parameters / weights
- The learned numbers in the model's matrices. A '70B model' has 70 billion of them. The weights are the model's long-term memory, frozen after training.
- Policy
- RL term for the behaviour being optimised: a mapping from situations to actions. After RL, an LLM's next-token distribution is a policy whose actions happen to be tokens.
- Prefill vs decode
- Prefill processes the whole prompt in parallel (compute-bound). Decode generates one token per pass (memory-bandwidth-bound). The reason output tokens cost more than input tokens.
- Pretraining
- Phase one: next-token prediction over trillions of tokens. Produces a base model with broad knowledge that autocompletes but doesn't follow instructions.
- Prompt caching
- Reusing the KV cache of a prefix the provider has recently processed, such as the system prompt and earlier turns of an agent session. Cheaper and faster, as long as the prefix stays identical.
- Prompt injection
- Untrusted text in the context (a web page, an email, a file) that contains instructions the model then follows. Hard to fix because data and instructions share one channel.
- Quantisation
- Storing weights in fewer bits (8, 4 or even fewer) to cut memory and speed up inference, at some quality cost. It's why a 70B model can run on a workstation.
- Query, key, value
- Three vectors computed from each token by matrix multiplication. Score = query · key; softmax of the scores gives weights; output = weighted sum of values.
- RAG
- Retrieval-augmented generation: search a document store and paste the relevant passages into the context before the model answers.
- Reasoning model
- A model trained, mostly with RLVR, to think at length before answering, e.g. o1, DeepSeek-R1 and their successors.
- Residual stream
- The running vector for each token that flows through the stack. Each layer reads it and adds its output (x = x + layer(x)), like writing to a shared bus.
- Reward hacking
- An RL-trained model exploiting a flaw in its reward signal, e.g. making tests pass by editing the tests rather than fixing the code.
- Reward model
- A separate model that scores answers, trained on preference comparisons. It stands in for a human grader at scale.
- RLHF
- Reinforcement learning from human feedback: people rank answers, a reward model learns their preferences, and the LLM is optimised against that reward model.
- RLVR
- RL with verifiable rewards: the reward comes from an automatic check, such as a matching math answer, passing tests or a finished agent task. The recipe behind reasoning models.
- RoPE (rotary position embedding)
- Encodes position by rotating query and key vectors by an angle that grows with position, so attention scores depend on the relative distance between tokens.
- Sampling / decoding
- Picking the actual next token from the distribution. Greedy decoding always takes the top token; sampling rolls weighted dice.
- Scaling laws
- The empirical finding that loss falls as a smooth power law in parameters, data and compute. It made frontier training a predictable engineering bet.
- Scratchpad
- Written-out intermediate tokens used as working memory. Each token adds another full pass through the network, giving the model serial computation beyond its fixed depth.
- SFT (supervised fine-tuning)
- Training on example conversations written by people or stronger models, so the model learns the assistant format.
- Slopsquatting
- Registering package names that models tend to hallucinate, so generated code with a non-existent dependency installs the attacker's package.
- Small language model (SLM)
- The same architecture at roughly 0.5–15B parameters, usually distilled and quantised. Fast, cheap and runs locally, but with less knowledge and weaker long reasoning.
- Softmax
- Turns a vector of scores into probabilities: exponentiate each score, then divide by the sum.
- Speculative decoding
- A small draft model proposes several tokens and the large model verifies them in one parallel pass. Faster generation with the same output distribution.
- State-space model (SSM)
- A recurrent-style layer (e.g. Mamba) that compresses the past into a fixed-size state. Cost is linear in sequence length, but recall is lossy.
- Superposition
- Packing many more features than dimensions into a vector space using nearly orthogonal directions. Why individual neurons usually respond to several unrelated things.
- Sycophancy
- The tendency to agree with the user and abandon correct answers under pushback, absorbed from preference training.
- Temperature
- The logits are divided by this number before softmax. Below 1 sharpens the distribution (more deterministic); above 1 flattens it (more random).
- Test-time compute
- Spending more computation at inference (longer reasoning, several attempts, voting) to get better answers. A second scaling axis alongside training compute.
- Test-time training (TTT)
- Updating some weights during inference, so what the model reads gets compressed into 'fast weights' instead of living only in the context window.
- Token
- The unit a model reads and writes: usually a word piece of 3–4 characters. Context limits and API prices are counted in tokens.
- Tokenizer
- Fixed preprocessing that maps text to integer token IDs and back. Most use BPE (byte-pair encoding): start from bytes, then repeatedly merge the most frequent adjacent pair.
- Top-p (nucleus) sampling
- Keep only the most likely tokens whose probabilities add up to p, renormalise, and sample from those. Cuts off the long tail of nonsense.
- Transformer
- The 2017 architecture behind nearly all LLMs: a stack of identical blocks, each with an attention layer and an MLP, joined by residual connections.
- Vocabulary
- The set of tokens a model knows, typically 100k–250k entries. Every output step is a choice among all of them.
- World model / JEPA
- Models that learn to predict how a situation evolves, in an abstract representation space rather than pixel by pixel or token by token. LeCun's Joint Embedding Predictive Architecture is one approach.