Around the model: small models, decision models, harnesses
Real AI products are teams, not one big brain. Who does what, and where the new decision models fit.
When you use an AI product, you’re rarely talking to one model. Behind the scenes there is usually a small team, each member doing what it’s best and cheapest at:
- Large frontier models do the open-ended thinking: understanding the task, planning, writing code.
- Small language models are the same kind of model made much smaller. They’re fast, cheap, private, and can run on a laptop or phone, but they know less and are weaker at long reasoning. They’re good for summarising, sorting and extracting.
- Decision models are new in autumn 2026. They don’t write text at all. You ask a narrow question with fixed answers (“allow, ask a human, or block?”) and get back a probability for each option, in about a tenth of a second. TypeSafe AI released Jev, the model that started the decision-model category, in mid-September 2026Nov 2026, and AWS followed with an open version.
- The harness is ordinary software: the loop that passes messages, runs tools, enforces permissions and keeps the conversation. Agent frameworks such as the Strands Agents SDK live here. (The Strands project also released a decision model, Strands Decider, which is why the names blur.)
Click through the steps of a typical agent loop to see who handles what:
Try it: a safety gate for an AI’s actions
An AI agent proposes a command. A decision model scores how risky it is, and plain, auditable code applies the rule. The model supplies a judgement; the policy stays in code you can read.
Illustrative probabilities, not output from a real decision model.
Engineer depthDecision models as a category
A decision model keeps the entire pretrained transformer and swaps only the unembedding layer from the stack chapter. Instead of scoring ~200k vocabulary tokens, a small pointer head scores your options in a single forward pass. The torso keeps everything it learned about language; the model just loses the ability to ramble. Question types are typically predicate (yes/no), choice (one of N) and score.
state: "Help! My payouts have been failing for 3 days!" choice: Which team should handle this? = billing, sales, retail
Question types: predicate (yes/no), choice (one of N), score (a value on a scale). Example adapted from the Strands Decider README; probabilities are illustrative.
| Model | From | Access | Notes |
|---|---|---|---|
| Jev | TypeSafe AI, Sept 2026 | Closed, API | Started and named the category; TypeSafe calls it a "System One" model. |
| Strands Decider 2B | AWS Strands Labs, Oct 2026 | Open weights, Apache 2.0 | AWS Strands Labs released Strands Decider 2B on 1 October 2026 under Apache 2.0: a Qwen3.5-2B torso whose text head is replaced by a pointer head, with Gemma-based variants from E2B to 26B-A4BNov 2026. |
| Laya | Nandakishor M., 2025 | Open | An earlier non-autoregressive decision model resolving typed decisions in one pass. |
Benchmarks in a two-month-old category: Strands' own table lists Decider 2B at 0.78 on JevBench, while an independent run on the 231 public tasks got 72.3%Nov 2026. Numbers depend on who ran them; measure on your own decisions.
Harnesses and MCP. a Strands agent boils down to a model, a system prompt and a list of tools, and the model itself decides when to call which toolNov 2026. Harnesses run tools, manage context and retries, and increasingly connect to tool servers over MCP. The common deployment pattern is a cascade: a cheap model or decision model handles everything, acts on confident answers, and escalates the uncertain tail.
SLMs in practice. Usually 0.5–15B parameters, made capable through distillation and runnable locally through quantisation. Typical roles: summarising a 50k-token log before it reaches the expensive model, extraction, classification, on-device features with data that must not leave the machine.
Checkpoint. What does a decision model such as Jev or Strands Decider return, compared with a normal LLM?