Machinery
Reading path

Sources & freshness

Every fact in this guide that can go out of date is listed here with its sources, the date a human last checked it, and how often it needs re-checking. Facts past their re-check date fade on the page until they are verified again.

ClaimStatusVerifiedRe-checkSources
a 2026 survey reports diffusion language models reaching several-fold speed-ups at performance comparable to autoregressive models emerging fresh 2026-11-21 every 3 months
AMI Labs has no public product yet research fresh 2026-11-21 every 3 months
AWS Strands Labs released Strands Decider 2B on 1 October 2026 under Apache 2.0: a Qwen3.5-2B torso whose text head is replaced by a pointer head, with Gemma-based variants from E2B to 26B-A4B emerging fresh 2026-11-21 every 3 months
Strands' own table lists Decider 2B at 0.78 on JevBench, while an independent run on the 231 public tasks got 72.3% emerging fresh 2026-11-21 every 3 months
one evaluation found a decision model matched a frontier model's verdicts noticeably more often than it matched the true labels emerging fresh 2026-11-21 every 3 months
another found Jev answering "cheap" for 99.5% of model-routing requests, no better than a constant emerging fresh 2026-11-21 every 3 months
today typically 128k to 1M+ tokens shipping fresh 2026-11-21 every 6 months
the largest and most capable open-weight models are still autoregressive decoder-only transformers shipping fresh 2026-11-21 every 6 months
researchers at TU Munich have critically compared sub-quadratic alternatives to attention on cost, benchmarks and fundamental limits research fresh 2026-11-21 every 6 months
their hallucination behaviour is still largely unexplored emerging fresh 2026-11-21 every 6 months
ByteDance's In-Place TTT (ICLR 2026 oral) reuses the MLP's final projection as fast weights, so it can be added without retraining from scratch research fresh 2026-11-21 every 6 months
NVIDIA's TTT-E2E compresses the context a model reads into its weights through next-token prediction, aiming for full attention's quality at recurrent-model latency research fresh 2026-11-21 every 6 months
TypeSafe AI released Jev, the model that started the decision-model category, in mid-September 2026 emerging fresh 2026-11-21 every 6 months
a fixed vocabulary of roughly 100k–250k entries shipping fresh 2026-11-21 every 12 months
DeepSeek-V3, for example, has 671B parameters in total but uses only about 37B per token shipping fresh 2026-11-21 every 12 months
since late 2024, reinforcement learning with verifiable rewards has produced reasoning models (o1, DeepSeek-R1 and successors) shipping fresh 2026-11-21 every 12 months
Yann LeCun's AMI Labs raised a $1.03B seed round in March 2026 to build JEPA-based world models research fresh 2026-11-21 every 12 months
Karpathy's AutoResearch (March 2026) is a short script in which an agent changes training code, runs quick experiments and keeps what works emerging fresh 2026-11-21 every 12 months
a Strands agent boils down to a model, a system prompt and a list of tools, and the model itself decides when to call which tool shipping fresh 2026-11-21 every 12 months
a Llama-3-70B-style setup (80 layers, 8 KV heads × 128 dimensions, fp16) needs about 0.31 MiB of KV cache per token evergreen evergreen 2026-11-21 evergreen
Kalai and Vempala showed that a calibrated model must hallucinate at some rate on facts it rarely saw in training evergreen evergreen 2026-11-21 evergreen
Anthropic's 2025 attribution-graph work traced two-hop reasoning and rhyme planning inside Claude 3.5 Haiku evergreen evergreen 2026-11-21 evergreen
Othello-GPT, trained only on move sequences, built an internal representation of the board evergreen evergreen 2026-11-21 evergreen
information in the middle of long contexts is used least reliably ("lost in the middle", Liu et al. 2023) evergreen evergreen 2026-11-21 evergreen