Fifteen Models Ran Football Clubs for Twenty Years, and Size Didn't Decide It

Source: FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents

Paper was published on August 19, 2026

This episode was AI-generated on August 20, 2026. The script was written by an AI language model and the host voices were synthesized by Eleven Labs. The producer is not affiliated with Anthropic or Eleven Labs.

Fifteen frontier models were each handed a football club and twenty simulated years to run it — and across a sevenfold spread in token spend, paying more bought nothing. The winner was one of the three cheapest models on the board, and the ranking didn't even exist until year fifteen. What actually separated them were three habits so boring you'd be embarrassed to put them on a slide.

Key Takeaways

• Why token spend, model size, and price predicted nothing here: a −0.19 correlation with a p-value of 0.5, with a previous flagship finishing above its successor
• The three behaviors that tracked score in all three worlds — tapering slow-payoff spend late, not letting cash sit idle, and opening contract renewals about eighteen months early
• The failure every single model shared: the oracle needs 1.0 offers per signing, the field median is 30, and the worst model needs 73 — hundreds of rejections and the price boundary is never located
• The notebook moment the authors treat as the thesis: a model diagnoses its own idle-cash problem at year ten and again at year nineteen, then ends holding roughly two billion in idle cash
• Why every dynasty in the solo track is an artifact — in the shared Arena, ten different models win the title and the reigning champion keeps it in two of nineteen transitions
• Where the evidence outruns the framing: three seeds, no Arena error bars, and an 'idle cash' finding that partly measures agreement with the benchmark's own scoring weights

• 00:00 — Eighteen million, agreed one raise at a time
  The cold open — a frontier model negotiating its club into insolvency — sets up the central puzzle: fifteen models survived twenty years, four of six humans went broke, and the biggest, most expensive reasoning didn't win.
• 01:05 — Does paying for more thinking buy anything?
  The intuition that bigger models plus longer thinking equals better agents collapses against a sevenfold spread in token spend that correlates with score at −0.19.
• 01:44 — A game with no judge and no memory
  How the benchmark works: a deterministic sixteen-club simulation with ~374 decision stops and 26 tools, arithmetic-only scoring, and a fresh conversation at every stop where the only thing carried forward is a self-written notebook.
• 03:45 — Random scores below doing nothing
  The anchors at both ends — a random script at −17, inaction at 0, a disciplined hand-written manager at 17, and an information-privileged oracle at 95.5 — against the best model's 90.94 while blind.
• 05:17 — At year five, the board was noise
  Rank correlation with the final order is 0.19 at year five and 0.78 by year fifteen — one model leads at years five and ten and finishes twelfth, which means a shorter horizon would have ranked a different set of models.
• 06:24 — Three boring habits, and nothing else
  Replaying every run surfaces six behavioral metrics, of which three hold sign across all worlds: endgame tapering, cash deployment, and renewal lead time — with total spending itself uncorrelated with score.
• 08:49 — It knew the answer and didn't act
  The two failures nobody escapes — never learning the market's acceptance prices despite hundreds of rejections, and the execution gap where a model writes the correct plan in its notebook twice and still ends with two billion in idle cash.
• 11:42 — When rivals fight back, dynasties disappear
  The shared Arena breaks the solo track's rich-get-richer dynasties — ten models win titles, the champion holds on twice in nineteen transitions — plus the six first-time human players and what they did that no model did.
• 15:25 — Where the framing outruns the evidence
  The steelman critique: three seeds with huge variance, an Arena with no error bars, an idle-cash finding that partly measures the benchmark's own scoring weights, and an oracle ceiling that's a script rather than an optimum.

Recommended Reading

• Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents — The closest precedent to FM-Bench's core finding — a simple simulated business run over very long horizons where frontier models stay coherent on short tasks but derail over time, and where capability tier fails to predict who survives.
• Measuring AI Ability to Complete Long Tasks — METR's task-length framing is the quantitative backdrop for Eric's warning that the leaderboard you're reading was measured on episodes too short to see the failure that bites you.
• MemGPT: Towards LLMs as Operating Systems — If the notebook is the whole capability under test, this is the systems-side alternative — explicit paging of memory in and out of context, i.e. exactly the retrieval plumbing FM-Bench deliberately refused to provide.
• Voyager: An Open-Ended Embodied Agent with Large Language Models — A direct counterpoint to the episode's sharpest failure — hundreds of rejected bids and the acceptance threshold never learned — via an agent whose growing skill library is designed to convert past experience into reusable competence.