Agents Deployed

Explained series Episode 8 about 11 min

Step fourteen: why long AI agent runs fall apart

Watch on YouTube

About this episode

How errors build up in long-running AI agents: what context poisoning is, the maths of compounding errors over many steps, and the common failure modes, from hallucinations and step repetition to premature termination. Then what helps: clear tool errors, verification steps, small tasks, handoffs, retry and step limits, and human review.

Written by Kiran Krishna. Narrated with an AI voice. · LinkedIn

In this episode

  1. 0:01In 2025, Anthropic let Claude run a small shop in its own office, for about a month.
  2. 1:03Here's the plan.
  3. 1:14So what actually happens at step fourteen?
  4. 3:53So how likely is a bad step like that, somewhere in a long run?
  5. 5:48So does this happen in real runs?
  6. 8:02So what actually helps?
  7. 10:12So why do long runs fall apart, when every step looks fine?

Sources check every one

  1. Anthropic, Project Vend: Can Claude run a small shop? (June 2025: the shop's results, the metal cubes, the blazer; the made-up account and "Sarah") anthropic.com
  2. Google, Gemini 2.5 technical report (2025: context poisoning; the hours spent looking for TEA in Pokémon) arxiv.org
  3. Anthropic, Building effective agents (December 2024: compounding errors) anthropic.com
  4. Sinha et al. 2025, The Illusion of Diminishing Returns (ICLR 2026: self-conditioning; models that think held up much better) arxiv.org
  5. METR, Measuring AI Ability to Complete Long Software Tasks (2025: failed runs of a 2023 and a 2024 model read by hand; the task length an agent manages half the time) arxiv.org
  6. METR, Time Horizon 1.1 (January 2026: the chart's numbers) metr.org
  7. METR, Task-Completion Time Horizons of Frontier AI Models (the chart's numbers) metr.org
  8. Andon Labs, Vending-Bench (Backlund & Petersson 2025: the run that believed its orders had arrived and closed the business; failures not tied to the context filling up) arxiv.org
  9. Andon Labs, Vending-Bench 2 (top models steady across a simulated year) andonlabs.com
  10. Cemri et al. 2025, Why Do Multi-Agent LLM Systems Fail? (UC Berkeley, v3 October 2025: over 1,600 runs, 14 kinds of failure; step repetition; premature termination) arxiv.org
  11. Anthropic, Effective harnesses for long-running agents (November 2025: declaring the job done; one feature at a time, tested before it counts as done) anthropic.com
  12. Chroma, Context Rot (July 2025: models do worse as the input gets longer) trychroma.com
  13. Anthropic, Writing effective tools for agents (September 2025: a tool that cuts a list short should say so) anthropic.com
  14. Anthropic, Project Vend: Phase two (December 2025: checking prices and delivery times, a newer model and better tools) anthropic.com
  15. Anthropic, Harness design for long-running application development (March 2026: harness as the word for the code around the model) anthropic.com
  16. OpenAI, A practical guide to building agents (2025: when to bring in a person) cdn.openai.com