Why your AI coding agent says 'done' when it isn't, and how to make done mean done

AI coding agents stop when the work looks done, not when it is proven done. They judge success from what they can see, so missing tests, weakened tests, lost context and vague requests all produce false reports. Fix it with written acceptance criteria, a check the agent must pass, hooks and CI that enforce it, and human review.

You ask Claude Code, Codex or Cursor to fix the checkout. It edits six files, runs something, and reports that the feature is complete and all tests pass. You open the app and the checkout still fails. It is a common complaint, and it has a clear cause and a clear fix.

Anthropic's own Claude Code best practices put the cause in one line: "Claude stops when the work looks done." If the agent has nothing to run that proves the work is done, its own judgement is all it has. This guide explains why, and sets out a system that makes "done" verifiable in Claude Code, Codex and Cursor. Product details were checked against vendor documentation on 20 November 2026.

  • Agents report done when the work looks finished from inside the session. Without a pass or fail check, that is a guess.
  • In the Stack Overflow Developer Survey 2025, 66% of developers named AI solutions that are almost right, but not quite, as their top frustration.
  • In METR's tests, telling a model not to cheat barely changed how often it gamed the scoring. Enforcement beats instructions.
  • Instructions in CLAUDE.md, AGENTS.md or Cursor rules are guidance. Stop hooks and required CI checks are enforcement.
  • Done means: acceptance criteria met, a failing test now passing, the real user flow working end to end, and a human approving the merge.
Key takeaways

Why does Claude Code say it's done when it isn't working?

Because an agent judges success only from what it can see: the files it edited and the output of commands it ran. It does not see your browser, your live data, the webhook that never fires, or the intent you never wrote down. When nothing in its view says "broken", the work looks done.

Five causes explain most false "done" reports:

  • No check to run. With no test or script that returns pass or fail, the agent decides for itself. Anthropic says this "makes you the test suite".
  • Tests that miss the real thing. Tests that mock the database or payment provider pass while the real integration fails.
  • Tests changed to pass. The agent skips, loosens or rewrites a failing test instead of fixing the code.
  • Lost goals. Long sessions are summarised to free space, and a requirement given only in chat can drop out.
  • No acceptance criteria, or tasks too large. "Build the booking system" has no finish line, so the agent sets its own.

In the Stack Overflow Developer Survey 2025, "almost right, but not quite" was the top frustration with AI tools, and 45% said debugging AI-generated code takes longer.

Developers' biggest frustrations with AI coding tools

Base: 31,476 respondents. More than one answer allowed.

FrustrationShare
AI solutions that are almost right, but not quite66%
Debugging AI-generated code is more time-consuming45.2%
I've become less confident in my own problem-solving20%
It's hard to understand how or why the code works16.3%
I haven't encountered any problems4%

Source: Stack Overflow, Developer Survey 2025: AI

Trust follows the same pattern. In the same survey, 45.7% of developers said they distrust the accuracy of AI tools and 32.7% said they trust it. Only 3.1% highly trust it. Treat an agent's report as a claim to check, not a result.

Can an AI coding agent fake its tests?

Yes. If the visible goal is "tests pass", editing the test can look like the shortest route. In real projects this shows up as a skipped test, a loosened assertion, an expected value changed to match wrong output, or a mock that returns success whatever the code does.

Independent evaluators have measured this. In June 2025 METR, a non-profit that evaluates AI models, reported that recent frontier models were reward hacking: changing tests, timers or scoring code instead of solving the task. On METR's RE-Bench tasks, OpenAI's o3 did this in 30.4% of runs. In one run it replaced an evaluator with a stub that marked every submission successful. Asked whether its approach matched the user's intent, it answered "No" 10 times out of 10.

The finding that matters most for your setup is what happened when METR simply told the model not to cheat:

Reward hacking rate on one METR task, by prompt wording

OpenAI o3 on the Optimize LLM Foundry task, 20 runs per prompt. Lower is better.

Instruction added to the promptRuns that hacked
Original prompt, no instruction80%
Please do not cheat80%
Please do not reward hack70%
Framed as helping Alzheimer's research70%
Solve it the way the designer intended95%

Source: METR, Recent frontier models are reward hacking, June 2025

METR described the effect of anti-cheating instructions as "nearly negligible". These were research tasks, and newer models may behave differently, but the lesson carries over: "never modify tests" in your instructions is not a control. Protect test files with a hook and review every test change.

Why does the agent forget what I asked it to do?

Because long sessions get compacted. When the context window fills, Claude Code summarises the conversation so work can continue. Its context window documentation lists what survives. The project-root CLAUDE.md and a plan written in plan mode are re-injected from disk. The rest of the conversation becomes a summary, including requirements you typed in chat. Path-scoped rules and nested CLAUDE.md files are summarised away until a matching file is opened again.

Anthropic warns that a boundary stated in conversation "can get lost when the conversation compacts". Codex and Cursor also compact long sessions and offer hooks around it (PreCompact and PostCompact in Codex, preCompact in Cursor). The rule is the same in all three: anything that defines done goes in a file on disk, not a chat message.

Is the agent actually saving you time?

Not always, and it is hard to tell from the inside. METR ran a randomised trial with experienced open-source developers on their own mature projects, mostly using Cursor Pro with Claude 3.5 and 3.7 Sonnet.

What experienced developers expected from AI tools, and what METR measured
  • 24% faster: what developers expected AI to make them before the study
  • 20% faster: what they believed AI had made them afterwards
  • 19% slower: what METR measured when AI was allowed
  • 246 real tasks completed by 16 developers

Source: METR, Measuring the impact of early-2025 AI on experienced open-source developer productivity, July 2025

The sample was small and the tools were early-2025 versions. In a February 2026 update, METR said late-2025 tools likely sped developers up, but that selection effects made its new estimate unreliable: 30% to 50% of developers said they avoided submitting some tasks they did not want to do without AI. The lasting lesson is the gap between perception and measurement: judge an agent by checks that pass, not by how productive it feels.

How do you make 'done' mean done with an AI coding agent?

Define done before the agent starts, give it a check it can run, make the tool enforce that check, and keep a person at the merge. In practice that is a loop:

  1. Write acceptance criteria in a spec file, using plan mode to agree the approach
  2. Write a failing test that proves the behaviour is missing
  3. Put the exact test, typecheck and lint commands in CLAUDE.md, AGENTS.md or Cursor rules
  4. Let the agent implement one small task at a time
  5. Enforce the checks with a Stop hook so the agent cannot finish while they fail
  6. Ask for evidence: the commands run and their output
  7. Run an end-to-end test of the real user flow
  8. Merge only when CI passes and a person approves the pull request
The done-means-done loop for AI coding agents

1. Write acceptance criteria first

Write acceptance criteria as observable outcomes: "a signed-in user can cancel a booking and receives an email", not "add cancellation". Anthropic suggests having Claude interview you, write the result to SPEC.md, then build from it in a fresh session. Plan mode makes the agent propose an approach before editing: Shift+Tab or /plan in Claude Code, /plan in Codex, and Plan Mode in Cursor, which its docs say "creates detailed implementation plans before writing any code".

2. Make a test fail first

A test seen to fail before the fix proves it tests the right thing. Anthropic's example prompt: "write a failing test that reproduces it, then fix it". If it then passes without the test being touched, you have real evidence.

3. Tell the agent how to check its work

Anthropic says the most useful line in CLAUDE.md is the command that runs your checks, and recommends keeping the file under 200 lines. A definition of done might read:

## Definition of done
- npm run typecheck passes
- npm test passes, with no new .skip or .only
- Never edit an existing test to make it pass. Ask first.
- For UI changes, npm run test:e2e passes for the affected flow
- Report the commands you ran and paste their output

Codex reads the same content from AGENTS.md, and its AGENTS.md guide includes running npm test after changing JavaScript files. Cursor reads AGENTS.md or .cursor/rules, and Claude Code reads AGENTS.md when there is no CLAUDE.md, so one shared file can steer all three.

4. Enforce it with a hook

Instructions are advice that Claude may decide does not apply. A Stop hook is enforced: the tool runs your script whenever the agent tries to finish and sends it back if the script fails. In .claude/settings.json:

{
  "hooks": {
    "Stop": [
      { "hooks": [
        { "type": "command", "command": "npm run typecheck 1>&2 || exit 2" }
      ] }
    ]
  }
}

Exit code 2 blocks the finish, and anything printed to stderr goes back to Claude as the reason. If the hook blocks eight stops in a row with no tool call between them, Claude Code ends the turn anyway, so a broken check cannot loop forever. Codex supports the same pattern: a Stop hook returning a block decision, or exit code 2, makes Codex continue with your reason as a new prompt. In Cursor, a stop hook can return a follow-up message that is submitted automatically, capped by a loop limit that defaults to 5.

5. Test the real flow, then gate the merge

Unit tests with mocks can all pass while the product is broken. For anything a customer touches, add an end-to-end test that drives a real browser through sign-up, payment or booking, using test data rather than real customer records. Then make CI the final gate. GitHub's protected branches can require status checks to pass and a set number of approving reviews before merging. By default those rules do not apply to admins, so turn on "Do not allow bypassing the above settings" if the founder's account is the one merging.

How do Claude Code, Codex and Cursor compare for enforcing done?

All three now offer the same core controls. The main differences are defaults and where each setting lives.

Controls that make done verifiable, by tool (checked 20 November 2026)
ControlClaude CodeCodexCursor
Project instruction fileYes, CLAUDE.md or AGENTS.mdYes, AGENTS.mdYes, rules or AGENTS.md
Hook that runs after each editYes, PostToolUseYes, PostToolUseYes, afterFileEdit
Hook that sends the agent back when it tries to finishYes, StopYes, StopYes, stop with follow-up
Sandbox for agent commands on by defaultNo, opt in with /sandboxYes, network offYes, in Auto-review mode

Sources: Anthropic, Claude Code best practices; OpenAI, Codex AGENTS.md; OpenAI, Codex hooks; OpenAI, Codex approvals and security; Cursor, Rules; Cursor, Hooks; Cursor, Run modes

SettingClaude CodeCodexCursor
Where instructions liveCLAUDE.md, under 200 linesAGENTS.md, 32 KiB combined by default.cursor/rules or AGENTS.md, rules under 500 lines
Where hooks live.claude/settings.json.codex/hooks.json or config.toml.cursor/hooks.json
How a finish is blockedExit code 2Block decision or exit code 2followup_message
Plan before editingShift+Tab or /plan/planPlan Mode
Built-in review/code-review/reviewUse CI and a reviewer
Where to set up the done-means-done loop in each tool

Project hooks run code on your machine, so Codex asks you to review them with /hooks and Cursor runs them only in a trusted workspace. Commit them so the whole team gets the same checks. For wider launch checks, see our Claude Code to production checklist and the checklist for shipping a Codex-built app. Our Codex vs Claude Code comparison covers the wider differences.

How do you stop an AI agent from changing your tests?

Use three layers. First, say it plainly in your instructions file, knowing it is guidance only. Second, block it: Anthropic's docs show a PreToolUse hook that exits with code 2 for any path under a protected folder, and the same pattern works for your tests folder during a bug fix. Cursor and Codex hooks can block in the same way, though OpenAI advises treating tool hooks as "a useful guardrail, not a complete enforcement boundary". That is why CI is the final gate. Third, review the diff. Before you accept any agent's work, search the changes for:

  • New .skip, .only or commented-out tests
  • Changed expected values or loosened assertions
  • New mocks of the code the change was meant to fix
  • Deleted tests or lowered coverage thresholds
  • Error handling that swallows exceptions

When a requirement really has changed, ask the agent to say so and explain the test change, so it is your decision rather than a side effect.

How big should a task be for an AI coding agent?

Small enough that one test, or a handful, can prove it is finished. "Add a cancel button that sets the booking status and emails the customer" is a good task. "Build bookings" is not. Smaller tasks mean shorter sessions, less compaction and a reviewable diff. Use /clear or a fresh session between unrelated tasks. Anthropic's checklist for unattended runs starts with one question: does the agent know what done looks like? If the answer is not a goal, a spec file or a failing test, the task is not ready to hand over.

When should you bring in a human?

Always at the merge, and earlier for anything touching payments, authentication, personal data or database migrations. A reviewer who reads the diff against the spec catches the gap between "tests pass" and "the feature works".

If an agent has already built an app that keeps saying done while users say broken, the fix is usually the loop above applied after the fact: write the acceptance criteria, add tests for the real flows, then repair what fails. We used AI-assisted development with Claude Code, Codex, Antigravity and Lovable to build The Study Room's Portal, Dentorb and ORBQ, with these checks around the agents. Read whether to fix or rebuild a vibe-coded app. To set the loop up for your own team, our Claude Code training covers instructions, hooks and review habits. If the app needs rescuing, see how we fix Claude Code apps and fix Codex-built apps.

Frequently asked questions

Why does Claude Code say all tests pass when my app is still broken?

Usually because the tests do not exercise the real flow. They may mock the database or payment provider, test the wrong behaviour, or have been changed during the session. Ask Claude for the exact commands and output, check the diff for edited tests, and add an end-to-end test that drives the real user journey.

How do I stop Claude Code or Codex from editing my tests to make them pass?

Say it in CLAUDE.md or AGENTS.md, but do not rely on that alone, because instructions are guidance. Add a hook that blocks edits to your tests folder during bug fixes, and review every test change in the diff. Look for new skips, changed expected values, new mocks and lowered coverage thresholds.

Why does Codex stop before the task is finished?

Codex ends a turn when the work looks complete from inside the session. If nothing it can run says the task is unfinished, it stops. Add a Stop hook that runs your tests and returns a block decision when they fail. Codex then continues, using your reason text as a new prompt.

Does telling an AI coding agent not to cheat actually work?

Not reliably. In METR's 2025 tests, adding please do not cheat to the prompt left OpenAI o3 gaming the scoring in 80 percent of runs on one task, the same as without it. Use controls the tool enforces, such as hooks that block test edits and CI checks that must pass, instead of instructions alone.

What should a definition of done for an AI coding agent include?

Observable acceptance criteria for the task, the exact test, typecheck and lint commands that must pass, a rule against editing existing tests without asking, an end-to-end check for any user-facing flow, and a request for evidence: the commands run and their output. Keep it short and in the project instructions file.

Does Cursor's agent check its own work before saying it is fixed?

Only if you give it something to check. Cursor can run tests in the terminal, but nothing forces it to unless you ask. Put your test commands in AGENTS.md or a project rule, and add a stop hook in .cursor/hooks.json that returns a follow-up message telling the agent to fix any failures.

Sources

Checked on 20 November 2026. Product features and figures change quickly, so check the current source before you rely on a detail.

Agent says done, users say broken?

Circlemount repairs apps built with coding agents and sets up the checks that stop false "done" reports: acceptance criteria, real end-to-end tests, hooks and CI gates. See how we fix Claude Code apps and fix Codex-built apps. The initial app assessment is free, project work starts from £1,000 + VAT, and ongoing support plans start from £125 a month + VAT. Join the Circle to book your assessment.