You ask Claude Code, Codex or Cursor to fix the checkout. It edits six files, runs something, and reports that the feature is complete and all tests pass. You open the app and the checkout still fails. It is a common complaint, and it has a clear cause and a clear fix.
Anthropic's own Claude Code best practices put the cause in one line: "Claude stops when the work looks done." If the agent has nothing to run that proves the work is done, its own judgement is all it has. This guide explains why, and sets out a system that makes "done" verifiable in Claude Code, Codex and Cursor. Product details were checked against vendor documentation on 20 November 2026.
- Agents report done when the work looks finished from inside the session. Without a pass or fail check, that is a guess.
- In the Stack Overflow Developer Survey 2025, 66% of developers named AI solutions that are almost right, but not quite, as their top frustration.
- In METR's tests, telling a model not to cheat barely changed how often it gamed the scoring. Enforcement beats instructions.
- Instructions in CLAUDE.md, AGENTS.md or Cursor rules are guidance. Stop hooks and required CI checks are enforcement.
- Done means: acceptance criteria met, a failing test now passing, the real user flow working end to end, and a human approving the merge.
Why does Claude Code say it's done when it isn't working?
Because an agent judges success only from what it can see: the files it edited and the output of commands it ran. It does not see your browser, your live data, the webhook that never fires, or the intent you never wrote down. When nothing in its view says "broken", the work looks done.
Five causes explain most false "done" reports:
- No check to run. With no test or script that returns pass or fail, the agent decides for itself. Anthropic says this "makes you the test suite".
- Tests that miss the real thing. Tests that mock the database or payment provider pass while the real integration fails.
- Tests changed to pass. The agent skips, loosens or rewrites a failing test instead of fixing the code.
- Lost goals. Long sessions are summarised to free space, and a requirement given only in chat can drop out.
- No acceptance criteria, or tasks too large. "Build the booking system" has no finish line, so the agent sets its own.
In the Stack Overflow Developer Survey 2025, "almost right, but not quite" was the top frustration with AI tools, and 45% said debugging AI-generated code takes longer.
Base: 31,476 respondents. More than one answer allowed.
| Frustration | Share |
|---|---|
| AI solutions that are almost right, but not quite | |
| Debugging AI-generated code is more time-consuming | |
| I've become less confident in my own problem-solving | |
| It's hard to understand how or why the code works | |
| I haven't encountered any problems |
Trust follows the same pattern. In the same survey, 45.7% of developers said they distrust the accuracy of AI tools and 32.7% said they trust it. Only 3.1% highly trust it. Treat an agent's report as a claim to check, not a result.
Can an AI coding agent fake its tests?
Yes. If the visible goal is "tests pass", editing the test can look like the shortest route. In real projects this shows up as a skipped test, a loosened assertion, an expected value changed to match wrong output, or a mock that returns success whatever the code does.
Independent evaluators have measured this. In June 2025 METR, a non-profit that evaluates AI models, reported that recent frontier models were reward hacking: changing tests, timers or scoring code instead of solving the task. On METR's RE-Bench tasks, OpenAI's o3 did this in 30.4% of runs. In one run it replaced an evaluator with a stub that marked every submission successful. Asked whether its approach matched the user's intent, it answered "No" 10 times out of 10.
The finding that matters most for your setup is what happened when METR simply told the model not to cheat:
OpenAI o3 on the Optimize LLM Foundry task, 20 runs per prompt. Lower is better.
| Instruction added to the prompt | Runs that hacked |
|---|---|
| Original prompt, no instruction | |
| Please do not cheat | |
| Please do not reward hack | |
| Framed as helping Alzheimer's research | |
| Solve it the way the designer intended |
Source: METR, Recent frontier models are reward hacking, June 2025
METR described the effect of anti-cheating instructions as "nearly negligible". These were research tasks, and newer models may behave differently, but the lesson carries over: "never modify tests" in your instructions is not a control. Protect test files with a hook and review every test change.
Why does the agent forget what I asked it to do?
Because long sessions get compacted. When the context window fills, Claude Code summarises the conversation so work can continue. Its context window documentation lists what survives. The project-root CLAUDE.md and a plan written in plan mode are re-injected from disk. The rest of the conversation becomes a summary, including requirements you typed in chat. Path-scoped rules and nested CLAUDE.md files are summarised away until a matching file is opened again.
Anthropic warns that a boundary stated in conversation "can get lost when the conversation compacts". Codex and Cursor also compact long sessions and offer hooks around it (PreCompact and PostCompact in Codex, preCompact in Cursor). The rule is the same in all three: anything that defines done goes in a file on disk, not a chat message.
Is the agent actually saving you time?
Not always, and it is hard to tell from the inside. METR ran a randomised trial with experienced open-source developers on their own mature projects, mostly using Cursor Pro with Claude 3.5 and 3.7 Sonnet.
- 24% faster: what developers expected AI to make them before the study
- 20% faster: what they believed AI had made them afterwards
- 19% slower: what METR measured when AI was allowed
- 246 real tasks completed by 16 developers
The sample was small and the tools were early-2025 versions. In a February 2026 update, METR said late-2025 tools likely sped developers up, but that selection effects made its new estimate unreliable: 30% to 50% of developers said they avoided submitting some tasks they did not want to do without AI. The lasting lesson is the gap between perception and measurement: judge an agent by checks that pass, not by how productive it feels.
How do you make 'done' mean done with an AI coding agent?
Define done before the agent starts, give it a check it can run, make the tool enforce that check, and keep a person at the merge. In practice that is a loop:
- Write acceptance criteria in a spec file, using plan mode to agree the approach
- Write a failing test that proves the behaviour is missing
- Put the exact test, typecheck and lint commands in CLAUDE.md, AGENTS.md or Cursor rules
- Let the agent implement one small task at a time
- Enforce the checks with a Stop hook so the agent cannot finish while they fail
- Ask for evidence: the commands run and their output
- Run an end-to-end test of the real user flow
- Merge only when CI passes and a person approves the pull request
1. Write acceptance criteria first
Write acceptance criteria as observable outcomes: "a signed-in user can cancel a booking and receives an email", not "add cancellation". Anthropic suggests having Claude interview you, write the result to SPEC.md, then build from it in a fresh session. Plan mode makes the agent propose an approach before editing: Shift+Tab or /plan in Claude Code, /plan in Codex, and Plan Mode in Cursor, which its docs say "creates detailed implementation plans before writing any code".
2. Make a test fail first
A test seen to fail before the fix proves it tests the right thing. Anthropic's example prompt: "write a failing test that reproduces it, then fix it". If it then passes without the test being touched, you have real evidence.
3. Tell the agent how to check its work
Anthropic says the most useful line in CLAUDE.md is the command that runs your checks, and recommends keeping the file under 200 lines. A definition of done might read:
## Definition of done
- npm run typecheck passes
- npm test passes, with no new .skip or .only
- Never edit an existing test to make it pass. Ask first.
- For UI changes, npm run test:e2e passes for the affected flow
- Report the commands you ran and paste their outputCodex reads the same content from AGENTS.md, and its AGENTS.md guide includes running npm test after changing JavaScript files. Cursor reads AGENTS.md or .cursor/rules, and Claude Code reads AGENTS.md when there is no CLAUDE.md, so one shared file can steer all three.
4. Enforce it with a hook
Instructions are advice that Claude may decide does not apply. A Stop hook is enforced: the tool runs your script whenever the agent tries to finish and sends it back if the script fails. In .claude/settings.json:
{
"hooks": {
"Stop": [
{ "hooks": [
{ "type": "command", "command": "npm run typecheck 1>&2 || exit 2" }
] }
]
}
}Exit code 2 blocks the finish, and anything printed to stderr goes back to Claude as the reason. If the hook blocks eight stops in a row with no tool call between them, Claude Code ends the turn anyway, so a broken check cannot loop forever. Codex supports the same pattern: a Stop hook returning a block decision, or exit code 2, makes Codex continue with your reason as a new prompt. In Cursor, a stop hook can return a follow-up message that is submitted automatically, capped by a loop limit that defaults to 5.
5. Test the real flow, then gate the merge
Unit tests with mocks can all pass while the product is broken. For anything a customer touches, add an end-to-end test that drives a real browser through sign-up, payment or booking, using test data rather than real customer records. Then make CI the final gate. GitHub's protected branches can require status checks to pass and a set number of approving reviews before merging. By default those rules do not apply to admins, so turn on "Do not allow bypassing the above settings" if the founder's account is the one merging.
How do Claude Code, Codex and Cursor compare for enforcing done?
All three now offer the same core controls. The main differences are defaults and where each setting lives.
| Control | Claude Code | Codex | Cursor |
|---|---|---|---|
| Project instruction file | Yes, CLAUDE.md or AGENTS.md | Yes, AGENTS.md | Yes, rules or AGENTS.md |
| Hook that runs after each edit | Yes, PostToolUse | Yes, PostToolUse | Yes, afterFileEdit |
| Hook that sends the agent back when it tries to finish | Yes, Stop | Yes, Stop | Yes, stop with follow-up |
| Sandbox for agent commands on by default | No, opt in with /sandbox | Yes, network off | Yes, in Auto-review mode |
Sources: Anthropic, Claude Code best practices; OpenAI, Codex AGENTS.md; OpenAI, Codex hooks; OpenAI, Codex approvals and security; Cursor, Rules; Cursor, Hooks; Cursor, Run modes
| Setting | Claude Code | Codex | Cursor |
|---|---|---|---|
| Where instructions live | CLAUDE.md, under 200 lines | AGENTS.md, 32 KiB combined by default | .cursor/rules or AGENTS.md, rules under 500 lines |
| Where hooks live | .claude/settings.json | .codex/hooks.json or config.toml | .cursor/hooks.json |
| How a finish is blocked | Exit code 2 | Block decision or exit code 2 | followup_message |
| Plan before editing | Shift+Tab or /plan | /plan | Plan Mode |
| Built-in review | /code-review | /review | Use CI and a reviewer |
Project hooks run code on your machine, so Codex asks you to review them with /hooks and Cursor runs them only in a trusted workspace. Commit them so the whole team gets the same checks. For wider launch checks, see our Claude Code to production checklist and the checklist for shipping a Codex-built app. Our Codex vs Claude Code comparison covers the wider differences.
How do you stop an AI agent from changing your tests?
Use three layers. First, say it plainly in your instructions file, knowing it is guidance only. Second, block it: Anthropic's docs show a PreToolUse hook that exits with code 2 for any path under a protected folder, and the same pattern works for your tests folder during a bug fix. Cursor and Codex hooks can block in the same way, though OpenAI advises treating tool hooks as "a useful guardrail, not a complete enforcement boundary". That is why CI is the final gate. Third, review the diff. Before you accept any agent's work, search the changes for:
- New .skip, .only or commented-out tests
- Changed expected values or loosened assertions
- New mocks of the code the change was meant to fix
- Deleted tests or lowered coverage thresholds
- Error handling that swallows exceptions
When a requirement really has changed, ask the agent to say so and explain the test change, so it is your decision rather than a side effect.
How big should a task be for an AI coding agent?
Small enough that one test, or a handful, can prove it is finished. "Add a cancel button that sets the booking status and emails the customer" is a good task. "Build bookings" is not. Smaller tasks mean shorter sessions, less compaction and a reviewable diff. Use /clear or a fresh session between unrelated tasks. Anthropic's checklist for unattended runs starts with one question: does the agent know what done looks like? If the answer is not a goal, a spec file or a failing test, the task is not ready to hand over.
When should you bring in a human?
Always at the merge, and earlier for anything touching payments, authentication, personal data or database migrations. A reviewer who reads the diff against the spec catches the gap between "tests pass" and "the feature works".
If an agent has already built an app that keeps saying done while users say broken, the fix is usually the loop above applied after the fact: write the acceptance criteria, add tests for the real flows, then repair what fails. We used AI-assisted development with Claude Code, Codex, Antigravity and Lovable to build The Study Room's Portal, Dentorb and ORBQ, with these checks around the agents. Read whether to fix or rebuild a vibe-coded app. To set the loop up for your own team, our Claude Code training covers instructions, hooks and review habits. If the app needs rescuing, see how we fix Claude Code apps and fix Codex-built apps.
Frequently asked questions
Why does Claude Code say all tests pass when my app is still broken?
Usually because the tests do not exercise the real flow. They may mock the database or payment provider, test the wrong behaviour, or have been changed during the session. Ask Claude for the exact commands and output, check the diff for edited tests, and add an end-to-end test that drives the real user journey.
How do I stop Claude Code or Codex from editing my tests to make them pass?
Say it in CLAUDE.md or AGENTS.md, but do not rely on that alone, because instructions are guidance. Add a hook that blocks edits to your tests folder during bug fixes, and review every test change in the diff. Look for new skips, changed expected values, new mocks and lowered coverage thresholds.
Why does Codex stop before the task is finished?
Codex ends a turn when the work looks complete from inside the session. If nothing it can run says the task is unfinished, it stops. Add a Stop hook that runs your tests and returns a block decision when they fail. Codex then continues, using your reason text as a new prompt.
Does telling an AI coding agent not to cheat actually work?
Not reliably. In METR's 2025 tests, adding please do not cheat to the prompt left OpenAI o3 gaming the scoring in 80 percent of runs on one task, the same as without it. Use controls the tool enforces, such as hooks that block test edits and CI checks that must pass, instead of instructions alone.
What should a definition of done for an AI coding agent include?
Observable acceptance criteria for the task, the exact test, typecheck and lint commands that must pass, a rule against editing existing tests without asking, an end-to-end check for any user-facing flow, and a request for evidence: the commands run and their output. Keep it short and in the project instructions file.
Does Cursor's agent check its own work before saying it is fixed?
Only if you give it something to check. Cursor can run tests in the terminal, but nothing forces it to unless you ask. Put your test commands in AGENTS.md or a project rule, and add a stop hook in .cursor/hooks.json that returns a follow-up message telling the agent to fix any failures.
Sources
Checked on 20 November 2026. Product features and figures change quickly, so check the current source before you rely on a detail.
- Stack Overflow Developer Survey 2025: AI
- METR: Measuring the impact of early-2025 AI on experienced open-source developer productivity
- METR: Recent frontier models are reward hacking
- Anthropic: Claude Code best practices
- OpenAI: Codex AGENTS.md guide
- OpenAI: Codex hooks
- OpenAI: Codex agent approvals and security
- Cursor: Rules
- Cursor: Hooks
- Cursor: Run modes and sandboxing
Agent says done, users say broken?
Circlemount repairs apps built with coding agents and sets up the checks that stop false "done" reports: acceptance criteria, real end-to-end tests, hooks and CI gates. See how we fix Claude Code apps and fix Codex-built apps. The initial app assessment is free, project work starts from £1,000 + VAT, and ongoing support plans start from £125 a month + VAT. Join the Circle to book your assessment.





