AI Agent Verification: How to Stop Coding Agents from Declaring Done Too Early

What it is: A practical guide to stopping AI coding agents from declaring “done” when the work isn’t actually done. Covers the two layers of verification (self-check and external), Anthropic’s three-agent harness with a dedicated Evaluator, the Spec Kit acceptance-criteria pattern, and the objective signals that beat self-grading.
Who it is for: Developers using Claude Code, Cursor, Codex, or GitHub Copilot who’ve watched the agent confidently say “task complete” when the test suite is red, the feature is half-built, or the actual goal was never achieved.
Best if: You want the patterns Anthropic and HumanLayer actually use in production, plus the Claude Code hooks and Spec Kit setup that enforces them.
Skip if: You haven’t tried letting an agent work on a real task yet. Start with How to Use AI and Why AI Coding Agents Fail first. Want one practical AI workflow every morning? Subscribe to our free daily newsletter.

Why do AI agents declare done before they’re actually done?

Anthropic names this failure mode explicitly in their official harness-design writeup: “agents tend to respond by confidently praising the work — even when, to a human observer, the quality is obviously mediocre.” That’s the root cause. The agent has been trained to be helpful and confident; when it grades its own output, it grades it generously. Always.

Three forces compound the problem:

• Sycophancy bias. Most modern coding models are trained with reinforcement learning that rewards agreement and confidence. Asking the agent “are you done?” gets a confident yes by default. The training does not push back against itself.
• Context anxiety. Cognition (the team behind Devin) documented this in Claude Sonnet 4.5: as the context window fills, the model “proactively summarizes its progress and becomes more decisive about implementing fixes to close out tasks” — even when the budget isn’t actually exhausted. The model also “consistently underestimates how many tokens it has left,” producing premature shortcuts and incomplete work.
• The “looks plausible” stopping condition. Without an explicit Definition of Done, the agent invents one. The invented one is always lenient. Walking Labs’ Lecture 9 calls the gap between the agent’s confidence and its actual correctness the verification gap.

Real examples you’ve probably seen:

• A login form is rendered. The submit handler was never wired. Agent says done.
• An API endpoint is built and registered in the router. No controller actually calls it. Agent says done.
• A test file is created. It imports the new module. It asserts True. Agent says done.
• The agent runs npm test, gets a long red wall, and says “core functionality is working — minor test issues remaining.” Agent says done.

All four are the same underlying failure: the agent grades itself, the agent grades generously, no objective signal stops it.

What are the two layers of verification?

Verification works at two levels. Most production harnesses use both.

• Layer 1 — Self-verification. The agent checks its own work before claiming done. Runs tests. Exercises the new code path. Writes a smoke test and runs it. This is what a thoughtful developer does on their own work; it catches the obvious mistakes.
• Layer 2 — External verification. A separate process (or a separate agent) checks the work. The verifier doesn’t trust the agent’s claim of done; it measures objective signals directly. This is what Anthropic’s three-agent harness calls the Evaluator role.

Anthropic is explicit that “separating the agent doing the work from the agent judging it” is one of the strongest reliability levers available. Self-verification alone is not enough — the same sycophancy bias that made the agent claim done in the first place makes it claim its self-verification passed.

What are the concrete self-verification patterns?

Five patterns that work, in order of effectiveness:

• Test-suite-must-pass rule. Hardcode it in your AGENTS.md or CLAUDE.md: “Before claiming a task is complete, run the full test suite and confirm zero failures. If any tests fail, do not declare done — fix them or escalate.” Anthropic’s own Claude Code guidance phrases this as “Only mark features as passing after careful testing.”
• Run the actual code path you just built. Not just compile. Invoke. Claude Code 2.1’s /goal command does this automatically — it writes code, runs tests, debugs failures, and re-runs until a defined completion state is reached.
• End-to-end smoke test as the last step. A single integration test that exercises the user flow the new code is supposed to support. If the smoke test fails, the work isn’t done, no matter what the unit tests say.
• Explicit completion criteria written upfront. Before the agent writes a line of code, it writes (or you write) the acceptance criteria: “the form submits successfully,” “the endpoint returns 200 with the expected payload,” “the new column appears in the dashboard.” GitHub’s Spec Kit calls these “explicit conditions that must be met for a feature to be considered complete … transforming specifications from passive documents into active quality gates.”
• Self-critique pass. Asking the agent to critique its own answer and revise helps for prose. It mostly fails on code, because the agent will reaffirm its earlier work. Use this as a final polish step, not as your primary verification.

What are the concrete external-verification patterns?

External verification is where the reliability story really gets won. Five patterns:

• The Evaluator agent. Anthropic’s three-agent harness uses a separate agent dedicated to grading. The Evaluator drives Playwright MCP to navigate live pages, click through UI flows, hit API endpoints, inspect database state, then scores each acceptance criterion. Hard thresholds: if any falls below, the sprint fails and the Generator gets specific feedback.
• The sprint contract. Before any code gets written, the Generator and Evaluator agree on what the Generator will build and how success will be verified. Anthropic frames this as “The generator proposed what it would build and how success would be verified, and the evaluator reviewed that proposal.” Verification is locked before work starts — the agent can’t move the goalposts mid-task.
• Independent supervisor session. Claude Code 2.1’s /goal workflow uses a second, independent Claude session to review the final repository state and confirm the goal was actually achieved before the human is notified. The reviewer never saw the work being done — only the end result.
• Spec-driven development. GitHub’s open-source Spec Kit uses a four-phase loop: Spec → Plan → Tasks → Implement. Each phase produces a Markdown artifact the agent must satisfy; quality checklists become the gate.
• Human as final verifier. Cursor’s diff review and Claude Code’s per-change approval workflow both put the human in the loop on every meaningful change. Slow, but unbeatable as a last line of defense.

The pattern that scales: use sprint contracts for definition, an Evaluator agent for routine verification, and human review on the final commit before main.

What completion-criteria mistakes do beginners make?

• Treating “it compiles” as “it works.” Compilation tests grammar, not behavior. A program that compiles can do entirely the wrong thing.
• “Tests pass” without confirming the new test actually exercises the new feature. Agents will write a test that imports the new module and asserts True. The test passes. The feature doesn’t work. Always inspect new tests for what they actually assert.
• Manual smoke tests the human forgets to run. If the criterion isn’t machine-verifiable, it isn’t a criterion. Walking Labs’ Lecture 9 puts this bluntly: machine-verifiable conditions only — tests pass, lint clean, type checks pass, build green.
• Trusting the agent’s own claim of done. Don’t ask the agent. Ask the test suite. Ask the build. Ask Playwright. The agent’s opinion of its own work is the noisiest signal in the room.
• Conflating “no errors” with “correct output.” A function that returns the wrong number with no exception is still wrong. Verify outputs, not just the absence of crashes.

What does Anthropic officially recommend?

Anthropic’s engineering team is unusually direct on this. From their harness-design article:

• Self-evaluation is unreliable because the agent “confidently prais[es] the work — even when quality is obviously mediocre.”
• The fix is structural: separate the role. Don’t ask the same agent to grade itself.
• Calibrate the Evaluator with few-shot examples and explicit scoring criteria.
• From the Claude Agent SDK design notes: “agents that can check and improve their own output are fundamentally more reliable — they catch mistakes before they compound, self-correct when they drift, and get better as they iterate.”
• The reliability loop they describe: gather context → take action → verify work → repeat, with ground truth coming from the environment, not the agent’s opinion.

Trust but verify is the right framing. Trust the agent to do the work. Verify the work with something the agent can’t lie to.

Which tools actually enforce verification?

• Playwright (and Playwright MCP server). The gold standard for browser-level verification — the Evaluator agent literally drives a real browser, clicks the buttons, and reads the rendered output. Anthropic’s three-agent harness uses Playwright MCP for exactly this. If your project has a frontend, this is the highest-leverage verification tool you can adopt.
• Claude Code hooks. Three hook types matter for verification: PreToolUse (fires before a tool runs, can block destructive commands), PostToolUse (fires after a tool runs — the natural slot for “ran an edit → now run the test suite / linter / type checker”), and Stop (fires when the agent claims done — the slot for the final verification gate). The hooks turn AGENTS.md rules into machine-enforced rules.
• OpenAI Codex. Codex commits its changes and “provides verifiable evidence of its actions through citations of terminal logs and test outputs.” In multi-agent Codex workflows, a dedicated Tester agent reads acceptance criteria and verifies other agents’ output.
• Spec Kit (GitHub). Open-source toolkit for spec-driven development. Specs become acceptance gates the implementation has to pass.
• CI pipelines. Old, boring, still effective. Any agent change should run through CI before reaching main — tests, type checks, lint, build, security scan.
• Human diff review. Cursor’s diff view, Claude Code’s per-edit approval, GitHub PR review. The cheapest external verifier for solo developers.

Why doesn’t “ask the agent if it’s done” work?

Modern coding models are trained on reinforcement-learning signals that reward confidence and agreement. The same training that makes the agent helpful and decisive also makes it agree with itself.

Real consequence: asking the agent “is this done?” almost always gets a confident yes. Asking “are you sure?” often gets a more emphatic yes. Asking “critique your work” usually produces a critique that praises the strong points of the work without flagging the real problems. Anthropic puts this plainly: agents “skew positive when grading their own work.”

The shift, then, is from asking the agent to measuring objective signals it can’t bend:

• Does pytest exit 0?
• Does npm run build produce the expected artifacts?
• Does the Playwright snapshot match?
• Does the new API endpoint return the expected payload when invoked?
• Does git diff show changes inside the files the spec said should change?
• Does the type checker pass?

Every one of those is a yes/no signal the agent can’t talk its way around. That’s the definition of “done” worth using.

Frequently asked questions

Should I always use a separate verifier agent?

For high-stakes work, yes. For a quick one-file edit or an experimental script, no — you’d burn more time setting up the verifier than the work itself. The right rule is: any task that takes more than 30 minutes or touches code that goes to production deserves an external verifier. Anything shorter, self-verification plus your own diff review is fine.

What’s the cheapest way to add external verification?

A Claude Code Stop hook that runs your test command. Two lines of configuration. The hook fires when the agent claims done; if the test command exits non-zero, the agent gets a “tests failed” message instead of a success acknowledgement. That single rule eliminates most premature-done failures for solo developers.

Do I need Playwright?

If your project has a frontend that real users will touch, yes. Playwright (or its MCP server variant) is the cleanest way to verify “the user’s actual flow works” without putting a human in the loop on every change. If your project is backend-only, your existing API test framework (Postman, Insomnia, raw curl, or whatever you use) plays the same role.

What if my codebase doesn’t have tests yet?

Add one. Just one. A single end-to-end smoke test that exercises the most important user flow. Without any tests at all, the agent has nothing to fail against and self-verification becomes meaningless. The first smoke test is the highest-ROI hour you can spend on agent reliability.

Is “agents grading agents” really better than self-grading?

Yes — provided the verifier has a different role definition and different success criteria from the builder. Anthropic’s three-agent harness gives the Evaluator a fundamentally different job (drive the browser, score against the spec) from the Generator (write the code). The role separation is what makes the verifier honest. Two instances of the same agent doing the same job won’t catch each other’s mistakes; an Evaluator with a different brief and different tooling will.

You may also like

• Why AI Coding Agents Fail
• What is AGENTS.md?
• Long-Running Claude Code Tasks
• CLAUDE.md: The File That Makes Claude Code Smarter
• Claude Code Beginners Guide
• Claude Code vs Cursor vs Copilot
• AI Glossary

Sources

• Anthropic — Harness design for long-running application development (three-agent harness, generator/evaluator separation)
• Anthropic — Effective context engineering for AI agents (gather / act / verify loop)
• Anthropic — Building agents with the Claude Agent SDK
• Anthropic — Claude Code best practices
• Cognition — Devin / Sonnet 4.5 context anxiety
• Walking Labs — Lecture 9: Why agents declare victory too early
• InfoQ — Anthropic three-agent harness (April 2026)
• GitHub Spec Kit — Spec-driven development
• Anthropic — Claude Code hooks guide
• OpenAI Codex CLI documentation
• Claude Code — Grokipedia

Last reviewed: May 2026. AI coding agents and verification tooling change every quarter — verify specifics on the vendor pages above before relying on them.