The agent shipped it and nobody read it

An AI agent can produce a working-looking site faster than any human can review it. The defects live in the gap between the agent saying it is done and a stranger being able to use it.

article By 6 min read

An agent finishes a session with a summary. Eleven files changed, three routes added, the build passes, the dev server returns 200 on every page it visited. The summary is accurate. Every sentence in it is true.

Then somebody who was not in that session opens the site in a browser they did not configure, on a host that is not localhost, without the session's environment, and gets a 500.

Nothing lied. The agent checked what it could reach: the routes it had just touched, from the machine it was working on, in the state that machine was already in. That is a much smaller set of conditions than a stranger arrives with, and the difference between the two sets is where the defects now live.

What the difference actually contains

These are not exotic failures. They are the ordinary consequence of writing code in a session that holds state a visitor does not have.

Pages that render for the author and 500 for everyone else. The agent hit the route while a record it had just created was still in the database, while a feature flag was still on, while it was still authenticated from an earlier step. The page is a function of that context. Delete the context and the function throws.

Links to routes that were renamed mid-session. An agent working through a long task renames things as its understanding improves — that is a virtue, not a bug. But the rename lands in the router and in nine of the eleven templates that referenced it. The two it missed are the ones that were not open at the time. A 404 does not fail the build.

Forms that post nowhere. The markup is correct, the fields are labelled, the submit button is styled, and the action points at /api/contact while the handler was registered at /api/v1/contact. Or the handler exists and only accepts POST from an origin the deployed hostname is not on. The form looks finished in every way a screenshot can show, and the failure only happens after somebody types into it.

Assets referenced at a path that only exists locally. An image at /uploads/hero.png that lives in the working tree and was never part of the build output. On the developer's machine it resolves. In the container it is a 404, which most browsers render as a broken layout rather than an error.

A redirect loop between the framework and the proxy. The application router is configured to add a trailing slash and 301s /pricing to /pricing/. nginx has a rewrite that strips it and 301s back. Each component is doing exactly what it was configured to do; together they never terminate. A check that follows redirects and reports the final status code has nothing to report until its hop limit runs out.

There is a common shape here. In every case the code is defensible, the tests that exist pass, and the failure requires an environment nobody who wrote it was standing in.

Why review does not close the gap

The obvious answer is to read the diff. Two things get in the way.

The first is arithmetic. A session can produce more changed lines than a person will read carefully in an afternoon, and it can do that again before lunch. Reviewing capacity did not grow when writing capacity did, and no amount of discipline closes a ratio.

The second is that human review, even done properly, is aimed at a different target. Code review is good at finding logic that is wrong, structure that will not survive the next change, and decisions that need discussing. It is quite bad at noticing that the third link in a footer points at a path that no longer exists, or that a page depends on a session variable that a first-time visitor does not have. Those are not judgement calls. They are facts about the deployed site, and they are only observable by making a request to it.

Verification has to become automatic because authorship already did

If code is now written at machine speed, then the only check that keeps up is one that also runs at machine speed, and the only useful vantage point is outside.

The difference matters more than it sounds. A test suite runs in an environment built to make it pass — fixtures loaded, environment variables set, network stubbed. That is the correct design for a test suite, and it is exactly the environment that hides all five failures above. An unauthenticated HTTP request from another machine has none of that context, which is precisely why it is worth making.

Of the checks HarpyWatch runs, a handful are aimed squarely at this class of defect:

Check What it catches that a session cannot
http-uptime The route answers at all, from outside, with no session
dom-content The page is up and the content is not a stack trace
broken-links The nine templates that were updated, and the two that were not
redirect-chain The loop between the framework and the proxy
browser-render Pages that only assemble once JavaScript runs, and do not
mixed-content Subresources the browser will silently refuse to load

These are the same requests a visitor makes, made on a schedule by something that carries none of the session's context.

What this does not do

An external check will not tell you the code is good. It has no opinion on your architecture, it cannot see that a function is quietly O(n²), and it will not catch a bug that requires a specific sequence of user actions to reach. It observes surfaces.

It is also a thing you have to configure and keep correct. A dom-content check asserting a string that legitimately changed is a false alarm, and an alert that cries wolf gets muted, which is the same as not having it.

It cannot tell you that a page is wrong in the semantic sense. A dom-content check knows whether a string you nominated is present. It does not know that the pricing table now shows last quarter's numbers. Somebody who knows what the numbers should be still has to look.

What it does is remove an entire category of failure from the list of things a human has to hold in their head — the category where the site is broken in a way that any request would have revealed, and no request was made. That category is small per incident and enormous in aggregate, and it grows in direct proportion to how fast you are shipping.

The honest version of the rule

You do not have to trust the agent less. The summary at the end of the session was true, and being suspicious of it is not a strategy — you cannot re-read your way out of a volume problem.

You have to make the site prove itself to something that was not in the room. Continuously, because the next session starts in twenty minutes, and from outside, because the inside is where all the assumptions live.

That does not cover everything, and the section above says what it misses. It covers the part that scales with your shipping rate, which is the part that has stopped being manageable by hand. HarpyWatch runs those requests on a schedule and tells you when the answer changes.

Cite this

The HarpyWatch team. “The agent shipped it and nobody read it”. The Nest, HarpyWatch, 2 September 2026 UTC. https://harpywatch.com/blog/the-agent-shipped-it-and-nobody-read-it

More from The Nest

The agent will make the check pass

Given a red signal, an agent optimises for the signal. That is not a flaw in any particular tool — it is what optimisation means, and it changes which signals are worth having.

article 6 min read

Your monitor cannot live in the thing it monitors

A health check inside your own stack goes silent at exactly the moment it had something to say. Here are the failure domains an internal check shares with the site, and what an external one sees instead.

article 6 min read