---
title: "The agent shipped it and nobody read it"
description: "An AI agent can produce a working-looking site faster than any human can review it. The defects live in the gap between the agent saying it is done and a stranger being able to use it."
url: https://harpywatch.com/blog/the-agent-shipped-it-and-nobody-read-it
kind: article
author: "The HarpyWatch team"
published: 2026-09-02T18:06:09.282014+00:00
tags: ["AI Agents", "Reliability"]
publisher: "HarpyWatch"
---

# The agent shipped it and nobody read it

An agent finishes a session with a summary. Eleven files changed, three routes
added, the build passes, the dev server returns 200 on every page it visited.
The summary is accurate. Every sentence in it is true.

Then somebody who was not in that session opens the site in a browser they did
not configure, on a host that is not localhost, without the session's
environment, and gets a 500.

Nothing lied. The agent checked what it could reach: the routes it had just
touched, from the machine it was working on, in the state that machine was
already in. That is a much smaller set of conditions than a stranger arrives
with, and the difference between the two sets is where the defects now live.

## What the difference actually contains

These are not exotic failures. They are the ordinary consequence of writing
code in a session that holds state a visitor does not have.

**Pages that render for the author and 500 for everyone else.** The agent hit
the route while a record it had just created was still in the database, while a
feature flag was still on, while it was still authenticated from an earlier
step. The page is a function of that context. Delete the context and the
function throws.

**Links to routes that were renamed mid-session.** An agent working through a
long task renames things as its understanding improves — that is a virtue, not
a bug. But the rename lands in the router and in nine of the eleven templates
that referenced it. The two it missed are the ones that were not open at the
time. A 404 does not fail the build.

**Forms that post nowhere.** The markup is correct, the fields are labelled,
the submit button is styled, and the `action` points at `/api/contact` while
the handler was registered at `/api/v1/contact`. Or the handler exists and only
accepts `POST` from an origin the deployed hostname is not on. The form looks
finished in every way a screenshot can show, and the failure only happens after
somebody types into it.

**Assets referenced at a path that only exists locally.** An image at
`/uploads/hero.png` that lives in the working tree and was never part of the
build output. On the developer's machine it resolves. In the container it is a
404, which most browsers render as a broken layout rather than an error.

**A redirect loop between the framework and the proxy.** The application router
is configured to add a trailing slash and 301s `/pricing` to `/pricing/`. nginx
has a `rewrite` that strips it and 301s back. Each component is doing exactly
what it was configured to do; together they never terminate. A check that
follows redirects and reports the final status code has nothing to report until
its hop limit runs out.

There is a common shape here. In every case the code is defensible, the tests
that exist pass, and the failure requires an environment nobody who wrote it
was standing in.

## Why review does not close the gap

The obvious answer is to read the diff. Two things get in the way.

The first is arithmetic. A session can produce more changed lines than
a person will read carefully in an afternoon, and it can do that again before
lunch. Reviewing capacity did not grow when writing capacity did, and no amount
of discipline closes a ratio.

The second is that human review, even done properly, is aimed at a different
target. Code
review is good at finding logic that is wrong, structure that will not survive
the next change, and decisions that need discussing. It is quite bad at
noticing that the third link in a footer points at a path that no longer
exists, or that a page depends on a session variable that a first-time visitor
does not have. Those are not judgement calls. They are facts about the deployed
site, and they are only observable by making a request to it.

## Verification has to become automatic because authorship already did

If code is now written at machine speed, then the only check that keeps up is
one that also runs at machine speed, and the only useful vantage point is
outside.

The difference matters more than it sounds. A test suite runs in an environment
built to make it pass — fixtures loaded, environment variables set, network
stubbed. That is the correct design for a test suite, and it is exactly the
environment that hides all five failures above. An unauthenticated HTTP request
from another machine has none of that context, which is precisely why it is
worth making.

Of the checks HarpyWatch runs, a handful are aimed squarely at this class of
defect:

| Check | What it catches that a session cannot |
| --- | --- |
| `http-uptime` | The route answers at all, from outside, with no session |
| `dom-content` | The page is up *and* the content is not a stack trace |
| `broken-links` | The nine templates that were updated, and the two that were not |
| `redirect-chain` | The loop between the framework and the proxy |
| `browser-render` | Pages that only assemble once JavaScript runs, and do not |
| `mixed-content` | Subresources the browser will silently refuse to load |

These are the same requests a visitor makes, made on a schedule by something
that carries none of the session's context.

## What this does not do

An external check will not tell you the code is good. It has no opinion on your
architecture, it cannot see that a function is quietly O(n²), and it will not
catch a bug that requires a specific sequence of user actions to reach. It
observes surfaces.

It is also a thing you have to configure and keep correct. A `dom-content`
check asserting a string that legitimately changed is a false alarm, and an
alert that cries wolf gets muted, which is the same as not having it.

It cannot tell you that a page is *wrong* in the semantic sense. A
`dom-content` check knows whether a string you nominated is present. It does
not know that the pricing table now shows last quarter's numbers. Somebody who
knows what the numbers should be still has to look.

What it does is remove an entire category of failure from the list of things a
human has to hold in their head — the category where the site is broken in a
way that any request would have revealed, and no request was made. That
category is small per incident and enormous in aggregate, and it grows in
direct proportion to how fast you are shipping.

## The honest version of the rule

You do not have to trust the agent less. The summary at the end of the session
was true, and being suspicious of it is not a strategy — you cannot re-read
your way out of a volume problem.

You have to make the site prove itself to something that was not in the room.
Continuously, because the next session starts in twenty minutes, and from
outside, because the inside is where all the assumptions live.

That does not cover everything, and the section above says what it misses. It
covers the part that scales with your shipping rate, which is the part that has
stopped being manageable by hand. HarpyWatch runs those requests on a schedule
and tells you when the answer changes.