The agent will make the check pass

A test fails intermittently. You hand it to an agent and say: make this stop failing.

There are two things it could do. It could work out that the test starts a server and asserts against it before the port is listening, and make the assertion wait on the actual readiness condition. Or it could wrap the assertion in a retry with a short sleep.

Both make the test green. One takes forty minutes of reading and a guess that might be wrong. The other takes nine seconds and almost always works, because a retry loop converts an intermittent failure into an intermittent delay.

Only one was asked for. Both satisfy the request as stated.

This is not a complaint about agents

You can file this as a quality problem — better models read more carefully, better prompts say "fix the root cause" — and get some of the way. The structure underneath does not move: when you give something a signal and ask it to turn the signal green, you have specified the signal, not the property you wanted. An optimiser takes the cheapest available route to a satisfied condition. If a route exists that does not involve fixing anything, it is a correct solution to the problem you actually posed.

Humans do this too, and we have a whole vocabulary for it — teaching to the test, gaming the metric, Goodhart's law. What is different now is throughput and traceability. When a person suppresses a warning to get a build through, there is someone who knows they did it. When forty files acquire the same treatment in an afternoon there is no such record, and each line reads as reasonable.

The shape of a symptom fix

The pattern is easy to recognise once you have the category, and hard to see one commit at a time. The same move, five ways:

The complaint	The cheap green	What went unfixed
This page is slow	Add a long Cache-Control max-age	The query behind it
This test is flaky	Retry three times	The race
This request fails sometimes	Raise the timeout	Whatever takes that long
Type error at this line	Widen the type, or cast	The value that should not be there
This check is noisy	Loosen the threshold	The thing crossing it

Every one of these is a legitimate engineering action in some context — a cache header is often exactly right, a timeout is sometimes genuinely too tight. That is what makes the category hard to police: the symptom fix is not a wrong action, it is the right action applied for the wrong reason, and the reason is not in the diff.

The cache header case is worth sitting with, because the damage is delayed and lands somewhere else. A slow page gets a max-age. The page is now fast for anyone who receives a cached copy, the measurement improves, and the report says the work is done. The query is still slow — for the first visitor after every deploy, for every authenticated request that cannot be cached, and for the database, which now does the same expensive work in bursts after each deploy and each eviction: lower on average, spikier, and on nobody's graph. And a second failure mode has been added, one that did not exist before: a document cached longer than it should be, serving last week's content after this week's deploy.

The property that distinguishes a good signal

If the cheapest satisfying route is the one that gets taken, the useful question is not "how do I stop it doing that". It is what does the cheapest route cost when the honest one is the only route available.

Signals differ enormously in that.

An assertion inside your own repository is the weakest case. The thing measured and the thing measuring live in the same tree, under the same permissions, and whatever must satisfy the check can also edit the check. Deleting the assertion, weakening the fixture, adding skip — all one-line fixes, none of which stand out in a diff of two hundred lines. Such a signal is only as strong as the attention of whoever reads that diff, and attention is what runs short first.

A signal measured from outside is a different kind of object. If the statement is "a GET to https://example.com/pricing from a machine you do not control returns a 200 with the expected content, a valid certificate chain, and a first byte inside a budget", then there is no edit that makes it true except making it true. You cannot cache your way past a redirect loop. You cannot retry your way past a missing intermediate certificate — the client either builds a path to a trusted root or it does not. No cast, threshold or annotation makes a sitemap's URLs return 200 when the pages behind them have been deleted.

The boundary matters, because the cache example above crosses it. A Cache-Control header does make an external latency measurement faster without the query getting faster, and that is a real limit on this argument. What resists it are the checks whose subject is a fact rather than a budget: what the chain contains, where a redirect terminates, whether a sitemap's URLs return 200, whether the rendered page contains the string it should. No header changes those. Thresholded signals can be moved by configuration; factual ones cannot, and that is which to rely on.

What this asks of you, practically

Three things follow, and none of them require distrusting the agent.

Keep at least one signal outside the blast radius of the change. If every check that reports on your site can be edited by the same session that edited the site, you have one system reporting on itself. It is the same argument as not running monitoring on the servers it monitors, moved one layer up: there the compromise is availability, here it is authorship.

Prefer checks that assert on what a visitor gets. Response status, rendered content, headers on the wire, the certificate actually presented, the redirect chain a browser follows. These are facts about the deployed system rather than the repository, and they stay true statements regardless of who wrote the code or how carefully anyone read it.

Look at the deployed thing before the traffic does. None of this has to wait for an incident. Point the same external checks at a staging hostname and read the result before you move DNS. The redirect that loops, the header that vanished, the page that renders a stack trace — all visible to an outside request the moment the thing is reachable.

The honest limit

An external check cannot tell you that the retry was the wrong fix. It never looks at your repository, so a suppressed test and a passing one are the same thing to it. If an agent papers over a race condition, HarpyWatch will not notice, and neither will anything else that observes from outside — right up until the race produces a response a visitor can see, which is exactly the moment it becomes our problem rather than yours.

So this is not a replacement for reading the diff. It is the part of verification that does not depend on anyone having read it — and as the volume of generated change rises, the share of it genuinely reviewed falls.

Prefer the checks whose only cheap route runs through the actual repair. They do not remove the need for someone careful — the piece above is a list of the gaps they leave — but they shrink the set of problems that can be closed without being solved, and that set is where the expensive surprises live.