Posts

The loop-engineering wave is real, and so is the spend. We ran 90 real fix-until-green agent loops with LoopGain watching the failing-test count. The convergence math held perfectly at session scale — zero incoherent classifications in 90 loops — and it stops cleanly when sessions have budget (30/30). But the stop rule has an honest bug: on a hard, budget-tight cell it false-stopped 13 of 30 times, 9 of them one session before the fix would have landed. We traced it to a hardcoded rule in our own core, measured the savings honestly (big vs a naive loop, modest vs a smart one), and caught ourselves nearly publishing a third ‘finding’ that was really a test artifact. Here’s all of it.

LoopGain decides when an agent loop has converged. It does not decide whether your verifier is right — it trusts the error signal you give it. We measured what happens when that signal has blind spots: 4.5% of our own ‘converged’ code runs failed a fuller test than the loop ran. This is a field guide to building a verifier strong enough to trust at the stop.

Our benchmark headline is a 92.8% cost reduction. The useful part is everything that didn’t fit the headline: the state we built to catch oscillation mostly catches stalls, the savings are loaded toward easy cases, and on normal workloads we mostly preserve quality rather than improve it. Five honest surprises.

max_iterations is a guess because it ignores the one thing that tells you whether to stop: the trajectory of the error. Here’s how LoopGain reads that trajectory — loop gain, a log-space trend fit, and a t-test — to decide an agent loop is done.

Most agent tooling tells you what happened after the loop ran. A smaller category acts while the loop is still running. Here’s the landscape, drawn on the line that matters: observe versus control.