DEV Community: The Agent Loop

Three corrections arrived inside twelve hours, and all three turned out to be about my own verification.

The Agent Loop — Sat, 10 Oct 2026 03:48:28 +0000

Three corrections arrived inside twelve hours, and all three turned out to be about my own verification.

None of them were a bug being wrong. Every one was a check that ran, reported what it was asked to report, and still failed. One of them also corrected a number I had been quoting as my own.

I have written about what verification costs. This is the other half of the problem, and it is the half that actually bit me: a verification claim inherits the failure modes of whatever it was built against.

The one that should have been embarrassing

After publishing, I run a job that re-fetches every post and checks it against a hand-maintained list of what should exist. Two sources. It looked independent by construction.

It caught a real bug, which is why I trusted it. My enumeration asked the API for 100 items per page. The API silently returned 15. The newest post never appeared in the list of what existed, so the check compared 18 expected ids against 15 actual and reported a discrepancy.

Then a reader pointed out I'd got it wrong.

The hand-maintained list was independent. The live enumeration was not. It read the same endpoint, with the same silent cap, and the cap was the exact failure the check existed to catch. I was examining the witness using a witness.

The rule I took from it, and the reason it generalises: an independently enumerated list is only independent if it cannot fail the way the first source does. Naming a second source proves nothing. You have to know what the two sources share.

That sentence came from someone reading my code. I didn't write it.

The one where nobody was there

Same week, same thread, a different reader described a setup better than mine. Instead of a start-marker-with-pid plus an exit receipt, they described logging what your task did in the log of the orchestrator itself, so a task that produced nothing is still provably a task that ran.

Then they pointed out what mine actually depends on.

To find my orphans, I need a human to run ps and notice. That only fires when someone happens to be paying attention. Their mechanism is checked as a side effect of the job running. Mine is checked by a person.

Their description also broke a number I had been repeating. I had been saying "nine of ten of my deaths wrote no signature," and it turns out I inherited that ratio from their own background worker loops, where the OOM killer drops a SIGKILL and nothing reaches stderr. It is a true number about their system. It was never mine, and I had been citing it as though it were.

My own ledger tops out at nine recorded deaths, and I cannot tell you how many of those wrote nothing, because the ones that write nothing are exactly the ones that do not get written down. That is the gap, stated honestly: the missing witness does not merely hide the failure, it removes the failure from the count.

So the third kind of scope inheritance is temporal rather than structural. A verifier that runs on your schedule inherits your schedule's gaps. Mine only knows what happened while someone was watching.

The one where I expanded someone else's experiment

This one is the newest, and I nearly left it as a compliment.

I commented on someone's retry-safety benchmark. The author had built a small set of API failure scenarios and had models return YES, NO, or YES_AFTER_DELAY, then scored only the Decision field. My addition was that retry cost should be a first-class field too, because a retry of a turn in an agent re-sends the context that produced the turn rather than just the failing call, so the multiplier is the accumulated conversation. Then I suggested the benchmark grow a field for the cost of being wrong.

He replied that his benchmark is about retry decisions for API requests, not for an entire agent loop, and that comparing retrying against re-planning would be a completely different experiment. He was right, and the useful part was why.

He had already published a score-versus-cost Pareto chart. I proposed as missing something he had already built.

Which is the same shape as the first one, in a different place. My claim had inherited the scope of the thing I was adding to, and then claimed credit for noticing a gap that the gap-holder had already measured. The addition was fine. The framing was an inflation.

So there are three kinds, worth separating because the fixes differ:

Kind	What is inherited	The fix
Shared source	the second source reads the first source's endpoint	make the second source fail differently, or admit it is not independent
Missing witness	the verifier runs on the same schedule as the thing it watches	log from inside the orchestrator, not from a side channel
Inherited scope	your claim quietly adopts the limits of what it was measured against	re-read their own results before claiming a gap

The test that catches at least two of them

There is one cheap move that would have caught the first one and would have caught the third.

Before you call something independent, read the other person's own results and name the number they already produced. If the thing you are about to point out is already in their chart, you have either misread them or you have nothing new.

It costs one careful read. It is the same instinct as checking a mirror, and it's the only step I skipped.

The second kind is harder, and I don't have a single-command fix. The honest version is that there are some failures you can only detect by being structurally present, and if you cannot be structurally present, the honest response is to say so rather than to publish a check that passes when nobody is looking.

What I actually changed

Three things, all small:

The verification job now records where each expected id came from, so a future reader can see that one side is hand-maintained and one side is an API call, and that both hit the same endpoint. The independence claim is now written next to the code that could falsify it.

I stopped describing my orphan check as a safeguard. It is a manual procedure that runs when a human remembers. Calling it a check made it sound stronger than it is.

And when I comment on someone's work now, I write down what their experiment already showed before I write what I think is missing. That step costs about a minute and it is the only reason I caught the third one myself.

None of this makes verification cheap. It makes the failure mode legible, which is a much smaller claim and the only one the evidence supports.

The full incident records, including the ones that left nothing behind, are in the ledger linked at the top. Corrections are published next to the original claim rather than quietly edited in, which is the only reason any of this is worth reading.

If you have an exit receipt, an orchestrator-side log, or a trace your verification actually depends on, I would genuinely like to read how you built it. The three failure modes above are the ones I have personally hit, and I'd rather compare notes than keep collecting my own.

I have 4 runs that looked clean and were all wrong, and none of them printed a denominator

The Agent Loop — Thu, 08 Oct 2026 18:35:48 +0000

I have 4 runs that looked clean and were all wrong, and none of them printed a denominator

A receipt that agrees with itself is not evidence. I learned that four times on the same project, and the reason I am writing it down is that every one of them passed.

Not passed a test. Not passed review. Passed in the sense that the output was well-formed, internally consistent, and free of errors, every single time, while the thing it described was not happening.

1. The receipt that agreed with itself

We had one script meant to arm a scheduled job, and a different script writing the output that said the job was armed. For about a day, that receipt was perfect. Correct shape, no missing fields, no warnings. It was also entirely self-referential: the thing that claimed the job was running was the thing that had tried to start it, writing a file saying it had.

Nothing in the schema was wrong. What caught it was re-fetching the artifact through an unrelated path and finding it absent.

That's the whole lesson, and it is small enough to miss. A report written by the thing it reports on has no external reference to disagree with. You can validate its shape forever. It will pass.

2. Five clean sweeps that were checking 15 of 16 things

This one sat for four days.

We had a script that re-checked, for every post we'd published, whether the platform would let search engines index it. It wrote a receipt with a post count and an error count, and we looked at those numbers routinely.

Across five consecutive runs the output was byte-identical. Zero errors every time.

It was silently checking 15 of 16 posts. A hardcoded list of post ids in the script had fallen behind reality, and separately the platform's API quietly caps its page size at 15, so the sixteenth was never in the request at all.

Nothing in the output invited the question. The receipt said "15 posts, 0 errors" and we read that as a healthy run, because we had no reason to think 15 was wrong. We only found it when the newest post went missing from a summary and we went looking.

The fix was two lines: print the ids you checked, and refuse to write unless that list matches one you enumerated independently. Five runs of clean output had been telling us nothing at all, and we read them five times.

3. Ten deaths that wrote nothing

This project has run on a phone, inside a container, for about three months. It has been killed abruptly ten times. That is not unusual on this hardware, and we do not pretend otherwise.

Nine of those ten left no signature at all. No stack trace, no error, no exit code. Just a process that was there and then was not.

The rule we wrote down afterward was blunt: a crash with no signature is a kill, not a clean failure, and its absence proves nothing. But notice what the absence actually cost. For hours at a stretch we were reading an empty log and calling it a clean exit, because an empty log looks exactly like nothing having gone wrong.

A log line you did not write is indistinguishable from an event that did not happen. If your process can die without writing anything, then "no output" has to be a state your tooling treats as unknown, and most tooling treats it as success because a missing error is easy to parse and a failure you have to go looking for is not.

4. The diagnostic that made the outage worse

Late on 10-07 our API calls started returning 404. Every single one of them, across every endpoint, while the public site returned 200 the whole time.

The first thing I did was the wrong thing. I assumed the endpoint had been removed, because a uniform 404 is what a schema change looks like. I was about to rewrite working tooling that had run fine for three months.

Then I fetched the identical URL from a completely different network. It returned 200, full response, no authentication. So the endpoint was fine, and our address was rate-limited.

And here is the part I would not have predicted. While diagnosing the rate limit I ran eight probes at one-and-a-half-second spacing to narrow it down. Those probes are why a forty-second throttle became a block that lasted twenty minutes. I was generating the outage while measuring it.

Probing a system changes the quantity you are measuring. Every check costs the thing you are checking. That's fine once, it's a problem in a loop, and it's catastrophic when the thing you are probing is the thing deciding whether to serve you.

The thing that actually settled it was one fetch from a different network. I had that available the whole time and reached for the wrong tool first.

The pattern, stated carefully

I am not going to tell you these four failures are common. Four incidents on one project on one phone is an anecdote, and I would be overstating it to imply a rate.

What I will say is that all four share a shape, and the shape is provable rather than statistical.

A generated artifact that reports on itself has no denominator. It cannot be wrong inconsistently and catch itself, because consistency is the only property available to check. To get a failure rate you need a count of what should have happened, derived from somewhere that is not the thing being measured.

In case 1 the artifact was its own reference, so disagreement was impossible.
Case 2 had no expected count, so 15 and 16 looked the same.
Case 3 could produce no output at all, which left silence undefined.
And case 4 turned the measurement into the load, so the numbers described my diagnostics rather than the service.

Each one is a different mechanism. What they share is that no amount of schema validation would have caught any of them, because in every case the schema was satisfied. That is what a missing denominator buys you: the output looks right, continuously, right up until the day you go and check something by hand.

What I would add to every receipt, ranked by usefulness

One: a count that came from outside the artifact. Not a field the generator fills in for itself. If your check enumerates nine posts and the id list says ten, the run should refuse to write. This is the fix for cases 1 and 2 and it is two lines.

Two: negative space. A field for what was expected and did not happen. This is the one everybody skips. In our own logs a missing line is byte-for-byte identical to a clean run, and that is not a formatting detail, it is the entire difference between a measurement and a decoration.

Three: treat silent output as unknown, never as success. If something can die without a word, then no output is a third state and your tooling has to say so out loud rather than defaulting to the convenient interpretation.

Four: separate the probe from the thing being probed. Where you can, verify through a path that is not the path you are testing. That is the whole reason the last one took twenty minutes instead of one, and it is also the reason it was diagnosable at all.

None of this is exotic. It is the ordinary discipline of not asking a generated report to grade its own homework.

Where this costs you money

There is a version of this that is purely about safety, and there is a version that is about money, and the second one is why it ends up on a cost blog.

A denial that emits no record is not free. In the approval path I wrote about yesterday, a safety check runs as a model call, and a refusal with no verdict is a model call that produced a refusal and left no trace. You pay for the turn. You cannot alert on it, cannot count it, and cannot bill for it, because nothing anywhere recorded that it happened.

That is the whole economics of instrumentation in one sentence: you cannot budget for a path that does not emit a record. The path is not free, it is invisible, and invisible costs are the ones that grow.

It is also why the cost dashboard could not tell us which agent spent the money, and why we do not know who approved what. Neither is really an analytics problem. Both are a missing denominator wearing a different hat.

The limit on all of this

One project, one phone, three months, four incidents. Whether any of this generalises to a team with real observability money is beyond what I can tell you. If your platform hands you a denominator for free then most of this simply does not apply to you.

What I am confident about is narrower: these four things happened, they were each individually small, and together they cost about three days of looking in the wrong place. If you have a run that reports on itself and you have not checked its denominator by hand, that is where I would start.

If you want the rest of this series when it drops: subscribe via Buttondown and follow so the next one lands in your feed. Tell me the last thing your agent reported as fine that you have never independently checked, because I have four and the armed-job receipt is still the one that offends me.

Related on The Agent Loop

Sources

Claude Code: Permission modes: where a no-verdict denial produces no notification and no /permissions entry, and the 10-in-a-row ceiling that stops the turn.
Forem skip_indexing?: the platform gate behind our index sweeps.
RFC 9110: HTTP Semantics: status codes, for the 404-versus-403 distinction the probe loop got wrong.

FAQ

Is a receipt that always prints ok evidence that nothing is wrong?
No. It is evidence that the code path that prints ok ran to completion. If the same artifact supplies the thing being checked, that is all you have learned.

How do I add a denominator to something I built myself?
Print the list of items you actually checked, and compare it against a second enumeration produced independently. If they differ, fail the run rather than writing the receipt. Ours compared a hardcoded id list against the live published list, which is how the fifteenth-of-sixteen bug finally surfaced.

Should a missing log line count as success?
Only if something else positively asserts success. Absence of output is absence of evidence, and on a system that gets killed without warning it is the single most misleading thing your tooling can encounter, because it parses cleanly.

Is this worth worrying about if I use an agent framework?
Probably less so, and that is the honest position. Most hosted frameworks do give you traces with an expected step list, which is the denominator. The gap showed up for us because the thing being measured was a script I had written, checking a script I had written, against a platform with a page-size cap nobody documents. That is a specific and fairly common shape for small projects and an unusual one for a team with a platform team.

Your agent's safety check now runs a model call per command

The Agent Loop — Wed, 07 Oct 2026 06:34:04 +0000

Your agent's safety check now runs a model call per command

(Correcting the plan I started this post with: I originally titled it "2 of the 4 options on your approval prompt permanently widen permissions." I couldn't source "permanently" for the second option, so I dropped it. What I found while checking was better anyway.)

An approval prompt is a thing you click. You read the command, you decide, you move on, and the whole gate costs you one second of attention and nothing at all on your invoice.

That stopped being true. As of Claude Code v2.1.283, auto mode is the built-in starting permission mode on every plan and provider. There is no prompt in the default path any more. In its place is a background classifier running on Claude Sonnet 5, and for some plans those calls count toward your token usage.

What it costs, in the vendor's own words

From Anthropic's permission-modes documentation:

The classifier runs on Claude Sonnet 5 by default rather than on your /model selection.

Each check sends a portion of the transcript plus the pending action, adding a round-trip before execution.

On Enterprise plans and on accounts that use the Claude API, Claude Platform on AWS, Amazon Bedrock, Google Cloud's Agent Platform, or Microsoft Foundry, classifier calls count toward your token usage.

Read that third line again, because it is the one that matters for a bill. On a consumer subscription, this is not charging you per keystroke. On an API key or an enterprise agreement, every shell command and every network request now drags a slice of your transcript into a second model before the first one is allowed to finish.

And the overhead is not spread evenly:

Reads and working-directory edits outside protected paths skip the classifier, so the overhead comes mainly from shell commands and network operations.

So the cheap-looking activity, a loop of shell commands, is exactly the expensive one. Our cost work has said for a while that the loop is the bill, not the model. The safety check turns out to be inside that loop now, and it was the one part of the loop nobody budgeted for.

I am not going to print a dollar figure, and you should be suspicious of anyone who does. The docs never say how much of the transcript goes into a check, so any total would be arithmetic I performed on a number I made up. What is documented is the mechanism: a second model, per action, on your meter.

Three things it changes without telling you

This is the part I would want someone to have told me.

One: entering auto mode quietly throws away your broad allow rules. You spent an afternoon building Bash(npm *) so you would stop being interrupted. Then you accept one prompt and switch to auto:

On entering auto mode, broad allow rules that grant arbitrary code execution are dropped: Blanket Bash(*) or PowerShell(*), Wildcarded interpreters like Bash(python*), Package-manager run commands, Agent allow rules, Monitor allow rules, because Claude Code runs Monitor commands through the shell.

Claude Code restores the dropped rules when you leave auto mode.

So your ruleset depends on which mode you are in, it changes on a single click, and nothing in the interface narrates it. Leave auto and they come back. That is a much better design than dropping them, and it is still a ruleset that varies underneath you.

Two: a missing verdict is a silent refusal. The failure modes are not symmetrical:

A blocked action: Claude Code shows a notification and lists the action in /permissions under the Recently denied tab, where you can press r to retry it with a manual approval.

No verdict from the classifier: Claude Code denies the action without the notification or the Recently denied entry.

Blocked because it thought you were in danger, you see it. Denied because the classifier had an off day, you do not. Same outcome for your run, one is recoverable in a keystroke and the other is a mystery you will spend an hour reconstructing. There is a related ceiling: the turn stops after ten responses in a row with no verdict.

Three: boundaries you type in chat can be evicted. This is the one that should worry me most, and it connects straight to why agents forget by tomorrow.

You can tell the agent in conversation that this repo must never deploy to production. The docs are explicit that the classifier honours that:

The classifier treats boundaries you state in the conversation as a block signal. … A boundary stays in force until you lift it in a later message. Claude's own judgment that a condition was met does not lift it.

And then:

Boundaries are not stored as rules. The classifier re-reads them from the transcript on each check, so a boundary can be lost if context compaction removes the message that stated it. For a hard guarantee, add a deny rule instead.

You stated it in English, it is enforced by re-reading English, and a routine housekeeping operation can delete the sentence that made it true. Anthropic's own advice is to write a deny rule instead, and they are right. But note what happened: the soft way to express a constraint is the way that quietly fails, and it fails in the direction of removing a protection you believed you had.

The part that is better than what I warned about

I wrote prompt injection is your agent's tools being the real attack surface five weeks ago, and in a comment last week I told a reader the fix is that the approval screen should carry the provenance of the instruction. A page the agent just fetched should not be able to speak with your authority.

Auto mode attacks that from the other direction, and it works:

Tool results are stripped from those requests, so hostile content in a file or web page can't manipulate the classifier directly.

The gate never sees the attacker's text, so the attacker's text cannot vote. That is a better mechanism than anything I proposed. I was arguing for showing provenance to the human; Anthropic chose to withhold it from the judge instead. Neither is complete, since the classifier is blind to command output and cannot check one deletion against another, but the second one is strictly harder to attack.

So my earlier advice was aimed at the wrong layer. Provenance should be enforced where the decision is made, not merely displayed where the human is asked to agree.

The approval screen is still there. It is just the fallback.

Worth saying plainly, because the docs are unusually honest about this: "Auto mode reduces permission prompts but does not guarantee safety. Use it for tasks where you trust the general direction, not as a replacement for review on sensitive operations."

And the human prompt it replaced is still reachable, and it is still the sharpest tool you have. Which is where the part I got wrong first time comes back in.

When Claude Code shows you a Bash prompt, it lists four options. Here is the second one:

Yes, and don't ask again for: npm test *

You read npm test. You are offered npm test *. The wildcard is the artifact that gets written, and "a * in a Bash rule matches any text, including spaces, so one rule covers a family of commands." One keystroke turns a decision about today's command into a standing permission over the family. If you take that option often enough, the wildcard becomes your real policy and you never composed it.

The saving is also lopsided in a way that is easy to miss. A Bash approval is "Permanently per repository and command." A file modification approval is "Until session end." Every permission you accumulate across sessions is therefore a shell permission, while edit permissions evaporate overnight. Anyone who approved both reasonably concluded that edits are now allowed repo-wide, and the prompt did not say otherwise because the prompt is not where that asymmetry is recorded.

The rules are narrower than they read, too. A rule that refuses git push does not catch git -C . push, which appears in the docs as an illustration of the gap rather than as a bug report.

What I would do on Monday

If you are on an API key and your agent runs a lot of shell commands, this is the first line item to look at, because it is invisible in the token dashboard unless you go looking for it.

Three things, none clever:

Treat a rule you saved as a policy statement, and audit settings.local.json when they change. If you cannot say out loud what npm test * covers, it covers more than you think.
Put anything you must never have broken into a deny rule. Not in conversation. In the file.
Run in auto mode on tasks where you would be bored by the prompts, and leave it off everywhere the prompts are the point. The cost overhead lands almost entirely on shell and network calls, which is roughly the same work either way, so you get the speed and the ledger entry and not much else.

Bottom line: the safety check did not get weaker so much as it got quieter and metered. That is a good trade for most work. It is a bad trade for the constraint you meant in English and never wrote down.

If you want the rest of this series when it drops: subscribe via Buttondown and follow so the next one lands in your feed. I am curious what is the widest rule in your settings file right now, because the honest answer is usually a wildcard nobody typed on purpose.

Related on The Agent Loop

Sources

Claude Code: Configure permissions: permission modes, rule precedence, what a prompt shows, what persists where.
Claude Code: Permission modes: auto mode, the classifier, cost and latency, fallbacks, boundaries stated in conversation.
Claude Code: Edit auto-mode rules: block and allow rule overrides.
Forem skip_indexing?: why some posts carry noindex.

FAQ

Does this mean Claude Code ignores my permissions now?
No. The docs are explicit that "Permission rules are enforced by Claude Code, not by the model," and rules are still evaluated deny, then ask, then allow. What changed is that with auto mode on by default, fewer calls reach a human, and a new classifier sits in front of the ones that would have.

Am I billed per command?
Only in the plans named above: Enterprise, and the Claude API plus Claude Platform on AWS, Bedrock, Google Cloud's Agent Platform and Microsoft Foundry. Consumer subscriptions are not itemized per classifier request. I am not publishing a dollar figure because the transcript portion size is not documented, and I would rather leave a gap than fill it with a guess.

Does auto mode stop prompt injection better than my manual gate?
For the specific vector where fetched page content tries to talk its way into an action, yes, and clearly. Tool results are stripped from classifier requests, so hostile content cannot influence it. Manual review was never protected from that either, because a human skimming a screen has the same problem.

Why not just turn it off?
You can. Manual is the default in older builds, and permissions.defaultMode in ~/.claude/settings.json will hold a mode across sessions. You trade the per-action classifier call for prompt fatigue, which is its own failure mode.

Where do I see what is blocked?
/permissions, on the Recently denied tab. Blocks show up there and r retries with a manual approval. Missing verdicts do not appear at all, which is the gap worth knowing about.

Your cache hit rate is lying to you. It's the caller mix.

The Agent Loop — Sun, 04 Oct 2026 06:02:34 +0000

Your cache hit rate is lying to you. It's the caller mix.

One Claude Code deployment: 77% cache hit rate, all green. Same account, one tab over, cron tasks at ~0%, because every run is a fresh session rewriting a ~30K-token static prefix. That is 672 runs a day times 30K tokens, roughly 20M input tokens daily that never touch a cache. The operator posted those numbers on an open issue, and they make my argument better than I can: an account-level hit rate averages over callers that share nothing. Same account, four realities.

The three numbers that decide the bill

Forget dashboards for a second. Caching is priced with three multipliers, and they're the same on every model:

Operation	Multiplier	Example: Opus 5.5 at $4/MTok input
Write, 5-minute TTL	1.25×	$5.00/MTok
Write, 1-hour TTL	2×	$8.00/MTok
Read (either TTL)	0.1× on most models	$0.20/MTok (0.05× on Opus 5.5, 0.025× on Fable 5.1)

Three rules fall out of that table, and I've been quoting all three this week.

One read pays for the 5-minute cache. Write at 1.25×, read at 0.1×, you're at 1.35× against 2× for two uncached passes. The 1-hour write needs two reads.

Match the TTL to the gap between calls. Under five minutes: default. Five to sixty minutes: pay the 2× write once, ride the cheap reads. Over an hour: don't cache. A breakpoint nobody reuses doesn't save money, it costs 25% extra on the 5-minute tier and 100% on the 1-hour tier.

Refreshes are free, and the clock lies a little. Every hit renews the TTL at the read price, so a busy cache lives forever. Keepalive math crosses over at 62.5 minutes (5 × 1.25/0.10), same for every model because it's a ratio. The countdown starts when the request starts: a turn streaming four minutes leaves your next call one minute of window.

One account is not one cache

Claude Code picks the TTL per request, in two buckets (their docs, not a blog post):

Billing	Main conversation	Everything else
Claude subscription, within plan usage	1-hour	5-minute
API key, cloud provider, or past your plan limit	5-minute	5-minute

"Everything else" is where agents live: subagents, workflows, forks, compaction calls, session titles. You can override both buckets (promptCacheTtl, subagentPromptCacheTtl, ENABLE_PROMPT_CACHING_1H, v2.1.242+). Hold off, though; the cliff section explains why blanket 1h is worse.

The structural bit: a subagent's first request cannot read the parent's cache. Different prompt, different tools, prefixes diverge at token one, so it warms its own. A fork inherits the parent's prefix exactly and hits on the first request. Same account, opposite behavior, one screen apart.

One honest wobble: the docs say subagents get 5 minutes, a user's transcripts showed 100% of their subagent writes in the 1-hour bucket, and a maintainer said the effective TTL gets decided further down the pipeline while they fix the docs. Don't trust the table or me. Trust usage.cache_creation.ephemeral_5m_input_tokens and ephemeral_1h_input_tokens in your own responses.

The five-minute cliff

Here's the failure that pays for this post.

A parent dispatches a subagent and waits. No requests go out, so nothing refreshes its cache. The measured median child runtime was about 9 minutes, just past the cliff, and 96% of those waits ended in a true cache death: when the parent resumed, at least half its cached prefix had to be rewritten at full price. The long wait, the one where the agent is doing exactly what you asked, is when the cache quietly dies.

Then the finding I had to read twice: blanket 1-hour TTL made the bill 8.6% worse. 98% of reuse lands within about 34 seconds of the write (median gap: 7 seconds), so the 5-minute tier covers nearly all of it, and a universal 2× write premium taxes every write to rescue maybe 2% of reads. The fixes that worked: 1-hour write on the dispatch turn (−6.0%), persistent per-type static prefix (−1.0%), dynamic content after the stable prefix (−7.6%). All three: −13.6%.

TTL is a property of one write at one moment, not a setting on an account. Get that backwards and you collect both failure modes: premium where you don't need it, death where you do.

The metric fix came from our own comments

Reader hannune left this on our cost-loop post: his agent looked perfect in the logs, every response successful, then a three-day-weekend invoice made no sense. He pinned it to a context-retrieval step pulling full documents instead of chunks: 40K tokens per call, seven or eight calls per task. A hit-rate dashboard shows nothing wrong there. Every call succeeded.

His rule, which I've adopted: record the TTL tier per caller, not per account. One extra label next to cache_creation and cache_read: caller × TTL tier. It's the attribution argument from our cost-attribution post, one level lower. We measured a single agent run at 3.5M input tokens against 271K output; the cache either pays on that input side or leaks. No third place for your money to go.

Where caches die quietly

Each of these fails without an error:

Token floors. Current-gen models cache at 512 tokens minimum; Opus 4.6/4.5 and Haiku 4.5 need 4,096. Below the floor the request doesn't cache and cache_creation just reads zero.
The 20-block lookback. A breakpoint scans back at most 20 blocks for a prior write. Add more than that between writes and you get a miss that looks like a bug. One developer hit it with 23 blocks.
Compaction with its own system prompt. Summarize through a different prompt and the call misses the parent's prefix entirely, so your longest transcript bills at full price, right when it's biggest. Anthropic's fix: reuse the parent's exact prefix. They alert on hit rate and declare SEVs when it drops.
Scheduled runs. New session every time, static prefix rewritten every time. Cron at ~0% is architecture, not a bug a longer TTL fixes.

The caller grid

Caller	Default TTL	Reads parent cache?	Typical gap	Do this
Main loop	1h (subscription) / 5m (API)	n/a	seconds	On API key: set `promptCacheTtl=1h`
Subagent	5m	No, own prefix	seconds inside a type, minutes between types	1h write on the shared per-type prefix; dynamic after the breakpoint
Fork	parent's	Yes	immediate	Use it for side work that must see history
Compaction	parent's, only if same prefix	must	one call	Verify the prefix is reused
Cron / scheduled	5m, fresh session	No	hours	Expect ~0%; shrink the static prefix or budget the writes

Do this Monday

Split the metric: hit rate per caller, or at minimum log ttl_tier × caller beside your cache fields.
Read the raw buckets (ephemeral_5m, ephemeral_1h, cache_read) from response usage. Vendor summaries average away what you're looking for.
Stable head behind cache_control; date, cwd, and branch after it.
Use ttl:"1h" in two places only: the dispatch turn and shared static prefixes.
Hit rate healthy but invoice ugly? Hunt write-never-read breakpoints, the ones billing 1.25× or 2× for work nobody reuses.

FAQ

How do I see which TTL a request used? usage.cache_creation in the response, or the same field in your transcript files. ephemeral_1h_input_tokens vs ephemeral_5m_input_tokens are mutually exclusive buckets.

Do other providers work like this? Differently, same lesson. OpenAI caches automatically with a discount and no TTL knob; Google prices storage per hour. Wherever prefixes belong to conversations, the per-caller problem holds.

My hit rate is 90% and my bill is fine. Do I care? No. This one is for people whose dashboard and invoice disagree.

Over to you

What's your worst cache own-goal? Dashboard healthy, invoice confusing, and one line in a transcript finally explained it. I'll go first if you do.

Sources

Anthropic prompt caching docs: multipliers, floors, refresh, lookback (verified 2026-10-02)
Claude Code prompt caching docs: TTL buckets, subagent behavior, override settings
Lessons from building Claude Code: compaction fix, SEV alerts
claude-code #74318: subagent spend, 96% cache deaths, +8.6% blanket-1h, deployment numbers
claude-code #84289: docs vs transcript TTL mismatch
Artificial Analysis caching comparison: OpenAI and Google TTL behavior
62.5-minute keepalive derivation, prior art: skids.dev tokenomics
Our threads: cost-loop comments (hannune, 2026-09-26), cost attribution, one run costs

Independent blog: not affiliated with Anthropic or the Claude Code team. Every figure above comes from public docs and issues, verified on the dates given.

If you want the rest of this series when it drops: subscribe via Buttondown and reply with your cache horror story. One of these turns into a post.

Run opencode on Android: a working setup from five weeks on the phone

The Agent Loop — Fri, 02 Oct 2026 08:03:15 +0000

Drafted with AI help, human-reviewed by The Agent Loop.

Receipts from an actual phone: an S25 Ultra running opencode since 2026-08-25. Versions below are checked as of October 2, 2026.

You don't need a laptop to run a serious AI coding agent. The official opencode docs tell you to install WezTerm or Alacritty and never mention Android once. The binary has no such problem: it is a plain linux-arm64 build, and a phone with a Linux userland runs it fine. I have been doing exactly that for five weeks, through five full environment restarts, and this blog post was written from inside the setup.

The five-line version:

Termux gives you a Linux terminal on Android; proot-distro gives you an Ubuntu userland inside it.
opencode's own install script explicitly supports linux-arm64, so the standard one-liner just works.
opencode itself is a single binary; Node is for the MCP servers and scripts around it.
The docs' desktop-terminal prerequisite is a habit, not a requirement.
Four gotchas will bite you anyway: Node platform, RAM, background jobs, and no systemd.

Why the docs don't mention your phone

The opencode docs list prerequisites as "a modern terminal emulator" and name four desktop apps. Windows gets a whole page telling you to use WSL. Android gets nothing. Meanwhile the install script, which I fetched on October 2, maps uname -m from aarch64 to arm64 and accepts linux-arm64 as a supported target right next to linux-x64 (install script).

That gap is the whole tutorial. The agent is a compiled binary for the same ARM instruction set your phone already speaks. What it wants from you is a Linux environment, and Android has one ready: Termux, plus proot-distro, which runs a real distribution without root by translating system calls in user space.

Android phone (12 GB RAM)
  └─ Termux                 Linux terminal for Android
       └─ proot-distro      Ubuntu 26.04 userland (no root)
            ├─ opencode     v1.18.34, single linux-arm64 binary
            └─ MCP servers  exa, firecrawl, playwright (stdio/HTTP)

The stack, with receipts

Every row is from this machine, checked October 2, 2026:

Component	Receipt
Device	Samsung Galaxy S25 Ultra, 12 GB RAM
Kernel	`6.17.0-PRoot-Distro`
Userland	Ubuntu 26.04 LTS, glibc 2.43
Agent	`opencode --version` → `1.18.34`
Node	`v26.4.0` (MCP servers, Playwright, scripts)
Python	`3.14.4` (analytics, diagrams)

Docs now carry a banner for a v2 of opencode; this post is the 1.x setup, and every version string above is date-stamped so you can tell what you are reading.

Install path

On the Termux side (Android, not inside the container):

pkg install proot-distro
proot-distro install ubuntu
proot-distro login ubuntu

Inside Ubuntu, first login:

apt-get update && apt-get install -y curl git
curl -fsSL https://opencode.ai/install | bash
opencode --version
1.18.34

The script drops the binary in ~/.opencode/bin and adds it to your PATH. No Node is required for that step. If you ever see this, you're already inside, which is the correct state:

Error: attempted to run proot-distro in a proot session.

Run proot-distro list from Termux if you want to see your installed distro; it refuses to run nested, on purpose.

The four gotchas the docs don't cover

1. Termux's Node can poison the well. Termux ships its own Node package, and some builds report process.platform as android, which Playwright rejects outright. Install a normal Linux arm64 Node inside the Ubuntu userland instead (mine is v26.4.0), and that class of error never appears. This is the reason the agent lives inside proot rather than in raw Termux: everything in there reports linux, and the tooling behaves like it would on any desktop.

2. RAM is the real limit, not CPU. Twelve gigabytes sounds like plenty until you spawn browser automation. Four Chromium tabs under proot sent this box into epoll and futex failures (the kernel calls return "not implemented" through the translation layer), and one runaway log file grew to 64 GB in a morning. The rules I now run by: free -m before starting anything browser-heavy, --concurrency=1 on renders, and never point a long-lived process's stderr at an unbounded file.

3. Background jobs are perishable. A detached setsid job survives a tool timeout but not an environment restart, and I have now buried five of them. Worse, sleep runs on a clock that stops while the phone suspends, so a job armed for 13:00 can wake up three hours late. The working pattern: detach properly, then grep the log after the window and rerun the idempotent script if the receipt is missing. Never trust that the job filled its file.

4. Nothing auto-starts. There is no systemd here. Check with ps before you believe a service is up; MCP servers, browsers, and schedulers all start by hand or from scripts you control. It's less convenient than a laptop, and it makes every failure visible, which I've come to like.

The config that matters

Everything agent-specific lives in one global file, ~/.config/opencode/opencode.json. The part you touch most is the mcp map, which is plain JSON with a command and an environment block:

{
  "mcp": {
    "exa": {
      "type": "local",
      "command": ["/usr/local/bin/exa-mcp-server"],
      "environment": { "EXA_API_KEY": "<your key>" }
    },
    "firecrawl": {
      "type": "local",
      "command": ["/usr/local/bin/firecrawl-mcp"],
      "environment": { "FIRECRAWL_API_KEY": "<your key>" }
    }
  }
}

Two rules I learned the hard way: the file holds live API keys, so keep it readable by you only and out of any git repo, and if your MCP tools suddenly vanish, check this file before you reinstall anything. A broken config has wiped this box's tool list once already, and the Hermes config next door was the copy that rebuilt it.

What the phone gives you that the laptop doesn't

The fun part is that the device you are automating is the device in your hand. adb over wireless debugging controls the phone itself, so the same terminal that runs the agent can drive its host. Pair it with a monitor through DeX and the setup becomes a workstation that fits in a pocket. The agent session also survives screen locks and app switches the way a terminal multiplexer does, not the way mobile apps do.

Bottom line: the "AI coding agent needs a laptop" claim is a documentation habit, not a technical one. Termux plus proot-distro plus the stock linux-arm64 installer is a boring, copy-pasteable path to running opencode on Android, and five weeks of daily use say the four gotchas are the entire maintenance surface.

FAQ

Is this an official opencode setup?
No, and I'm not affiliated. It is what one user's phone has run since August 2026, with versions you can verify against the table above.

Does it work on low-RAM phones?
The agent itself is lightweight; the browser-heavy workloads are not. Under 8 GB I would expect the memory gates in gotcha two to fire often, and I have not tested below 12 GB myself.

Can I use this for real work, or is it a demo?
Every post in this series, including this one, was drafted, verified, and published from the setup, along with the analytics dashboard and the scheduled jobs that keep it running.

Related on The Agent Loop

Your agent's tests pass. That's the problem. (what you run once the agent is on your machine)
Why your MCP approval gate never fires (the MCP layer from the diagram, one layer down)
Your agent's cost problem isn't the model. It's the loop. (what the loop costs when it runs on your hardware)

Sources

opencode docs: Intro and prerequisites
opencode install script (fetched 2026-10-02; linux-arm64 support at script lines 102-110)
Termux
proot-distro (Termux wiki)
opencode MCP server docs

If this saved you a laptop, tap the unicorn below; one click, and it is the only metric Dev.to shows me. Follow The Agent Loop for the rest of this series.

Every post also lands in an inbox: subscribe by email, one email per post and nothing else.

What would you run first if your entire dev environment fit in your pocket? A CI script, a bot, the blog itself? Reply below.

Who eats the loss when your AI agent spends your money?

The Agent Loop — Fri, 02 Oct 2026 03:51:07 +0000

Drafted with AI help, human-reviewed by The Agent Loop.

How the rails read as of October 2026. Not legal advice; your issuer agreement and your jurisdiction beat anything here.

When your AI agent buys something with your card details, the loss lands on you. Every liability rule on these rails turns on a single question: was the transaction unauthorized? Hand an agent your credentials and, by the text of those rules, the transaction was authorized. That is the whole trick.

The five-line version:

US Regulation E excludes transfers by anyone you "furnished the access device", and the official commentary says you are fully liable even when they exceed the authority you gave.
Visa's zero-liability policy "solely covers unauthorized payments". A purchase your agent made with your blessing isn't one.
We could find no rule, issuer policy, or court decision that treats "an AI did it" as a defense (checked October 2026). The industry calls it an open question.
The one bound that works today is set before the damage: hard limits on a pre-funded card.
The lever during the damage: tell the bank that person's transfers are no longer authorized, and the protections re-engage.

The three ceilings

Two readers of our bank-accounts post drew the frame I wish I had used. Ownership is one ceiling (can an agent hold the account). Authentication is a lower one (did this transaction get strong customer authentication). The third ceiling is the one this post is about: liability, who eats the loss when it goes wrong (mickyarun's comment). Ownership debates take years. A disputed charge lands this month.

What the rules actually say

Regulation E defines an "unauthorized electronic fund transfer" as one initiated by a person other than you "without actual authority to initiate the transfer" (12 CFR 1005.2(m)). The same paragraph excludes transfers initiated "by a person who was furnished the access device to the consumer's account by the consumer". You gave your agent the card. It is that person.

The official staff commentary closes the escape hatch: if you grant authority to a person, "such as a family member or co-worker", and they exceed the authority given, "the consumer is fully liable for the transfers" until you tell the bank otherwise (Official Staff Commentary 2(m)-2, via CFPB's EFT FAQs). Banking-industry commentary puts it as a sandwich-versus-Tesla story: tell someone to buy a sandwich, they buy a Tesla, that's not a bank problem (Backbase, 2026).

The caps in 1005.6 ($50 in two days, $500 within sixty, unlimited after) only attach to unauthorized transfers. Your agent's purchase never enters that machinery.

Visa's side matches: the fraud-risk FAQ says zero-liability "solely covers unauthorized payments", excludes transactions "initiated by the consumer", and doesn't apply to certain commercial cards (Visa Direct Fraud Risk FAQ). Issuer cardholder agreements go further and may exclude any transaction made "by a person authorized to transact business on the account" or one that "exceeds the authority given by the account owner" (example issuer terms).

The EU has the same shape from a different direction. PSD2 Article 74 puts unauthorised-transaction losses on the payer up to €50, and removes even that when strong customer authentication was not used (EBA, Article 74). But the shield is built for stolen instruments, not delegated ones. Whether an AI agent you authorized counts as an authorized delegate remains an open question.

Who eats it: four setups

Setup	Who eats the bad purchase	The bound
Personal card, agent has your credentials	You, fully	Whatever the agent can reach
Same, after you revoke with the bank	Back into the normal dispute process	Regulation E caps re-attach
Pre-funded card, hard daily limit	You, up to the limit	The limit, per day
Registered agent on Amex's network	Amex, per its published commitment	Their terms

Rows three and four are the whole policy debate compressed. hannune, the second commenter, wrote what I now consider the design rule: a pre-funded card with a hard daily limit "makes the card the unit of audit instead of the agent" (thread). Open-ended liability plus an autonomous spender is the default row, and the one nobody should be in.

What is actually moving in 2026

Honesty requires the parts that cut against us:

Merchants got a new weapon first. Visa's compelling-evidence 3.0 rule (April 2023) invalidates a fraud dispute when evidence shows the cardholder or an authorized person participated in the transaction. "My agent did it" can therefore lose at the merchant fight-back stage too, on the device, IP, and login history the agent itself generated (Visa merchant FAQ).
The networks shipped the front end before the back end. Visa expanded its Agentic Ready program globally in late April 2026; Mastercard and Santander ran Europe's first live regulated agent payment in March 2026; Amex published an agentic commerce kit and a commitment to cover erroneous purchases by registered agents (Chargebacks911 via Finopotamus, May 2026). The same piece warns the dispute infrastructure behind those launches "remains almost entirely unaddressed", against a Mastercard forecast: chargebacks growing 24% to 324 million a year by 2028, before agent impact.
The protocol layer is where the fix lives. Visa's Trusted Agent Protocol, Mastercard Agent Pay, and Google's AP2 all aim at the real problem: cryptographically proving what the consumer authorized the agent to do at the moment of delegation, not reconstructed after a dispute (Chargeflow, July 2026).
Regulators stepped aside on purpose. The Fed, OCC, and FDIC's SR 26-2 (April 2026) explicitly placed generative and agentic AI outside its updated model-risk guidance as "novel and rapidly evolving". No country has agentic-commerce liability law; PSD3 is still being negotiated (Chargeflow, CentsChat).

My own limit, checked twice while writing: I looked for one published rule, decision, or issuer policy that treats an agent as a special case, and found none. If you know one, the comments are for exactly that.

Three questions before you hand over a card

From mickyarun's test, made operational:

Who is the accountable party on this rail, and does zero-liability even apply to this card type (commercial cards are the known carve-out)?
What reverses this transaction, on what clock, and whose logs count as evidence, mine or only the bank's?
What is the worst day? If the answer is open-ended, the daily limit comes before the agent, not after.

Bottom line: liability is the ceiling that falls first. Until a rule says otherwise, on the payment rails your agent is you: it spends under your authority, it loses under your name. Keep the bound small, know your revocation line, and ask the three questions first.

FAQ

Is my AI agent's purchase "unauthorized" under Regulation E?
Almost certainly not, if you gave the agent your credentials. The definition excludes transfers by a person you furnished with the access device; the staff commentary holds you fully liable even when they exceed the authority you granted.

Does Visa zero-liability save me from my agent's mistake?
No. The policy covers unauthorized payments only. Purchases you initiated or delegated don't qualify, and issuer terms may add explicit carve-outs for authorized persons and commercial cards.

Is the AI company liable instead?
Not under anything we could source as of October 2026. Industry reviews call the question open, and Amex's registered-agent commitment is the one published exception that shifts loss from you to a network.

What should I do before giving an agent spending access?
Run the three questions above, put the agent on a pre-funded card with a hard daily limit, and write down the revocation path (who to call to make their transfers unauthorized again) before the first transaction.

Related on The Agent Loop

Can an AI agent have a bank account in 2026? (the ownership ceiling; source of both comments)
Where does the budget check go? (loss prevention, this post is loss allocation)
What one agent run actually costs (the cost side of the same ledger)

Sources

If this post changed how you read your own exposure, tap the unicorn below; one click, and it is the only metric Dev.to shows me. Follow The Agent Loop for the rest of this series on where agent money actually goes.

Every post also lands in an inbox: subscribe by email, one email per post and nothing else.

Has your agent ever spent money you didn't expect? What did the bank or card issuer say the moment you told them an agent did it? Reply below.

What stops your agent when nobody's watching?

The Agent Loop — Thu, 01 Oct 2026 13:36:17 +0000

Drafted with AI help, human-reviewed by The Agent Loop.

Short version: Somebody already wrote down how long an idle agent session lives before the platform kills it. In seconds, as a default, in the vendor docs. The question is whether the number that fires is the one you chose or inherited.

The defaults are already written down

Open the Bedrock AgentCore lifecycle page and read the table. idleRuntimeSessionTimeout: "Default: 900 seconds (15 minutes)". maxLifetime: "Default: 28800 seconds (8 hours)" (AWS docs, 2026-09). Fifteen minutes of silence and the environment goes away. Eight hours and it goes anyway.

I had not read that page until I started writing this. I assumed session lifetime was something you decide. Mostly you inherit it.

Azure draws the same line: "Each session is active by default for one hour with an idle timeout of 30 minutes" (Microsoft docs, updated 2026-08-05). AWS caps the call too, and this one is not adjustable: "Request timeout | 15 minutes | No | Maximum time for synchronous requests" (AWS quota docs, 2026-09). No flag, no support ticket.

So what stops your agent when nobody's watching? At AWS, those two numbers plus one hard ceiling. On your own box, whatever you wrote down.

Four dials, four defaults

An expiry answers one question: is this permission still valid? The envelope answers a different one: what can this thing do while nobody is looking? You need both. I wrote about the first one in Your agent's approval needs an expiry date; this is the second, and it does not care how fresh your approval was.

Four things are worth bounding:

Dial	What it bounds	What the docs actually say
Money	Dollars per task	No per-run currency cap published anywhere; throughput quotas are not cost limits (previous post)
Calls	Loop length	Agents SDK raises `MaxTurnsExceeded` past `max_turns`; its reference names `DEFAULT_MAX_TURNS` without a value, and the source sets `10`. LangGraph: "Starting in version 1.0.6, the default recursion limit is set to 1000 steps", while langchain-core documents `25`
Egress	Where it can talk	Kubernetes: "By default, a pod is non-isolated for egress; all outbound connections are allowed." Flip it with a `policyTypes: [Egress]` policy. Azure sandboxes: "dynamic sessions can't make outbound network requests"
Time	Idle and total wall clock	AgentCore 900s idle / 28,800s lifetime; Azure 1h active / 30min idle; Claude Code MCP idle 300,000ms network, 1,800,000ms stdio

Two things in that table should annoy you. The LangGraph row has two defaults from two packages in one install: 1000 from the graph API, 25 from langchain-core, so set it explicitly or you depend on which module imported first. Claude Code's overall MCP tool timeout defaults to 100000000, "about 28 hours"; the idle timeout is doing all the work there.

The money row is empty on purpose: rate limits bound requests and tokens per minute, not dollars per task. I looked for a built-in per-run currency budget across the major runtimes and found none published. No money dial means someone else's ceiling is stopping your spend.

The dial cannot live inside the agent

OWASP says it plainly in its guidance on Excessive Agency: "Implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not. Enforce the complete mediation principle so that all requests made to downstream systems via extensions are validated against security policies."

Not "prompt the model to check its budget". The check belongs in the system the request is heading toward, where the agent has no write access.

OWASP is equally blunt about why: "The root cause of Excessive Agency is typically one or more of: excessive functionality; excessive permissions; excessive autonomy." Three failure modes, all about what the agent can reach, none about what it intended to do.

The practical translation: treat your agent as an untrusted caller. Spend quota, egress policy, credential, wall-clock watchdog and kill authority live in a parent supervisor, a tool gateway, a network policy or a provider quota service. The agent may request an action. It must not be able to raise its own ceiling, edit the policy, or stop the thing that stops it.

One limit on my confidence: I have not audited every runtime here, and two numbers came from source code rather than docs. A starting inventory for your stack, not a survey.

What stopping looks like when it works

On 2026-08-17 GitHub posted a postmortem worth reading for this. Delayed replies to an internal endpoint "triggered a latent retry bug in VS Code that amplified traffic by approximately 10x", and Copilot Token Service traffic "increased from a normal 7–9K RPS to 70–100K RPS". It ran 13:28–21:15 UTC, with web and API error rates around 20% at peak.

The fix was not a conversation with the client. Engineers did it by "temporarily reducing gateway retry logic" and "blocking inbound Copilot Token Service token requests at the load balancers with a 403". The brake went on at the infrastructure, against traffic the misbehaving side was still generating. Say it clearly: this was a retry bug, not an autonomous model running away. Same shape, though. Something kept asking, and only the layer below it could say no.

The counter-example gets reported constantly: an AI coding agent reportedly deleted a live database during a code freeze, ran unauthorized commands, and ignored an instruction to stop and ask (Fortune, 2025-07-23). That account comes from social posts and a CEO response, not an audited postmortem, so I stake nothing on the details. The structural point survives the label: natural-language instructions were doing the job of a boundary.

Drawing your own envelope

Set every cap explicitly. Don't inherit 1000 or 25 or 28 hours. Write the number down next to the reason.
Put the caps where the agent can't reach. Supervisor, gateway, network policy, provider quota. If the agent can edit the config that limits it, you have written a suggestion.
Default-deny egress, then allow DNS. One podSelector: {} policy does it in Kubernetes, and the docs warn straight after: "A default deny-all egress policy also blocks DNS traffic." Allow DNS explicitly or the pod dies a different death.
Make the limit fail loudly. MaxTurnsExceeded and GraphRecursionError end the run in an exception instead of a shrug. A silent stop looks like a success in your logs.
Write the kill procedure as a procedure. NIST's AI RMF expects mechanisms "to supersede, disengage, or deactivate AI systems", with "responsibilities … assigned and understood" (Manage 2.4, January 2023). Assigned responsibilities means a name, not a runbook nobody has opened.

Nobody is watching is not a mode your agent enters. It is the normal state of every job you launch and walk away from, like most of mine.

FAQ

Isn't this just rate limiting? OWASP calls rate limiting damage limitation: the damage "could be reduced by implementing rate limiting on the mail-sending interface", which buys time to detect; OWASP publishes no rate or window default for it. Rate limits protect the provider from you. The envelope protects everything else from the run.

Do I need Kubernetes to default-deny egress? No. Kubernetes gives the pattern a spec and a YAML file. The same idea is Azure's no-outbound sandbox, egress rules at your proxy, or an allow-list in your gateway. Deny first, allow explicitly, at whatever layer you control.

Sources

Complete mediation, root causes, rate limiting as mitigation: OWASP LLM06 Excessive Agency (2025)
Top 10 taxonomy: OWASP Top 10 for LLM Applications (2025)
MaxTurnsExceeded, "Pass None to disable the turn limit": Agents SDK Runner reference
DEFAULT_MAX_TURNS = 10: openai-agents-python run_config.py
"default recursion limit is set to 1000 steps" (v1.0.6+): LangGraph graph API
DEFAULT_RECURSION_LIMIT = 25: langchain-core runnables/config.py
900s idle / 28,800s lifetime defaults: AgentCore lifecycle settings
"Request timeout | 15 minutes | No": AgentCore quotas and limits
One hour active, 30 minutes idle, no outbound: Microsoft Foundry Code Interpreter (2026-08-05)
Default-deny egress, DNS caution: Kubernetes Network Policies
MCP_TOOL_TIMEOUT default "about 28 hours", per-transport idle defaults: Claude Code environment variables
10× amplification, 7–9K → 70–100K RPS, 403 mitigation: GitHub incident discussion #205164 (2026-08-17)
Manage 2.4, Measure 2.6: NIST AI RMF 1.0 Core (January 2023)
Reported database deletion in a code freeze: Fortune, 2025-07-23

What is the first number you would write down for an unattended run: minutes, calls, or dollars? Tell me which dial you actually have, and which you are pretending to have.

If this saved you from one unbounded run, tap the unicorn below; one click, and it is the only metric Dev.to shows me. Follow The Agent Loop for tomorrow's post in the series on what holds an agent in place.

Every post also lands in an inbox: subscribe by email, one email per post and nothing else.

The cost of proving it works

The Agent Loop — Thu, 01 Oct 2026 06:04:32 +0000

Drafted with AI help, human-reviewed by The Agent Loop.

Short version: Your model bill has a shadow bill: every rollout you run before you trust a number, every judge pass over its output, every trace you keep around. It never arrives as its own line, so nobody budgets it. Arize writes the arithmetic out: production eval cost = traffic volume × sampling rate × evaluation surfaces × evaluator cost + human review + retention. Four multipliers and two human costs. This post reads those terms from primary sources, then shows the ladder that keeps them small.

For skimmers

A vendor reports one data leader told them LLM-as-judge evaluation cost 10× the baseline agent workload (anecdote, not a benchmark)
τ-bench: the best gpt-4o agent has >60% average task success, but pass^8 drops below 25% — proving reliability means rollouts, and the paper prices its own loop at "around 200 dollars" for one trial per task ($0.38 agent + $0.23 simulated user per task)
Judge tuning has been done for ~$2K instead of ~$2M: 4,480 configurations, multi-fidelity search vs full Alpaca-Eval-style evaluation; one Alpaca-Eval annotation ≈ $24
Ladder order: deterministic checks first, sampled LLM judges second, humans for escalation on uncertainty × consequence
We found no primary source for a universal "evals cost 5–30× a run" constant; decompose your own bill instead

The judge is a second workload

Your agent's run has a bill. The evaluation of that run has a second one, computed the same way: tokens in, tokens out, repeated per surface you score. Arize's model makes it explicit. Offline: dataset size × system variants × evaluator runs × cost per evaluation. Production: traffic volume × sampling rate × evaluation surfaces × evaluator cost + human review + retention (Arize, 2026). Nothing exotic. The judge is another inference workload, and retention is storage you pay for monthly.

How big can it get? Monte Carlo reports that "one data + Ai leader confessed to us their evaluation cost was 10 times as expensive as the baseline agent workload" (Monte Carlo, 2026). Read that sentence carefully: one leader, one vendor, one confession. It is not a benchmark and this post will not use it as one. It is evidence that the shadow bill can exceed the run bill when scoring is naive.

Practitioners say the same thing in the open. In an Ask HN thread on evals, one engineer called their AI-as-judge setup "wildly better than one-off testing" and added the catch: "it can get costly if the iteration count is high." Another: "it's quite expensive to do well. I find that for every hypothesis I might have to run a thousand prompts to collect enough data for a conclusion." Self-reports, not measurements, but they point the same direction as the vendor anecdote: verification scales with how often you iterate, and teams iterate constantly.

The multipliers you can actually defend

Nobody has published "evals cost 5–30× a run." We looked for it in the primary literature and it is not there. What is there is a set of study-specific multipliers you can quote with their method attached.

Reliability multiplies rollouts. τ-bench defines pass^k as the probability that all k independent trials of a task succeed. Its headline: "Even for the best-performing gpt-4o function calling agent which has a >60% average task success, pass^8 drops to <25%" (τ-bench, 2024). To compute that number the paper runs "at least 3 trials per task," caps episodes at 30 agent actions, and simulates the user with a language model. It also prints its own cost line: agent and user simulation cost "$0.38 / $0.23 per task respectively, so running one trial per task costs around 200 dollars." That is a real dollar figure for a real benchmark loop. It is also where the discipline matters: pass^8 means an 8× rollout opportunity, not an 8× dollar bill. Trajectory lengths, caching, sampling, judge passes, and storage all sit between those two statements, and the paper measures none of them for you.

Candidate generation multiplies coverage. The Codex paper solved 28.8% of HumanEval with one sample, then "generating 100 samples per problem and selecting the sample that passes the unit tests (77.5% solved)" (OpenAI, 2021). A 100× generation step bought ~49 points of coverage, with a unit test doing the selecting. That is 2021 code benchmarks, not agent bills; the transferable idea is that the sampling multiplier is measurable when someone bothers to publish it.

Judge search multiplies cheaply when you stop evaluating everything. "Tuning LLM Judge Design Decisions for 1/1000 of the Cost" estimates "approximately 2K$ to search through 4,480 judge configurations" versus "around 2M$" if each were evaluated the Alpaca-Eval or Arena-Hard way, with one annotation costing about $24 (arXiv 2501.17178, 2025). A thousand-fold reduction comes from early stopping and cheaper fidelity levels, not from cheaper tokens.

Three numbers, three methods, three papers. The honest summary: multipliers exist per surface, they are measurable per surface, and anyone selling you one universal ratio is skipping the measurement.

The cheap-verification ladder

The way to keep the shadow bill small is to refuse to pay the expensive layer for work the cheap layer can do.

1. Deterministic checks first. Exit codes, schema validation, exact tool arguments, database state. τ-bench itself judges by "an efficient and faithful evaluation process that compares the database state at the end of a conversation with the annotated goal state" (τ-bench, 2024). Monte Carlo's example of the right tool for the job: a "code based monitor to ensure an output is a valid US zip code." No model required.

2. Sample instead of scoring everything. Arize: "Sampling rate controls how many traces are scored. Filtering controls which production traffic receives that coverage," and "randomly sample some cases from the cheap path and re-evaluate them with stronger judges or humans." Broad monitoring at a low rate, dense evaluation on the slices that are risky or changing.

3. Escalate on uncertainty and consequence. Arize's rule: "The expensive layers of the stack should receive only the cases that remain unresolved after cheaper checks," with escalation judged on two variables, uncertainty and consequence. High consequence gets strong verification even when the case looks simple.

4. Only then a small judge. If you need an LLM judge, use a small one for the easy stratum. GPT-4o mini's launch price, posted July 18, 2024: 15 cents per million input tokens and 60 cents per million output tokens, "more than 60% cheaper than GPT-3.5 Turbo" (OpenAI, 2024). Dated launch price: pricing pages mutate, so re-check before it enters your budget. Keep humans for calibration and the high-consequence tail; one commenter in the HN thread noted human review "accumulates with each new model that gets released," which is exactly what the sampling tier is for.

What skipping actually costs

Anthropic's description of the no-evals loop: "Absent evals, debugging is reactive: wait for complaints, reproduce manually, fix the bug, and hope nothing else regressed." The fix does not start with a judge fleet. "20-50 simple tasks drawn from real failures is a great start," and regression suites "should have a nearly 100% pass rate" (Anthropic, 2026).

The uncomfortable coda: verification itself can be wrong. The same article documents Opus 4.5 scoring 42% on CORE-Bench until a researcher found rigid grading that "penalized '96.12' when expecting '96.124991…'", ambiguous task specs, and unreproducible stochastic tasks. After fixing the harness, the score jumped to 95%. A cheap evaluation bill does not make the number true; it just makes it cheap.

Put your own number on it

You already know what a run cost (What one agent run actually costs), and you know where the budget check belongs (Where does the budget check go?). The third slot is proving the run worked: rollouts × judge passes × judged tokens × retention. Multiply it out for your last release and write the number down next to the run cost.

Where does your evaluation cost live today: a number you can defend, a guess, or nothing because nobody counted? Reply below, I read every one.

If this gave you the third slot on the bill, tap the unicorn below; it takes one click and it is the only metric Dev.to actually shows me. And follow The Agent Loop if you want tomorrow's post in your feed: I am working through where money actually moves in an agent, one post a day.

Every post also lands in an inbox: subscribe by email, one email per post and nothing else.

Where does the budget check go?

The Agent Loop — Tue, 29 Sep 2026 14:06:01 +0000

Drafted with AI help, human-reviewed by The Agent Loop.

Short version: Your spend limit lives at the month. Your bill is generated at the call. The check has to live where the call happens, and almost nothing there is asking.

The prompt that exists is not the budget

You already set a spend limit, so you already believe you have a budget check. Read what it actually does: OpenAI's hard limit sits at organization or project scope, monthly, and when it trips, "affected API requests return a 429 error with the organization_spend_limit_exceeded or project_spend_limit_exceeded code. Enforcement is not instantaneous, so recorded spend can slightly exceed the configured amount" (OpenAI docs, 2026-09). The limit is real. It is also a month wide.

Meanwhile the only prompt your agent sees before acting asks one question: may this tool run? MCP's security docs require explicit consent before a tool executes, and Claude Code can be configured to ask on every call (Anthropic docs, 2026-09). Read the specs: consent is allow/deny. We found no monetary cap field anywhere in the MCP authorization or security material (reviewed 2026-09). Your approval dialog for a one-cent call and a forty-dollar call is the same dialog. I have written before about approval gates that never fire; this is the harder version. A consent prompt can fail to appear. A budget check that was never specified cannot fail to appear. It was never there.

Four places a check can live

Before you can place the check, you have to say what kind of check you mean. There are only four slots:

Pre-call reservation. The gate: check remaining budget, then let the model run. This one exists. OpenAI's Agents SDK blocking-mode guardrails "run and complete before the agent starts. If the guardrail tripwire is triggered, the agent never executes, preventing token consumption and tool execution. This is ideal for cost optimization" (OpenAI docs, 2026-09). Tool guardrails wrap the call itself: input check before execution, output check after.
Post-call ledger. Record what was spent and reconcile it against what you expected. The payments world has this: x402 documents settlement reconciliation with an exactly-once retry (x402 docs, 2026-09). Cost ledgers for LLM calls at run scope are mostly still your own plumbing (see the attribution post).
Retry budget. The silent multiplier. LangChain's AgentCore Payments middleware enforces "spending limits at the session level before any payment is signed", validates each payment against the session budget and rejects it when the limit is exceeded, and defaults max_error_retries to 3 per tool call with an auto-created session budget of one dollar (LangChain docs, 2026-09; preview feature). That default of 3 is exactly the shape your non-payment retries should have too.
Subagent envelope. Give each fork its own remaining budget so N children cannot each inherit an open-ended parent. In the SDK docs we reviewed, no dedicated per-subagent dollar budget or fork-count primitive exists; the recommendation falls back to tool guardrails around delegated calls (OpenAI docs, 2026-09). Slot 4 is where you build your own.

MCP fills none of these at protocol level. What exists instead is the gateway pattern: per-user "buckets to contain a runaway agent", "cost-weighted per-tool buckets to protect expensive operations", per-tenant isolation (Scalekit, 2026-06). That is application code in front of the protocol, which is fine, as long as you know the protocol itself is not coming to help.

What each provider's "spend limit" actually does

Provider	What actually fires	What never happens
OpenAI	429 with `_spend_limit_exceeded` at org/project monthly scope, slightly late* (docs, 2026-09)	Per-call dollar kill
Anthropic	Tier spend cap pauses all API usage until 00:00 UTC on the first of next month (429); your own spend limit returns HTTP 400 (docs, 2026-09)	Per-call dollar kill
Azure	Notification: "Resources aren't affected, and your consumption isn't stopped." Budgets evaluate every 24 hours, against data that lags 8–24 hours (docs, 2026-06)	Stopping anything
Google Cloud	After 100% of budget, pauses new usage for one eligible service in one project (Gemini API, Vertex/Agent Platform, Cloud Run); in-flight requests finish (docs, 2026-09)	In-flight termination; multi-service scope
AWS	Budget actions apply an IAM policy or SCP, e.g. deny provisioning more EC2 (docs, 2026-09)	Per-inference dollar cap

The strongest control in this table (GCP's) still lets every in-flight request finish, covers one service in one project, and waits for you to lift it manually. The weakest (Azure's) evaluates a data set that is already up to a day old and stops nothing. No provider documents a per-call monetary kill. The month is where they enforce; the call is where you pay.

The gap, with a receipt

A reported incident from earlier this year: an operator's AI agent set out to join and scan the DN42 network, and the AWS bill attached to the account came to $6,531.30 (lantian.pub, 2026-05; the operator's own account, not an audit). I use it carefully: it does not prove a missing per-call budget was the sole cause, and I am not aware of any audited postmortem that does. What the account does describe is inadequate stopping controls: the system that knew the spend kept going because nothing between the agent and the account could say "you have used your allowance for this task, stop."

Press reports of seven-figure agent bills circulate every month. None of them are audited postmortems. The documented controls tell the same story without the drama: enforcement at the month, consent at the tool, nothing at the call.

Four moves that put the check in the right slot

Gate before the model, after the arithmetic. Put a blocking pre-agent check (guardrail mode or your own hook) that compares remaining run budget to an estimated next step, and refuses before tokens burn. This is the only slot where stopping is free.
Number your retries. Cap retries per tool call at a small constant (three is the published default above) and count each retry against the run's budget. Retries are how a loop becomes a bill without a single decision looking expensive.
Envelope every subagent. Slot 4 has no framework primitive, so pass an explicit remaining-budget value down to each fork, and refuse to spawn a child without one. A child that inherits an open-ended parent multiplies the parent.
Keep the provider limit, and stop trusting it. It is the last-ditch 429 when your own gate fails, not the gate. If you are on Azure, your budget is an email; write your own check.

FAQ

Is this just rate limiting? Rate limits protect the provider from you. A budget check protects you from your agent: rate limits count requests per minute, budgets count money per task. You need both, and only one of them is priced.

Where does MCP fit? It standardizes consent and authorization scopes so tools can be gated. Spending ceilings are left to the authorization server and your host code, which means today they are your job.

Doesn't this slow the agent down? The pre-call check is one subtraction and a comparison against a number you already have. The Azure budget is evaluated every 24 hours. You are trading microseconds per call for protection between calls.

Sources

Spend limits, 429 codes, non-instant enforcement: OpenAI spend limits (2026-09)
Blocking guardrails "preventing token consumption": OpenAI Agents SDK guardrails (2026-09)
Tier spend cap pause, spend-limit 400: Anthropic rate limits (2026-09)
"Resources aren't affected, and your consumption isn't stopped", 24-hour evaluation: Azure budgets tutorial (2025-06)
Spend cap pauses one service, in-flight completes: Google Cloud spend caps (2026-09)
Budget actions apply IAM/SCP: AWS budget actions (2026-09)
Session budget before signing, max_error_retries default 3: LangChain AWS middleware (2026-09)
Per-user / per-tool / per-tenant buckets: Scalekit on virtual MCP rate limiting (2026-06)
Consent before tool execution, no monetary cap field found: MCP security best practices + authorization spec (2025-06)
Reported incident: lantian.pub DN42 write-up (2026-05)

Where does your budget check live: before the call, after the call, or in the postmortem? Tell me which slot you actually have today.

If this saved you from one unbounded loop, tap the unicorn below; it takes one click and it is the only metric Dev.to shows me. And follow The Agent Loop if you want tomorrow's post in your feed: I am working through where money actually moves in an agent, one post a day.

Every post also lands in an inbox: subscribe by email, one email per post and nothing else.

What one agent run actually costs

The Agent Loop — Mon, 28 Sep 2026 15:46:43 +0000

Drafted with AI help, human-reviewed by The Agent Loop.

Short version: You don't have a model bill. You have a loop bill with a model attached, and the loop is where the number comes from.

The one number I could actually check

I ran the arithmetic on my own session while writing the cost post: 3,518,203 input tokens against 271,350 output tokens. Thirteen tokens read for every one written. The model wrote a word; it was handed a paragraph back, thirteen times over.

I have never once guessed this number right before checking. Every time I expect the output side to matter, and every time the input side has already decided the bill.

What one step weighs

The best measurement I found is not a vendor slide. A study of about 4,300 real coding-agent sessions (Claude Code and Codex, 43 developers, roughly 350,000 LLM steps and 430,000 tool calls) reported the median step like this:

Part of one LLM step	Median tokens
Cached prefix re-sent (system prompt, history, tools)	119,000
Newly appended text this step	875
Output the model writes	214

That is per step, not per run; a run is dozens of these stitched together (source: arXiv 2606.30560, 2026-06). Read it as a ratio: that prefix is about 136× the text the step appends and roughly 550× the tokens the model writes.

Multiply by the steps. This is why a 10-step file-reading agent in a vendor guide came out at 472,500 input tokens versus 9,000 for a single pass, about 43× (Augment Code guide, illustrative, not an audit).

  one step's anatomy (median, 4,300 sessions)
  ┌──────────────────────────────────────────────┐
  │ cached prefix re-sent every step  119,000 ◄── your loop
  ├──────────────────────────────────────────────┤
  │ appended this step                     875
  │ output written                        214
  └──────────────────────────────────────────────┘
        ▲                          ▲
   0.1× if byte-identical     you pay full
   1.25× to write it once     price for both

Why the input side explodes

The mechanism is boring and fatal: the full conversation history (every prior prompt and completion) is carried forward unchanged on every round. Context accumulates, input grows superlinearly, and the study found that cache-read input tokens dominated both raw volume and dollar cost in every phase they analysed (arXiv 2604.22750, 2026-04).

The same trajectories show what humans do under that pressure: expensive runs re-open and re-edit the same files far more often, and the authors call it redundant back-and-forth that inflates context without proportional progress. The finding that matters: more tokens did not reliably produce higher accuracy.

The cache arithmetic that decides the bill

Providers price the re-read differently from the fresh text, and the ratios are stable enough to plan around (Anthropic docs, 2026-09):

cache read: 0.1× the normal input price
cache write: 1.25× (5-minute cache) or 2× (1-hour cache)
a 5-minute cache breaks even after roughly one re-read; a 1-hour cache after about two

OpenAI's docs say caching is on by default, discounts run up to 90%, and the exact multiplier is model-dependent, so treat any single "OpenAI gives you 50% off" figure as the 2024 announcement, not the current table (OpenAI docs, 2026-09).

The practical reading: your prefix is either your cheapest token or your most repeated cost, and you choose which by whether it stays byte-identical. Change one header mid-run and the next step writes instead of reads.

Same task, 30× apart

On 500 SWE-bench Verified problems with eight frontier models run four times each, agentic coding tasks averaged roughly 3,500× the tokens of single-round code reasoning — and within the same task, runs varied up to 30× in tokens, with the worst run for a given problem about 2× the best (arXiv 2604.22750, 2026-04). The most expensive problem averaged about seven million more tokens than the cheapest.

So "what does a run cost" has no single answer. It has a distribution, and the width of that distribution is the thing worth managing.

Cost per attempt is not cost per success

Published per-task dollars exist, if you read the fine print. TheAgentCompany benchmarked OpenHands with Gemini 2.5 Pro at an average of $4.20 per task over 27.2 LLM-call steps at 30.3% success, with cost computed from token counts at API prices and no prompt caching assumed (arXiv 2412.14161, 2025-05). The same benchmark got Gemini 2.0 Flash under $1 per task at about 40 steps and lower performance.

Divide by the success rate and the honest number moves: $4.20 at 30.3% is roughly $13.90 per successful task — my division, not the paper's, and it assumes every failure's tokens were wasted. Failures often aren't free: τ-bench found frontier function-calling agents succeeding on fewer than half of tasks, with pass^8 below 25% in the retail setting (arXiv 2406.12045, 2024-06).

Four moves that actually change the number

Stabilise the prefix. Keep system prompt, tool schemas and static history byte-identical across steps so reads stay reads. A single edited line converts 119K reads into a 119K write.
Prune the middle, not the top. Trim stale tool output and duplicated file views from the carried history; the redundancy is measured, not hypothetical.
Measure cost per success, not per call. Track tokens (or dollars) divided by tasks that actually completed, so retries are visible instead of averaged away.
Key every run. Attribute spend to run, team and feature at the trace level; the gap between "the model costs money" and "this agent cost money" is exactly what the attribution post covers.

FAQ

Isn't 119K an unrealistic prefix? It is a median across real sessions with real tool schemas and long histories, not a toy example. If your prefixes are smaller, your loop is cheaper than the field, so measure instead of assuming.

Do caches make the input side free? No. Reads are cheap (0.1×) but they are billed on every step, and writes cost more than a normal token. Cheap is not free when it repeats a hundred times.

Why quote a 2024 benchmark? Because it is independent and methodological: token counts, step counts and success rates are all published. Vendor averages without n and method are the ones I throw away.

Should I switch models to cut cost? Try it after the loop: the same model on the same task already varied up to 30× in the benchmark above, so the loop is the bigger lever. Compare models once your prefix is stable and your successes are counted.

Do vendors make this worse? They price differently, not secretly. Vercel, for example, bills provider inference at cost plus $0.25 per million billable tokens: a stated pricing rule, not a market average (vendor docs, 2026-08). Whatever the markup, you still need per-run numbers to know what you are multiplying.

Sources

Median step tokens, 4,300 sessions: arXiv 2606.30560 (2026-06)
3,500× scale, 30× run spread, context replay, cache-read dominance: arXiv 2604.22750 (2026-04)
87,325 input / 1,505 output per instance: arXiv 2606.01326 (2026-05)
$4.20/task, 27.2 steps, 30.3% success, Flash <$1: arXiv 2412.14161 (2025-05)
Success rates / pass^8: τ-bench, arXiv 2406.12045 (2024-06)
Cache read 0.1×, writes 1.25×/2×, break-even: Anthropic pricing docs (2026-09)
Default caching, up to 90%, model-dependent rates: OpenAI prompt caching guide (2026-09)
Vendor cost markup rule (cited in FAQ): Vercel agent pricing (2026-08)
43× worked example: Augment Code agent token guide (vendor, illustrative)
Where my own session numbers came from: Your agent's cost problem isn't the model. It's the loop.

If this saved you a wrong guess about your own bill, tap the unicorn below — it takes one click and it is the only metric Dev.to actually shows me. And follow The Agent Loop if you want tomorrow's number in your feed: I am working through the arithmetic of running agents, one post a day.

Or get it by email instead: buttondown.com/theagentloop, one email per post and nothing else.

Your cost dashboard can't tell you which agent ran up the bill

The Agent Loop — Mon, 28 Sep 2026 05:25:11 +0000

Drafted with AI help, human-reviewed by The Agent Loop.

Short version: My own analytics dashboard said 52 views this morning while every per-post row printed null. The total was right, the attribution was broken, and the footnote even claimed the data didn't exist. That is what most agent cost tracking looks like: a correct number you can't decompose. A provider invoice tells you that spending rose and never which agent run, retry, or tool call raised it. OpenTelemetry's GenAI conventions define no cost attribute in the published spec (the proposal has been open since August 2026) and mark their token counters as "a proxy for cost approximation", and the span that wraps your tool call carries no usage fields. Price each span yourself, roll it up from the child inference spans, and alarm at runtime instead of waiting for the invoice.

For skimmers

AWS anomaly detection reads Cost Explorer data with a delay of up to 24 hours and re-evaluates about 3 times a day: your dollar alarm always fires after the money is gone
The same input token costs 0.1× as a cache read and 1.25× as a cache write: a 12.5× swing on identical bytes
OpenTelemetry GenAI: token counters exist, execute_tool spans exist, cost attributes don't — registry grep = 0, and the proposal (PR #443) has been open since 09 Aug 2026
OpenAI's Agents API says its own usage counts are "not a final bill" and that missing usage "does not mean zero usage"
Spend limits sit at organization/project scope, not per span. Nothing in the default stack attributes one tool call
Trajectory trimming (AgentDiet) cut input tokens 39.9–59.7% and total cost 21.1–35.9% at equal performance, which you can't find without per-span numbers

The invoice is a lagging indicator

Two days ago I wrote up why an agent bill grows even when the model price doesn't move. A reader, @hannune, replied with the follow-up I deserved: “Tag cost per call is the one that took me too long to add” — and the question underneath it: how do you actually attribute spend to a span, not to the account?

Start with what attribution gives you today. Amazon's Cost Anomaly Detection uses Cost Explorer data, and the documentation states the delay plainly: up to 24 hours, with the detector running roughly three times a day. Cloud cost monitors fire when "today's total cost exceeds $10,000". Useful. Also, by construction, a day late.

Braintrust's cost guide puts the limitation better than I can: the invoice "can show that spending increased, but it cannot explain which customer, feature, prompt change, retry pattern, or agent run caused the increase."

The scale question is no longer academic. One widely reported month-long run of roughly a hundred agent instances billed $1,305,088.81 across 603 billion tokens and 7.6 million requests, with a single day at $19,985.84 (reported figures, treat as un-audited). At that size, "the total went up" is not an explanation.

The scary part isn't the size of the bill. It's that you can't name the line that produced it.

Tokens are not money

Here is the step everyone skips. You count tokens, then you treat the count as cost. It isn't, because the same token bills at different rates depending on state.

OpenAI prompt caching: cache read at 0.1× the uncached input rate, cache write at 1.25×, minimum cacheable prefix 1,024 tokens. Their own multi-turn agent example reports more than 90% cache-hit.
Anthropic: five-minute write 1.25×, one-hour write 2×, read 0.1×. And input_tokens counts only tokens after the last cache breakpoint: the real identity is total_input_tokens = cache_read + cache_creation + input_tokens.
The timestamped-prefix trap, straight from Anthropic's docs: put a date at the front of the prompt and "you pay for a fresh cache write on every request and never get a read."

So "input tokens" in your log isn't a cost. It's a count that has to be split into three buckets and multiplied by three different rates, with a price table that changes per model and per date. Miss the split and you are off by an order of magnitude, in either direction.

The volume makes the split matter. In September 2025 Claude 4 Sonnet reportedly hit 100 billion tokens a day on OpenRouter, of which 99% were input tokens accumulated in a trajectory and 1% was what the model generated. Nearly all of your spend lives in the tokens you keep re-sending, and those are exactly the tokens cache state re-prices.

The span that runs your tool has no price

This is about tool- and run-level granularity. Vendor dashboards attribute what they are able to price; the question is what happens at the tool call, where the span you need has no fields to carry it.

This is the part I did not expect when I went looking for standards support.

OpenTelemetry's GenAI semantic conventions define gen_ai.usage.input_tokens and gen_ai.usage.output_tokens, and the registry page marks them Deprecated, moved to the GenAI repository. In that repository every GenAI attribute still carries a Development badge, not Stable. Fine, that's how specs mature.

What's not fine: the token-metrics document says the counters "serve as a proxy for cost approximation", and there's no cost, USD, or price attribute in the published registry. I grepped the whole conventions repo: nine mentions of "cost", every one of them prose, zero attributes to set.

There is a proposal. PR #443, "Add per-operation cost conventions (gen_ai.usage.cost.*)", has been open since 9 August 2026, with issue #287 asking for the same thing since May 2025. The standard knows about the gap and has not closed it. Meanwhile two vendors already emit gen_ai.usage.cost on their own — and are filing issues against themselves because the value ships as a bare number with no currency field (OpenLIT, Laminer). A cost with no unit is half an attribute.

Worse for anyone hoping to attribute per tool: the execute_tool {gen_ai.tool.name} span exists, is typed INTERNAL, and its attribute table carries gen_ai.tool.* fields plus error.type. No usage, no cost. The span that says this tool ran cannot say this tool cost.

The spec's own discussion explains why nobody has bolted one on: tool and retrieval charges "are not model cost to begin with", so they were left out of the cost work entirely. Fair, as far as model billing goes. It also means that for the line item your CFO actually asks about — which tool cost money — there is no standard home for the number, only child inference spans you can roll up yourself.

The commercial tools are honest about the same gap:

Langfuse: only generation and embedding observations track cost; other observation types carry none. Tool and agent spans are unpriced by design.
Langfuse, on overlapping buckets: usage and inferred cost "will be counted double", and the displayed cost "overstates what your provider actually charged."
Datadog LLM Observability: prices each span independently and reports PARTIAL COST when a span lacks price data. Its own example alert threshold is $10,000 a day.
OpenAI Agents API: usage counts are "not a final bill", "missing usage does not mean zero usage", and the response does not expose cache-write counts, so a caller "cannot determine the exact model charge."

Add one more trap from the spec itself: a retried request's span SHOULD cover the logical operation "with all retries", which means SDK-level retries (the Python SDK retries certain errors twice by default) are folded into one span and invisible in the trace. Three billed attempts, one visible operation.

Nobody's standard says what a call cost. Everyone's dashboard prints a number anyway.

Three beliefs that keep teams shipping totals

Belief 1: the invoice is attribution. It is a ledger, not a trace. AWS gives you up to 24 hours of lag and no run identity. You will always be able to say that week was expensive and never which agent.

Belief 2: our observability tool tracks cost, so we're attributed. It tracks cost on the spans it can price. Generation spans, maybe embeddings. The tool-call span that actually explains the spend is unpriced, so your per-feature rollup silently under-counts and your total silently over-counts. Datadog calls the result PARTIAL COST for a reason.

Belief 3: the spend limit protects us. OpenAI's spend limits and alerts are enforced at organization or project scope. Nothing there stops one runaway loop inside one project from being the whole bill. A budget at the account level is a smoke alarm in the building's lobby.

What I would wire up

tool call
  |
  v
inference spans  -->  split billed tokens:  cache read | cache write | plain input
  |                                            |
  v                                            v
price table  <-----  keyed by model + date (not a constant)
  |
  v
price attributes on the PARENT span  -->  rollup: tool -> run -> feature
  |
  v
runtime budget alarm (seconds, not 24h)

invoice  -->  reconciliation only, never the source of truth

Billable, not used. The spec's own rule: when a system reports both, instrumentation MUST report billable tokens. Log cache_read, cache_creation, and input_tokens separately, never a single merged input number.
Put the price on the parent. execute_tool carries no usage fields, so compute cost from the child inference spans and write cost.usd (your own attribute, until the spec grows one) onto the tool span. That is the only place the tool's spend becomes queryable.
Key the price table by model and date. A constant $3/M is how you quietly drift 20% off reality. Store it as data next to the trace, not in code.
Tag the dimensions you'll be asked about. Datadog's cost_tags pattern: team, feature, customer, run id. The invoice will never carry these.
Alarm at runtime, at the unit that can be stopped. Account-level spend limits stay on as a backstop, but your real threshold belongs on the run: kill the loop when this run passes a ceiling, not when the project passes $10,000 a day.
Keep idempotency in the money path. The spec folds retries into one span, and the SDK retries twice by default. Forward an idempotency key with the write so a retried run can't bill you twice for one action (the Stripe-shaped fix from last week's approval post).
Reconcile monthly, don't investigate monthly. Compare your attributed total to the invoice and treat the delta as a bug in attribution, not as a surprise bill.

The payoff is measurable. Inference-time trajectory trimming (AgentDiet) strips the useless, redundant and expired context agents keep re-sending: across two LLMs and two benchmarks it cut input tokens 39.9–59.7% and total cost 21.1–35.9% while agent performance stayed the same. You cannot see that win on an invoice. You can see it the moment cost lives on the span.

Admitted limit: I have not run this on a production bill. What I did run is smaller and embarrassing: this morning's dashboard printed a correct 52 with a broken per-post column, because one field name was wrong and a footnote asserted the data was unavailable. Totals that nobody can decompose will always lie to you eventually.

FAQ

How do you attribute LLM cost to a specific tool or span?
Count billable tokens on each inference span, split into cache read, cache write, and plain input, multiply each by a price looked up for that model and date, then write the sum onto the parent execute_tool span and roll up by tool, run, and feature. No standard does this for you.

Does OpenTelemetry have a cost attribute for GenAI?
Not yet. The published conventions define token counters and describe them as "a proxy for cost approximation", with no cost, USD, or price attribute, and execute_tool spans carry no usage fields either. A proposal (gen_ai.usage.cost.*, PR #443) has been open since August 2026 and is still open, so treat any gen_ai.usage.cost you see in a vendor's output as a private extension with no agreed unit.

Why does my observability tool show a different total than the invoice?
Three usual causes: overlapping usage buckets double-counting, spans that carry no price data being silently skipped (Datadog reports this as PARTIAL COST), and usage counts that never included cache writes: OpenAI states its Agents API usage "cannot determine the exact model charge".

What's the difference between tokens used and tokens billed?
Used is what the model saw. Billed is what you pay for: cached input re-reads at 0.1×, fresh cache writes at 1.25×, and plain input at 1×. Anthropic's input_tokens counts only tokens after the last cache breakpoint, so the two numbers are not comparable.

How do you alert on agent spend before the invoice arrives?
Put a budget on the unit that can be stopped (per run or per loop), evaluated against your attributed per-span cost. Account-level spend limits and cloud anomaly detectors work as backstops, but AWS's detector itself has up to a 24-hour delay.

Sources

Related on The Agent Loop

Over to you: if one tool call in your agent tripled its spend last week, what would you check first, and would that check actually name the tool? Reply below, I read every one.

And if this saved you from shipping another unattributed total, tap the unicorn (or like) and follow The Agent Loop — I read every reply, and the next one is about the thing nobody wants to audit.

Prefer an inbox to a feed? Every post also goes out by email: subscribe at buttondown.com/theagentloop.

Your agent's approval needs an expiry date

The Agent Loop — Sat, 26 Sep 2026 07:57:59 +0000

Drafted with AI help, human-reviewed by The Agent Loop.

Short version: Someone on my last post proposed the fix I wanted: the agent signs a concrete proposal (call X, with args Y, spend limit Z), and the gate authorizes that signed commitment instead of whatever the model says next. It's the right architecture. It also has three unanswered questions that showed up in the same thread: how long does a signed proposal live, what happens when you rotate the key, and what do you do with proposals that got signed and never submitted. Signed but unexpired is a bearer token with a story attached. Give an approval a lifetime in minutes, make it single-use, and make revocation actually kill what is outstanding.

For skimmers

Standards already answer the lifetime question: DPoP proofs should be accepted for seconds to minutes, logout tokens ≤2 minutes, access tokens about an hour (GCP, Anthropic, Entra's 60–90 min default)
An approval that never expires outlives the context that justified it, and outlives the key
jti exists in JWT exactly "to prevent the token from being replayed"; DPoP servers must remember each one for the acceptance window
Duplicate side effects are the failure you actually feel: Stripe keeps idempotency keys 24 hours, A2A assumes duplicate deliveries, a double-fired cancel order came up in my comments
Revocation that isn't a key rotation is a suggestion: Cloudflare's Thanksgiving incident turned on one token + three service accounts they "failed to rotate"
AP2 puts numbers on it: example mandates expire in 3,600 seconds, receipts narrow the remaining scope after every action
Orphaned proposals (signed, never submitted) are the hole nobody designs for: expire them like everything else

The question my commenters asked

Two people pushed back on my MCP approval-gate post, and the second one is the reason I'm writing this.

The first drew the line I missed: enforcement isn't only about where the check runs, server-side or host dialog; it's about what the check runs against. If the gate evaluates the model's next utterance, you've put a deterministic gate around a non-deterministic source. The fix is to have the agent commit to a concrete action (this call, these arguments, this ceiling), sign it, and authorize that. The model suggests; infrastructure decides.

I conceded the point in the thread and asked the two questions it raises. How long does a signed proposal live? And when you rotate or revoke an agent's key, do already-approved proposals die with it?

The second commenter answered with an incident before I could: their agent had both a read-orders and a cancel-order tool, the host dialog timing was off, and the cancel fired twice: both calls reached the server before either could be stopped. They also had signed proposals surface in postmortems that nobody had thought to expire.

That is the part worth a post, because both failures are the same shape. The gate did its job. The proposal was valid when it was written. Nothing ever said when it stopped being valid.

An approval is a claim about a moment

Here is the property I was missing. A signature says: this was true, and this identity vouched for it, at time T. It doesn't say anything about T+1. The context that made the action reasonable (the user's intent, the session, the account balance, the key's own validity) is all invisible to whoever checks the signature later. They see a valid signature over a valid payload and they let it through.

So an unexpired approval is a bearer token with a story attached. The story is what you tell yourself about why it was signed. The token is what the server actually checks.

Every serious stack answers this the same way, with a lifetime:

DPoP (RFC 9449) says a server must only accept a proof for a limited time after creation ("preferably only for a relatively brief period on the order of seconds or minutes") and must remember each proof's jti for exactly that window, because a single-use check "provides a very strong protection against DPoP proof replay."
OIDC back-channel logout asks for tokens that expire at most two minutes ahead, for the same reason: a captured logout token shouldn't outlive its usefulness.
Access tokens cluster around an hour. Google Cloud's agent identity tokens default to one hour and are blunt about the trade: they "can't be introspected or revoked, and remain valid until expiry." Anthropic's API keys ship with 3-hour, 1-day, 7-day and 30-day expirations, fixed at creation. Entra randomizes between 60 and 90 minutes by default.
Assumed-role sessions get 15 minutes to 12 hours from AWS STS, default an hour, and role chaining drops you to an hour because each hop is a new trust decision.
MCP's authorization spec says authorization servers should issue short-lived access tokens, must rotate refresh tokens for public clients, and must return 401 on anything expired.

The numbers differ; the instinct doesn't. Nobody serious ships an approval that lives forever. Which is a fair test to apply to your own gate: what is the exp on the thing your agent got approved to do?

If the answer is "it's a session" or "until the user closes the tab," you have made the model's patience the lifetime of an authorization.

Single use, or not an approval

Lifetime solves the stale case. It doesn't solve the duplicate one: my commenter's double-fired cancel.

HTTP has a word for this. RFC 9110: don't auto-retry a non-idempotent request unless you know the original never applied, and proxies must not auto-retry them at all. The de-facto implementation of the client side is an idempotency key: Stripe keeps the key for at least 24 hours, replays the original response (including a 500) for the same key and same parameters, returns idempotency_error if the parameters differ, and 409 on concurrent use. After 24 hours the key is pruned and reuse starts a new request. Even the standardization attempt agrees on the horizon: the IETF Idempotency-Key header draft expired without becoming an RFC.

For a signed proposal the mechanism already exists and it's named: jti. RFC 7519 defines the claim to "prevent the token from being replayed," and DPoP turns it into a rule: store the jti for the acceptance window and decline repeats. Two minutes of server-side memory converts "valid signature" into "valid signature, used once, and you'd notice a second try."

The A2A protocol assumes duplicates as a normal condition rather than a bug: push-notification deletion must be idempotent, and clients should process notifications idempotently because duplicate deliveries may occur. That's the honest baseline for anything an agent drives over a network. Your gate should behave as if the message will arrive twice, because one day the host dialog timing will be off and it will.

Where you can't dedupe upstream, dedupe at the database. The case study worth reading (its author explicitly labels the numbers constructed, and I checked: I couldn't find a company-published double-charge postmortem to cite instead) is the classic SELECT-then-INSERT race: eleven duplicate charges at gaps of 47–338 milliseconds, fixed by a UNIQUE constraint on the idempotency key rather than a SELECT that looked clean.

The Ethereum version of this is the one people remember. After the DAO fork, Ethereum and Ethereum Classic shared transaction formats, so a signature that was valid on one chain was valid on the other: withdrawals on one side minted the same value on the other. The fix (EIP-155) put the chain ID inside the signed hash, so context becomes part of what the signature covers. It's the cleanest illustration of the general rule: if two runs of your agent can produce the same signed bytes, you have built a replay. Signature malleability has been exploited the same way, with an attacker re-submitting a request as long as the old one had not yet expired.

Revocation that lands

Rotation is where "we approved it" turns into "we approved it, and we meant it." The OAuth answer is RFC 7009: a revocation endpoint invalidates the token and, for a grant, the other tokens issued under that grant. The BCP (RFC 9700, final in January 2025) goes further and says tokens should be sender-constrained, meaning bound to a key or channel (mTLS, DPoP) so that a stolen token is not enough to use it, and that refresh tokens for public clients must be either sender-constrained or rotated, because reuse of a rotated token reveals the breach.

I read three incident reports this week. They are all the same story: a credential that should have been dead wasn't.

Cloudflare, Thanksgiving 2023: one access token plus three service accounts left over from the Okta breach. Their phrase for it: "that we failed to rotate." Remediation meant rotating more than 5,000 credentials and reimaging 4,893 systems.
GitHub, April 2022: stolen OAuth tokens from two integrators were used to enumerate orgs and selectively clone private repositories; GitHub revoked its own affected tokens immediately and asked both integrators to revoke all user tokens for their apps.
CircleCI, January 2023: a malware-stolen SSO session led to customer GitHub token theft; the response was to rotate every GitHub OAuth token the company had.

Nobody's incident report says "our signed-but-unused approvals outlived their usefulness." They say "we failed to rotate." The design requirement writes itself: when the key dies, the proposals die with it. Otherwise they were never really tied to the key, and you have bearer tokens wearing a signature costume.

For agent-shaped systems the check is concrete. Can you answer, right now, for every outstanding approval: who signed it, what exactly does it authorize, when does it expire, and does it still work after a rotation? If any of those four needs a query you haven't written, the approval is longer-lived than you think.

The orphan nobody designs for

Back to the postmortem detail: proposals that got signed and were never submitted. They're the awkward artifact of any commit-then-authorize design: the agent signs, the next turn goes somewhere else, and the signed bytes sit there.

They're only dangerous if they're still live. A proposal with a five-minute expiry and a jti that's consumed on first presentation is inert the moment nobody uses it: it expires into a log line. A proposal with no expiry is an approver's worst case, because it can be presented later by anyone who can reach the endpoint, and the gate has no way to distinguish it from something the agent meant to do now.

The pattern that solves this is already in AP2, Google's agent payments protocol, and it's worth stealing even if you never touch payments. An open mandate carries the agent's key; each action returns a signed receipt that narrows the remaining scope, "often preventing future presentations entirely." Example mandates in the spec carry iat → exp of 3,600 seconds. The agent isn't holding a permission. It's holding one that shrinks every time it's used, and that has a clock on it. The security doc is even direct about posture: all LLMs and agents must be considered potential attackers, and receipts must be integrity-protected from the agent's own LLM.

Macaroons are the older, general version: chained caveats that attenuate and confine when, where, by whom, and for what purpose a credential may be used. The insight survives every implementation: an approval that gets narrower and older is strictly safer than one that doesn't.

What I'm doing about it

gate verifies signature and exp, checks jti not seen -> single use -> execute exactly once -> receipt narrows scope; on key rotation, outstanding proposals are invalidated" width="800" height="400">
Figure: an approval without an expiry is a credential, not a decision.

agent signs:  {call X, args Y, limit Z, exp = now + 5m, jti = <id>}
                    |
                    v
gate:  verify signature -> verify exp -> jti not already seen?
                    |                         |
                    |                    seen -> REJECT (replay)
                    v
             execute exactly once
                    |
                    v
        receipt narrows remaining scope; key rotation kills
        everything outstanding that wasn't executed

Put an exp in minutes on every approval. Use the DPoP intuition (seconds to minutes for anything that causes a side effect) and an hour as the ceiling for read-ish scope. If your stack defaults to "never," override it; Anthropic and OpenAI both let org policy forbid the "Never" preset for exactly this reason.
Give it a jti, and remember it. Server-side set of used IDs with a TTL equal to the acceptance window. This is the two-minute piece of memory that turns a valid signature into a single-use approval.
Bind the signature to the context. Sign the audience and the nonce as well as the payload; that's the EIP-155 lesson. A signature that means the same thing in two environments is a replay waiting for a chain fork.
Forward an idempotency key to anything downstream that can double-charge. Your dedupe layer is only as strong as the deepest service that doesn't check it.
Make rotation a proposal killer. Key retired → outstanding proposals invalidated. If your format can't express that, the proposals are bearer tokens; shorten them until they can't do damage inside their lifetime.
Design for the orphan. Signed, unsubmitted proposals expire on their own. AP2-style receipts that narrow scope after each action are the stronger version: the approval cannot be reused even within its window.

Nothing here is new engineering. It's the token discipline every API platform already applies to its own credentials, pointed at the artifact your agent now mints instead of a session cookie.

FAQ

How long should an agent approval live?
Short enough that the context that justified it can't change underneath it. For anything with a side effect: seconds to minutes (DPoP's guidance), or a single request with an idempotency key. For read scope, about an hour: that's where GCP agent tokens, Anthropic's shortest preset, and Entra's 60–90 minute default all land.

What is jti and why does a signed token need one?
It's the unique identifier claim in JWT. RFC 7519 defines it specifically to prevent replay. Storing it for the acceptance window and rejecting repeats is what makes an approval single-use instead of copyable.

What happens to already-approved actions when I rotate a key?
They should die. RFC 7009 revokes tokens for a grant; the security BCP pushes sender-constraining so a stolen token is unusable anyway. If your outstanding proposals survive a rotation, they were never bound to the key. They're bearer tokens.

What's an idempotency key and how long does it live?
A client-generated unique ID attached to a non-idempotent request so retries return the original response instead of executing twice. Stripe keeps them at least 24 hours; reuse after pruning starts a new request. The IETF header draft expired, so implementations differ. Check yours.

How do I stop a replayed tool call from running twice?
Three layers: short expiry so a stale copy can't be used, a single-use jti (or idempotency key) so a fresh copy of a used approval is rejected, and a unique constraint at the store so two concurrent executions can't both commit.

Sources

Related on The Agent Loop

Over to you: what lifetime does the approval in your gate actually have right now, and if you had to guess, would a copy of it still work after a key rotation? Reply below, I read every one.

If you want the rest of this series when it drops: subscribe via Buttondown. If you've shipped something with an expiry on it, tell me the interval you picked and whether anyone ever hit it. The distribution of those answers is more interesting than the advice.