The delegation tax: measuring what it costs to let cheap models do the work

Half the internet will tell you there’s a free lunch in agentic coding: put a frontier model in charge, and let a $0 local model do the typing. I wanted to know whether the meter agreed. So over four days in July I ran a pre-registered A/B experiment. One project spec, built end to end by two agent setups that differed by a single committed policy file. Arm A sent every subtask to the cloud. Arm B was the free-lunch setup: a frontier orchestrator (Claude Sonnet 5) handing its coding to a $0 local model (Qwen3.6-35B on an M1 Max), paying cloud rates only for coordination and review. I wrote and committed the acceptance suites before any run. Ten runs later, every run of both arms had passed every check.¹ The models were never the interesting variable. The bill was, and it says the thing you actually pay for is trust.

The first matched pair told a tidy story. The hybrid cost $1.42 and 41 minutes. The all-cloud arm cost $1.15 and 9 minutes. Call it a 24% premium plus a 4.5× wall-clock penalty for the privilege of using a free model.¹ Then the variance runs landed and killed that tidy story, by a rule I’d registered in advance for exactly this purpose. The second hybrid run cost $0.68 and ran in 9 minutes. Cheapest and fastest run of any arm, on an identical harness.¹

Here’s what the meter actually reads. At this project size, two things move per-run cost 2–4×: which files the orchestrator decides to delegate, and how the repair dice land. The cloud-versus-local axis moves it less than that. By three runs per arm the means had inverted outright. Hybrid $1.13, all-cloud $1.65, and the two arms’ cost ranges overlap.¹ Almost nobody meters that overhead. It’s where the delegation money actually goes.

The honest baseline: you’re competing with cached tokens

Most local-savings math starts from list price. List price is the wrong denominator. Agentic workloads are overwhelmingly input-heavy and overwhelmingly repetitive. The same system prompt, tool definitions, and conversation history get resent on every request. One measured Claude Code workload came to 100.9 million tokens across 1,289 requests. Of that, 99.4% were input tokens, and 84% of the input was prompt-cache reads. Caching alone cut the bill by roughly 74%.²

Cache reads on the Claude API bill at 0.1× the input rate.³ Run that arithmetic and a long-running agent’s effective input cost lands at a small fraction of list. So a local model isn’t displacing $2-per-million-token input. It’s mostly displacing $0.20-per-million cache reads, and the real prize is the ~16% slice of fresh input. The experiment’s own ledgers show the same shape: one build session read 2.16 million cached input tokens for $0.43.¹ Ignore caching and your savings estimate overstates the local opportunity several-fold, before the first delegation even happens. (One disclosure on the denominations: these runs billed at Sonnet 5’s introductory $2/$10 per million tokens, in effect through August 31, 2026, after which standard pricing is $3/$15.³)

One more baseline habit worth stating. The provider dashboard is ground truth. Your harness’s self-reported cost is a lower bound. Reconciling billed-versus-computed spend daily, the harness under-recorded by 0.33% on a clean day and 13% on a day with killed runs. Every run also quietly billed an extra ~$0.0018 for a title-generation call that never showed up in the harness’s own accounting.¹ Small numbers. But if you’re writing an article about cost measurement, or trusting your own dashboards to make a routing decision, the direction of the error matters.

The rig, and the rules that kept it honest

The experiment ran on waybill, a small agent orchestrator with a cost ledger I built for the purpose. It was built partly by the same delegation patterns it exists to measure, which is where most of the war stories below come from. Everything is public: ledgers, the routing policies, and the acceptance suites.¹ If you want to check the arithmetic yourself, the repo’s README and SETUP.md cover the rig, the arm policy files, and how to reproduce a run.¹

The rules did the heavy lifting. Acceptance suites were committed before any arm ran, with a declared repair rule for suite defects. Every repair I applied was parse-level; the substance of a check never changed. Both arms got byte-identical specs, guidance files, and OS-level sandboxes. The only difference between them was which model each agent role resolved to. Run ledgers are append-only. And one rule earned its keep twice over:

Never trust a subagent’s self-report, in either direction. Not because the models lie, but because nothing in the loop makes truth load-bearing.

I earned that rule empirically, from both sides of the cloud/local divide. The local model’s failure mode was the silent no-op. It returns a confident completion message and writes nothing to disk. Or worse, it writes two import lines of a 480-line file. The failure was stochastic and size-correlated: clean at 355 output lines, partial at ~480, with small tasks running 15-for-16 clean across the same days.¹ The cloud model’s failure mode was worse, in a way I find genuinely funny. A Sonnet subagent asked to run verification checks reported detailed check results without executing anything. The session log shows zero tool calls. It billed $0.02 for the fiction.¹ Symmetric lesson, symmetric rule: the orchestrator runs every acceptance check itself, and artifacts on disk are the only admissible evidence.

The quieter version of the same lesson. Across the build, the notes logged four separate instances of code that passed tsc strict mode and was still wrong: swapped arguments, silently tightened validation, a CLI positional that came through as undefined at runtime.¹ Type checks and tests are gates, not verdicts. That’s why I graded the arms exclusively on pre-registered, mechanically-run acceptance checks.

What the ledgers say

Here’s the full matched-harness matrix as it stands. Two seeds: a small CRUD service over SQLite, and an MCP server over a ledger file format, picked as a thin-training-data difficulty probe. Every run passed every check.¹

That’s nine matched runs. A tenth sits outside the matrix: arm A’s run 1, a no-guidance solo build with no committed AGENTS.md. It also passed 16/16. It’s the solo baseline the cost comparisons below lean on.¹

Quality never differentiated. Not once, on either seed, did an arm fail a check the other passed. So the interesting columns are dollars, wall clock, and what the orchestrator chose to do. At three runs per arm on Seed A, the all-cloud arm averages $1.65 and 12.6 minutes. The hybrid averages $1.13 and 22.0 minutes. Cheaper on the mean, slower on the mean. And the within-arm spread, 2.1–2.2× in each arm, is still wider than the gap between the arms. That’s the finding.¹

Start with the anatomy of the expensive hybrid run. Its orchestrator emitted 43,500 output tokens of specifications, repair diffs, and coordination. That’s more than the 34,700 output tokens the all-cloud arm’s first run spent building the entire project solo.¹ Two of its three local delegations came back needing repairs, and every repair is frontier output at $10 per million tokens. The big delegation was a 60,400-token test suite. It ran 29 minutes on local hardware and came back implementing a subtly different API contract than the spec: integer cents where the spec said decimal amounts. The orchestrator then corrected it file-wide.¹ That’s the dangerous failure class. Not loud crashes, but quiet spec drift, caught only because something downstream was paid to look. And it wasn’t a one-off. Across four hybrid runs, delegated code came back with the same cents-versus-decimal money-representation mismatch three times.¹ Recurrence is what makes it a tax and not bad luck.

Now the counter-evidence, because the ledgers cut against my framing too. Delegation itself wasn’t the tax. The all-cloud arm’s delegating run beat its own solo run on cost, $1.15 versus $1.23, because fresh child contexts read 3.3× less cache than one long-running solo context.¹ And the cheap hybrid run delegated exactly one file, the test suite again, got it back clean, and finished at $0.68. Same policy, same spec, same hardware as the $1.42 run. The difference was delegation shape and repair luck. Nothing else.

Two more ledger lines deserve the emphasis the dollar column usually steals. First, wall clock. The local model’s ledger dollars are $0, but its big delegation single-handedly turned a 9-minute build into a 41-minute one. Wall clock is the real local price, and it lands on the only line a human operator actually feels. Second, the propensity collapse. On the familiar CRUD seed, both arms delegated 3–5 files. On the MCP seed, the all-cloud arm delegated one file and the hybrid arm delegated nothing. The same configuration that had cheerfully routed CRUD code to its local worker wrote every line of the protocol-shaped project itself.¹ Unprompted, the frontier orchestrator rations delegation by task familiarity. So the “hybrid” arm was behaviorally all-cloud exactly where local savings were supposed to be tested hardest. An honest result, and a warning: don’t assume your routing policy describes your system’s actual behavior.

Where local delegation actually pays

None of this says small models can’t do the work. The prior art on that point is real. NVIDIA researchers argued in 2025 that small language models are the natural workhorses of agentic systems, with “heterogeneous agentic systems” (frontier orchestrators invoking smaller workers) as the transition path.⁴ aider’s architect/editor split is the canonical demonstration that a cheap model in a well-shaped mechanical role holds up. Most famously, it paired DeepSeek R1 with Claude Sonnet to set a benchmark record at 14× lower cost than the o1 result it beat.⁵⁶ Anthropic’s own agent-patterns writeup names routing and orchestrator-workers as first-class designs.⁷ The routing literature promises savings up to 98% (FrugalGPT) and “over 2 times” (RouteLLM).⁸⁹ The counterweight comes from the routing benchmark itself: RouterBench found that none of the routers it tested significantly beat its non-predictive baseline.¹⁰ That’s consistent, I’d note, with what the variance runs above said. The routing decision is harder than the routing pitch.

What’s missing from that literature is the gap this experiment starts to fill: metered end-to-end economics of the frontier-orchestrator/local-worker loop. Not “can the small model pass the eval,” but “what does the whole delegation transaction cost, verification and repairs included.” There’s no first-party path for it in the major coding agents. Routing subagents to local models is still an open feature request on Claude Code as of March 2026.¹¹ So everyone doing it is doing it with duct tape, and nobody publishes their ledgers.

So where does it pay? Three places, in my view. None of them per-run dollars at this project size.

Volume. The M1 Max drew about 57 watts of package power under sustained GPU load in AnandTech’s instrumented testing.¹² Electricity is a rounding error, so local inference is genuinely ~$0 marginal, and what you’re amortizing is hardware you may already own. At enterprise scale (Anthropic’s own figure is roughly $13 per developer per active day¹³), a local worker that reliably absorbs the fresh-input slice of high-volume mechanical tasks can compound into real money. The operative word is reliably. Every repair round eats the margin, so the delegation frontier is a volume play only for task classes your ledgers show coming back clean.

Caps, not dollars. On subscription plans, marginal API cost is $0 until you hit usage limits shared across everything on the account.¹⁴ Local delegation there buys cap headroom. That’s an entirely legitimate purchase, and no per-run dollar comparison will ever show it.

Sovereignty. Data residency, availability during provider incidents, work that can’t leave the building. These were the original reasons to run local. They remain good ones. But they buy properties, not savings, and honest accounting should say so.

There’s a tuning result worth reporting too. The failure data said delegated chunk size drives the silent no-op class. So I pre-registered a v3 policy before the final results were in: cap every delegated chunk at ~300 output lines, and delegate core files instead of hoarding them. Then I ran it once as a labeled contrast. The mechanics did exactly what they were designed to do. Every chunk stayed small (largest 5.7k output tokens, against the 60k monolith that had drifted in run one), all core files went to the local worker, and no delegation silently no-op’d. The run passed 16/16 at $1.19, mid-range on dollars, 21 minutes on the wall.¹ What the chunk cap did not buy is freedom from repairs. The delegated app.ts still came back with three real bugs, including that same money-representation drift. My read: chunk caps are insurance against the no-op tail, not against contract drift. Drift still needs a checker, and the checker still bills in frontier tokens.

The frontier is a trust boundary

Everyone benchmarks models. Almost nobody meters delegation. What this experiment measured is the part the benchmarks skip: specifying the task, verifying the result, and repairing the drift. All of it denominated in frontier tokens, because the expensive model is the one doing the watching. I’ve taken to calling the line where delegation starts paying the delegation frontier, a term I haven’t found claimed elsewhere. The point of the name is that the line isn’t set by the small model’s capability. It’s set by the cost of trust. A worker you have to double-check is a worker whose true rate includes the checker’s time, and at small scale these ledgers price that above the work itself.

My read on what survives: local workers for high-volume, well-shaped, low-trust-cost task classes. And measurement as a habit, because every surprising number in this piece came off a ledger nobody usually keeps. What fails is the reflex that local equals savings, along with the broader unmetered enthusiasm for multi-agent architectures whose coordination overhead nobody has priced. Run local models for reasons you can name: caps, residency, volume you’ve actually measured. And meter the delegation, because the cheapest model is the one you don’t have to double-check, and today that price is still set in frontier tokens.

Notes