The Catalogue Tax: Why I Still Hand My Agents a CLI

I did an uncomfortable audit this week: I counted the MCP servers in my Claude Code config. Browser automation, web scraping, Cloudflare, documentation search, my own Composer platform — the list has grown the way browser tabs grow, one reasonable decision at a time.

Meanwhile my actual working habit points the other way. When I want an agent to do something real, I hand it a command-line tool — gh, gcloud, even DoorDash's agent CLI when breakfast was on the line. The MCP servers sit in config; the CLIs do the work.

I've defended that habit on instinct. Now there's data. A group of researchers at three Spanish universities just published The Scaffolding Matters More Than the Interface — the first controlled comparison of MCP versus CLI tool use I've seen that actually isolates the variables: seven agent harnesses, five models, one fixed GitHub task, every run verified by inspecting the repository instead of trusting the agent's word for it.

The numbers back the instinct. They also complicate it, in ways worth being honest about.

A Bill You Pay Every Turn

Start with the mechanism, because everything else follows from it.

A language model retains nothing between requests. Every turn of an agent conversation re-sends the whole history — plus, if you've attached an MCP server, the entire tool catalogue. Every schema, every parameter description, every turn. A task that takes twenty exchanges transmits that catalogue twenty times.

A CLI takes the opposite bet. The harness sends one schema — here is a shell — and maybe a line noting that git and gh are installed. Everything else rides on what the model already knows about command-line tools, which is a lot, because decades of man pages, tutorials, and Stack Overflow answers are baked into its weights. The knowledge was paid for once, at training time. The MCP catalogue is paid for again on every request.

That's the theory. The study measured what it costs in practice.

What MCP Costs in the Harnesses We Actually Use

The researchers attached the official GitHub MCP server — 44 tool schemas covering issues, branches, commits, and pull requests — to five popular harnesses, and routed every request through a proxy that recorded exactly what got sent.

Four of the five, Claude Code among them at the version tested, shipped all 44 schemas with every single request. Claude Code actually sent 74 schemas per request on that arm once you count its own built-in tools. The one exception was Hermes, which sends just seven: a gateway pair that can list and invoke tools by name, with the other 42 descriptions fetched only when the model asks.

Same MCP server. Same 44 operations available. The median completed run cost 216,986 input tokens under eager delivery and 70,836 under lazy delivery — a factor of 3.1, from nothing but when the schemas get transmitted.

Zoom out to the whole experiment and the direction holds: across all completed runs on the MCP-capable harnesses, the MCP arm consumed roughly twice the input tokens of the CLI arm — 187,954 versus 119,916 median. Prompt caching softens the bill (providers re-bill repeated context at a discount), but even after applying a 90% cache discount the MCP arm still cost about 20% more.

Two details from the study sharpen the picture. First, output tokens were nearly constant across every configuration — 824 to 1,367 — while input varied by a factor of twenty. The task demands the same amount of generation no matter what; what varies wildly is how much context the harness makes the model carry to produce it. Second, that input is more than a line item on the bill. It's prefill — text the model must chew through before it says a word, on every turn. Fatter requests are slower requests, cache or no cache.

The tax isn't the protocol. The tax is the catalogue riding along on every exchange — and most harnesses, today, let it ride.

The CLI Alternative Costs One Line

Here's the same unit of work, both ways. The CLI version:

gh pr create --title "Fix flaky retry test" --body "Closes #42"

One schema in context (the shell), one line of output from the model, knowledge from the weights.

The MCP version invokes create_pull_request — a perfectly good tool. To make that call, the eager harnesses first placed the schemas for create_pull_request and its 43 siblings into the request:

{
  "name": "create_pull_request",
  "description": "Open a new pull request...",
  "inputSchema": {
    "properties": {
      "owner": {...}, "repo": {...}, "title": {...},
      "body": {...}, "head": {...}, "base": {...},
      "draft": {...}, "maintainer_can_modify": {...}
    }
  }
}

…times 44, times every turn of the conversation, whether the turn touches GitHub or not.

Did the agents actually need the catalogue? No. The study's cleanest result: two of the seven harnesses ship no MCP client at all — a shell, a few file tools, and gh is all their agents ever get — and they completed every run in the main experiment. Nothing about this class of work required MCP. The same operations were sitting behind a CLI the whole time, pre-learned and free.

Failure Is Where MCP Really Bleeds

My favorite finding in the paper is the one about failure, because it's the one nobody advertises.

Both arms failed at the same rate — three runs of nineteen each. The failures were not the same price. The CLI arm's three failures happened on cheap configurations and totaled six cents. The MCP arm's three landed on expensive ones and totaled $0.41. Add it up and 12.9% of every dollar spent on the MCP arm bought no completed work, versus 2.2% on the CLI arm.

The general shape is worse than the split. The median failed run in the study consumed 170,937 input tokens against 83,600 for the median completed one. An agent that fails doesn't stop early and save you money — it works hard, spends twice what a successful run would, and delivers nothing. When each attempt is carrying a 44-schema catalogue on every turn, the flailing is simply priced higher.

The only rows in the study with zero wasted spend? The two shell-only harnesses. They never failed at all.

The Harness Multiplies Everything

The catalogue tax is per-turn, which means the harness driving the conversation multiplies it. A chatty harness that burns turns re-reading context pays the tax over and over; a lean one pays it a handful of times. This is where the study's headline result lives, and it's the part that made me wince.

Same task, same models, command-line arm only — no MCP server attached to anything:

The thinnest harness finished the job in 14,660 tokens. The heaviest — Claude Code, the one writing this very post — took 410,797. A 28× spread with the interface held constant. The five general-purpose harnesses carry the machinery of open-ended work into every task, including tasks that need almost none of it, and that generality tax dwarfs anything the interface does.

The extreme case: a 27-billion-parameter open-weight model running on a single local workstation completed the whole task in 17,416 tokens under a thin harness. The same model, same task, under a heavy harness with the MCP server attached: 2,418,828 tokens — 139× more, most of it spent re-reading a catalogue while it flailed through 108 tool calls. The model was never the constraint. The software wrapped around it was.

One more thing the paper demonstrates almost by accident. The researchers drove all seven harnesses from a single shell script — reset fixture, invoke agent, verify repo. That orchestration was only possible because every harness exposes a CLI entry point. There is no equivalent way to script seven agents through their MCP catalogues. The composability that lets an agent be piped, scripted, retried, and put in a cron job is the same composability that lets an agent use ordinary programs. That property doesn't show up in a token count, and it's worth more than the token counts.

Fair Play: What This Doesn't Prove

I promised honesty about the complications, so here they are.

Within a single harness, the paired MCP-versus-CLI ratios were all over the map — 0.43× to 29× — and in several harnesses the MCP arm was actually cheaper than the CLI arm. The authors flat-out call the within-harness comparison inconclusive. The "MCP is 35× more expensive" figure that's been making the rounds isn't a property of the protocol; it was somebody's harness, somebody's catalogue, somebody's task. Retire that number.

The fix is also already shipping. Recent Claude Code versions defer MCP tool schemas behind an on-demand search — my sessions now start with a list of tool names and fetch full schemas only when needed, which is the Hermes trick arriving in the mainstream. Anthropic's own engineering work points the same direction: stop presenting every tool definition up front. The lazy-delivery comparison in the paper rests on a single harness, so treat the 3.1× as a strong hint rather than settled law — but the protocol's stewards are clearly betting the same way.

Where does MCP genuinely earn its keep? Services with no mature CLI, and agents that can't know their toolset in advance. A discoverable, standardized catalogue is exactly the right design when the agent might meet anything. My argument was never MCP bad. It's narrower: when a good CLI already exists, the catalogue describes work the agent already knows how to do — and you're paying to re-teach it every turn.

The Trade I'm Making

The CLI path isn't free, so let me price my side of the trade honestly.

CLIs mean maintenance. I install the tools, keep the auth fresh, absorb the occasional breaking flag change, and sometimes wrap ugly output so an agent can parse it. That's real work — paid in minutes, occasionally, by me. The catalogue tax is paid in tokens and latency, on every turn, forever, multiplied by however many agents I run. I know which subscription I'd rather hold.

So the working rules, post-paper:

Mature CLI exists → use it. The agent already knows gh, git, gcloud, aws. Free knowledge beats retransmitted knowledge.

No CLI → MCP, lazily. Some surfaces have no command-line story, and MCP is the right answer there. Check what your harness actually transmits per request. Eager catalogue delivery is a harness defect you should refuse to pay for in 2026.

Repeatable job → thin harness. For pipeline work — scheduled tasks, CI fixes, the stuff Composer dispatches all day — a general-purpose assistant is the wrong tool at any interface. The study's 5–28× gap between thin and heavy harnesses is the biggest number in the whole paper, and it's sitting there for anyone willing to build small.

Audit your own config the way the researchers audited harnesses. Count your MCP servers. Ask which catalogues are riding along on every session doing nothing. Mine didn't survive the audit intact.

The tokens are the bill. The maintenance is the subscription. I know which one I'd rather pay.

–Jeremy