I ran the same multi-agent research task through a 70B model and a 14B model on the same box last spring. The 14B won. Not on speed — on the actual answer. Cleaner tool calls, fewer hallucinated URLs, a final summary that didn't contradict itself halfway through.
That should not have happened. The 70B has more parameters, scored higher on every leaderboard I'd looked at, and ate twice the VRAM proving it. But the variable that moved was never the weights. It was the local-llm-orchestration layer wrapped around them — the harness that decides what context each call sees, how tool results get fed back, and what happens when a call returns garbage. If you run vLLM or Ollama and you're stitching agents together, this is the lesson nobody writes down: you are tuning the harness, and you think you're tuning the model.
What you actually control
There are three layers between your hardware and a finished answer, and they are not equally yours to change.
The model is mostly fixed. You picked it, you quantized it, you can swap it — but you don't move the needle inside it. You rent its behavior.
The prompt is yours, and everyone obsesses over it. It's the layer with a thousand blog posts.
The harness — the thing that loops, routes, stores memory, parses tool calls, and decides what the next call even sees — is also yours, and almost nobody treats it as a tunable surface. It's usually a pile of Python that someone wrote in an afternoon and never measured. That's the layer that beat my 70B with a 14B. The harness fed the smaller model a tighter, better-ordered context and refused to let a malformed tool call poison the next turn. The big model, run through a sloppier loop, kept tripping over its own history.
When people say "the model isn't good enough for agents," they are very often describing a harness that isn't good enough for the model.
The test that exposes it: web_search then web_fetch
Here's the cheapest way to see this for yourself. Build a two-step chain: the agent calls web_search, gets a list of results, then calls web_fetch on the most relevant URL and summarizes.
user: What changed in the latest release of <some library>?
→ web_search("library release notes")
→ [10 results returned to the model]
→ web_fetch("https://...")
→ [page text returned to the model]
→ final summary
This chain breaks in places that have nothing to do with model intelligence:
- The search returns 10 results, your harness dumps all 10 raw into context, and now the model is reasoning over 6,000 tokens of SEO sludge to pick one link.
- The model emits a tool call in a slightly different JSON shape than your parser expects, your parser throws, and your retry logic silently re-asks the same question with the error appended — so the model now thinks it failed and apologizes instead of fetching.
- The fetched page is 40k tokens, you blow past your KV-cache headroom, and the server truncates from the front, quietly deleting the original question.
None of those are model failures. Every one is a harness decision. Swap the model and they all still happen.
Three ways to orchestrate, judged honestly
I've shipped all three of these. Here's how they compare on the criteria that actually bite you later.
| Criterion | Raw Python loop | Heavyweight framework | Declarative config (YAML) |
|---|---|---|---|
| Transparency of context | Total — you wrote every line | Low — hidden prompt templating | High — context assembly is in one file |
| Debuggability when output degrades | High, if you logged | Painful — stack traces through 9 abstraction layers | Medium — you read the config, not the call stack |
| Vendor / framework lock-in | None | High — your logic lives in their abstractions | Low — config ports across runtimes |
| Model-swap cost | Manual edits everywhere | Often re-templated per provider | Change one field |
| Time to first working agent | Slow | Fast | Medium |
The framework wins the demo and loses the month. The moment your output quality drops, you need to see exactly what the model saw on the failing turn — and a framework that hides prompt assembly behind provider adapters turns a five-minute fix into an afternoon of print-debugging through code you didn't write. The raw loop gives you total visibility but you rebuild memory, retries, and tool parsing every time, and you'll do it slightly wrong each time.
The verdict that emerged for me, after enough 2 a.m. sessions: keep the orchestration logic declarative and external, keep the execution boring and inspectable, and never let a tool integration live somewhere you can't read it. The harness should be a thing you can audit, not a thing you trust.
How orchestration quietly degrades the answer
The failures are rarely loud. They look like this:
Context order. Models weight recent tokens heavily. If your harness appends tool results after the instruction, the model anchors on the data and drifts from the task. Reorder it and the same model gets sharper. Same weights, different answer.
Retry loops that lie. A failed parse re-prompted with the raw error teaches the model it's in a failure state. It starts hedging. Strip the error, re-issue clean, and quality returns.
Memory bleed. Shared scratch memory across agents feels efficient until one agent's half-finished reasoning contaminates another's context. Scope memory per agent and the contradictions stop.
You will not catch any of this on a benchmark number. You catch it by logging the exact final assembled context for every call and reading it like a transcript.
Do this tonight
Before you swap models again, instrument what you already have:
- Log the full assembled prompt sent on every call — not your template, the rendered string.
- Log every tool call before parsing and the parser's verdict, so format mismatches surface.
- Add a token counter at the point of send; alert when you cross ~70% of your KV-cache budget.
- On any failed turn, save the input/output pair to a flat file you can grep.
- Run the
web_search → web_fetchchain and read every transcript by hand once.
That's a couple hours of work and it will tell you more than a week of model shopping.
A caveat, because the honest version matters: none of this makes a 7B model do what a 70B does, and a great harness on starved hardware — single-digit tokens per second, no cache headroom — is still going to feel like wading through mud. The harness is leverage, not magic. It widens the gap between a model used well and the same model used badly, and that gap is usually larger than the gap between two models.
If your agent output is bad tonight, read the last prompt you actually sent before you blame the weights that received it.
