Multi-Agent Systems

Pradeep P
3 days ago
4 min read

Series: Modern System Design · Layer 6 — Modern systems

Layer 6 · Post 24 of 26

← Previous: Tool Calling → Next: AI Observability

Layer 6 — Modern systems · Post 86 of 88

Multi-agent systems split work across specialized agents that coordinate. The gain is modularity. The risk is chaos, cost, and loops.

What you'll learn

When multiple agents help (specialization, isolation) vs when one loop is enough
Orchestrator, handoff, and blackboard patterns you can actually operate
How cost, latency, and infinite debate show up as system-design failures

The idea in one minute

A multi-agent design is several agent loops (or roles) that hand work to each other instead of one model with twenty tools.

Each box has its own prompt, tools, and memory slice. Something you wrote still owns the graph: who may speak, when to stop, what is shared.

This is microservices for prompts. You get isolation. You also get distributed-systems bugs: extra hops, inconsistent state, retries that fork reality.

Why it matters

One mega-prompt with every tool is hard to eval and easy to jailbreak into the wrong capability. Splitting "search the corpus" from "send email" means the researcher cannot email. That is a security and quality story, not a demo aesthetic.

The bill is multiplicative. Three agents chatting for eight rounds is 24 model calls. Interviewers will ask why two agents, not how many logos you used.

How it works

Patterns that work

Orchestrator (supervisor). A router agent picks a specialist or a workflow step. Specialists do not talk in a free-for-all. The supervisor merges results and talks to the user. This is the default. It is a state machine with LLM steps.

Handoff. Agent A finishes and passes a structured payload (not a novel) to agent B. A is done. No debate.

Blackboard / shared store. Agents read and write a task document (findings, open questions). A scheduler decides who runs next. Useful for research; needs a lock or you get lost updates.

Verifier. A second model checks the first: citations exist, JSON validates, policy holds. Narrow, cheap. This is often the highest-ROI "second agent."

What you share

Share typed artifacts: Claim[], Source[], Plan. Do not pipe full chain-of-thought between agents as if it were an API. You will copy hallucinations with high fidelity.

Each agent gets only the tools it needs. The writer does not get refund. The refund agent does not get browse_web.

Coordination rules

A global step budget for the whole graph, not only per agent.
No cycles unless you cap them (critic → writer → critic at most twice).
Deterministic merge: if two specialists disagree, a rule or a verifier decides — not an unbounded argument.
Identity: logs must say which agent spent tokens (post 87).

When not to

If the task is "retrieve then answer," RAG is enough. If the steps are known, use a workflow engine. Multi-agent is for different permissions, models, or evals — not a swarm for its own sake.

A simple example

A vendor RFP assistant:

Orchestrator splits: "extract requirements," "search our docs," "draft response."
Extractor (small model, no network) returns a JSON list of must-haves.
Researcher (RAG tools only) attaches source_ids.
Writer (no search) drafts from that packet.
Verifier checks every MUST has a citation. Fail → writer once more, then stop with gaps listed.

The user sees one answer plus sources. Cost is bounded: extractor cheap, researcher medium, writer once or twice, verifier cheap. A brainstorming "swarm" of eight would 10× tokens and still miss a MUST.

Common mistakes

Chat as the bus. Agents posting prose at each other. You cannot test it. Pass schemas.

Too many roles. Researcher, planner, critic, ethicist, poet. You added latency and a new failure mode (they politely agree on a wrong fact).

Shared god tools. Isolation was the point.

No orchestrator. A fully connected mesh of agents is a meeting. Meetings do not have SLOs.

Forgetting the user. Multi-agent can spend 90 seconds "aligning." Stream a status, or do not split the path.

How this shows up in real systems

Customer support: triage agent vs billing agent vs policy agent, often just routed tools behind one loop.
CrewAI / AutoGen / LangGraph multi-node: frameworks; you still design the graph.
Human + model: the most important extra "agent" is often a reviewer in the UI.
MoE / mixture of models is routing (post 78), not a multi-agent product.

Tool calling (previous) is the handshake with the world. Multi-agent is the handshake between policies. You will not operate either without observability.

Recap

Split agents for isolation and eval, not for theater; default to an orchestrator and typed handoffs.
Cap the graph, forbid unbounded debate, share artifacts not novels.
A verifier agent is often enough; a swarm rarely is.

If you cannot see which agent spent tokens and why the answer is wrong, you cannot ship this. That is AI observability.

Series: Modern System Design · Layer 6 — Modern systems

Layer 6 · Post 24 of 26

← Previous: Tool Calling → Next: AI Observability