Multi agent collaboration: the two agents that agreed on the wrong answer

A support team is two agents and an orchestrator. The writer drafts an answer from the help pages. The reviewer checks the draft. The orchestrator runs both and sends the reply when they agree. On Thursday a customer asked how long the refund window is on the Pro plan. The writer searched, found the 2024 policy page that says 30 days, and missed the March 2026 changelog that cut it to 14 days for Pro. The orchestrator passed the draft to the reviewer. The reviewer wrote: "Checked. 30 days is correct." The reply went out. The run ended with status success and the log said agreement 2 of 2.

The customer asked for a refund on day 20 and was refused. Two agents had agreed, and both were wrong. The trace below is eight lines.

Look at line 5. The orchestrator gave the reviewer the draft and nothing else. No question, no sources, no changelog. A reviewer that sees only a conclusion can do one of two things: agree with it, or guess. It agreed, in confident words, because that is what a model does with a plausible sentence and no evidence.

The deeper problem is that the two agents were not independent. Same model, same help pages, and the second one read the first one's answer. Their errors were correlated, so a second vote added almost no information. The Berkeley study "Why do multi agent LLM systems fail?" (Cemri and others, March 2025) looked at seven frameworks over more than 200 tasks and sorted the failures into 14 modes in three groups: specification problems, inter agent misalignment, and weak task verification. This run is the third group. The verifier existed, and it verified nothing.

Give the reviewer the question and the sources, not the draft. Make it derive its own answer, then compare the two answers in code. If they match, send. If they differ, the run stops and a person reads both. Disagreement is the signal you built the second agent to produce, so do not hide the evidence from it.

Then ask whether the second agent is needed at all. The check that would have caught this is not a model: the two pages have dates, and the newer one wins. A one line rule beats a second opinion that costs a full model call. Anthropic's post on its research system (June 2025) is the best public account of a team that does earn its cost: a lead agent on Claude Opus 4 with subagents on Claude Sonnet 4 beat a single Opus 4 agent by 90.2 percent on their internal research eval. The same post says the win came from breadth, many subagents searching different things at once, and that domains where every agent needs the same context, or where the steps depend on each other, are not a good fit. A refund question is one lookup. It did not need a team.

When a team is the right call, write each subagent a job description: the objective, the output format, the tools it may use, and where its job ends. Vague tasks make subagents duplicate work or wander. The same post describes early versions spawning 50 subagents for simple queries.

When the round is "single agent or a team, defend it", start with the shape of the work. A team earns its cost when the work splits into parts that can run at the same time and do not need each other's context. It costs more tokens, adds more things that can break, and the agents can agree on a wrong answer. If an interviewer shows you a run like this one: "Line 5. The reviewer got the conclusion and no evidence, so its agreement meant nothing. I would give it the sources, compare the two answers in code, and route disagreement to a person. And I would ask whether a date check makes the reviewer unnecessary."