When an agent is the wrong answer: the script that should have been a script

Every night a job takes the day's orders as a CSV export, renames six columns, converts the date column to ISO format, and drops the file where the invoicing system picks it up. The team built it as an agent: a prompt, a file tool, a model call per run. It worked for months. Then one night the export had dates in two formats, the model decided the whole column should match the one it recognised, and it rewrote every date in US order. The invoicing import accepted the file and billed every order on the wrong day. Nobody noticed for three runs, because each run ended with status success.

The same job as a script is a short file that does exactly the same thing every night. When it meets a date it cannot parse, it throws, and someone looks.

Three properties were traded away and nothing was gained. Cost: the model call runs every night whether the input needs judgment or not, and on Anthropic's numbers agents use about 4 times the tokens of a chat, teams about 15 times. Determinism: the same input can produce a different output on different nights, so you cannot replay a run and get the same answer, and you cannot write a test that pins the output. Debuggability: when the script fails you get a line number and a stack trace; when the agent fails you get a plausible paragraph and a success status.

Anthropic's "Building effective agents" (December 2024) says it in one sentence: find the simplest solution possible, and only increase complexity when needed. The post reserves agents for open ended problems where the number of steps cannot be predicted and a fixed path cannot be hardcoded. A column rename has a fixed path. The job needed an agent for nothing, and paid for it in every property that matters in production.

The field is not unanimous about where the line sits, and better models keep moving it. But a deterministic job should not pay a model to be nondeterministic.

Before you build an agent, answer five questions. Any no is a reason to write code instead. One: is the number of steps unknown before the run starts? If you can write the steps down, write them down as code. Two: does the input vary in ways a rule cannot cover? Fixed shape, fixed rule. Three: does the task need a judgment a rule cannot make? Renaming columns is not judgment. Deciding whether a support ticket is angry is. Four: can a wrong output be caught cheaply? If not, the cost of the model being wrong is yours. Five: is the cost per run justified by what the output is worth, when it runs every night?

If the answer is yes to all five, build an agent, and start with the simplest shape: one model call with a tool, then a workflow with fixed steps and gates, and only then a loop that decides its own steps.

The second question in this round is about the loop's shape when you do need one. ReAct (Yao and others, 2022) interleaves a thought, an action and an observation, one step at a time; each tool result changes the next decision. Plan and execute, in the style of Plan-and-Solve (Wang and others, 2023) and the orchestrator worker pattern, writes the whole plan first and then runs it, with the plan visible before anything costs money. Choose ReAct when each result changes what to do next: debugging, open ended search. Choose plan and execute when the steps are knowable up front, when a person or a check should approve the plan before tools run, and when parallel workers can take the steps. Often the honest answer is both: plan at the top, then a short ReAct loop inside each step with a tight budget.

Say the properties by name: cost, determinism, debuggability. "This is a fixed transformation. I would write it as a script with a schema check on input and output, and put a model only at the point where the input is unpredictable, for example to classify an unknown date format and ask a person." For the loop question: "Plan and execute, because the steps are known and I want to check the plan before spending. ReAct inside a step only if the result of one tool call changes the next." An interviewer asking this wants to hear you decline an agent when one is not needed. No is the senior answer.