Exception handling: the 500 the agent hid from the user

An account agent has two tools: cancel_subscription and send_email. A customer asked it to cancel and confirm. The cancel call returned HTTP 500 with the body "upstream timeout". The agent then called send_email with a confirmation, replied "Done", and ended with status success. Thirty days later the card was charged again. The customer had an email from the company saying it would not be.

Nothing in the run was flagged. The 500 was in the trace, in plain sight, at line 3. The run was marked success because the last step succeeded, and the last step was sending an email.

Line 4 is the mistake. The agent took an error as if it were a result and carried on. Three things let that happen. The tool returned the error as text in the same channel as a good result, so to the model it was just another string. The prompt had no rule about errors, so the model had to decide, and a model asked to cancel and confirm will tend to confirm. And the confirmation email depended on nothing: it did not read the subscription state, it repeated what the plan said should be true.

A 500 is a specific kind of error. It says the server failed, not that the request was wrong. The cancel may have gone through before the timeout, or it may not have. The agent had no way to know, and it did not try. Compare a 404 or a 422: those say the request itself was wrong, and retrying the same call will fail the same way. Handling them all the same is as bad as ignoring them.

Type the errors. A tool result is either a value or an error with a class: retryable (500, 502, 503, 504, timeouts), not retryable (400, 404, 422), or needs a person (403, anything about money). The loop handles the class before the model sees anything. Retryable errors get retried with exponential backoff and jitter, the approach AWS wrote up in 2015, with a cap of a few attempts and respect for a Retry-After header when the server sends one. A retried write needs an idempotency key, the pattern Stripe's API uses, so a cancel that did go through before the timeout is not applied twice and a retry is safe.

When the retries run out, stop. Do not let a later step run on an unverified state. In this run that means no email until a get_subscription call returns status cancelled. Read after write is the rule for any side effect that matters: confirm from the system of record, not from the plan. If the state cannot be confirmed, the reply to the customer is honest and short: "I could not complete the cancellation. I have opened a ticket and you will hear back within one business day." Then open the ticket. The customer's trust survives a failed call. It does not survive a false confirmation.

Protect the rest of the system too. If the billing service is returning 500s, the fiftieth retry is not going to help, and it adds load to a service that is already down. A circuit breaker, the Azure architecture pattern, opens after a run of failures and fails fast for a while. Google's SRE book makes the same point about retries: clients that keep retrying an overloaded service make the overload worse, so retries need a budget. During that time the agent tells customers the truth and queues the work.

Finally, make the run's status mean something. A run with an unhandled error inside it is not a success, whatever the last step did. Mark it failed, log the error class, and alert on the rate of replies sent after an error. The recall question in the tool use challenge is this exact rule: surface the error or retry with corrected input, never answer as if the call succeeded.

"Line 4. The cancel returned a 500 and the agent sent the confirmation anyway, because the error came back as plain text with no rule attached. I would classify tool errors in the loop, retry 5xx with backoff and an idempotency key, stop when retries run out, verify the subscription state before any confirmation, and tell the customer plainly when it failed." If they push on retries, say which codes you retry and which you do not, and mention the circuit breaker. That is the difference between knowing the word and having run the service.