Tool Calling Patterns That Don't Fall Apart in Production

Tool calling breaks in production for boring reasons: schemas the model can't fill reliably, retries that double-charge a customer, parallel calls whose results get dropped or misordered. The model is rarely the weak link in 2026 — the plumbing is. The fixes are correspondingly old-fashioned: flat schemas with enums, idempotency keys derived from the call ID, every result returned (including the failures), and error messages written for the model to act on rather than for a human to read in a stack trace.

Schema design: flat, boring, described

Models fill flat schemas far more reliably than nested ones. Every level of nesting, every oneOf, every clever polymorphic shape raises the malformed-call rate. Concrete rules that have held up:

• One level deep where possible. {"city": ..., "start_date": ..., "end_date": ...} beats {"location": {"city": ...}, "range": {"start": ...}}.
• Enums over free strings for anything with a closed set. A JSON Schema enum on a status field eliminates a whole class of creative inputs.
• Descriptions say when to call, not just what it does. "Look up an order by ID. Call this whenever the user references a specific order, before answering anything about it" measurably improves call rates over "Fetches order data." Current models are conservative about reaching for tools; the trigger condition belongs in the description.
• Use strict/constrained modes where the provider offers them — schema-enforced tool arguments remove malformed JSON entirely, leaving you only the semantic errors. The same machinery as structured outputs generally, and worth the setup.
• Cap the tool count. Past roughly 20 tools in context, selection accuracy degrades and token overhead grows. Consolidate near-duplicate tools, and past that, use deferred loading / tool search rather than cramming schemas in.

Parallel calls: all results, one message

Models emit multiple tool calls in a single turn, and your harness should execute them concurrently — that's the free latency win. The contract on the way back is where implementations quietly go wrong: return every result, in one message, matched by call ID. Two specific bugs to avoid:

• Splitting results across multiple messages. It works, but it teaches the model that calls resolve one at a time, and it will stop parallelising — a silent regression you won't notice for weeks.
• Dropping a failed call. If one of four calls fails, the model needs three results and one error, not three results.

{
  "role": "user",
  "content": [
    {"type": "tool_result", "tool_use_id": "call_1",
     "content": "{\"status\": \"created\", \"invoice\": \"inv_812\"}"},
    {"type": "tool_result", "tool_use_id": "call_2",
     "content": "Error: 'due' must be ISO 8601 (YYYY-MM-DD); got '3rd of May'",
     "is_error": true}
  ]
}

Only parallelise what's actually parallel-safe: mark read-only tools as such in your harness and serialise anything with side effects. The model doesn't know your consistency requirements; the harness has to.

Idempotency: assume every call happens twice

Between network retries, harness restarts, and models occasionally emitting the same call twice in one loop, a side-effectful tool will be invoked more than once with the same intent. The standard fix transfers directly: derive an idempotency key from the tool-call ID, which is unique per emitted call but stable across retries of that call.

def create_invoice(args: dict, tool_call_id: str) -> str:
    return billing.invoices.create(
        **args,
        idempotency_key=f"llm-{tool_call_id}",
    )

For internal tools without idempotency support, fake it: a table keyed by call ID storing the first result, returned verbatim on replays. Reads are free; every write needs a key. This is the difference between "the agent retried" being a log line versus a refund.

Errors the model can act on

A tool that raises an exception gives the harness a stack trace the model never benefits from. Return errors as results with is_error set, and write them like validation messages: state the field, the constraint, and the received value — the invoice example above is the shape. Given that, current models repair the call on the next turn most of the time; given KeyError: 'due', they flail.

Bound the repair loop: two retries per tool call, then surface the failure. And set per-tool timeouts — a hung tool call stalls the entire agent turn, which is far more user-visible than a fast failure. Track repair rate per tool in your tracing setup; a tool whose calls need repair more than ~5% of the time has a schema or description problem, and that metric finds it before your users do.

Partial failure is a UX problem, not just a retry problem

A five-step agent action that dies at step four should not present as a generic failure. Patterns that work:

• Stream tool activity. Show which tool is running as it runs; users tolerate 20-second agent turns when they can see progress, and the trace doubles as an explanation when something breaks.
• Report completed work distinctly from failed work. "Created the issue and assigned it; couldn't post the Slack notification (channel not found)" — the model writes this summary naturally if your tool results carry the information.
• Never silently re-run side-effectful sequences from the top. Resume from the failed step using the idempotency layer, or hand control back to the user.

The failure catalogue

Bottom line

Design tools like a public API for a fast, literal-minded junior developer: flat inputs, explicit contracts, idempotent writes, errors that state what to change. Keep the failure catalogue above as a review checklist for every new tool, and wire repair-rate and timeout metrics in from day one. Get the plumbing right and the same tool definitions will survive every model swap and every architecture change you make above them.