Most of the attention in agentic AI goes to an agent’s reasoning capabilities. The model can figure out the right plan, break a goal into steps and decide what to do next. Far less attention is given to the last step of that process: actually executing the action the model decided on. This “last mile” is often taken for granted, despite being the step that can cause agentic systems to quietly break in production.
What tool calling actually unlocks
Tool calls give an agent hands. It is the difference between a model that can only describe what it would do and one that actually does it, whether querying a database, filing a ticket, running a calculation or calling another service, then coming back with a real result instead of a guess. For an agent, tool calling is the mechanism through which reasoning becomes action.
That also makes it prone to failure. Each step’s output feeds the next step’s input. An agent that plans five steps but can reliably execute only three of them may fail the task even if its reasoning was sound.
The formatting problem
Models can call tools in different ways. If the tool call is not formatted in the shape a parser expects, the response can fall apart.
Some models use specific tags or tokens to mark the call, while others require a particular JSON shape and field names. A parser has to know which shape a given model is going to produce. This becomes harder when a model mixes reasoning text with the tool call, placing the part needed for execution next to text that is not meant to be parsed as instructions.
Fig. 1: XML-style tags wrapping JSON, plain JSON, and function-style tool calls:
# Model A: XML-style tag wrapping a JSON payload
<tool_call>{"name": "get_weather", "arguments": {"city": "Boston"}}</tool_call>
# Model B: plain JSON, different field name for the same concept
{"tool_calls": [{"function": {"name": "get_weather", "parameters": {"city": "Boston"}}}]}
# Model C: a special token followed by a function-call-style string
[TOOL_CALL] get_weather(city="Boston")
Why this hits agentic systems harder than chat
In a single chat exchange, a badly parsed tool call is one bad answer. In an agent working through a multistep task, it is the first domino. The next step’s plan depends on what the previous step actually did. It does not matter what the model intended to do, only how the tool call was parsed and executed.
Get one step wrong along the way and everything downstream can reason from a world that no longer matches reality, with no obvious signal that anything went wrong.
That compounding is the real last-mile problem. Common failure modes include:
- Incorrect tag style: A parser tuned for one model’s tag style can miss a tool call from a model that does not use those tags. The call never fires and the agent’s plan silently loses a step.
- Mismatched labels: A parser expecting the label “arguments” may receive “parameters” instead. It may pass an empty or incorrect set of values, so the tool runs with the wrong input and the agent proceeds as if the step succeeded.
- Varying call loads: A parser built to expect one tool call per response can grab the first call in a multicall response and silently drop the rest.
- Reasoning mixed with the call: If a parser does not draw the boundary correctly, it can extract arguments from surrounding reasoning text instead of the actual call.
An agent does not have a human reading over its shoulder at every step and checking that each tool call did the right thing before moving on. That is part of the point of an agent. It is also why a silent failure at the last mile can be more expensive here than in an ordinary chat interaction.
Why this keeps breaking silently, even after it worked
Swap in a different model for cost or performance reasons and an agent that was working can start failing to call tools, or can call them with the wrong data. Nothing in the logs necessarily looks like an obvious error because nothing crashed. The system may simply execute the wrong action, or no action, instead of behaving according to plan.
An update to a model already in use can create the same problem. A small change to how the model formats a tool call can break a parser hardcoded to the old shape. The visible failure may then surface several agent steps later, after other actions have already run on top of the bad result.
What serving needs to do about it
A serving engine needs to understand the tool-call format a model produces and remain resilient when model versions change. In practice, it should be able to:
- recognize and correctly parse each model family’s native tool-call format, including tag-based, JSON-based and special-token variants;
- normalize field-name differences such as “arguments” and “parameters” instead of assuming one shape;
- handle multiple tool calls in a single response, not just the first;
- separate reasoning text from the actual tool call before extracting arguments; and
- stay correct across model-version changes so an upstream update does not silently break a dependent pipeline.
These are not peripheral concerns for agentic systems. They determine whether an agent keeps doing what it was designed to do as models and dependencies change.
Grace Ableidinger is a Developer Advocate at Red Hat.
Sawyer Bowerman is an AI Developer Advocate at Red Hat.
Editor’s note: This contributed article has been lightly edited for clarity and TNGlobal house style. The substance of the author’s contribution has been preserved.
Share your perspective: TNGlobal welcomes contributed insights and expert commentary from across Asia’s technology and innovation ecosystem. Submit a contribution for editorial consideration, or explore more conversations in our TNGlobal INSIDER and TNGlobal Q&A and Interviews archive.
Featured image: Tara Winstead on Pexels