Tool Call Syntax Rewriting

AbstractCore can preserve and rewrite tool-call syntax so downstream clients can consume tool calls consistently, even when models emit different tag formats.

Two Related Features

  • Python API (tool_call_tags): preserve/rewrite tool-call markup inside response.content (mostly for prompted-tool models).
  • HTTP Server (agent_format): convert/synthesize tool-call syntax for HTTP clients while keeping tool_calls structured.
  • Canonical representation: use structured tool_calls for execution; rewriting is formatting, not execution.

1) Python API: tool_call_tags

Default behavior (recommended)

  • tool_call_tags=None (default): tool-call markup is removed from response.content for clean UX.
  • Tool calls are still surfaced in response.tool_calls (structured dicts).

When to set tool_call_tags

Set tool_call_tags when a downstream consumer needs tool calls in text form (for example, a runtime that parses tool calls from assistant content).

Supported values

  • qwen3 → <|tool_call|>...JSON...</|tool_call|>
  • llama3 → <function_call>...JSON...</function_call>
  • xml → <tool_call>...JSON...</tool_call>
  • gemma → ```tool_code ...JSON... ```
  • Custom tags: "START,END" or "MYTAG" (auto-wrapped).

Example (non-streaming)

from abstractcore import create_llm

tool = {
    "name": "get_weather",
    "description": "Get weather for a city",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}

llm = create_llm("ollama", model="qwen3:4b-instruct")
response = llm.generate(
    "Weather in Paris?",
    tools=[tool],
    tool_call_tags="llama3",
)

print(response.content)     # contains <function_call>...</function_call>
print(response.tool_calls)  # structured dicts for host/runtime execution

Example (streaming)

tool_calls = []

for chunk in llm.generate(
    "Weather in Paris?",
    tools=[tool],
    stream=True,
    tool_call_tags="llama3",
):
    print(chunk.content, end="", flush=True)
    if chunk.tool_calls:
        tool_calls.extend(chunk.tool_calls)

2) HTTP Server: agent_format

When using the OpenAI-compatible server (/v1/chat/completions), request a target syntax format with agent_format.

Important: The server runs in passthrough mode by default (execute_tools=false): it returns tool calls; it does not execute them.

Supported values

  • auto (default): auto-detect using User-Agent + model patterns
  • openai, codex, qwen3, llama3, xml, gemma, passthrough

Example (curl)

curl -sS http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ollama/qwen3:4b-instruct",
    "messages": [{"role": "user", "content": "Weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather for a city",
        "parameters": {"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}
      }
    }],
    "agent_format": "codex"
  }'

See also: Server Guide for endpoint details and provider/model naming.

Notes

  • tool_call_tags and agent_format are formatting controls, not tool execution.
  • Use response.tool_calls (Python) or message.tool_calls (server/OpenAI format) as the canonical tool-call representation.

Related Documentation

Tool Calling System

Universal tools across all providers

HTTP Server Guide

OpenAI-compatible REST API

API Reference

Sync/async, streaming, response objects