Tool calling and function calling
The OpenAI function calling API (June 2023, updated to "tool calling" November 2023) and Anthropic's tool use API (April 2024) both implement the same core mechanism: the model outputs a structured JSON object specifying which function to call and with what arguments, instead of generating free text. The runtime executes the function and feeds the result back as a new message. The model never executes code — it produces a JSON declaration of intent, and your code handles execution.
OpenAI tool calling
A tool is defined as a JSON schema describing the function name, description, and parameters. The model receives these schemas alongside the conversation and can choose to emit a tool call instead of a text response.
import openai
import json
client = openai.OpenAI()
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": (
"Get current weather for a city. "
"Returns temperature in Celsius, conditions, and humidity."
),
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g. 'San Francisco' or 'London'",
},
"units": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature units. Defaults to celsius.",
},
},
"required": ["city"],
},
},
},
{
"type": "function",
"function": {
"name": "search_restaurants",
"description": (
"Search for restaurants in a city by cuisine type. "
"Returns name, rating (1-5), price range ($-$$$$), and address."
),
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string"},
"cuisine": {"type": "string", "description": "e.g. 'italian', 'japanese'"},
"max_price": {
"type": "integer",
"description": "Maximum price level 1-4, where 1=$ and 4=$$$$",
},
},
"required": ["city", "cuisine"],
},
},
},
]
def execute_tool(name, args):
if name == "get_weather":
return json.dumps({
"city": args["city"],
"temperature": 18,
"conditions": "partly cloudy",
"humidity": 72,
})
elif name == "search_restaurants":
return json.dumps({
"results": [
{"name": "Delfina", "rating": 4.5, "price": "$$", "address": "3621 18th St"},
{"name": "Flour + Water", "rating": 4.7, "price": "$$$", "address": "2401 Harrison St"},
]
})
return json.dumps({"error": f"Unknown tool: {name}"})
messages = [
{"role": "user", "content": "What's the weather in SF, and find me Italian restaurants there?"}
]
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
)
# The model may call multiple tools in parallel
assistant_message = response.choices[0].message
messages.append(assistant_message)
print(f"Tool calls: {len(assistant_message.tool_calls)}")
for tc in assistant_message.tool_calls:
print(f" {tc.function.name}({tc.function.arguments})")
result = execute_tool(tc.function.name, json.loads(tc.function.arguments))
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": result,
})
# Second call: model synthesizes tool results into a response
final = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
)
print(f"\nResponse: {final.choices[0].message.content}")GPT-4o will typically issue both tool calls in a single response (parallel tool calling). The model recognizes that the weather lookup and restaurant search are independent and emits two tool calls in one assistant message. Each tool result is returned as a separate tool message with a matching tool_call_id. The model then synthesizes both results into a coherent response.
Anthropic tool use
Anthropic's API uses a different message structure but the same pattern. Tools are defined in the API call, the model returns tool_use content blocks, and results are sent back as tool_result blocks.
import anthropic
import json
client = anthropic.Anthropic()
tools = [
{
"name": "get_stock_price",
"description": (
"Get the current stock price for a ticker symbol. "
"Returns price in USD, daily change percentage, and market cap."
),
"input_schema": {
"type": "object",
"properties": {
"ticker": {
"type": "string",
"description": "Stock ticker symbol, e.g. 'AAPL', 'GOOGL'",
},
},
"required": ["ticker"],
},
},
]
def get_stock_price(ticker):
prices = {"AAPL": 195.23, "GOOGL": 178.45, "MSFT": 420.10}
price = prices.get(ticker.upper(), 0)
return {"ticker": ticker.upper(), "price": price, "change_pct": 1.2, "market_cap": "3.0T"}
response = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
tools=tools,
messages=[{"role": "user", "content": "What's Apple's stock price?"}],
)
# Process tool use blocks
tool_results = []
for block in response.content:
if block.type == "tool_use":
print(f"Tool call: {block.name}({json.dumps(block.input)})")
result = get_stock_price(**block.input)
tool_results.append({
"type": "tool_result",
"tool_use_id": block.id,
"content": json.dumps(result),
})
# Send results back
final = client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
tools=tools,
messages=[
{"role": "user", "content": "What's Apple's stock price?"},
{"role": "assistant", "content": response.content},
{"role": "user", "content": tool_results},
],
)
print(f"\nResponse: {final.content[0].text}")Tool descriptions are the most important prompt you will write
The tool description is the primary signal the model uses to decide which tool to call and how to construct arguments. A bad description causes wrong tool selection, missing arguments, and hallucinated parameter values. The model has no access to the function implementation — it sees only the name, description, and parameter schema.
Effective descriptions follow a pattern:
- First sentence: what the tool does — "Search the product database by name, SKU, or category."
- Return value: what the caller gets back — "Returns: product_id, name, price_usd, in_stock (boolean), warehouse_location."
- Parameter descriptions: constraints and examples — For a
dateparameter: "ISO 8601 format, e.g. '2024-03-15'. Must be within the last 90 days." - Edge cases: what happens on failure — "Returns an empty array if no products match. Returns an error if the category does not exist."
Bad descriptions:
"search stuff"— The model cannot distinguish this from other search tools."Gets data from the database"— Which database? What data? What format?- No parameter descriptions — The model will guess argument formats and often guess wrong. A
datefield without format guidance will receive inconsistent formats ("March 15","2024-03-15","03/15/2024").
Forcing and disabling tools
Both APIs support controlling tool selection:
- Auto (default) — The model decides whether to call a tool or respond with text. OpenAI:
tool_choice="auto". Anthropic:tool_choice={"type": "auto"}. - Required — The model must call at least one tool. OpenAI:
tool_choice="required". Anthropic:tool_choice={"type": "any"}. - Specific tool — Force a specific tool call. OpenAI:
tool_choice={"type": "function", "function": {"name": "get_weather"}}. Anthropic:tool_choice={"type": "tool", "name": "get_weather"}. - None — Disable all tools. OpenAI:
tool_choice="none". Useful for the final synthesis step where you want the model to produce text, not another tool call.
Forcing tools is essential for deterministic first steps. A RAG agent should always retrieve documents before answering — tool_choice={"type": "function", "function": {"name": "search_docs"}} on the first call guarantees this.
Error handling when tools fail
Tools fail. APIs time out. Databases return connection errors. Search indices return empty results. The agent must handle these gracefully, or it will spin in a retry loop or hallucinate an answer.
def execute_tool_safely(name, args, timeout=10):
try:
func = TOOLS.get(name)
if not func:
return json.dumps({
"error": f"Tool '{name}' not found. Available: {list(TOOLS.keys())}"
})
result = func(**args)
return json.dumps({"success": True, "data": result})
except TimeoutError:
return json.dumps({
"error": f"Tool '{name}' timed out after {timeout}s. Try simplifying the query."
})
except Exception as e:
return json.dumps({
"error": f"Tool '{name}' failed: {type(e).__name__}: {str(e)}"
})The error message must be informative enough for the model to adjust its approach. "Error" is useless. "Tool 'search' timed out after 10s. Try simplifying the query." gives the model actionable information. In practice, models recover from well-described errors ~85% of the time by rephrasing queries or trying alternative tools (internal benchmarks from production agent systems).
Parallel tool calls
When the model identifies independent operations, it can emit multiple tool calls in a single response. GPT-4o and Claude both support this. The weather + restaurant example above demonstrates this: the model recognizes that weather and restaurant search do not depend on each other and issues both calls simultaneously.
Parallel execution reduces latency. If each tool call takes 500ms, sequential execution of 3 calls takes 1.5s. Parallel execution takes 500ms — a 3x improvement. In production, wrap parallel tool calls in asyncio.gather or concurrent.futures.ThreadPoolExecutor:
import concurrent.futures
def execute_parallel_tools(tool_calls):
results = {}
with concurrent.futures.ThreadPoolExecutor(max_workers=len(tool_calls)) as pool:
futures = {
pool.submit(execute_tool, tc.function.name, json.loads(tc.function.arguments)): tc
for tc in tool_calls
}
for future in concurrent.futures.as_completed(futures):
tc = futures[future]
results[tc.id] = future.result()
return resultsNot all tool calls should be parallelized. If tool B's input depends on tool A's output, they must be sequential. The model usually handles this correctly — it will emit a single tool call for the first step, wait for the result, then emit the dependent call. If the model incorrectly parallelizes dependent calls, add explicit ordering instructions to the system prompt.