I Constructed a Device-Calling Agent in Python. Right here’s How I Debugged It

is spectacular when it really works. However when it fails, the ultimate reply will not be sufficient. You want to see the perform it selected, the arguments it despatched, and the end result your code returned.

In case you have constructed a small agent round a product API, a retrieval endpoint, a database lookup, a climate service, or a Mannequin Context Protocol (MCP) instrument, you could have most likely seen the identical downside. The mannequin says it checked one thing. Possibly it did. Possibly it despatched malformed arguments. Possibly the API returned no end result. Possibly the mannequin request failed earlier than your instrument ran in any respect. With out the message historical past, instrument arguments, returned payload, and closing reply in a single place, you might be trusting a narrative as a substitute of inspecting a run.

OpenAI’s documentation lays out 4 steps: you describe a instrument, the mannequin requests it, you run the instrument, and you come the end result. That sample is helpful, however the first demo usually skips the half that issues as soon as an agent touches actual work. Are you able to show what the mannequin requested, what Python executed, what failed validation, and whether or not the ultimate reply used the returned information?

This text is constructed round that debugging downside. The agent calls public APIs, validates instrument arguments with JSON Schema, returns compact instrument payloads, data every instrument step, catches mannequin request failures, and might write the identical run to a Weights & Biases (W&B) Weave hint for assessment. JSON means JavaScript Object Notation. On this article, JSON Schema is the contract that tells Python which instrument arguments are legitimate earlier than any instrument runs.

The tutorial downside is easy: construct an agent that may reply a climate query solely by calling actual instruments, then make each step inspectable. Climate is standing in for the service-backed duties builders normally give brokers: test a bundle, retrieve a buyer report, search for stock, worth an order, or name an inside API. The helpful query is whether or not the model-selected perform name truly occurred and returned usable information.

The reader takeaway is direct: cease judging a tool-calling agent by the ultimate reply alone. Decide it by the mannequin request, schema validation, Python execution, compact instrument end result, error path, and closing reply collectively.

The loop is price constructing immediately as soon as earlier than adopting a bigger agent framework or wiring the identical instruments into MCP. Frameworks and MCP servers are helpful when you could have many instruments, routing guidelines, state, retries, or group conventions. The purpose right here is to grasp the message circulate earlier than abstraction hides it.

By the top, you’ll have a script that may:

• outline instruments with JSON Schema for an OpenAI mannequin
• run a bounded instrument calling loop
• validate instrument names and arguments earlier than execution
• maintain instrument outputs compact earlier than returning them to the mannequin
• return a structured error if the mannequin request itself fails
• seize run proof in console output and Weave

That’s the article’s edge. You get a runnable message loop you may examine, break, confirm, and later change or wrap with MCP or an agent framework from a place of understanding.

The run ought to reply 4 questions

A tool-calling agent could reply, “I checked the API,” however the helpful questions begin after that sentence:

1. Which instrument did the mannequin request?
2. What arguments did it ship?
3. What did Python return?
4. Did the ultimate reply use the returned information or disguise a failure?

The tutorial builds a small agent round these questions. The consumer asks whether or not to hold an umbrella in Lagos. The mannequin has to request a metropolis lookup, obtain coordinates, request climate, obtain a forecast, and reply from that returned information. Each step is printed and might be traced.

For those who can examine this small loop, the identical behavior carries into extra critical service-backed brokers. A refund agent ought to present the order lookup, coverage test, refund choice, and closing message. A doc agent ought to present the search question, retrieved passages, and reply. An MCP instrument ought to nonetheless present the instrument title, arguments, end result, and error path.

What a instrument calling agent truly does

A instrument calling agent in Python is a loop pushed by a big language mannequin, or LLM. It lets the mannequin request named capabilities, obtain their outcomes, and proceed with up to date messages.

That sounds near a chatbot, however the conduct is completely different. A chatbot receives textual content and returns textual content. A instrument calling agent receives textual content, could request motion, waits on your utility to execute that motion, reads the instrument end result, after which decides what to say or do subsequent.

The essential items are plain engineering objects:

• The mannequin decides whether or not it wants a instrument.
• The instrument is a Python perform owned by your utility.
• The schema specifies the arguments the instrument accepts.
• The messages are the working report of the consumer request, mannequin instrument requests, instrument outcomes, and closing reply.
• The agent loop is your Python code that retains the method working till the mannequin stops requesting instruments.

A fundamental perform name is one request and one end result. An agent loop is the repeated model. The mannequin can ask for one instrument, learn the end result, ask for one more instrument, and maintain going till it has sufficient context.

The climate instance makes use of two instruments:

• geocode_city, which turns a metropolis title into latitude, longitude, and nation.
• get_weather, which turns latitude and longitude right into a compact climate report.

In an actual utility, these capabilities name exterior APIs. A bundle instrument would possibly name a transport supplier. A flight instrument would possibly name an airline standing service. A buying instrument would possibly name a list system. On this article, the instruments use Nominatim from OpenStreetMap for geocoding and Open-Meteo for climate information. These companies maintain the instance actual whereas nonetheless being sufficiently small to learn. Additionally they don’t require API keys for this tutorial run, so the one key you want is the OpenAI key used for the mannequin name.

The message loop this text exposes

Your app sends the consumer message and the listing of obtainable instruments to the mannequin. If the mannequin wants a instrument, it returns a structured request. Your Python utility reads that request, runs the matching perform, sends the end result again as a instrument message, and asks the mannequin to proceed.

The newer OpenAI Responses API follows the identical concept. It additionally helps inbuilt instruments, together with internet search and file search, so the instrument might be offered by OpenAI or by your individual utility.

The primary helpful concept is easy: the mannequin chooses, however your code executes.

That boundary issues. Your utility ought to nonetheless resolve whether or not a requested instrument exists, whether or not the arguments match the schema, whether or not the decision is allowed, how a lot information to return, what to log, and when to cease the loop.

Why begin with out a framework

It’s price constructing one customized Python loop earlier than adopting a bigger agent framework. The primary model teaches you what is simple to examine. When you perceive the uncooked message circulate, you may make a greater choice about whether or not a framework removes complexity or hides it.

The identical concept applies to MCP. The official MCP documentation describes MCP as a typical means for purposes to offer context to giant language fashions. An MCP server can expose instruments, sources, and prompts to an AI shopper, however the design questions stay the identical: What arguments are legitimate? What ought to the instrument return? What occurs when the mannequin request fails? What occurs when the instrument returns no end result?

For this tutorial, the principle path makes use of the OpenAI Python software program growth package, or SDK. Native runtimes that assist instrument calling, together with Ollama, comply with the identical sample with completely different response codecs. The purpose right here is to construct the loop as soon as the place each object is seen.

Create one folder and arrange the atmosphere

Create one working folder for this text, then run each setup and execution command from that folder. The folder will comprise the digital atmosphere and openai_tool_calling_agent.py. Use Python 3.11 or newer. You do not want a graphics processing unit, or GPU, as a result of the principle path calls a hosted OpenAI mannequin. The geocoding and climate instruments use public APIs that don’t require their very own keys.

On macOS or Linux, open a terminal in that working folder and run:

python3.11 -m venv .venv
supply .venv/bin/activate
python -m pip set up openai requests weave jsonschema

On Home windows PowerShell, open the identical working folder and run:

py -3.11 -m venv .venv
.venvScriptsactivate
python -m pip set up openai requests weave jsonschema

In the identical activated terminal, set your OpenAI key earlier than the mannequin run. On macOS or Linux, run:

export OPENAI_API_KEY="your_api_key_here"

On Home windows PowerShell, run:

$env:OPENAI_API_KEY="your_api_key_here"

In the identical activated terminal, if you would like Weave tracing, log in to W&B:

wandb login

Confirm the Python model:

python --version

Anticipated output will look just like this:

Python 3.11.9

Save the whole runnable script

Save the next code as openai_tool_calling_agent.py in the identical working folder the place you created .venv. That is the one file readers have to create. The sections after the code clarify the design decisions, however they don’t add any further required code.

import argparse
import json
import os
from typing import Any

import requests
from jsonschema import ValidationError, validate
from openai import OpenAI, OpenAIError

attempt:
    import weave
besides ImportError:
    weave = None


REQUEST_TIMEOUT = 10
USER_AGENT = "tool-calling-agent-python/1.0"
MODEL = os.getenv("OPENAI_MODEL", "gpt-4.1")


def geocode_city(metropolis: str) -> dict[str, Any]:
    response = requests.get(
        "https://nominatim.openstreetmap.org/search",
        params={"q": metropolis, "format": "jsonv2", "restrict": 1, "addressdetails": 1},
        headers={"Person-Agent": USER_AGENT},
        timeout=REQUEST_TIMEOUT,
    )
    response.raise_for_status()
    outcomes = response.json()
    if not outcomes:
        return {"error": f"Metropolis not discovered: {metropolis}"}

    first = outcomes[0]
    handle = first.get("handle", {})
    return {
        "metropolis": first.get("title", metropolis),
        "nation": handle.get("nation"),
        "latitude": float(first["lat"]),
        "longitude": float(first["lon"]),
    }


def get_weather(latitude: float, longitude: float, metropolis: str) -> dict[str, Any]:
    response = requests.get(
        "https://api.open-meteo.com/v1/forecast",
        params={
            "latitude": latitude,
            "longitude": longitude,
            "present": "temperature_2m,precipitation,rain,weather_code",
            "each day": (
                "weather_code,temperature_2m_max,temperature_2m_min,"
                "precipitation_sum,precipitation_probability_max"
            ),
            "forecast_days": 2,
            "timezone": "auto",
        },
        timeout=REQUEST_TIMEOUT,
    )
    response.raise_for_status()
    information = response.json()
    present = information.get("present", {})
    each day = information.get("each day", {})

    def tomorrow_value(subject: str) -> Any:
        values = each day.get(subject) or []
        return values[1] if len(values) > 1 else None

    return {
        "metropolis": metropolis,
        "temperature_c": present.get("temperature_2m"),
        "precipitation_mm": present.get("precipitation"),
        "rain_mm": present.get("rain"),
        "weather_code": present.get("weather_code"),
        "tomorrow_weather_code": tomorrow_value("weather_code"),
        "tomorrow_temperature_max_c": tomorrow_value("temperature_2m_max"),
        "tomorrow_temperature_min_c": tomorrow_value("temperature_2m_min"),
        "tomorrow_precipitation_sum_mm": tomorrow_value("precipitation_sum"),
        "tomorrow_rain_chance_percent": tomorrow_value("precipitation_probability_max"),
    }


TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "geocode_city",
            "description": "Find latitude, longitude, and country for a supported city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "City name, such as Lagos, London, or New York.",
                    }
                },
                "required": ["city"],
                "additionalProperties": False,
            },
        },
    },
    {
        "kind": "perform",
        "perform": {
            "title": "get_weather",
            "description": "Get a compact climate report for a identified location.",
            "parameters": {
                "kind": "object",
                "properties": {
                    "latitude": {"kind": "quantity"},
                    "longitude": {"kind": "quantity"},
                    "metropolis": {"kind": "string"},
                },
                "required": ["latitude", "longitude", "city"],
                "additionalProperties": False,
            },
        },
    },
]

TOOL_REGISTRY = {
    "geocode_city": geocode_city,
    "get_weather": get_weather,
}

SCHEMAS_BY_TOOL = {
    instrument["function"]["name"]: instrument["function"]["parameters"]
    for instrument in TOOLS
}


def compact_tool_result(end result: dict[str, Any]) -> dict[str, Any]:
    if "error" in end result:
        return {"error": end result["error"]}

    allowed_keys = {
        "metropolis",
        "nation",
        "latitude",
        "longitude",
        "temperature_c",
        "precipitation_mm",
        "rain_mm",
        "weather_code",
        "tomorrow_weather_code",
        "tomorrow_temperature_max_c",
        "tomorrow_temperature_min_c",
        "tomorrow_precipitation_sum_mm",
        "tomorrow_rain_chance_percent",
    }
    return {key: worth for key, worth in end result.objects() if key in allowed_keys}


def execute_tool_call(tool_name: str, tool_args: dict[str, Any]) -> dict[str, Any]:
    if tool_name not in TOOL_REGISTRY:
        return {"error": f"Unknown instrument: {tool_name}"}

    attempt:
        validate(occasion=tool_args, schema=SCHEMAS_BY_TOOL[tool_name])
    besides ValidationError as exc:
        return {"error": "Invalid instrument arguments", "particulars": exc.message}

    attempt:
        return TOOL_REGISTRY[tool_name](**tool_args)
    besides Exception as exc:
        return {"error": "Device execution failed", "particulars": str(exc)}


def maybe_trace(title):
    if weave is None:
        return lambda fn: fn
    return weave.op(title=title)


@maybe_trace("run_agent")
def run_agent(user_prompt: str, max_turns: int = 4) -> dict[str, Any]:
    shopper = OpenAI()
    messages = [
        {
            "role": "system",
            "content": (
                "You are a concise weather assistant. "
                "Call tools only when they add facts needed for the answer."
            ),
        },
        {"role": "user", "content": user_prompt},
    ]
    transcript: listing[dict[str, Any]] = []

    for flip in vary(max_turns):
        attempt:
            response = shopper.chat.completions.create(
                mannequin=MODEL,
                messages=messages,
                instruments=TOOLS,
            )
        besides OpenAIError as exc:
            return {
                "mannequin": MODEL,
                "user_prompt": user_prompt,
                "reply": "",
                "error": {
                    "kind": "model_request_failed",
                    "particulars": str(exc),
                },
                "transcript": transcript,
            }
        assistant_message = response.decisions[0].message
        messages.append(assistant_message)

        tool_calls = assistant_message.tool_calls or []
        if not tool_calls:
            return {
                "mannequin": MODEL,
                "user_prompt": user_prompt,
                "reply": assistant_message.content material or "",
                "transcript": transcript,
            }

        for tool_call in tool_calls:
            tool_name = tool_call.perform.title
            tool_args = json.masses(tool_call.perform.arguments)
            raw_result = execute_tool_call(tool_name, tool_args)
            tool_result = compact_tool_result(raw_result)

            transcript.append(
                {
                    "flip": flip + 1,
                    "instrument": tool_name,
                    "arguments": tool_args,
                    "end result": tool_result,
                }
            )
            messages.append(
                {
                    "position": "instrument",
                    "tool_call_id": tool_call.id,
                    "content material": json.dumps(tool_result),
                }
            )

    return {
        "mannequin": MODEL,
        "user_prompt": user_prompt,
        "reply": "I couldn't end as a result of the agent reached its instrument name restrict.",
        "transcript": transcript,
    }


def confirm() -> dict[str, Any]:
    bad_arguments = execute_tool_call("get_weather", {"metropolis": "Lagos"})
    unknown_tool = execute_tool_call("lookup_package", {"tracking_id": "123"})
    schema_names = sorted(SCHEMAS_BY_TOOL)
    return {
        "standing": "okay",
        "mannequin": MODEL,
        "instruments": schema_names,
        "bad_arguments_check": bad_arguments,
        "unknown_tool_check": unknown_tool,
    }


def principal() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--mode", decisions=["verify", "run"], default="run")
    parser.add_argument(
        "--prompt",
        default="Ought to I carry an umbrella in Lagos tomorrow?",
    )
    parser.add_argument("--weave-project", default="")
    args = parser.parse_args()

    if args.mode == "confirm":
        print(json.dumps(confirm(), indent=2))
        return

    if args.weave_project:
        if weave is None:
            elevate RuntimeError("Set up weave earlier than utilizing --weave-project.")
        weave.init(args.weave_project)

    end result = run_agent(args.immediate)
    print(json.dumps(end result, indent=2))


if __name__ == "__main__":
    principal()

Run a preflight test earlier than spending tokens

Run this command from the identical working folder and the identical activated terminal. It checks imports, the instrument registry, JSON Schema validation, and unknown instrument dealing with with out utilizing the OpenAI API.

python openai_tool_calling_agent.py --mode confirm

Captured output from my preflight run:

{
  "standing": "okay",
  "mannequin": "gpt-4.1",
  "instruments": [
    "geocode_city",
    "get_weather"
  ],
  "bad_arguments_check": {
    "error": "Invalid instrument arguments",
    "particulars": "'latitude' is a required property"
  },
  "unknown_tool_check": {
    "error": "Unknown instrument: lookup_package"
  }
}

That is the primary cause to construct the loop your self. Earlier than the mannequin is concerned, you may test that the Python layer refuses an unknown instrument and catches a malformed get_weather name.

Run the agent towards actual APIs

After setting OPENAI_API_KEY, run the total instrument calling loop from the identical working folder and activated terminal:

python openai_tool_calling_agent.py --mode run --prompt "Ought to I carry an umbrella in Lagos tomorrow?"

The script prints JSON. It consists of the mannequin title, the consumer immediate, the ultimate reply, and a transcript of every instrument name. In my profitable climate run, the mannequin requested geocode_city first, then get_weather, then wrote the ultimate reply from the compact climate payload.

For readability, the identical captured run is formatted under as a step-by-step hint:

[MODEL]
gpt-4.1

[USER PROMPT]
Ought to I carry an umbrella in Lagos tomorrow?

[OPENAI REQUESTS TOOL turn 1]
{
  "arguments": {
    "metropolis": "Lagos"
  },
  "instrument": "geocode_city"
}

[PYTHON RUNS geocode_city]
{
  "metropolis": "Lagos",
  "nation": "Nigeria",
  "latitude": 6.4550575,
  "longitude": 3.3941795
}

[OPENAI REQUESTS TOOL turn 2]
{
  "arguments": {
    "metropolis": "Lagos",
    "latitude": 6.4550575,
    "longitude": 3.3941795
  },
  "instrument": "get_weather"
}

[PYTHON RUNS get_weather]
{
  "metropolis": "Lagos",
  "precipitation_mm": 0.0,
  "rain_mm": 0.0,
  "temperature_c": 27.7,
  "tomorrow_precipitation_sum_mm": 6.5,
  "tomorrow_rain_chance_percent": 84,
  "tomorrow_temperature_max_c": 28.9,
  "tomorrow_temperature_min_c": 24.6,
  "tomorrow_weather_code": 80,
  "weather_code": 3
}

[FINAL ANSWER]
Sure, you must carry an umbrella in Lagos tomorrow. There's a excessive likelihood of rain (84%) with about 6.5 mm of precipitation anticipated.

That output is the audit path. The mannequin didn’t magically know Lagos climate. It requested coordinates, your Python code fetched them, the mannequin requested a forecast, your Python code fetched a compact forecast, and the ultimate reply used that returned information.

Run one messy immediate

A clear run proves the loop can end. It doesn’t show the loop is nice to debug when one thing goes flawed.

I attempted a messier immediate subsequent:

OPENAI_MODEL=gpt-4.1-mini python openai_tool_calling_agent.py --mode run --prompt "Ought to I carry an umbrella in Xqznotacity tomorrow? If that place will not be actual, inform me what failed."

The immediate was meant to train the geocoder path with a spot that ought to not exist. The primary failure appeared sooner than that: the mannequin request itself returned a server error. Earlier than including the OpenAIError handler, this crashed the script with a stack hint. After the change, the agent returned a structured failure:

{
  "mannequin": "gpt-4.1-mini",
  "user_prompt": "Ought to I carry an umbrella in Xqznotacity tomorrow? If that place will not be actual, inform me what failed.",
  "reply": "",
  "error": {
    "kind": "model_request_failed",
    "particulars": "Error code: 500 ... server_error"
  },
  "transcript": []
}

This can be a higher edge case than I anticipated. The primary boundary in a instrument calling agent is the mannequin request. If that fails, the appliance ought to return a helpful error as a substitute of hiding the issue behind a generic crash.

Add a Weave hint

To report the identical run in Weave, run this command from the identical working folder and activated terminal:

python openai_tool_calling_agent.py --mode run --weave-project wb-authors/tool-calling-agent-python --prompt "Ought to I carry an umbrella in Lagos tomorrow?"

The hint is helpful as a result of it preserves the sequence that issues: consumer immediate, mannequin instrument request, validated Python name, compact instrument end result, and closing reply. I captured the output above from my very own run, and the identical run is logged on this Weave hint. The Weave tracing docs describe the identical assessment sample for logged mannequin calls.

Earlier than you take into account this primary model completed, assessment the run output and the Weave hint. You need to see the mannequin request geocode_city, Python run the Nominatim lookup, the mannequin request get_weather, Python run the Open-Meteo forecast name, and the mannequin write a solution from the compact climate payload. That visibility is the baseline to protect earlier than including extra instruments, frameworks, evaluations, or dashboards.

How the script maps to the agent loop

The script has six items.

First, geocode_city and get_weather are slim instruments that decision actual companies. Nominatim turns a metropolis into coordinates, and Open-Meteo turns these coordinates right into a forecast. Nominatim additionally asks public purchasers to ship a transparent consumer agent string, which is why the script units USER_AGENT.

Second, TOOLS describes these capabilities with JSON Schema. The schema is the contract the mannequin sees. It says which perform exists, what arguments it accepts, which fields are required, and whether or not additional fields are allowed.

Third, TOOL_REGISTRY and execute_tool_call maintain execution inside your utility. The mannequin can request a instrument, however Python decides whether or not the instrument is understood, whether or not the arguments match the schema, and what structured error to return when one thing is flawed.

Fourth, compact_tool_result removes fields the mannequin doesn’t want. Device outputs are messages again to the mannequin, not full API dumps. Compact payloads make the reply cheaper to supply and simpler to examine later.

Fifth, run_agent retains a bounded loop. It sends messages and power schemas to the mannequin, receives instrument calls, executes the matching Python capabilities, appends instrument outcomes, and stops when the mannequin returns a standard reply or the loop reaches max_turns.

Sixth, the OpenAIError handler turns mannequin request failures into structured output. That retains server errors, connection failures, or authentication errors seen to the caller as a substitute of burying them in a traceback.

That’s the fundamental loop most groups want to grasp earlier than adopting a bigger framework. Frameworks are simpler to guage after you could have seen the uncooked message circulate as soon as.

Reliability begins the place the loop is seen

Manufacturing instrument calling brokers fail on the edges. A mannequin request can fail earlier than a instrument is chosen. Arguments arrive within the flawed format. A instrument occasions out. A mannequin picks the flawed perform. A name repeats with out including data. The entire script handles the primary guardrails immediately: mannequin request errors, schema validation, unknown instrument errors, request timeouts, compact instrument outcomes, and a loop restrict.

A helpful preflight test is already constructed into --mode confirm. It calls get_weather with solely a metropolis, although the schema requires latitude, longitude, and metropolis. The script returns a structured error as a substitute of working a damaged instrument name. It additionally checks that an unknown instrument title returns a structured error.

That small preflight path issues. It lets readers test the Python layer earlier than they spend cash on mannequin calls, and it offers groups a spot so as to add extra checks later. The messy immediate provides the opposite aspect of the reliability story: dwell mannequin calls can fail too, so the agent ought to make that failure seen.

The place this leaves the agent

You now have the essential instrument calling agent sample in Python: outline slim instruments, describe them with schemas, let the mannequin request them, execute solely identified capabilities, return compact outcomes, deal with mannequin request failures, and maintain the loop bounded.

The climate instance is just one use case. The identical loop can name a doc search instrument, a buyer database, a pricing service, a check runner, or an inside workflow API.

Do one factor earlier than you add extra instruments: run the agent with a immediate that ought to fail. Ask for an unsupported metropolis. Break one argument. Return an outsized payload. Watch what the loop does.

That can inform you greater than one other clear, completely satisfied path.

The identical intuition applies past instrument calling. In a follow-up piece on debugging coding brokers, I apply the identical evidence-first strategy to an agent that edits UI code, and recording what it inspected, patched, and verified as a substitute of trusting its personal “mounted”.

The helpful lesson is larger than climate. Device calling begins as a loop you may examine earlier than it turns into an structure choice. As soon as the loop is seen, a framework or MCP choice turns into simpler: undertake one when it removes repeated routing, state, retries, instrument packaging, or observability work, after you perceive what it hides. The ultimate reply is just one a part of the run.

Chosen Sources