OpenTelemetry GenAI: Instrument AI Brokers With out Sending Information to the Cloud

Why Observability for AI Brokers Is Totally different

Observability for AI brokers presents challenges that conventional APM instruments by no means dealt with. OpenTelemetry GenAI semantic conventions provide a vendor-neutral framework to deal with them, and this tutorial exhibits the way to run the whole pipeline regionally with out sending a single byte to the cloud. In contrast to deterministic API calls or database queries, AI agent outputs fluctuate throughout similar inputs. Multi-step reasoning chains create branching execution paths which are tough to hint with standard request-response fashions. Token consumption straight impacts value, making per-call monitoring a monetary concern quite than a nice-to-have. And prompts themselves usually comprise PII, proprietary enterprise logic, or delicate buyer knowledge that compliance frameworks bar from leaving managed infrastructure.

Conventional APM instruments can measure latency and error charges, however they lack the semantic understanding to seize mannequin identifiers, token utilization breakdowns, or operation sorts like chat versus embeddings. This hole leaves groups blind to the metrics that really matter for GenAI workloads.

Token consumption straight impacts value, making per-call monitoring a monetary concern quite than a nice-to-have.

This tutorial delivers a completely native, cloud-free instrumentation pipeline. It covers establishing an OpenTelemetry Collector and Jaeger in Docker, instrumenting a Node.js AI agent with GenAI semantic conference attributes, and constructing a React dashboard to visualise traces. Every part runs on the developer’s machine.

What Are the OpenTelemetry GenAI Semantic Conventions?

The Commonplace Attributes You Have to Know

The OpenTelemetry GenAI semantic conventions outline a standardized set of span attributes for AI and huge language mannequin operations. The core attributes embrace:

• gen_ai.system: Identifies the AI supplier, similar to openai, anthropic, or a customized native mannequin identifier.
• gen_ai.request.mannequin: Specifies the mannequin being referred to as, for instance gpt-4o or claude-sonnet-4.
• Token utilization attributes gen_ai.utilization.input_tokens and gen_ai.utilization.output_tokens seize token counts from the supplier’s response, enabling value monitoring and utilization evaluation.
• The operation sort goes in gen_ai.operation.title, distinguishing chat from embeddings or different name sorts.

Structure Overview: Absolutely Native Observability Stack

The structure follows a three-component linear pipeline. A Node.js AI Agent sends telemetry by way of the OTel SDK to an OpenTelemetry Collector operating regionally, which then exports traces to a neighborhood Jaeger occasion. The OTel SDK devices software code and creates spans; the Collector receives, processes, and routes that telemetry to Jaeger, which shops and visualizes traces by way of its internet UI.

The vital level: zero knowledge leaves the developer’s machine or personal community. The configuration contains no cloud endpoints, no API keys to exterior observability platforms, and no third-party knowledge processors within the chain.

Conditions:

• Node.js 18+ (18.x or 20.x LTS beneficial)
• npm 9+
• Docker Engine ≥ 24.x and Docker Compose v2 (the docker compose CLI plugin)
• An OpenAI API key (or any LLM supplier key) with billing enabled and obtainable quota
• Ports 4317, 4318, 13133, and 16686 free on localhost (verify with lsof -i :4317)

Setting Up the Native OpenTelemetry Collector and Jaeger

Docker Compose Configuration

Create a docker-compose.yml and an accompanying otel-collector-config.yml within the mission root:

providers:
  otel-collector:
    picture: otel/opentelemetry-collector-contrib:0.103.0
    command: ["--config", "/etc/otel-collector-config.yml"]
    volumes:
      - ./otel-collector-config.yml:/and many others/otel-collector-config.yml
    ports:
      - "4317:4317"   
      - "13133:13133" 
    depends_on:
      jaeger:
        situation: service_healthy

  jaeger:
    picture: jaegertracing/all-in-one:1.57.0
    ports:
      - "16686:16686" 
      - "4318:4318"   
    
    healthcheck:
      check: ["CMD", "wget", "--spider", "-q", "http://localhost:16686"]
      interval: 5s
      timeout: 3s
      retries: 5

Notice: Pin picture tags to particular variations (as proven above) quite than utilizing newest. Upstream breaking modifications — such because the elimination of the jaeger exporter sort from collector-contrib in v0.104.0 — can silently break your pipeline. Verify Docker Hub for the most recent secure launch tags when beginning a brand new mission.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  batch:
    timeout: 5s
    send_batch_size: 1024
    send_batch_max_size: 512

exporters:
  otlphttp/jaeger:
    endpoint: "http://jaeger:4318"
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/jaeger]

Verifying the Native Stack

Run docker compose up -d to start out each providers. Affirm Jaeger is accessible by opening http://localhost:16686 in a browser. Confirm the Collector’s well being by hitting http://localhost:13133 — a wholesome response ({"standing":"Server obtainable"}) confirms the Collector course of is operating. To confirm the gRPC receiver particularly, verify the Collector logs for "Beginning OTLP gRPC receiver" output:

docker compose logs otel-collector | grep -i "grpc"

Instrumenting a Node.js AI Agent with OpenTelemetry GenAI Attributes

Putting in Dependencies

First, initialize a mission and create a bundle.json with pinned dependency variations:

npm init -y

Then set up the required packages:

npm set up @opentelemetry/sdk-node@0.52.1 @opentelemetry/api@1.9.0 
  @opentelemetry/exporter-trace-otlp-grpc@0.52.1 @opentelemetry/sdk-trace-node@1.25.1 
  @opentelemetry/assets@1.25.1 openai@4.56.0

Including a React Dashboard for Actual-Time Agent Metrics

Fetching Hint Information from Jaeger’s API

CORS Notice: Jaeger’s API doesn’t allow CORS by default. Browser fetch() calls from a distinct origin might be blocked. To work round this, both run the React app utilizing a dev server proxy (e.g., Vite’s server.proxy choice to proxy /api requests to http://localhost:16686), or begin Jaeger with the flag --query.additional-headers 'Entry-Management-Permit-Origin: *' added to the Jaeger service command in docker-compose.yml.

To scaffold a React mission for the dashboard:

npm create vite@newest dashboard -- --template react
cd dashboard
npm set up

Place the next element in src/AgentTraceViewer.jsx:

import { useState, useEffect } from "react";

export default perform AgentTraceViewer() {
  const [traces, setTraces] = useState([]);
  const [error, setError] = useState(null);

  useEffect(() => {
    const controller = new AbortController();

    fetch("/api/traces?service=ai-agent&restrict=20", { sign: controller.sign })
      .then((res) => {
        if (!res.okay) throw new Error(`Jaeger API error: ${res.standing} ${res.statusText}`);
        return res.json();
      })
      .then((knowledge) => )
      .catch((err) => {
        if (err.title !== "AbortError") {
          setError(err.message);
        }
      });

    return () => controller.abort();
  }, []);

  if (error) {
    return <p model={{ colour: "pink" }}>Did not load traces: {error}p>;
  }

  return (
    <desk model={{ borderCollapse: "collapse" }}>
      <thead>
        <tr>
          <th>Timestampth><th>Mannequinth><th>Enter Tokensth>
          <th>Output Tokensth><th>Latencyth><th>Standingth>
        tr>
      thead>
      <tbody>
        {traces.map((row) => (
          <tr key={`${row.traceID}-${row.timestamp}`}>
            <td>{row.timestamp}td><td>{row.mannequin}td><td>{row.inputTokens}td>
            <td>{row.outputTokens}td><td>{row.length}td><td>{row.standing}td>
          tr>
        ))}
      tbody>
    desk>
  );
}

If utilizing Vite, configure the dev proxy in vite.config.js to ahead /api requests to Jaeger:

export default {
  server: {
    proxy: {
      "/api": {
        goal: "http://localhost:16686",
        changeOrigin: true,
        timeout: 5000,
        configure: (proxy) => {
          proxy.on("error", (err) => {
            console.error("[vite proxy] Jaeger unreachable:", err.message);
          });
        },
      },
    },
  },
};

Extending the Dashboard

Implementation Guidelines: Your Full Reference

Infrastructure

1. ☐ Docker Compose with OTel Collector + Jaeger operating regionally (pinned picture variations)
2. ☐ OTel Collector config: OTLP receiver → batch processor → OTLP HTTP exporter to Jaeger
3. ☐ No exterior endpoints configured: all knowledge stays native

Instrumentation

4. ☐ Node.js OTel SDK initialized with OTLP gRPC exporter to localhost:4317
5. ☐ gen_ai.system attribute set on all AI spans
6. ☐ gen_ai.request.mannequin attribute set on all AI spans
7. ☐ gen_ai.operation.title attribute set (chat, embeddings, and many others. — this tutorial demonstrates chat)
8. ☐ gen_ai.utilization.input_tokens captured from API response
9. ☐ gen_ai.utilization.output_tokens captured from API response
10. ☐ Father or mother-child span hierarchy for multi-step agent flows
11. ☐ Error recording with span.setStatus() and span.recordException()

Verification

12. ☐ Jaeger UI accessible and exhibiting traces with GenAI attributes
13. ☐ (Optionally available) React dashboard consuming Jaeger API for customized views

Frequent Pitfalls and Troubleshooting

Tracing file load order issues. If tracing.js just isn’t required earlier than the agent code runs, the tracer supplier is not going to be registered and spans will silently fail to seize. All the time use --require ./tracing.js.

You see OTLP export failed or no spans arrive? Verify for a protocol mismatch between SDK and Collector. The exporter-trace-otlp-grpc bundle speaks gRPC. If the Collector is configured for http/protobuf, or vice versa, the connection fails silently. Match each side. The gRPC exporter expects an http:// URL or a naked host:port — utilizing an unsupported scheme like grpc:// causes the SDK to drop spans with out error.

OpenAI just isn't a constructor. The openai npm bundle v4+ makes use of named exports underneath CommonJS. Use const { OpenAI } = require("openai"); (not const OpenAI = require("openai")).

When token counts are lacking, verify whether or not you’re utilizing streaming mode. Some LLM suppliers omit the utilization object from streamed responses. If response.utilization is undefined, the instrumentation can’t set token attributes. All the time null-guard response.utilization earlier than accessing token fields, and verify supplier documentation for stream-specific choices that embrace utilization knowledge.

The BatchSpanProcessor buffers spans and flushes them on a timer (default 5 seconds). In brief-lived scripts, the runtime drops all buffered spans if the method exits earlier than the flush fires.

The BatchSpanProcessor buffers spans and flushes them on a timer (default 5 seconds). In brief-lived scripts, the runtime drops all buffered spans if the method exits earlier than the flush fires. All the time name supplier.shutdown() (or the exported shutdownTracing() perform) earlier than course of exit to drive a flush.

Attribute title instability. The GenAI semantic conventions are experimental. Pin @opentelemetry bundle variations in bundle.json to keep away from breakage when attribute names change between releases.

If the Collector exits instantly after beginning, verify docker compose logs otel-collector for errors. A typical trigger is utilizing a deprecated exporter sort (such because the jaeger exporter, which was faraway from collector-contrib in v0.104.0). Use the otlphttp/jaeger exporter as proven on this tutorial.

No service seems within the Jaeger dropdown. Confirm that spans are reaching Jaeger by checking the Collector logs and confirming the URL in tracing.js makes use of http://localhost:4317 (not grpc://).

By no means move API keys inline on the command line. Inline OPENAI_API_KEY=sk-... node ... instructions are recorded in shell historical past (~/.bash_history) and visual in course of listings. Use a .env file (added to .gitignore) and supply .env as a substitute.

What Comes Subsequent

This tutorial delivers full native observability for AI brokers utilizing OpenTelemetry GenAI semantic conventions, with no knowledge leaving managed infrastructure. Because the conventions transfer towards their stability goal, attribute names will stabilize and library authors will ship auto-instrumentation for common LLM SDKs. Groups can put together now by adopting the guide instrumentation patterns proven right here, then migrate to auto-instrumentation because the ecosystem catches up. Past traces, the pure subsequent step is metrics. Begin with aggregating token prices per mannequin — essentially the most instant operational win — then layer on latency distributions and utilization anomaly alerts as your instrumentation matures.

The vital level: zero knowledge leaves the developer’s machine or personal community. The configuration contains no cloud endpoints, no API keys to exterior observability platforms, and no third-party knowledge processors within the chain.