Skip to content

OTel GenAI Semantic Conventions

Key Points

  • OpenTelemetry has GenAI semantic conventions for AI/LLM observability — standardized attribute names.
  • Microsoft.Extensions.AI's UseOpenTelemetry() middleware emits these automatically.
  • Standard fields: gen_ai.system, gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.response.id.
  • Vendor-portable dashboards: queries work across OpenAI, Anthropic, Google, etc.
  • Sensitive data: prompts/responses NOT included by default; opt-in with care.

Standard attributes

gen_ai.system: "openai" | "anthropic" | "google.gemini" | "azure.ai.openai" | ...
gen_ai.operation.name: "chat" | "embeddings" | "image_generation"

gen_ai.request.model: "gpt-4o-mini"
gen_ai.request.temperature: 0.7
gen_ai.request.max_tokens: 1000
gen_ai.request.top_p: 0.9

gen_ai.response.model: "gpt-4o-mini-2024-07-18"
gen_ai.response.id: "chatcmpl-..."
gen_ai.response.finish_reasons: ["stop"]

gen_ai.usage.input_tokens: 1234
gen_ai.usage.output_tokens: 567

Span events (with sensitive data enabled)

gen_ai.system.message    { content: "..." }
gen_ai.user.message      { content: "..." }
gen_ai.assistant.message { content: "..." }
gen_ai.tool.message      { content: "...", id: "...", name: "GetWeather" }
gen_ai.choice            { index: 0, finish_reason: "stop", message: {...} }

Setup

chat = chat.AsBuilder()
    .UseOpenTelemetry(sourceName: "MyApp.AI", configure: o =>
    {
        o.EnableSensitiveData = false;   // default: false
    })
    .Build();

Activity source name should be configured for OTel collector.

Pipeline

builder.Services.AddOpenTelemetry()
    .WithTracing(t => t
        .AddSource("MyApp.AI")
        .AddOtlpExporter())
    .WithMetrics(m => m
        .AddMeter("MyApp.AI")
        .AddOtlpExporter());

Dashboards

// App Insights — top models by cost
dependencies
| where customDimensions["gen_ai.system"] != ""
| extend model = tostring(customDimensions["gen_ai.request.model"])
| extend in_tokens = toint(customDimensions["gen_ai.usage.input_tokens"])
| extend out_tokens = toint(customDimensions["gen_ai.usage.output_tokens"])
| summarize total_in = sum(in_tokens), total_out = sum(out_tokens) by model

// Latency p99 per model
dependencies
| where name == "chat"
| summarize p99 = percentile(duration, 99) by model = tostring(customDimensions["gen_ai.request.model"])

Vendor portability

Same query works against OpenAI, Anthropic, Gemini — because attribute names are standardized.

Cost tracking

var inTokens = activity?.GetTagItem("gen_ai.usage.input_tokens") as long?;
var outTokens = activity?.GetTagItem("gen_ai.usage.output_tokens") as long?;
var model = activity?.GetTagItem("gen_ai.request.model") as string;

var cost = ComputeCost(model, inTokens, outTokens);

Pre-computed cost as custom metric:

private static readonly Histogram<double> _cost = _meter.CreateHistogram<double>("ai.cost.usd");
_cost.Record(cost, new TagList { { "model", model }, { "tenant", tenantId } });

Custom attributes

Add app-specific tags:

activity?.SetTag("app.feature", "chatbot");
activity?.SetTag("user.tier", "premium");
activity?.SetTag("tenant.id", tenantId);

Use to filter dashboards.

Distributed tracing across calls

LLM call within agent → tool calls → DB calls all linked via OTel:

[GET /chat]
└── [Agent.InvokeAsync]
    ├── [chat (gen_ai.system=openai)]
    ├── [Tool: SearchDB]
    │   └── [SQL Server query]
    └── [chat (gen_ai.system=openai)]   (final)

Click span → see all related operations.

Spec status

GenAI conventions are stable in OpenTelemetry as of 2026. Some fields experimental (e.g., for tool calls).

Senior considerations

  • Always add OTel for AI calls.
  • Don't enable sensitive data in prod (PII).
  • Do add custom tags for app context.
  • Cost dashboards save money — visibility = control.

Cross-references