Skip to content

Emit gen_ai.client.operation.exception via ILogger LoggerMessage on OpenTelemetry instrumentation classes#7379

Merged
stephentoub merged 12 commits intomainfrom
copilot/add-exception-event-handling
Mar 12, 2026
Merged

Emit gen_ai.client.operation.exception via ILogger LoggerMessage on OpenTelemetry instrumentation classes#7379
stephentoub merged 12 commits intomainfrom
copilot/add-exception-event-handling

Conversation

Copy link
Contributor

Copilot AI commented Mar 9, 2026

  • Implement shared OpenTelemetryLog class with [LoggerMessage] method
  • Add _logger field and OpenTelemetryLog.OperationException call to OpenTelemetryChatClient
  • Add _logger field and OpenTelemetryLog.OperationException call to OpenTelemetryEmbeddingGenerator
  • Add _logger field and OpenTelemetryLog.OperationException call to OpenTelemetrySpeechToTextClient
  • Add _logger field and OpenTelemetryLog.OperationException call to OpenTelemetryImageGenerator
  • Add _logger field and OpenTelemetryLog.OperationException call to OpenTelemetryHostedFileClient
  • Add _logger field and OpenTelemetryLog.OperationException call to OpenTelemetryRealtimeClientSession
  • Add _logger field and OpenTelemetryLog.OperationException call to OpenTelemetryTextToSpeechClient
  • Add tests for exception logging in all OpenTelemetry instrumentation classes
  • Move OpenTelemetryLog.cs to Common folder
  • Remove exception.* constants and ExceptionEventName from OpenTelemetryConsts
  • Remove trailing empty line in OpenTelemetryChatClient.cs
  • Change LoggerMessage Message to "gen_ai.client.operation.exception"
  • Add changelog entry for the PR
Original prompt

Background

The OpenTelemetry semantic conventions have added a new gen_ai.client.operation.exception event for recording exceptions during Generative AI client operations. This was merged in open-telemetry/semantic-conventions#3436.

The new convention specifies:

  • Event name: gen_ai.client.operation.exception
  • When: SHOULD be recorded when an exception occurs during GenAI client operations
  • Severity: SHOULD be WARN (severity number 13)
  • Attributes:
    • exception.type (conditionally required) — fully-qualified exception class name
    • exception.message (conditionally required) — the exception message
    • exception.stacktrace (recommended) — stack trace as a string

This is part of a broader initiative to migrate exception recording from span events to log-based events. The convention notes that exception.message may contain sensitive information.

What needs to change

1. All four OpenTelemetry instrumentation classes need to emit the exception event

The following classes all have ILogger? logger parameters in their constructors that are currently unused (suppressed with #pragma warning disable IDE0060):

  • OpenTelemetryChatClient in src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryChatClient.cs
  • OpenTelemetryEmbeddingGenerator<TInput, TEmbedding> in src/Libraries/Microsoft.Extensions.AI/Embeddings/OpenTelemetryEmbeddingGenerator.cs
  • OpenTelemetrySpeechToTextClient in src/Libraries/Microsoft.Extensions.AI/SpeechToText/OpenTelemetrySpeechToTextClient.cs
  • OpenTelemetryImageGenerator in src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryImageGenerator.cs

For each class:

a. Store and use the ILogger parameter

  • Add a private readonly ILogger? _logger; field
  • Store the logger parameter in the constructor: _logger = logger;
  • Remove the #pragma warning disable/restore IDE0060 around the constructor since the parameter is now used
  • Remove any comments that say the parameter is unused / exists for backwards compatibility

b. Emit the exception event in the TraceResponse method

In each class's TraceResponse method, when error is not null, in addition to the existing span status/tag logic, emit the exception event using Activity.AddEvent. Use Activity.AddEvent because the .NET ILogger does not natively support the OTel event model with event names in the way the convention requires (the event name must be gen_ai.client.operation.exception, and ILogger doesn't map cleanly to OTel log events with specific event names in the current .NET OTel SDK). Using Activity.AddEvent is the pragmatic choice that matches how .NET OTel instrumentation works today.

The event should be added to the activity like this:

if (error is not null && activity is not null)
{
    var exceptionTags = new ActivityTagsCollection
    {
        { "exception.type", error.GetType().FullName },
        { "exception.message", error.Message },
        { "exception.stacktrace", error.ToString() },
    };

    activity.AddEvent(new ActivityEvent(
        "gen_ai.client.operation.exception",
        tags: exceptionTags));
}

This should be placed inside the existing if (error is not null) block in TraceResponse, alongside (not replacing) the existing error.type tag and SetStatus(ActivityStatusCode.Error) calls.

c. Add the event name constant to OpenTelemetryConsts

In src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs, add to the GenAI.Client class:

public static class Operation
{
    public const string ExceptionEventName = "gen_ai.client.operation.exception";
}

And add exception attribute constants:

public static class Exception
{
    public const string Type = "exception.type";
    public const string Message = "exception.message";
    public const string Stacktrace = "exception.stacktrace";
}

Use these constants in all four classes instead of string literals.

2. Existing code reference for each class

OpenTelemetryChatClient — current error handling in TraceResponse (lines 710-715):

if (error is not null)
{
    _ = activity?
        .AddTag(OpenTelemetryConsts.Error.Type, error.GetType().FullName)
        .SetStatus(ActivityStatusCode.Error, error.Message);
}

OpenTelemetryEmbeddingGenerator — similar pattern in its TraceResponse method:

if (error is not null)
{
    _ = activity
        .AddTag(OpenTelemetryConsts.Error.Type, error.GetType().FullName)
        .SetStatus(ActivityStatusCode.Error, error.Message);
}

OpenTelemetrySpeechToTextClient — similar pattern in its TraceResponse method.

OpenTelemetryImageGenerator — similar pattern in its TraceResponse method.

3. Tests

Add or update tests for each class to verify that when an exception occurs:

  • The existing error.type tag and error status are...

This pull request was created from Copilot chat.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Microsoft Reviewers: Open in CodeFlow

Copilot AI changed the title [WIP] Add gen_ai.client.operation.exception event for exception handling Emit gen_ai.client.operation.exception Activity event on OpenTelemetry instrumentation classes Mar 9, 2026
@stephentoub stephentoub marked this pull request as ready for review March 10, 2026 00:49
@stephentoub stephentoub requested a review from a team as a code owner March 10, 2026 00:49
Copilot AI review requested due to automatic review settings March 10, 2026 00:49
@stephentoub stephentoub requested a review from tarekgh March 10, 2026 00:49
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the Microsoft.Extensions.AI OpenTelemetry instrumentation to emit a dedicated exception ActivityEvent (with exception.* tags) when operations fail, and adds unit tests across the AI client surface area to validate the new telemetry behavior.

Changes:

  • Add a new OpenTelemetry event name constant and exception.type / exception.message / exception.stacktrace tag constants.
  • Emit a gen_ai.client.operation.exception ActivityEvent on error for Chat, Image, Embeddings, Hosted Files, and Speech-to-Text telemetry wrappers.
  • Add new tests asserting the exception event is emitted and existing error tags/status are preserved.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs Adds constants for exception event name and exception.* tag keys.
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryChatClient.cs Emits exception ActivityEvent when a chat operation throws.
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryImageGenerator.cs Emits exception ActivityEvent when image generation throws.
src/Libraries/Microsoft.Extensions.AI/Embeddings/OpenTelemetryEmbeddingGenerator.cs Emits exception ActivityEvent when embedding generation throws.
src/Libraries/Microsoft.Extensions.AI/Files/OpenTelemetryHostedFileClient.cs Emits exception ActivityEvent when hosted file operations throw.
src/Libraries/Microsoft.Extensions.AI/SpeechToText/OpenTelemetrySpeechToTextClient.cs Emits exception ActivityEvent when speech-to-text operations throw.
test/Libraries/Microsoft.Extensions.AI.Tests/ChatCompletion/OpenTelemetryChatClientTests.cs Adds tests validating exception ActivityEvent for streaming/non-streaming chat calls.
test/Libraries/Microsoft.Extensions.AI.Tests/Image/OpenTelemetryImageGeneratorTests.cs Adds test validating exception ActivityEvent for image generation.
test/Libraries/Microsoft.Extensions.AI.Tests/Embeddings/OpenTelemetryEmbeddingGeneratorTests.cs Adds test validating exception ActivityEvent for embedding generation.
test/Libraries/Microsoft.Extensions.AI.Tests/Files/OpenTelemetryHostedFileClientTests.cs Adds test validating exception ActivityEvent for file upload.
test/Libraries/Microsoft.Extensions.AI.Tests/SpeechToText/OpenTelemetrySpeechToTextClientTests.cs Adds tests validating exception ActivityEvent for streaming/non-streaming STT calls.

@tarekgh
Copy link
Member

tarekgh commented Mar 10, 2026

@copilot should Activity.AddException get used instead of AddEvent? The difference is if there is any registered ActivityListner, can get the notification of the exception registration.

@tarekgh
Copy link
Member

tarekgh commented Mar 10, 2026

@stephentoub I confirmed as we previously discussed, according to the OTel plan, the exception should be reported through the log event and not the span/activity event. I think we should follow that. open-telemetry/semantic-conventions#3436 (comment)

@stephentoub
Copy link
Member

@tarekgh what does that look like? Are you saying this should use ILogger, and if so, can you share what the call site should look like?

@tarekgh
Copy link
Member

tarekgh commented Mar 10, 2026

I believe the following should do it. OpenTelelmtry exporter should handle the exception correctly and emit its parts according to the specs.

    [LoggerMessage(
        EventName = "gen_ai.client.operation.exception",
        Level = LogLevel.Warning,
        Message = "A GenAI client operation exception occurred.")]
    private static partial void LogOperationException(ILogger logger, Exception error);

and the call site will be like:

    if (error is not null)
    {
        _ = activity?
            .AddTag(OpenTelemetryConsts.Error.Type, error.GetType().FullName)
            .SetStatus(ActivityStatusCode.Error, error.Message);

        if (_logger is not null)
        {
            LogOperationException(_logger, error);
        }
    }

CC @lmolkova @CodeBlanch

@stephentoub
Copy link
Member

Activity.AddEvent is used rather than ILogger because the OTel event model requires a specific event name (gen_ai.client.operation.exception) that doesn't map cleanly to ILogger in the current .NET OTel SDK.

Ok, thanks. I'd initially planned on going that route, but copilot suggested it didn't work:

Activity.AddEvent is used rather than ILogger because the OTel event model requires a specific event name (gen_ai.client.operation.exception) that doesn't map cleanly to ILogger in the current .NET OTel SDK.

@copilot, please redo this PR, revising it to store the ILogger in the ctor into a field in each of the implementations, and then following the approach that @tarekgh outlines in #7379 (comment).

Copy link
Contributor Author

Copilot AI commented Mar 10, 2026

Activity.AddEvent is used rather than ILogger because the OTel event model requires a specific event name (gen_ai.client.operation.exception) that doesn't map cleanly to ILogger in the current...

Redone in 7aa2b78. All five OTel classes now store the ILogger in a _logger field and use the [LoggerMessage] pattern per @tarekgh's approach:

[LoggerMessage(
    EventName = "gen_ai.client.operation.exception",
    Level = LogLevel.Warning,
    Message = "A GenAI client operation exception occurred.")]
private static partial void LogOperationException(ILogger logger, Exception error);

Call site:

_ = activity?
    .AddTag(OpenTelemetryConsts.Error.Type, error.GetType().FullName)
    .SetStatus(ActivityStatusCode.Error, error.Message);

if (_logger is not null)
{
    LogOperationException(_logger, error);
}

Removed the exception.* constants and ExceptionEventName from OpenTelemetryConsts since the OTel exporter handles those automatically from the Exception parameter. All 512 tests pass, 0 warnings.

I believe the following should do it. OpenTelelmtry exporter should handle the exception correctly and emit its parts according to the specs.

...

Implemented exactly this pattern in 7aa2b78 across all five OTel classes. The ILogger is now stored from the constructor and LogOperationException is called at each error site.

Copilot AI changed the title Emit gen_ai.client.operation.exception Activity event on OpenTelemetry instrumentation classes Emit gen_ai.client.operation.exception via ILogger LoggerMessage on OpenTelemetry instrumentation classes Mar 10, 2026
@stephentoub
Copy link
Member

@copilot please also update the corresponding opentelemetry support for ITextToSpeechClient, and add a line for the whole PR to the Microsoft.Extensions.AI changelog file under the not yet released section.

Copy link
Contributor Author

Copilot AI commented Mar 12, 2026

@copilot please also update the corresponding opentelemetry support for ITextToSpeechClient, and add a line for the whole PR to the Microsoft.Extensions.AI changelog file under the not...

Done in fdedf65. OpenTelemetryTextToSpeechClient now stores the ILogger in a _logger field (removing the IDE0060 pragma) and calls OpenTelemetryLog.OperationException(_logger, error) in the error path. Added ExceptionLogged_Async test (streaming + non-streaming) and a changelog entry under NOT YET RELEASED.

@stephentoub
Copy link
Member

@copilot resolve the conflicts

Copilot AI requested a review from a team as a code owner March 12, 2026 02:40
Copy link
Contributor Author

Copilot AI commented Mar 12, 2026

@copilot resolve the conflicts

Resolved in c3ce851. Merged main and fixed the CHANGELOG.md conflict — our entry and the TTS middleware entry are both under "NOT YET RELEASED", with the new ## 10.4.0 section below.

Copilot AI and others added 12 commits March 11, 2026 22:56
…reporting per OTel convention

Store ILogger in _logger field in all five OpenTelemetry classes, add [LoggerMessage]
partial method with EventName="gen_ai.client.operation.exception", replace Activity.AddEvent
with LogOperationException call. Remove unused exception.* constants from OpenTelemetryConsts.
Update all tests to verify ILogger output via FakeLogCollector.

Co-authored-by: stephentoub <[email protected]>
@stephentoub stephentoub force-pushed the copilot/add-exception-event-handling branch from c3ce851 to ca86a76 Compare March 12, 2026 02:57
@stephentoub stephentoub enabled auto-merge (squash) March 12, 2026 02:57
@stephentoub stephentoub merged commit 4457a07 into main Mar 12, 2026
6 checks passed
@stephentoub stephentoub deleted the copilot/add-exception-event-handling branch March 12, 2026 04:03
jeffhandley pushed a commit to jeffhandley/extensions that referenced this pull request Mar 17, 2026
…penTelemetry instrumentation classes (dotnet#7379)

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: stephentoub <[email protected]>
Co-authored-by: tarekgh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants