Support invoke_workflow as an equivalent parent span to invoke_agent in FunctionInvokingChatClient#7382
Conversation
…tClient Co-authored-by: stephentoub <[email protected]>
invoke_workflow as an equivalent parent span to invoke_agent in FunctionInvokingChatClient
test/Libraries/Microsoft.Extensions.AI.Tests/ChatCompletion/FunctionInvokingChatClientTests.cs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull request overview
Updates Microsoft.Extensions.AI OpenTelemetry behavior so invoke_workflow parent spans are treated equivalently to invoke_agent in FunctionInvokingChatClient, aligning with newer GenAI semantic conventions.
Changes:
- Added
invoke_workflowoperation name constant in OpenTelemetry constants. - Generalized parent-span detection logic to match either
invoke_agentorinvoke_workflow(including “name suffix” forms like"invoke_workflow MyWorkflow"). - Added/extended unit tests to cover
invoke_workflowparity (span suppression, trace propagation, ActivitySource reuse, sensitive-data propagation, and near-miss cases).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs | Adds InvokeWorkflowName constant for GenAI operation name. |
| src/Libraries/Microsoft.Extensions.AI/ChatCompletion/FunctionInvokingChatClient.cs | Treats invoke_workflow as an equivalent grouping parent to invoke_agent via a shared display-name matcher helper. |
| test/Libraries/Microsoft.Extensions.AI.Tests/ChatCompletion/FunctionInvokingChatClientTests.cs | Adds comprehensive parity tests for invoke_workflow and extends non-match coverage. |
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/FunctionInvokingChatClient.cs
Outdated
Show resolved
Hide resolved
test/Libraries/Microsoft.Extensions.AI.Tests/ChatCompletion/FunctionInvokingChatClientTests.cs
Show resolved
Hide resolved
…ized tests Co-authored-by: stephentoub <[email protected]>
Head branch was pushed to by a user without write access
# Conflicts: # src/Libraries/Microsoft.Extensions.AI/ChatCompletion/FunctionInvokingChatClient.cs
tarekgh
left a comment
There was a problem hiding this comment.
@stephentoub I have resolved the merge conflict with the Realtime stuff that merged this morning. Thanks!
|
@copilot please fix the error: ##[error]src\Libraries\Microsoft.Extensions.AI\ChatCompletion\FunctionInvokingChatClient.cs(1279,1): error IDE0055: (NETCORE_ENGINEERING_TELEMETRY=Build) Fix formatting (https://learn.microsoft.com/dotnet/fundamentals/code-analysis/style-rules/ide0055) |
Co-authored-by: tarekgh <[email protected]>
…nt` in FunctionInvokingChatClient (dotnet#7382) Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: stephentoub <[email protected]> Co-authored-by: Tarek Mahmoud Sayed <[email protected]> Co-authored-by: Tarek Mahmoud Sayed <[email protected]>
InvokeWorkflowName = "invoke_workflow"constant toOpenTelemetryConsts.csIsActivityDisplayNameMatchhelper inFunctionInvokingChatClient.csand updateCurrentActivityIsInvokeAgentto also matchinvoke_workflowspansinvoke_workflowinvoke_workflowin existing tests rather than separate methodsOriginal prompt
Background
The OpenTelemetry semantic conventions PR open-telemetry/semantic-conventions#3249 (merged) adds a new
invoke_workflowoperation name to the GenAI semantic conventions. This represents a coordinated process composed of multiple agents or other operations involving generative AI — essentially a parent grouping span similar toinvoke_agent.Problem
Currently,
FunctionInvokingChatClienthas special behavior when it detects it is running inside aninvoke_agentspan:orchestrate_toolsspan — sinceinvoke_agentalready serves as a grouping spanActivitySource— so that childexecute_toolspans are properly parentedEnableSensitiveData— reads the custom property from the agent activityThe
invoke_workflowspan serves the same parent-grouping role, soFunctionInvokingChatClientshould treat it identically.Required Changes
1.
src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.csAdd a new constant after
InvokeAgentName:2.
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/FunctionInvokingChatClient.csUpdate
CurrentActivityIsInvokeAgentto also matchinvoke_workflowparent spans. The property currently checks ifActivity.Current.DisplayNamestarts with"invoke_agent"followed by end-of-string or a space. It should be generalized to also match"invoke_workflow"the same way. The cleanest approach:static bool IsActivityDisplayNameMatch(string? displayName, string operationName)that does the StartsWith + boundary checkCurrentActivityIsInvokeAgentto check bothInvokeAgentNameandInvokeWorkflowNameusing the helper (or rename the property to reflect both)This affects these call sites (no changes needed at call sites if the property itself is updated):
GetResponseAsync:using Activity? activity = CurrentActivityIsInvokeAgent ? null : _activitySource?.StartActivity(...)GetStreamingResponseAsync: same patternInstrumentedInvokeFunctionAsync:Activity? invokeAgentActivity = CurrentActivityIsInvokeAgent ? Activity.Current : null;3.
test/Libraries/Microsoft.Extensions.AI.Tests/ChatCompletion/FunctionInvokingChatClientTests.csAdd test cases mirroring existing
invoke_agenttests:DoesNotCreateOrchestrateToolsSpanWhenInvokeWorkflowIsParent— with[InlineData("invoke_workflow")],[InlineData("invoke_workflow my_workflow")],[InlineData("invoke_workflow ")]StreamingPreservesTraceContextWhenInvokeWorkflowWithNameIsParentUsesWorkflowActivitySourceWhenInvokeWorkflowIsParentSensitiveDataPropagatesFromWorkflowActivityWhenInvokeWorkflowIsParentCreatesOrchestrateToolsSpanWhenParentIsNotInvokeAgentto also testinvoke_workflow-like non-matches:"invoke_workflo","invoke_workflow_extra","invoke_workflowx"The following is the prior conversation context from the user's chat exploration (may be truncated):
User: Anything we need to do to support @open-telemetry/semantic-conventions/pull/3249 in Microsoft.Extensions.AI?
Assistant:
Let me investigate the PR and the current codebase to understand what changes might be needed.
Assistant: Now let me look more closely at the key files to understand how
invoke_agentis currently handled and whetherinvoke_workflowneeds similar support:Assistant: Now let me also check how the
CurrentActivityIsInvokeAgentproperty is used more broadly in the orchestrate_tools span creation logic:Assistant: Now let me also check how the non-streaming path uses this same pattern:
Assistant: Now let me also check the non-streaming
GetResponseAsyncfor the same pattern:Assistant: Now let me also check the non-streaming path for the
orchestrate_toolsspan:Assistant: Now let me find the specific usage in
GetResponseAsync(non-streaming) too:Assistant: Now I have a comprehensive understanding of the situation. Let me summarize my findings.
Analysis: Impact of
open-telemetry/semantic-conventions#3249onMicrosoft.Extensions.AIThe merged PR adds two things to the OpenTelemetry GenAI semantic conventions:
invoke_workflowoperation name — a new well-known value forgen_ai.operation.namegen_ai.workflow.nameattribute — a human-readable name for the workflowWhat currently exists in
dotnet/extensionsThere are two key areas in
Microsoft.Extensions.AIthat interact with theinvoke_agentoperation name and would need to be aware ofinvoke_workflow:1.
OpenTelemetryConsts.GenAI(OpenTelemetryConsts.cs)