Merge pull request #34089 from backstage/otel/mcp-tools-call

feat: Instrument MCP tool calls with semantically appropriate span

docs/ai/mcp-actions.md

@@ -302,3 +302,51 @@ The MCP Actions Backend emits metrics for the following operations:
 - `mcp.server.session.duration`: The duration of the MCP session from the perspective of the server
 
 See the [OpenTelemetry tutorial](../tutorials/setup-opentelemetry.md) to learn how to make these metrics available.
+
+## Tracing
+
+The MCP Actions Backend emits a trace span for each `tools/call` invocation via the [Tracing Service](../backend-system/core-services/tracing.md), following the [OpenTelemetry server-side MCP semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/mcp/#server). Each span uses the name `tools/call <toolname>`, server kind, and includes the standard MCP attributes (`mcp.method.name`, `gen_ai.tool.name`, `gen_ai.operation.name`). Known Backstage errors (such as `InputError` or `NotFoundError`) are caught and returned as `isError: true` tool responses — the span is marked with `error.type=tool_error` in that case. Unhandled exceptions are recorded automatically by the Tracing Service and the span status is set to `ERROR`.
+
+In addition to those attributes, the Tracing Service automatically attaches the authenticated principal's type as `backstage.principal.type` (one of `user`, `service`, or `none`). Each `tools/call` span is also attributed to the plugin that owns the invoked action via `backstage.plugin.id` (e.g. `catalog`, `scaffolder`) — overriding the default `mcp-actions` value so tracing backends can filter activity by the source plugin rather than by the MCP transport.
+
+### Baggage propagation
+
+The MCP Actions routers propagate OpenTelemetry context from the incoming HTTP request headers so that trace parent and baggage survive through the MCP transport layer. The following low-cardinality identifier entries from the OpenTelemetry [`gen_ai.*` attribute registry](https://opentelemetry.io/docs/specs/semconv/registry/attributes/gen-ai/), when set by the MCP client in baggage, are automatically forwarded as attributes on the `tools/call` span:
+
+- `gen_ai.agent.id`
+- `gen_ai.agent.name`
+- `gen_ai.conversation.id`
+- `gen_ai.provider.name`
+- `gen_ai.request.model`
+
+This enables tracing backends to correlate MCP tool invocations back to the originating agent, conversation, or model without additional configuration. Other `gen_ai.*` baggage entries are intentionally not forwarded — baggage may be set by arbitrary upstream callers, and a broad prefix filter would let clients smuggle high-cardinality or payload-shaped keys (e.g. `gen_ai.tool.call.result`, `gen_ai.prompt`) onto the span and bypass the [tool payload capture flag](#capturing-tool-arguments-and-results).
+
+### Capturing the authenticated end user
+
+The Tracing Service can additionally include the authenticated principal's identity as `enduser.id` (the user entity ref for a user principal, the service subject for a service principal). This is gated behind a backend-wide configuration flag and is **disabled by default**:
+
+```yaml title="app-config.yaml"
+backend:
+  tracing:
+    capture:
+      endUser: true # defaults to false
+```
+
+This flag is honored by every plugin that creates spans through the [Tracing Service](../backend-system/core-services/tracing.md), not just MCP Actions.
+
+### Capturing tool arguments and results
+
+When `mcpActions.tracing.capture.toolPayload` is enabled, the tool's input arguments and output result are recorded on the span as `gen_ai.tool.call.arguments` and `gen_ai.tool.call.result`.
+
+```yaml title="app-config.yaml"
+mcpActions:
+  tracing:
+    capture:
+      toolPayload: true # defaults to false
+```
+
+:::warning
+These attributes are marked Opt-In by the OpenTelemetry GenAI semantic conventions because they may contain sensitive information — entity payloads, scaffolder inputs, free-form text, and so on. Only enable this flag if your tracing backend's data handling is appropriate for the kinds of payloads your MCP tools accept and produce.
+:::
+
+See the [OpenTelemetry tutorial](../tutorials/setup-opentelemetry.md) to learn how to make these spans available.

docs/backend-system/core-services/tracing.md

@@ -100,6 +100,56 @@ The span object exposes:
 | `setAttribute(key, value)`     | Set a single attribute. Value is a primitive or array of primitives. |
 | `setStatus({ code, message })` | Set the span status. `code` is `'ok'`, `'error'`, or `'unset'`.      |
 
+## Context Propagation
+
+The tracing service exposes two sub-objects that mirror the corresponding namespaces in `@opentelemetry/api`:
+
+- `tracing.context` for context management (`active`, `with`).
+- `tracing.propagation` for context propagation (`extract`, `getBaggage`, `getActiveBaggage`).
+
+When your plugin handles a request through a transport or framework that doesn't automatically attach the caller's context to the work it runs (for example, a handler dispatched from a message-queue consumer, or a third-party transport like the MCP streamable HTTP transport that re-enters user code outside of Express's middleware chain), extract the trace parent and baggage from the inbound request's headers yourself and run the handler with that context active:
+
+```ts
+router.post('/', async (req, res) => {
+  const ctx = tracing.propagation.extract(
+    tracing.context.active(),
+    req.headers,
+  );
+  await tracing.context.with(ctx, () =>
+    transport.handleRequest(req, res, req.body),
+  );
+});
+```
+
+`propagation.extract` reads from a header-shaped record (`Record<string, string | string[] | undefined>`), so any source of headers — Express's `req.headers`, a Node.js `http.IncomingMessage`, or a payload field carrying serialized headers — works the same way.
+
+Any spans created inside the callback — including those from `startActiveSpan` — will be children of the propagated trace and will have access to the propagated baggage.
+
+The context returned by `propagation.extract` and `context.active` is an opaque handle: consumers pass it back into the API but do not introspect it.
+
+## Reading Baggage
+
+Use `propagation.getActiveBaggage()` to read baggage entries from the currently active context. This is useful for forwarding caller-set metadata onto your spans — for example, a request ID, tenant identifier, or feature-flag context that the caller propagated via baggage. The baggage is exposed as a flat list of entries — iterate through them to find the keys you care about:
+
+```ts
+const baggage = tracing.propagation.getActiveBaggage();
+for (const [key, entry] of baggage?.getAllEntries() ?? []) {
+  if (key === 'app.tenant.id') {
+    span.setAttribute('app.tenant.id', entry.value);
+  }
+}
+```
+
+Use `propagation.getBaggage(ctx)` when you already hold a specific context handle (for example, one returned by `propagation.extract`) and want to read its baggage without first activating the context.
+
+The returned object exposes:
+
+| Method            | Description                                  |
+| ----------------- | -------------------------------------------- |
+| `getAllEntries()` | Returns all entries as `[key, { value }][]`. |
+
+Both calls return `undefined` when no baggage is present. Single-key lookups are intentionally not provided — baggage is meant for bridging caller metadata onto spans or metrics, not as a general-purpose key-value store.
+
 ## Principal Enrichment
 
 When you supply either `credentials` or a `request`, the service adds principal-derived attributes to the span: