From 90007d5623687261e5212506e3855da43e1304af Mon Sep 17 00:00:00 2001 From: Kurt King Date: Mon, 23 Jun 2025 21:22:32 -0600 Subject: [PATCH 01/16] start bep for telemetry service Signed-off-by: Kurt King --- beps/0012-telemetry-service/README.md | 85 +++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 beps/0012-telemetry-service/README.md diff --git a/beps/0012-telemetry-service/README.md b/beps/0012-telemetry-service/README.md new file mode 100644 index 0000000000..ae7b269a7d --- /dev/null +++ b/beps/0012-telemetry-service/README.md @@ -0,0 +1,85 @@ +--- +title: Backstage Telemetry Service +status: implementable +authors: + - '@kurtaking' +owners: + - '@kurtaking' +project-areas: + - backend + - framework +creation-date: 2025-06-23 +--- + + + +# BEP: Backstage Telemetry Service + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) +- [Design Details](#design-details) +- [Release Plan](#release-plan) +- [Dependencies](#dependencies) +- [Alternatives](#alternatives) + +## Summary + +Add a core telemetry service to Backstage's backend system that provides a unified interface for metrics, tracing, and logging instrumentation. The service would offer OpenTelemetry-based telemetry capabilities with zero configuration defaults while allowing extensibility through plugin modules. + +## Motivation + +While individual plugins may implement their own metrics or tracing, there's no unified service that provides a standardized approach for telemetry and observability. This gap makes it difficult for operators to monitor Backstage deployments effectively and for plugin developers to add observability to their plugins consistently. + +### Goals + +- Consistent telemetry patterns across all plugins +- Zero-configuration defaults that work out of the box +- Extensible architecture for custom telemetry backends +- OpenTelemetry integration following industry standards +- Performance monitoring capabilities built into the platform + +### Non-Goals + +## Proposal + + + +## Design Details + + + +## Release Plan + + + +## Dependencies + + + +## Alternatives + + From 319199f77bab55f5fed7d4b7afb33d0d66d56334 Mon Sep 17 00:00:00 2001 From: Kurt King Date: Wed, 25 Jun 2025 08:29:40 -0600 Subject: [PATCH 02/16] doc: backstage telemetry service Signed-off-by: Kurt King --- beps/0012-telemetry-service/README.md | 44 +++++++++------------------ 1 file changed, 15 insertions(+), 29 deletions(-) diff --git a/beps/0012-telemetry-service/README.md b/beps/0012-telemetry-service/README.md index ae7b269a7d..a85edccad8 100644 --- a/beps/0012-telemetry-service/README.md +++ b/beps/0012-telemetry-service/README.md @@ -7,14 +7,11 @@ owners: - '@kurtaking' project-areas: - backend + - core - framework creation-date: 2025-06-23 --- - - # BEP: Backstage Telemetry Service - [Summary](#summary) @@ -33,10 +30,12 @@ Add a core telemetry service to Backstage's backend system that provides a unifi ## Motivation -While individual plugins may implement their own metrics or tracing, there's no unified service that provides a standardized approach for telemetry and observability. This gap makes it difficult for operators to monitor Backstage deployments effectively and for plugin developers to add observability to their plugins consistently. +There is currently no guidance for plugin authors when it comes to telemetry and while individual plugins may implement their own metrics or tracing, there's no unified service that provides a standardized approach for telemetry and observability. This gap makes it difficult for operators to monitor Backstage deployments effectively and for plugin developers to add observability to their plugins consistently. ### Goals +Provide a unified service that provides a standardized approach for telemetry where setup docs like [these](https://backstage.io/docs/tutorials/setup-opentelemetry/) are not needed. + - Consistent telemetry patterns across all plugins - Zero-configuration defaults that work out of the box - Extensible architecture for custom telemetry backends @@ -45,41 +44,28 @@ While individual plugins may implement their own metrics or tracing, there's no ### Non-Goals +- We will not backfill missing telemetry for existing plugins. +- We will not recreate the OTEL Node SDK + ## Proposal - +Following similar patterns to other core services, we will create a new `RootTelemetryService` responsible for initializing telemetry and app-wide concerns, and the creation of plugin-specific telemetry services. ## Design Details - +We will provide a thin wrapper around the OpenTelemetry Node SDK to provide a consistent interface while re-exporting the types from the SDK. This lets us provide a consistent telemetry interface while leveraging types and concepts already familiar to the community. + +The decision to leverage the SDK comes from the fact that OpenTelemetry is already the fundamental abstraction for metrics and tracing. ## Release Plan - +TBD ## Dependencies - +- There are one-off implementations of telemetry in the wild that may conflict with the proposed service. ## Alternatives - +1. Plugin authors continue to implement their own telemetry as they see fit. +2. Adopters continue using the [setup-opentelemetry](https://backstage.io/docs/tutorials/setup-opentelemetry/) docs as a guide. From af3260dd120ee25b32238132c51fd02e1eef41ee Mon Sep 17 00:00:00 2001 From: Kurt King Date: Wed, 25 Jun 2025 09:07:10 -0600 Subject: [PATCH 03/16] spruce up a bit more and include config Signed-off-by: Kurt King --- beps/0012-telemetry-service/README.md | 81 +++++++++++++++++++++++++-- 1 file changed, 75 insertions(+), 6 deletions(-) diff --git a/beps/0012-telemetry-service/README.md b/beps/0012-telemetry-service/README.md index a85edccad8..c421f3f2d3 100644 --- a/beps/0012-telemetry-service/README.md +++ b/beps/0012-telemetry-service/README.md @@ -20,13 +20,18 @@ creation-date: 2025-06-23 - [Non-Goals](#non-goals) - [Proposal](#proposal) - [Design Details](#design-details) + - [Configuration](#configuration) - [Release Plan](#release-plan) - [Dependencies](#dependencies) - [Alternatives](#alternatives) ## Summary -Add a core telemetry service to Backstage's backend system that provides a unified interface for metrics, tracing, and logging instrumentation. The service would offer OpenTelemetry-based telemetry capabilities with zero configuration defaults while allowing extensibility through plugin modules. +Add a core telemetry service to Backstage's framework that provides a unified interface for metrics, tracing, and logging instrumentation. The service would offer OpenTelemetry-based telemetry capabilities with zero configuration defaults while allowing extensibility through app configuration and plugin modules. + +- **Plugin Authors** gain a straightforward, documented way to address telemetry, and can focus on business logic instead of reimplementing telemetry plumbing. +- The **community** receives a standard approach to telemetry. +- **Backstage Operators/SRE/Platform Engineers** receive a consistent, reliable stream of metrics from all plugins and the core system, simplifying monitoring, alerting, and troubleshooting. They can enforce metrics standards, aggregate data, and build dashboards more easily. Organizations with their own observability or compliance requirements can override the core service to integrate with proprietary or custom telemetry solutions, without forking or modifying plugins. ## Motivation @@ -42,10 +47,17 @@ Provide a unified service that provides a standardized approach for telemetry wh - OpenTelemetry integration following industry standards - Performance monitoring capabilities built into the platform +We will include the following two plugins in the initial release to demonstrate the new telemetry service: + +- Replace existing catalog implementation with `TelemetryService` +- Replace existing scaffolder implementation with `TelemetryService` + ### Non-Goals -- We will not backfill missing telemetry for existing plugins. -- We will not recreate the OTEL Node SDK +- We will not backfill missing telemetry for existing plugins +- We will not refactor existing telemetry implementations OTHER THAN the explicility mentioned catalog and scaffolder implementations. +- We will not start by providing a full Backstage SDK (aka recreate the OTEL Node SDK). +- We will not refactor the existing LoggerService. Future work to unify the two would be ideal, but not a goal. ## Proposal @@ -53,17 +65,74 @@ Following similar patterns to other core services, we will create a new `RootTel ## Design Details -We will provide a thin wrapper around the OpenTelemetry Node SDK to provide a consistent interface while re-exporting the types from the SDK. This lets us provide a consistent telemetry interface while leveraging types and concepts already familiar to the community. +We will provide a thin wrapper around the OpenTelemetry Node SDK to provide a consistent interface while re-exporting the types from the SDK. This lets us provide a consistent telemetry interface while leveraging types and concepts already familiar to the community. Additionally, we will rely on the auto insstrumentation features of the SDK as much as possible. The decision to leverage the SDK comes from the fact that OpenTelemetry is already the fundamental abstraction for metrics and tracing. +```ts +interface TelemetryService { + createCounter(name: string, options?: MetricOptions): Counter; + // + + // provide escape hatch for advanced use cases (???) - todo: I don't like this if we don't have a good reason for it + getMeter(): Meter; +} + +interface RootTelemetryService extends TelemetryService { + forPlugin(pluginId: string): TelemetryService; + + // provide escape hatches for advanced use cases (???) + getClient(): NodeSDK; // give access to the SDK +} +``` + +### Configuration + +All OTEL configuration will be supported via the app's `app-config.yaml` file. + +```yaml +# this is not a complete example, but it shows the general structure +telemetry: + enabled: true + + resource: + serviceName: backstage-telemetry + serviceVersion: 0.0.1 + + metrics: + enabled: true + collection: + exportIntervalMillis: 15000 + + exporters: + - type: prometheus + enabled: true + config: + port: 9464 + + tracing: + enabled: false + + instrumentations: + http: false + knex: false + express: true + # ... +``` + ## Release Plan -TBD +1. Create a new `RootTelemetryService` that initializes telemetry and app-wide concerns. +2. Create the plugin-scoped `TelemetryService` that provides a telemetry service for plugins. +3. Release the telemetry service as alpha. +4. Refactor catalog and scaffolder implementations to use the new (alpha) `TelemetryService`. +5. Replace the existing setup docs by intergrating the new `RootTelemetryService` to `createBackend` +6. ...TBD ## Dependencies -- There are one-off implementations of telemetry in the wild that may conflict with the proposed service. +1. The root telemetry service MUST BE initialized as EARLY as possible to prevent dependents from receiving no op meters, while still allowing for app-wide configuration. +2. There are one-off implementations of telemetry in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. ## Alternatives From 06479e1da291c63851b0f4f8c330a1ff1ca0316d Mon Sep 17 00:00:00 2001 From: Kurt King Date: Thu, 26 Jun 2025 09:49:36 -0600 Subject: [PATCH 04/16] Add considerations, outstanding questions, fix typos Signed-off-by: Kurt King --- beps/0012-telemetry-service/README.md | 72 ++++++++++++++++++++------- 1 file changed, 54 insertions(+), 18 deletions(-) diff --git a/beps/0012-telemetry-service/README.md b/beps/0012-telemetry-service/README.md index c421f3f2d3..31cf275034 100644 --- a/beps/0012-telemetry-service/README.md +++ b/beps/0012-telemetry-service/README.md @@ -27,15 +27,21 @@ creation-date: 2025-06-23 ## Summary -Add a core telemetry service to Backstage's framework that provides a unified interface for metrics, tracing, and logging instrumentation. The service would offer OpenTelemetry-based telemetry capabilities with zero configuration defaults while allowing extensibility through app configuration and plugin modules. +Add a core telemetry service to Backstage's framework that provides a unified interface for metrics, tracing, and logging instrumentation. The service offers OpenTelemetry-based capabilities with zero configuration defaults while allowing extensibility through app configuration (e.g. `app-config.yaml`) and plugin modules. -- **Plugin Authors** gain a straightforward, documented way to address telemetry, and can focus on business logic instead of reimplementing telemetry plumbing. -- The **community** receives a standard approach to telemetry. -- **Backstage Operators/SRE/Platform Engineers** receive a consistent, reliable stream of metrics from all plugins and the core system, simplifying monitoring, alerting, and troubleshooting. They can enforce metrics standards, aggregate data, and build dashboards more easily. Organizations with their own observability or compliance requirements can override the core service to integrate with proprietary or custom telemetry solutions, without forking or modifying plugins. +## Considerations + +1. It's possible a single telemetry service is too encompassing and that more granular services are needed. For example, a `MetricsService` and `TracingService` which would work along side the existing `LoggerService` to solve for telemetry needs. ## Motivation -There is currently no guidance for plugin authors when it comes to telemetry and while individual plugins may implement their own metrics or tracing, there's no unified service that provides a standardized approach for telemetry and observability. This gap makes it difficult for operators to monitor Backstage deployments effectively and for plugin developers to add observability to their plugins consistently. +There is currently no guidance for plugin authors when it comes to telemetry and while individual plugins may implement their own metrics or tracing, there's no unified service that provides a standardized approach for telemetry and observability. + +By providing a core telemetry service: + +- **Plugin Authors** gain a straightforward, documented way to address telemetry, and can focus on business logic instead of needing to reimplement telemetry plumbing. +- The **community** receives a standard approach to telemetry. +- **Backstage Operators/SRE/Platform Engineers** receive a consistent, reliable stream of metrics from the core system, simplifying monitoring, alerting, and troubleshooting. They can enforce metrics standards, aggregate data, and build dashboards more easily. -**Organizations** with their own observability or compliance requirements can override the core service to integrate with proprietary or custom telemetry solutions, without forking or modifying plugins. ### Goals @@ -54,18 +60,20 @@ We will include the following two plugins in the initial release to demonstrate ### Non-Goals -- We will not backfill missing telemetry for existing plugins -- We will not refactor existing telemetry implementations OTHER THAN the explicility mentioned catalog and scaffolder implementations. +- Adding telemetry instrumentation to existing plugins that don't currently have it +- Only the catalog and scaffolder implementations will be refactored to use the new telemetry service. - We will not start by providing a full Backstage SDK (aka recreate the OTEL Node SDK). -- We will not refactor the existing LoggerService. Future work to unify the two would be ideal, but not a goal. +- We will not refactor the existing LoggerService. Future work to unify telemetry related concerns would be ideal, but not a goal. ## Proposal Following similar patterns to other core services, we will create a new `RootTelemetryService` responsible for initializing telemetry and app-wide concerns, and the creation of plugin-specific telemetry services. +A core concern is not initializing the telemetry service early enough in the backend lifecycle. Services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping functionality that will be called as early as possible in the backend lifecycle. + ## Design Details -We will provide a thin wrapper around the OpenTelemetry Node SDK to provide a consistent interface while re-exporting the types from the SDK. This lets us provide a consistent telemetry interface while leveraging types and concepts already familiar to the community. Additionally, we will rely on the auto insstrumentation features of the SDK as much as possible. +We will provide a thin wrapper around the OpenTelemetry Node SDK to provide a consistent interface while re-exporting the types from the SDK. This lets us provide a consistent telemetry interface while leveraging types and concepts already familiar to the community. Additionally, we will rely on the auto instrumentation features of the SDK as much as possible. The decision to leverage the SDK comes from the fact that OpenTelemetry is already the fundamental abstraction for metrics and tracing. @@ -86,12 +94,28 @@ interface RootTelemetryService extends TelemetryService { } ``` +### Bootstrapping + +A core concern is not initializing the telemetry service early enough in the backend lifecycle and services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping mechanism that will be called as early as possible in the backend lifecycle. + +Ideally, this is called at the start of the `createBackend` function [[ref](https://github.com/backstage/backstage/blob/ab539cc72bb503618df55815f24c4ba933607460/packages/backend-defaults/src/CreateBackend.ts#L73-L75)], but we need to gain access to the config file at minimum. Ideally the logger as well, but not as crucial. + +```ts +export function createBackend(): Backend { + // ... + // call the bootstrap function + Telemetry.bootstrap(); + // ... + return createSpecializedBackend({ defaultServiceFactories }); +} +``` + ### Configuration -All OTEL configuration will be supported via the app's `app-config.yaml` file. +All OTEL configuration will be supported via the app's `app-config.yaml` file. The config schema should be based on existing resources such as the [OTEL collector configuration](https://opentelemetry.io/docs/collector/configuration/). ```yaml -# this is not a complete example, but it shows the general structure +# this is not a complete example, but it shows the general idea telemetry: enabled: true @@ -123,18 +147,30 @@ telemetry: ## Release Plan 1. Create a new `RootTelemetryService` that initializes telemetry and app-wide concerns. -2. Create the plugin-scoped `TelemetryService` that provides a telemetry service for plugins. -3. Release the telemetry service as alpha. -4. Refactor catalog and scaffolder implementations to use the new (alpha) `TelemetryService`. -5. Replace the existing setup docs by intergrating the new `RootTelemetryService` to `createBackend` -6. ...TBD +1. Create the plugin-scoped `TelemetryService` that provides a telemetry service for plugins. +1. Release the telemetry service as alpha. +1. Refactor catalog and scaffolder implementations to use the new (alpha) `TelemetryService`. +1. Replace the existing setup docs by integrating the new `RootTelemetryService` to `createBackend` +1. Release the telemetry service +1. Offer a migration path for existing adopters to migrate to the new telemetry service. +1. Update all documentation to reference the new telemetry service. +1. Create follow-up action items to integrate the new telemetry service into the core system. +1. Fully deprecate all existing telemetry implementations like the existing Prometheus one-off implementations. ## Dependencies 1. The root telemetry service MUST BE initialized as EARLY as possible to prevent dependents from receiving no op meters, while still allowing for app-wide configuration. -2. There are one-off implementations of telemetry in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. +1. There are one-off implementations of telemetry in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. ## Alternatives 1. Plugin authors continue to implement their own telemetry as they see fit. -2. Adopters continue using the [setup-opentelemetry](https://backstage.io/docs/tutorials/setup-opentelemetry/) docs as a guide. +1. Adopters continue using the [setup-opentelemetry](https://backstage.io/docs/tutorials/setup-opentelemetry/) docs as a guide. + +## Outstanding Questions + +1. Are there enough Backstage-isms to build an additional layer of abstraction for metrics and tracing on top of the OpenTelemetry SDK? Do we re-export the types from the OpenTelemetry SDK or do we create our own? How do we handle breaking changes? +1. How do we guarantee that the telemetry service is initialized early enough in the backend lifecycle, while still allowing for app-wide configuration? +1. Should we provide an escape hatch for advanced use cases? +1. Should there be a single telemetry service or should we provide more granular services (e.g. adding separate `MetricsService` and `TracingService`)? +1. What core services could leverage the auto instrumentation features of the OpenTelemetry SDK? (e.g. `DatabaseService` could leverage the `knex` auto instrumentation, `HttpService` could leverage the `express` auto instrumentation, etc.) From 5201ad4d29e1c121bc8210a92225a4f8b855928c Mon Sep 17 00:00:00 2001 From: Kurt King Date: Thu, 26 Jun 2025 10:01:53 -0600 Subject: [PATCH 05/16] reorder sections Signed-off-by: Kurt King --- beps/0012-telemetry-service/README.md | 70 +++++++++++++++------------ 1 file changed, 40 insertions(+), 30 deletions(-) diff --git a/beps/0012-telemetry-service/README.md b/beps/0012-telemetry-service/README.md index 31cf275034..acc9d0026d 100644 --- a/beps/0012-telemetry-service/README.md +++ b/beps/0012-telemetry-service/README.md @@ -71,6 +71,40 @@ Following similar patterns to other core services, we will create a new `RootTel A core concern is not initializing the telemetry service early enough in the backend lifecycle. Services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping functionality that will be called as early as possible in the backend lifecycle. +## Configuration + +All OTEL configuration will be supported via the app's `app-config.yaml` file. The config schema should be based on existing resources such as the [OTEL collector configuration](https://opentelemetry.io/docs/collector/configuration/). + +```yaml +# this is not a complete example, but it shows the general idea +telemetry: + enabled: true + + resource: + serviceName: backstage-telemetry + serviceVersion: 0.0.1 + + metrics: + enabled: true + collection: + exportIntervalMillis: 15000 + + exporters: + - type: prometheus + enabled: true + config: + port: 9464 + + tracing: + enabled: false + + instrumentations: + http: false + knex: false + express: true + # ... +``` + ## Design Details We will provide a thin wrapper around the OpenTelemetry Node SDK to provide a consistent interface while re-exporting the types from the SDK. This lets us provide a consistent telemetry interface while leveraging types and concepts already familiar to the community. Additionally, we will rely on the auto instrumentation features of the SDK as much as possible. @@ -110,39 +144,15 @@ export function createBackend(): Backend { } ``` -### Configuration +### Plugins & Modules -All OTEL configuration will be supported via the app's `app-config.yaml` file. The config schema should be based on existing resources such as the [OTEL collector configuration](https://opentelemetry.io/docs/collector/configuration/). +WIP -```yaml -# this is not a complete example, but it shows the general idea -telemetry: - enabled: true + ## Release Plan From 5d634e2000421ad6dcef138d3da4fe8015f2280a Mon Sep 17 00:00:00 2001 From: Kurt King Date: Thu, 26 Jun 2025 22:29:19 -0600 Subject: [PATCH 06/16] Add separate document for naming conventions Signed-off-by: Kurt King --- beps/0012-telemetry-service/README.md | 4 + .../README.md | 113 ++++++++++++++++++ 2 files changed, 117 insertions(+) create mode 100644 beps/0013-telemetry-metrics-naming-conventions/README.md diff --git a/beps/0012-telemetry-service/README.md b/beps/0012-telemetry-service/README.md index acc9d0026d..c316d05ca8 100644 --- a/beps/0012-telemetry-service/README.md +++ b/beps/0012-telemetry-service/README.md @@ -128,6 +128,10 @@ interface RootTelemetryService extends TelemetryService { } ``` +### Naming Conventions + +Visit [0013-telemetry-metrics-naming-conventions](../0013-telemetry-metrics-naming-conventions/README.md) for the naming conventions. + ### Bootstrapping A core concern is not initializing the telemetry service early enough in the backend lifecycle and services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping mechanism that will be called as early as possible in the backend lifecycle. diff --git a/beps/0013-telemetry-metrics-naming-conventions/README.md b/beps/0013-telemetry-metrics-naming-conventions/README.md new file mode 100644 index 0000000000..c6cbc85d24 --- /dev/null +++ b/beps/0013-telemetry-metrics-naming-conventions/README.md @@ -0,0 +1,113 @@ +--- +title: OpenTelemetry Metrics Naming Conventions for Backstage +status: implementable +authors: + - '@kurtaking' +owners: + - '@kurtaking' +project-areas: + - framework + - telemetry +creation-date: 2025-06-26 +--- + +# BEP: OpenTelemetry Metrics Naming Conventions for Backstage + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) +- [Design Details](#design-details) +- [Release Plan](#release-plan) +- [Dependencies](#dependencies) +- [Alternatives](#alternatives) + +## Summary + +This proposal establishes standardized naming conventions for OpenTelemetry metrics across the Backstage ecosystem. It defines a hierarchical naming pattern that ensures consistency, discoverability, and compliance with OpenTelemetry semantic conventions while providing clear guidance for plugin and module authors. + +## Motivation + +Currently, Backstage has: + +- Mixed metric naming patterns: Legacy Prometheus metrics (`catalog_entities_count`) alongside newer OpenTelemetry metrics (`catalog.processing.queue.delay`) +- No standard pattern for plugin-specific metrics +- Difficulty correlating metrics across plugins and providers +- Some existing metrics don't follow OpenTelemetry semantic conventions + +We need to decided on a pattern that we can use for the telemetry service to obfuscate the underlying implementation details. + +### Goals + +- Establish consistent naming patterns across all Backstage metrics +- Ensure OpenTelemetry compliance with semantic conventions +- Provide clear guidance for plugin and service authors + +### Non-Goals + +- Breaking existing metrics immediately (backward compatibility maintained) +- Forcing all plugins to emit the same metrics +- Defining specific metric values or thresholds + +## Proposal + +Provide guidance for naming metrics for Backstage. + +## Design Details + +### Core Naming Pattern + +All Backstage metrics MUST follow this hierarchical pattern: + +`backstage.{scope}.{category}.{metric_name}` + +**Where:** + +- `backstage` is the root namespace for all Backstage metrics +- `{scope}` is the system scope (`pluginId`, `core` for core services) +- `{category}` is the logical grouping within the `{scope}` +- `{metric_name}` is the name of the metric + +### Scope + +The `scope` represents where it belongs in the Backstage ecosystem. + +- `pluginId` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) +- `core` - A core service metric (e.g. `backstage.core.database.connections.active`) + +#### Plugin Metrics + +Pattern: `backstage.{pluginId}.{category}.{metric_name}` + +```yaml +# Examples +backstage.catalog.entities.processed.total +backstage.scaffolder.tasks.completed.total +backstage.techdocs.builds.active +backstage.auth.sessions.active.total +``` + +#### Core Metrics + +Pattern: `backstage.core.{category}.{metric_name}` + +```yaml +# Examples +backstage.core.database.connections.active +backstage.core.scheduler.tasks.queued.total +backstage.core.http.requests.total +``` + +## Release Plan + +- Provide documentation for the naming conventions +- Release as part of the larger telemetry service proposal + +## Dependencies + +- [0012-telemetry-service](../0012-telemetry-service/README.md) + +## Alternatives + +Adopters implement their own metrics and naming conventions. From aba73ed31135422cc4704fe599aa29d96be48fdc Mon Sep 17 00:00:00 2001 From: Kurt King Date: Thu, 26 Jun 2025 22:39:24 -0600 Subject: [PATCH 07/16] correlate naming pattern back to design Signed-off-by: Kurt King --- beps/0012-telemetry-service/README.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/beps/0012-telemetry-service/README.md b/beps/0012-telemetry-service/README.md index c316d05ca8..f63e5c2cf4 100644 --- a/beps/0012-telemetry-service/README.md +++ b/beps/0012-telemetry-service/README.md @@ -130,7 +130,25 @@ interface RootTelemetryService extends TelemetryService { ### Naming Conventions -Visit [0013-telemetry-metrics-naming-conventions](../0013-telemetry-metrics-naming-conventions/README.md) for the naming conventions. +Visit [0013-telemetry-metrics-naming-conventions](../0013-telemetry-metrics-naming-conventions/README.md) for details on the naming conventions. + +The root telemetry service is responsible for initializing the global meter, namespaced to `backstage`. The root service will initialize a meter for each plugin that is registered based on the `pluginId`. The metric name will be the name of the metric as provided by the plugin author in the form `{category}.{name}`. + +```ts +const rootTelemetry = DefaultRootTelemetryService.fromConfig(config); +const pluginTelemetryService = rootTelemetry.forPlugin('my-plugin'); + +const metric = pluginTelemetryService.createCounter('my_category.my_metric'); + +metric.add(1); + +// ... +// metric is now available as `backstage.my-plugin.my_category.my_metric` +``` + +### References + +- [General naming considerations](https://opentelemetry.io/docs/specs/semconv/general/naming/#general-naming-considerations) ### Bootstrapping From a292cf3c0a86775b089894abc70a44aab362ab69 Mon Sep 17 00:00:00 2001 From: Kurt King Date: Thu, 3 Jul 2025 08:16:04 -0600 Subject: [PATCH 08/16] refactor: only focus on metrics service Signed-off-by: Kurt King --- beps/0012-metrics-service/README.md | 255 ++++++++++++++++++ beps/0012-telemetry-service/README.md | 208 -------------- .../README.md | 113 -------- 3 files changed, 255 insertions(+), 321 deletions(-) create mode 100644 beps/0012-metrics-service/README.md delete mode 100644 beps/0012-telemetry-service/README.md delete mode 100644 beps/0013-telemetry-metrics-naming-conventions/README.md diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md new file mode 100644 index 0000000000..8bb290d6d0 --- /dev/null +++ b/beps/0012-metrics-service/README.md @@ -0,0 +1,255 @@ +--- +title: Backstage Metrics Service +status: implementable +authors: + - '@kurtaking' +owners: + - '@kurtaking' +project-areas: + - core-framework + - observability +creation-date: 2025-06-23 +--- + +# BEP: Backstage Metrics Service + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) +- [Design Details](#design-details) + - [Configuration](#configuration) + - [Interface](#interface) + - [Naming Conventions](#naming-conventions) +- [Release Plan](#release-plan) +- [Dependencies](#dependencies) +- [Alternatives](#alternatives) + +## Summary + +Add a core `MetricsService` to Backstage's framework that provides a unified interface for metrics instrumentation. The service offers OpenTelemetry-based capabilities with support for app configuration (e.g. `app-config.yaml`). + +## Motivation + +There is currently no guidance for plugin authors when it comes to metrics instrumentation. While individual plugins may implement their own metrics, there's no standardized approach for metrics collection, naming, or configuration. + +By providing a core metrics service: + +- **Plugin Authors** and the **Community** gain a straightforward way to address metrics instrumentation and can focus on business logic instead of needing to reimplement metrics plumbing. +- **Backstage Admins** receive a reliable stream of metrics from the core system for monitoring, alerting, and troubleshooting. + +### Goals + +- Consistent metrics patterns across all plugins +- Aligned with OpenTelemetry industry standards +- Provide a familiar interface as other core services + +The catalog and scaffolder plugins will be updated to use the new metrics service in the initial release. + +### Non-Goals + +- Adding metrics to plugins that don't currently have it (outside of catalog and scaffolder) +- Providing a full Backstage telemetry SDK (aka recreate the OTEL Node SDK). +- Tracing and other telemetry concerns are out of scope for this BEP. +- Refactoring the existing `LoggerService`. Future work to unify observability related concerns would be ideal, but not a goal. + +## Proposal + +Following similar patterns to other core services, create a new `RootMetricsService` responsible for initializing metrics, root-level concerns, and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `serviceName` and optional `serviceVersion` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter). + +## Design Details + +### Configuration + + + +```yaml +backend: + metrics: + serviceName: backstage + # todo: this is optional - should we just default it to the current Backstage release and allow override? + serviceVersion: 0.0.1 +``` + +### Interface + +Provide a familiar interface as other core services while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. + +```ts +interface MetricsService { + // Synchronous instrumentation + createCounter(name: string, options?: MetricOptions): Counter; + createUpDownCounter(name: string, options?: MetricOptions): UpDownCounter; + createHistogram(name: string, options?: MetricOptions): Histogram; + createGauge(name: string, options?: MetricOptions): Gauge; + + // Asynchronous instrumentation + createObservableCounter( + name: string, + options?: MetricOptions, + ): ObservableCounter; + createObservableUpDownCounter( + name: string, + options?: MetricOptions, + ): ObservableUpDownCounter; + createObservableGauge(name: string, options?: MetricOptions): ObservableGauge; + + // Provide access to the underlying meter for advanced configuration. + getMeter(): Meter; + + // Additional convenience methods tbd... +} +``` + +The `RootMetricsService` is responsible for the global meter, namespaced to `backstage` or the `serviceName` and `serviceVersion` provided in the app's `app-config.yaml` file. The root metrics service will depend on as little as possible so that it can be initialized as early as possible. + +```ts +export const rootMetricsServiceFactory = createServiceFactory({ + service: rootMetricsServiceRef, + deps: { + rootConfig: coreServices.rootConfig, + }, + factory: ({ rootConfig }) => { + return DefaultRootMetricsService.fromConfig(rootConfig); + }, +}); +``` + +It also provides a method for creating a plugin-scoped `MetricsService` for each registered plugin. + +```ts +interface RootMetricsService extends MetricsService { + forPlugin(pluginId: string): MetricsService; +} + +class DefaultRootMetricsService { + // ... + + static fromConfig(config: Config): RootMetricsService { + return new DefaultRootMetricsService(config); + } + + forPlugin(pluginId: string): MetricsService { + return new PluginMetricsService({ + pluginId, + serviceName: this.serviceName, + serviceVersion: this.serviceVersion, + }); + } +} +``` + +Each plugin-scoped `MetricsService` acquires a meter when it is created by calling `metrics.getMeter` from the `@opentelemetry/api` package. The internals of the service ensure the metric name is namespaced to match the naming conventions outlined in the [naming conventions](#naming-conventions) section. + +```ts +class PluginMetricsService implements MetricsService { + // ... + constructor({ + pluginId, + serviceName, + serviceVersion, + }: PluginMetricsServiceOptions) { + this.pluginId = pluginId; + this.serviceName = serviceName; + this.meter = metrics.getMeter(serviceName, serviceVersion); + } + + // Ensures consistency across all metrics + private prefixMetricName(name: string): string { + return `${this.serviceName}.plugin.${this.pluginId}.${name}`; + } +} +``` + +### Naming Conventions + +All Backstage metrics MUST follow this hierarchical pattern: + +`backstage.{scope}.{scope_name}.{category}.{metric_name}` + +**Where:** + +- `backstage` is the root namespace for all Backstage metrics +- `{scope}` is the system scope (`pluginId`, `core` for core services) +- `{scope_name}` is the name of the scope (e.g. `my-plugin`, `catalog`, `scaffolder`, etc.) +- `{category}` is the logical grouping within the `{scope}` +- `{metric_name}` is the name of the metric + +The metric name will be the name as provided by the plugin author in the form `{category}.{name}`. + +```ts +const rootMetrics = DefaultRootMetricsService.fromConfig(config); +const pluginMetricsService = rootMetrics.forPlugin('my-plugin'); + +const metric = pluginMetricsService.createCounter('my_category.my_metric'); + +metric.add(1); + +// ... +// metric is now available as `backstage.plugin.my-plugin.my_category.my_metric` +``` + +#### Scope + +The `scope` represents where it belongs in the Backstage ecosystem. + +- `pluginId` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) +- `core` - A core service metric (e.g. `backstage.core.database.connections.active`) + +##### Plugin-Scoped Metrics + +Pattern: `backstage.plugin.{pluginId}.{category}.{metric_name}` + +```yaml +# Examples +backstage.plugin.catalog.entities.processed.total +backstage.plugin.scaffolder.tasks.completed.total +backstage.plugin.techdocs.builds.active +backstage.plugin.auth.sessions.active.total # todo: technically a core service and a backend plugin +``` + +##### Core Metrics + +Pattern: `backstage.core.{core_service}.{category}.{metric_name}` + +```yaml +# Examples +backstage.core.database.connections.active +backstage.core.scheduler.tasks.queued.total +backstage.core.http.requests.total +``` + +### References + +- [General naming considerations](https://opentelemetry.io/docs/specs/semconv/general/naming/#general-naming-considerations) +- [Acquiring a meter](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter) + +## Release Plan + +1. Create a new `RootMetricsService` that initializes metrics and app-wide concerns. +1. Create the plugin-scoped `MetricsService` that provides a metrics service for plugins. +1. Create alpha-related documentation to add to existing core service [docs](https://backstage.io/docs/backend-system/core-services/index). +1. Release the metrics service as alpha. +1. Refactor catalog and scaffolder implementations to use the new (alpha) `MetricsService`. +1. Release the metrics service +1. Offer a migration path for existing adopters to migrate to the new metrics service. +1. Update all documentation to reference the new metrics service. +1. Create follow-up action items to integrate the new metrics service into the core system. +1. Fully deprecate all existing metrics implementations like the existing Prometheus one-off implementations. + +## Dependencies + +1. The root metrics service MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters, while still allowing for configuration. +1. There are one-off implementations of metrics in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. + +## Alternatives + +Plugin authors continue to implement their own metrics as they see fit. diff --git a/beps/0012-telemetry-service/README.md b/beps/0012-telemetry-service/README.md deleted file mode 100644 index f63e5c2cf4..0000000000 --- a/beps/0012-telemetry-service/README.md +++ /dev/null @@ -1,208 +0,0 @@ ---- -title: Backstage Telemetry Service -status: implementable -authors: - - '@kurtaking' -owners: - - '@kurtaking' -project-areas: - - backend - - core - - framework -creation-date: 2025-06-23 ---- - -# BEP: Backstage Telemetry Service - -- [Summary](#summary) -- [Motivation](#motivation) - - [Goals](#goals) - - [Non-Goals](#non-goals) -- [Proposal](#proposal) -- [Design Details](#design-details) - - [Configuration](#configuration) -- [Release Plan](#release-plan) -- [Dependencies](#dependencies) -- [Alternatives](#alternatives) - -## Summary - -Add a core telemetry service to Backstage's framework that provides a unified interface for metrics, tracing, and logging instrumentation. The service offers OpenTelemetry-based capabilities with zero configuration defaults while allowing extensibility through app configuration (e.g. `app-config.yaml`) and plugin modules. - -## Considerations - -1. It's possible a single telemetry service is too encompassing and that more granular services are needed. For example, a `MetricsService` and `TracingService` which would work along side the existing `LoggerService` to solve for telemetry needs. - -## Motivation - -There is currently no guidance for plugin authors when it comes to telemetry and while individual plugins may implement their own metrics or tracing, there's no unified service that provides a standardized approach for telemetry and observability. - -By providing a core telemetry service: - -- **Plugin Authors** gain a straightforward, documented way to address telemetry, and can focus on business logic instead of needing to reimplement telemetry plumbing. -- The **community** receives a standard approach to telemetry. -- **Backstage Operators/SRE/Platform Engineers** receive a consistent, reliable stream of metrics from the core system, simplifying monitoring, alerting, and troubleshooting. They can enforce metrics standards, aggregate data, and build dashboards more easily. -**Organizations** with their own observability or compliance requirements can override the core service to integrate with proprietary or custom telemetry solutions, without forking or modifying plugins. - -### Goals - -Provide a unified service that provides a standardized approach for telemetry where setup docs like [these](https://backstage.io/docs/tutorials/setup-opentelemetry/) are not needed. - -- Consistent telemetry patterns across all plugins -- Zero-configuration defaults that work out of the box -- Extensible architecture for custom telemetry backends -- OpenTelemetry integration following industry standards -- Performance monitoring capabilities built into the platform - -We will include the following two plugins in the initial release to demonstrate the new telemetry service: - -- Replace existing catalog implementation with `TelemetryService` -- Replace existing scaffolder implementation with `TelemetryService` - -### Non-Goals - -- Adding telemetry instrumentation to existing plugins that don't currently have it -- Only the catalog and scaffolder implementations will be refactored to use the new telemetry service. -- We will not start by providing a full Backstage SDK (aka recreate the OTEL Node SDK). -- We will not refactor the existing LoggerService. Future work to unify telemetry related concerns would be ideal, but not a goal. - -## Proposal - -Following similar patterns to other core services, we will create a new `RootTelemetryService` responsible for initializing telemetry and app-wide concerns, and the creation of plugin-specific telemetry services. - -A core concern is not initializing the telemetry service early enough in the backend lifecycle. Services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping functionality that will be called as early as possible in the backend lifecycle. - -## Configuration - -All OTEL configuration will be supported via the app's `app-config.yaml` file. The config schema should be based on existing resources such as the [OTEL collector configuration](https://opentelemetry.io/docs/collector/configuration/). - -```yaml -# this is not a complete example, but it shows the general idea -telemetry: - enabled: true - - resource: - serviceName: backstage-telemetry - serviceVersion: 0.0.1 - - metrics: - enabled: true - collection: - exportIntervalMillis: 15000 - - exporters: - - type: prometheus - enabled: true - config: - port: 9464 - - tracing: - enabled: false - - instrumentations: - http: false - knex: false - express: true - # ... -``` - -## Design Details - -We will provide a thin wrapper around the OpenTelemetry Node SDK to provide a consistent interface while re-exporting the types from the SDK. This lets us provide a consistent telemetry interface while leveraging types and concepts already familiar to the community. Additionally, we will rely on the auto instrumentation features of the SDK as much as possible. - -The decision to leverage the SDK comes from the fact that OpenTelemetry is already the fundamental abstraction for metrics and tracing. - -```ts -interface TelemetryService { - createCounter(name: string, options?: MetricOptions): Counter; - // - - // provide escape hatch for advanced use cases (???) - todo: I don't like this if we don't have a good reason for it - getMeter(): Meter; -} - -interface RootTelemetryService extends TelemetryService { - forPlugin(pluginId: string): TelemetryService; - - // provide escape hatches for advanced use cases (???) - getClient(): NodeSDK; // give access to the SDK -} -``` - -### Naming Conventions - -Visit [0013-telemetry-metrics-naming-conventions](../0013-telemetry-metrics-naming-conventions/README.md) for details on the naming conventions. - -The root telemetry service is responsible for initializing the global meter, namespaced to `backstage`. The root service will initialize a meter for each plugin that is registered based on the `pluginId`. The metric name will be the name of the metric as provided by the plugin author in the form `{category}.{name}`. - -```ts -const rootTelemetry = DefaultRootTelemetryService.fromConfig(config); -const pluginTelemetryService = rootTelemetry.forPlugin('my-plugin'); - -const metric = pluginTelemetryService.createCounter('my_category.my_metric'); - -metric.add(1); - -// ... -// metric is now available as `backstage.my-plugin.my_category.my_metric` -``` - -### References - -- [General naming considerations](https://opentelemetry.io/docs/specs/semconv/general/naming/#general-naming-considerations) - -### Bootstrapping - -A core concern is not initializing the telemetry service early enough in the backend lifecycle and services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping mechanism that will be called as early as possible in the backend lifecycle. - -Ideally, this is called at the start of the `createBackend` function [[ref](https://github.com/backstage/backstage/blob/ab539cc72bb503618df55815f24c4ba933607460/packages/backend-defaults/src/CreateBackend.ts#L73-L75)], but we need to gain access to the config file at minimum. Ideally the logger as well, but not as crucial. - -```ts -export function createBackend(): Backend { - // ... - // call the bootstrap function - Telemetry.bootstrap(); - // ... - return createSpecializedBackend({ defaultServiceFactories }); -} -``` - -### Plugins & Modules - -WIP - - - -## Release Plan - -1. Create a new `RootTelemetryService` that initializes telemetry and app-wide concerns. -1. Create the plugin-scoped `TelemetryService` that provides a telemetry service for plugins. -1. Release the telemetry service as alpha. -1. Refactor catalog and scaffolder implementations to use the new (alpha) `TelemetryService`. -1. Replace the existing setup docs by integrating the new `RootTelemetryService` to `createBackend` -1. Release the telemetry service -1. Offer a migration path for existing adopters to migrate to the new telemetry service. -1. Update all documentation to reference the new telemetry service. -1. Create follow-up action items to integrate the new telemetry service into the core system. -1. Fully deprecate all existing telemetry implementations like the existing Prometheus one-off implementations. - -## Dependencies - -1. The root telemetry service MUST BE initialized as EARLY as possible to prevent dependents from receiving no op meters, while still allowing for app-wide configuration. -1. There are one-off implementations of telemetry in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. - -## Alternatives - -1. Plugin authors continue to implement their own telemetry as they see fit. -1. Adopters continue using the [setup-opentelemetry](https://backstage.io/docs/tutorials/setup-opentelemetry/) docs as a guide. - -## Outstanding Questions - -1. Are there enough Backstage-isms to build an additional layer of abstraction for metrics and tracing on top of the OpenTelemetry SDK? Do we re-export the types from the OpenTelemetry SDK or do we create our own? How do we handle breaking changes? -1. How do we guarantee that the telemetry service is initialized early enough in the backend lifecycle, while still allowing for app-wide configuration? -1. Should we provide an escape hatch for advanced use cases? -1. Should there be a single telemetry service or should we provide more granular services (e.g. adding separate `MetricsService` and `TracingService`)? -1. What core services could leverage the auto instrumentation features of the OpenTelemetry SDK? (e.g. `DatabaseService` could leverage the `knex` auto instrumentation, `HttpService` could leverage the `express` auto instrumentation, etc.) diff --git a/beps/0013-telemetry-metrics-naming-conventions/README.md b/beps/0013-telemetry-metrics-naming-conventions/README.md deleted file mode 100644 index c6cbc85d24..0000000000 --- a/beps/0013-telemetry-metrics-naming-conventions/README.md +++ /dev/null @@ -1,113 +0,0 @@ ---- -title: OpenTelemetry Metrics Naming Conventions for Backstage -status: implementable -authors: - - '@kurtaking' -owners: - - '@kurtaking' -project-areas: - - framework - - telemetry -creation-date: 2025-06-26 ---- - -# BEP: OpenTelemetry Metrics Naming Conventions for Backstage - -- [Summary](#summary) -- [Motivation](#motivation) - - [Goals](#goals) - - [Non-Goals](#non-goals) -- [Proposal](#proposal) -- [Design Details](#design-details) -- [Release Plan](#release-plan) -- [Dependencies](#dependencies) -- [Alternatives](#alternatives) - -## Summary - -This proposal establishes standardized naming conventions for OpenTelemetry metrics across the Backstage ecosystem. It defines a hierarchical naming pattern that ensures consistency, discoverability, and compliance with OpenTelemetry semantic conventions while providing clear guidance for plugin and module authors. - -## Motivation - -Currently, Backstage has: - -- Mixed metric naming patterns: Legacy Prometheus metrics (`catalog_entities_count`) alongside newer OpenTelemetry metrics (`catalog.processing.queue.delay`) -- No standard pattern for plugin-specific metrics -- Difficulty correlating metrics across plugins and providers -- Some existing metrics don't follow OpenTelemetry semantic conventions - -We need to decided on a pattern that we can use for the telemetry service to obfuscate the underlying implementation details. - -### Goals - -- Establish consistent naming patterns across all Backstage metrics -- Ensure OpenTelemetry compliance with semantic conventions -- Provide clear guidance for plugin and service authors - -### Non-Goals - -- Breaking existing metrics immediately (backward compatibility maintained) -- Forcing all plugins to emit the same metrics -- Defining specific metric values or thresholds - -## Proposal - -Provide guidance for naming metrics for Backstage. - -## Design Details - -### Core Naming Pattern - -All Backstage metrics MUST follow this hierarchical pattern: - -`backstage.{scope}.{category}.{metric_name}` - -**Where:** - -- `backstage` is the root namespace for all Backstage metrics -- `{scope}` is the system scope (`pluginId`, `core` for core services) -- `{category}` is the logical grouping within the `{scope}` -- `{metric_name}` is the name of the metric - -### Scope - -The `scope` represents where it belongs in the Backstage ecosystem. - -- `pluginId` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) -- `core` - A core service metric (e.g. `backstage.core.database.connections.active`) - -#### Plugin Metrics - -Pattern: `backstage.{pluginId}.{category}.{metric_name}` - -```yaml -# Examples -backstage.catalog.entities.processed.total -backstage.scaffolder.tasks.completed.total -backstage.techdocs.builds.active -backstage.auth.sessions.active.total -``` - -#### Core Metrics - -Pattern: `backstage.core.{category}.{metric_name}` - -```yaml -# Examples -backstage.core.database.connections.active -backstage.core.scheduler.tasks.queued.total -backstage.core.http.requests.total -``` - -## Release Plan - -- Provide documentation for the naming conventions -- Release as part of the larger telemetry service proposal - -## Dependencies - -- [0012-telemetry-service](../0012-telemetry-service/README.md) - -## Alternatives - -Adopters implement their own metrics and naming conventions. From 6e3b53e9a119403764e48c7db31967389a64430b Mon Sep 17 00:00:00 2001 From: Kurt King Date: Fri, 4 Jul 2025 23:47:35 -0600 Subject: [PATCH 09/16] refactor: start sdk for user based on config Signed-off-by: Kurt King --- beps/0012-metrics-service/README.md | 196 ++++++++++++++++++++-------- 1 file changed, 142 insertions(+), 54 deletions(-) diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md index 8bb290d6d0..c45a88e8ba 100644 --- a/beps/0012-metrics-service/README.md +++ b/beps/0012-metrics-service/README.md @@ -30,6 +30,8 @@ creation-date: 2025-06-23 Add a core `MetricsService` to Backstage's framework that provides a unified interface for metrics instrumentation. The service offers OpenTelemetry-based capabilities with support for app configuration (e.g. `app-config.yaml`). +This approach leverages industry standards while focusing the `MetricsService` on distinct Backstage concerns, following the same pattern as other core services (DatabaseService builds on Knex, LoggerService builds on Winston, HttpRouterService builds on Express, etc.). + ## Motivation There is currently no guidance for plugin authors when it comes to metrics instrumentation. While individual plugins may implement their own metrics, there's no standardized approach for metrics collection, naming, or configuration. @@ -41,16 +43,17 @@ By providing a core metrics service: ### Goals +- Plugin-scoped metric namespacing - Consistent metrics patterns across all plugins - Aligned with OpenTelemetry industry standards - Provide a familiar interface as other core services +- Standardized initialization and lifecycle management The catalog and scaffolder plugins will be updated to use the new metrics service in the initial release. ### Non-Goals - Adding metrics to plugins that don't currently have it (outside of catalog and scaffolder) -- Providing a full Backstage telemetry SDK (aka recreate the OTEL Node SDK). - Tracing and other telemetry concerns are out of scope for this BEP. - Refactoring the existing `LoggerService`. Future work to unify observability related concerns would be ideal, but not a goal. @@ -60,28 +63,85 @@ Following similar patterns to other core services, create a new `RootMetricsServ ## Design Details +## Integration with OpenTelemetry Auto-Instrumentation + +The `MetricsService` **complements** rather than duplicates auto-instrumentation by focusing on **application-level metrics** that only Backstage can provide. + +For example, the catalog plugin may want to track the number of entities processed by the `refresh` operation and the kind of entity being processed. + +```ts +// Auto-instrumentation provides (automatically): +// - http.server.requests.total{method="GET", route="/catalog/entities", status_code="200"} +// - http.server.request.duration{method="GET", route="/catalog/entities"} + +// MetricsService provides (manually): +const entityMetrics = metricsService.createCounter('entities.processed.total'); +entityMetrics.add(entities.length, { operation: 'refresh', kind: 'Component' }); +``` + ### Configuration - + collection?: { + exportIntervalMillis?: number; + }; + + exporters: Array<{ + type: 'prometheus' | 'otlp' | 'console'; + enabled: boolean; + config?: Record; + }>; + + autoInstrumentation: { + enabled: boolean; + exclude?: string[]; + }; +} +``` ```yaml backend: metrics: - serviceName: backstage - # todo: this is optional - should we just default it to the current Backstage release and allow override? - serviceVersion: 0.0.1 + enabled: true + + resource: + serviceName: backstage + serviceVersion: 0.0.1 + environment: production + + # Collection settings + collection: + exportIntervalMillis: 15000 + + exporters: + - type: prometheus + enabled: true + config: + port: 9464 + # ... + - type: console + enabled: true + config: + logLevel: debug + + autoInstrumentation: + enabled: true + exclude: + - 'http' ``` ### Interface -Provide a familiar interface as other core services while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. +Provide a thin wrapper around OpenTelemetry's API while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. ```ts interface MetricsService { @@ -109,7 +169,7 @@ interface MetricsService { } ``` -The `RootMetricsService` is responsible for the global meter, namespaced to `backstage` or the `serviceName` and `serviceVersion` provided in the app's `app-config.yaml` file. The root metrics service will depend on as little as possible so that it can be initialized as early as possible. +The `RootMetricsService` is responsible for initializing the OpenTelemetry SDK and creating plugin-scoped metrics services. The root metrics service will depend on as little as possible so that it can be initialized as early as possible. ```ts export const rootMetricsServiceFactory = createServiceFactory({ @@ -130,81 +190,109 @@ interface RootMetricsService extends MetricsService { forPlugin(pluginId: string): MetricsService; } -class DefaultRootMetricsService { - // ... +class DefaultRootMetricsService implements RootMetricsService { + private sdk: NodeSDK; static fromConfig(config: Config): RootMetricsService { - return new DefaultRootMetricsService(config); + const metricsConfig = config.getOptionalConfig('backend.metrics'); + + const sdk = new NodeSDK({ + resource: createResourceFromConfig(metricsConfig), + instrumentations: [ + getNodeAutoInstrumentations({ + ...getAutoInstrumentationConfig(metricsConfig), + }), + ], + metricReader: createMetricReadersFromConfig(metricsConfig), + }); + + sdk.start(); + + return new DefaultRootMetricsService(sdk); } + constructor(private sdk: NodeSDK) {} + forPlugin(pluginId: string): MetricsService { - return new PluginMetricsService({ - pluginId, - serviceName: this.serviceName, - serviceVersion: this.serviceVersion, - }); + return new PluginMetricsService(pluginId); + } + + async shutdown(): Promise { + await this.sdk.shutdown(); } } ``` -Each plugin-scoped `MetricsService` acquires a meter when it is created by calling `metrics.getMeter` from the `@opentelemetry/api` package. The internals of the service ensure the metric name is namespaced to match the naming conventions outlined in the [naming conventions](#naming-conventions) section. +Each plugin receives a metrics service that automatically namespaces all metrics to match the naming conventions outlined in the [naming conventions](#naming-conventions) section. ```ts +const metricsServiceFactory = createServiceFactory({ + service: metricsServiceRef, + deps: { + rootConfig: coreServices.rootConfig, + pluginMetadata: coreServices.pluginMetadata, + rootMetrics: coreServices.rootMetrics, + }, + factory: ({ rootMetrics, pluginMetadata }) => { + return rootMetrics.forPlugin(pluginMetadata.getId()); + }, +}); + class PluginMetricsService implements MetricsService { // ... - constructor({ - pluginId, - serviceName, - serviceVersion, - }: PluginMetricsServiceOptions) { - this.pluginId = pluginId; - this.serviceName = serviceName; - this.meter = metrics.getMeter(serviceName, serviceVersion); + constructor(private pluginId: string) { + this.meter = metrics.getMeter(`backstage.plugin.${pluginId}`); } - // Ensures consistency across all metrics + createCounter(name: string, options?: MetricOptions): Counter { + const fullName = this.prefixMetricName(name); + return this.meter.createCounter(fullName, options); + } + + // ... other interface methods + private prefixMetricName(name: string): string { - return `${this.serviceName}.plugin.${this.pluginId}.${name}`; + return `backstage.plugin.${this.pluginId}.${name}`; } } ``` +### Example + +````ts +const entitiesProcessed = metricsService.createCounter('entities.processed.total', { + description: 'Total entities processed during refresh', + unit: '{entity}', +}); + +entitiesProcessed.add(100); + +// ... +// metric is now available as `backstage.plugin.catalog.entities.processed.total` + + ### Naming Conventions -All Backstage metrics MUST follow this hierarchical pattern: +All Backstage metrics follow this hierarchical pattern: -`backstage.{scope}.{scope_name}.{category}.{metric_name}` +`backstage.{plugin|core}.{scope_name}.{category}.{metric_name}` **Where:** - `backstage` is the root namespace for all Backstage metrics -- `{scope}` is the system scope (`pluginId`, `core` for core services) -- `{scope_name}` is the name of the scope (e.g. `my-plugin`, `catalog`, `scaffolder`, etc.) +- `{scope}` is the system scope (either **plugin** or **core**) +- `{scope_name}` is the name of the plugin or core service (e.g. `my-plugin`, `catalog`, `scaffolder`, etc.) - `{category}` is the logical grouping within the `{scope}` -- `{metric_name}` is the name of the metric - -The metric name will be the name as provided by the plugin author in the form `{category}.{name}`. - -```ts -const rootMetrics = DefaultRootMetricsService.fromConfig(config); -const pluginMetricsService = rootMetrics.forPlugin('my-plugin'); - -const metric = pluginMetricsService.createCounter('my_category.my_metric'); - -metric.add(1); - -// ... -// metric is now available as `backstage.plugin.my-plugin.my_category.my_metric` -``` +- `{metric_name}` is the name of the metric as provided by the plugin author. #### Scope The `scope` represents where it belongs in the Backstage ecosystem. -- `pluginId` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) +- `plugin` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) - `core` - A core service metric (e.g. `backstage.core.database.connections.active`) -##### Plugin-Scoped Metrics +#### Plugin-Scoped Metrics Pattern: `backstage.plugin.{pluginId}.{category}.{metric_name}` @@ -214,9 +302,9 @@ backstage.plugin.catalog.entities.processed.total backstage.plugin.scaffolder.tasks.completed.total backstage.plugin.techdocs.builds.active backstage.plugin.auth.sessions.active.total # todo: technically a core service and a backend plugin -``` +```` -##### Core Metrics +#### Core Metrics Pattern: `backstage.core.{core_service}.{category}.{metric_name}` From a9ad6932a26c304bd2632621f0865764c8c42810 Mon Sep 17 00:00:00 2001 From: Kurt King Date: Tue, 8 Jul 2025 09:19:00 -0600 Subject: [PATCH 10/16] document wrapping sdk as the direction to head in Signed-off-by: Kurt King --- beps/0012-metrics-service/README.md | 53 +++++++++++++++++------------ 1 file changed, 32 insertions(+), 21 deletions(-) diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md index c45a88e8ba..c621a25faf 100644 --- a/beps/0012-metrics-service/README.md +++ b/beps/0012-metrics-service/README.md @@ -21,20 +21,19 @@ creation-date: 2025-06-23 - [Design Details](#design-details) - [Configuration](#configuration) - [Interface](#interface) - - [Naming Conventions](#naming-conventions) - [Release Plan](#release-plan) - [Dependencies](#dependencies) - [Alternatives](#alternatives) ## Summary -Add a core `MetricsService` to Backstage's framework that provides a unified interface for metrics instrumentation. The service offers OpenTelemetry-based capabilities with support for app configuration (e.g. `app-config.yaml`). +Add a core `MetricsService` to Backstage's framework to provide a unified interface for metrics instrumentation. The service offers OpenTelemetry-based capabilities with support for app configuration (e.g. `app-config.yaml`). This approach leverages industry standards while focusing the `MetricsService` on distinct Backstage concerns, following the same pattern as other core services (DatabaseService builds on Knex, LoggerService builds on Winston, HttpRouterService builds on Express, etc.). ## Motivation -There is currently no guidance for plugin authors when it comes to metrics instrumentation. While individual plugins may implement their own metrics, there's no standardized approach for metrics collection, naming, or configuration. +There is no guidance when it comes to metrics instrumentation. While individual plugins may implement their own metrics, there's no standardized approach for metrics collection, naming, or configuration. By providing a core metrics service: @@ -49,7 +48,7 @@ By providing a core metrics service: - Provide a familiar interface as other core services - Standardized initialization and lifecycle management -The catalog and scaffolder plugins will be updated to use the new metrics service in the initial release. +The catalog and scaffolder plugins will be updated to use the new metrics service in the initial alpha release. ### Non-Goals @@ -59,11 +58,20 @@ The catalog and scaffolder plugins will be updated to use the new metrics servic ## Proposal -Following similar patterns to other core services, create a new `RootMetricsService` responsible for initializing metrics, root-level concerns, and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `serviceName` and optional `serviceVersion` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter). +Following similar patterns to other core services, create a new `RootMetricsService` responsible for initializing metrics, root-level concerns, and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `service.name` and optional `service.version` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter). ## Design Details -## Integration with OpenTelemetry Auto-Instrumentation +### Integration with OpenTelemetry Auto-Instrumentation + +The `RootMetricsService` will automatically enable instrumentation for known libraries leveraged by the Backstage framework. + +- Express +- Knex +- Winston +- etc. + +Configuration will be provided to enable or disable auto-instrumentation via inclusion or exclusion lists. The `MetricsService` **complements** rather than duplicates auto-instrumentation by focusing on **application-level metrics** that only Backstage can provide. @@ -82,6 +90,7 @@ entityMetrics.add(entities.length, { operation: 'refresh', kind: 'Component' }); ### Configuration ```ts +// Not final, but this is the general idea... interface MetricsConfig { enabled: boolean; @@ -93,16 +102,18 @@ interface MetricsConfig { collection?: { exportIntervalMillis?: number; + // ... }; exporters: Array<{ - type: 'prometheus' | 'otlp' | 'console'; + type: 'prometheus' | 'otlp' | 'console' | '...'; enabled: boolean; config?: Record; }>; autoInstrumentation: { enabled: boolean; + include?: string[]; exclude?: string[]; }; } @@ -135,13 +146,15 @@ backend: autoInstrumentation: enabled: true + include: + - 'some other library' exclude: - - 'http' + - 'express' ``` ### Interface -Provide a thin wrapper around OpenTelemetry's API while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. +Provide a thin wrapper around OpenTelemetry's API while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. The OTEL maintainers state this is the recommended approach and that we can rely on `@opentelemetry/api` to be the source of truth and not to have breaking changes. ```ts interface MetricsService { @@ -162,17 +175,15 @@ interface MetricsService { ): ObservableUpDownCounter; createObservableGauge(name: string, options?: MetricOptions): ObservableGauge; - // Provide access to the underlying meter for advanced configuration. - getMeter(): Meter; - - // Additional convenience methods tbd... + // Future - add additional convenience methods as we learn more about the needs of the framework } ``` -The `RootMetricsService` is responsible for initializing the OpenTelemetry SDK and creating plugin-scoped metrics services. The root metrics service will depend on as little as possible so that it can be initialized as early as possible. +The `RootMetricsService` is responsible for initializing the OpenTelemetry SDK and creating plugin-scoped metrics services. If the end user wants to initialize their own SDK, they are responsible for initializing the OpenTelemetry SDK with their own configuration. The `RootMetricsService` is responsible for providing metrics to other root services and creating plugin-scoped metrics services. ```ts export const rootMetricsServiceFactory = createServiceFactory({ + // depends on as little as possible so that it can be initialized as early as possible. service: rootMetricsServiceRef, deps: { rootConfig: coreServices.rootConfig, @@ -223,7 +234,7 @@ class DefaultRootMetricsService implements RootMetricsService { } ``` -Each plugin receives a metrics service that automatically namespaces all metrics to match the naming conventions outlined in the [naming conventions](#naming-conventions) section. +Each plugin receives a metrics service that automatically namespaces all metrics to match the naming conventions outlined in the section below. ```ts const metricsServiceFactory = createServiceFactory({ @@ -270,7 +281,6 @@ entitiesProcessed.add(100); // ... // metric is now available as `backstage.plugin.catalog.entities.processed.total` - ### Naming Conventions All Backstage metrics follow this hierarchical pattern: @@ -322,20 +332,21 @@ backstage.core.http.requests.total ## Release Plan -1. Create a new `RootMetricsService` that initializes metrics and app-wide concerns. +1. Create a new `RootMetricsService` that initializes the OpenTelemetry SDK and creates plugin-scoped metrics services. 1. Create the plugin-scoped `MetricsService` that provides a metrics service for plugins. 1. Create alpha-related documentation to add to existing core service [docs](https://backstage.io/docs/backend-system/core-services/index). -1. Release the metrics service as alpha. -1. Refactor catalog and scaffolder implementations to use the new (alpha) `MetricsService`. -1. Release the metrics service +1. Release the metrics service under `@alpha`. +1. Mark all existing metrics implementations as deprecated. +1. Refactor catalog and scaffolder plugins to use the new (alpha) `MetricsService`. 1. Offer a migration path for existing adopters to migrate to the new metrics service. +1. Release the metrics service 1. Update all documentation to reference the new metrics service. 1. Create follow-up action items to integrate the new metrics service into the core system. 1. Fully deprecate all existing metrics implementations like the existing Prometheus one-off implementations. ## Dependencies -1. The root metrics service MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters, while still allowing for configuration. +1. The root metrics service MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters 1. There are one-off implementations of metrics in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. ## Alternatives From 86e1dfb411c4a5ffbbb6fa37a09b0708abf8f941 Mon Sep 17 00:00:00 2001 From: Kurt King Date: Wed, 9 Jul 2025 20:59:59 -0600 Subject: [PATCH 11/16] refactor: reorg sections for better flow of content Signed-off-by: Kurt King --- beps/0012-metrics-service/README.md | 169 ++++++++++++++-------------- 1 file changed, 83 insertions(+), 86 deletions(-) diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md index c621a25faf..9948f22cba 100644 --- a/beps/0012-metrics-service/README.md +++ b/beps/0012-metrics-service/README.md @@ -11,16 +11,22 @@ project-areas: creation-date: 2025-06-23 --- -# BEP: Backstage Metrics Service +## Table of Contents - [Summary](#summary) - [Motivation](#motivation) - [Goals](#goals) - [Non-Goals](#non-goals) - [Proposal](#proposal) +- [Naming Conventions](#naming-conventions) - [Design Details](#design-details) + - [References](#references) + - [Integration with OpenTelemetry Auto-Instrumentation](#integration-with-opentelemetry-auto-instrumentation) - [Configuration](#configuration) - [Interface](#interface) + - [Root Metrics Service](#root-metrics-service) + - [Plugin Metrics Service](#plugin-metrics-service) + - [Example](#example) - [Release Plan](#release-plan) - [Dependencies](#dependencies) - [Alternatives](#alternatives) @@ -58,24 +64,68 @@ The catalog and scaffolder plugins will be updated to use the new metrics servic ## Proposal -Following similar patterns to other core services, create a new `RootMetricsService` responsible for initializing metrics, root-level concerns, and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `service.name` and optional `service.version` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter). +Following similar patterns to other core services, create a new `RootMetricsService` responsible for initializing the OpenTelemetry SDK, root-level concerns, and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `service.name` and optional `service.version` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter). + +## Naming Conventions + +All Backstage metrics follow this hierarchical pattern: + +`backstage.{scope}.{scope_name}.{metric_name}` + +**Where:** + +- `backstage` is the root namespace for all Backstage metrics +- `{scope}` is the system scope (either **plugin** or **core**) +- `{scope_name}` is the name of the plugin or core service (e.g., `catalog`, `scaffolder`, `database`) +- `{metric_name}` is the hierarchical metric name as provided by the plugin author (e.g., `entities.processed.total`, `tasks.completed.total`) + +### Scope + +The `scope` represents where it belongs in the Backstage ecosystem. + +- `plugin` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) +- `core` - A core service metric (e.g. `backstage.core.database.connections.active`) + +### Plugin-Scoped Metrics + +Pattern: `backstage.plugin.{pluginId}.{metric_name}` + +```yaml +# Examples +backstage.plugin.catalog.entities.processed.total +backstage.plugin.scaffolder.tasks.completed.total +backstage.plugin.techdocs.builds.active +backstage.plugin.auth.sessions.active.total # todo: technically a core service and a backend plugin +``` + +### Core Metrics + +Pattern: `backstage.core.{core_service}.{metric_name}` + +```yaml +# Examples +backstage.core.database.connections.active +backstage.core.scheduler.tasks.queued.total +backstage.core.httpRouter.requests.total +``` ## Design Details +### References + +- [General naming considerations](https://opentelemetry.io/docs/specs/semconv/general/naming/#general-naming-considerations) +- [Acquiring a meter](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter) + ### Integration with OpenTelemetry Auto-Instrumentation -The `RootMetricsService` will automatically enable instrumentation for known libraries leveraged by the Backstage framework. +The `RootMetricsService` will automatically enable instrumentation for known libraries leveraged by the Backstage framework. Configuration will be provided to enable or disable auto-instrumentation via inclusion or exclusion lists. - Express - Knex - Winston - etc. -Configuration will be provided to enable or disable auto-instrumentation via inclusion or exclusion lists. - -The `MetricsService` **complements** rather than duplicates auto-instrumentation by focusing on **application-level metrics** that only Backstage can provide. - -For example, the catalog plugin may want to track the number of entities processed by the `refresh` operation and the kind of entity being processed. +The `MetricsService` **complements** rather than duplicates auto-instrumentation by focusing on **application-level metrics** that only Backstage can provide. For example, the catalog plugin may want to track the number of entities processed by the `refresh` operation and the kind of entity being processed. ```ts // Auto-instrumentation provides (automatically): @@ -107,7 +157,6 @@ interface MetricsConfig { exporters: Array<{ type: 'prometheus' | 'otlp' | 'console' | '...'; - enabled: boolean; config?: Record; }>; @@ -135,26 +184,19 @@ backend: exporters: - type: prometheus - enabled: true config: port: 9464 # ... - type: console - enabled: true - config: - logLevel: debug autoInstrumentation: enabled: true - include: - - 'some other library' - exclude: - - 'express' + exclude: ['express'] ``` ### Interface -Provide a thin wrapper around OpenTelemetry's API while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. The OTEL maintainers state this is the recommended approach and that we can rely on `@opentelemetry/api` to be the source of truth and not to have breaking changes. +Provide a wrapper around OpenTelemetry's API while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. ```ts interface MetricsService { @@ -179,9 +221,15 @@ interface MetricsService { } ``` +#### Root Metrics Service + The `RootMetricsService` is responsible for initializing the OpenTelemetry SDK and creating plugin-scoped metrics services. If the end user wants to initialize their own SDK, they are responsible for initializing the OpenTelemetry SDK with their own configuration. The `RootMetricsService` is responsible for providing metrics to other root services and creating plugin-scoped metrics services. ```ts +interface RootMetricsService { + forPlugin(pluginId: string): MetricsService; +} + export const rootMetricsServiceFactory = createServiceFactory({ // depends on as little as possible so that it can be initialized as early as possible. service: rootMetricsServiceRef, @@ -194,13 +242,7 @@ export const rootMetricsServiceFactory = createServiceFactory({ }); ``` -It also provides a method for creating a plugin-scoped `MetricsService` for each registered plugin. - ```ts -interface RootMetricsService extends MetricsService { - forPlugin(pluginId: string): MetricsService; -} - class DefaultRootMetricsService implements RootMetricsService { private sdk: NodeSDK; @@ -234,15 +276,16 @@ class DefaultRootMetricsService implements RootMetricsService { } ``` -Each plugin receives a metrics service that automatically namespaces all metrics to match the naming conventions outlined in the section below. +#### Plugin Metrics Service + +Each plugin receives a metrics service that automatically namespaces all metrics to match the naming conventions. ```ts const metricsServiceFactory = createServiceFactory({ service: metricsServiceRef, deps: { - rootConfig: coreServices.rootConfig, - pluginMetadata: coreServices.pluginMetadata, rootMetrics: coreServices.rootMetrics, + pluginMetadata: coreServices.pluginMetadata, }, factory: ({ rootMetrics, pluginMetadata }) => { return rootMetrics.forPlugin(pluginMetadata.getId()); @@ -255,11 +298,6 @@ class PluginMetricsService implements MetricsService { this.meter = metrics.getMeter(`backstage.plugin.${pluginId}`); } - createCounter(name: string, options?: MetricOptions): Counter { - const fullName = this.prefixMetricName(name); - return this.meter.createCounter(fullName, options); - } - // ... other interface methods private prefixMetricName(name: string): string { @@ -268,68 +306,23 @@ class PluginMetricsService implements MetricsService { } ``` -### Example +##### Example -````ts -const entitiesProcessed = metricsService.createCounter('entities.processed.total', { - description: 'Total entities processed during refresh', - unit: '{entity}', -}); +```ts +const entitiesProcessed = metricsService.createCounter( + 'entities.processed.total', + { + description: 'Total entities processed during refresh', + unit: '{entity}', + }, +); entitiesProcessed.add(100); // ... // metric is now available as `backstage.plugin.catalog.entities.processed.total` - -### Naming Conventions - -All Backstage metrics follow this hierarchical pattern: - -`backstage.{plugin|core}.{scope_name}.{category}.{metric_name}` - -**Where:** - -- `backstage` is the root namespace for all Backstage metrics -- `{scope}` is the system scope (either **plugin** or **core**) -- `{scope_name}` is the name of the plugin or core service (e.g. `my-plugin`, `catalog`, `scaffolder`, etc.) -- `{category}` is the logical grouping within the `{scope}` -- `{metric_name}` is the name of the metric as provided by the plugin author. - -#### Scope - -The `scope` represents where it belongs in the Backstage ecosystem. - -- `plugin` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) -- `core` - A core service metric (e.g. `backstage.core.database.connections.active`) - -#### Plugin-Scoped Metrics - -Pattern: `backstage.plugin.{pluginId}.{category}.{metric_name}` - -```yaml -# Examples -backstage.plugin.catalog.entities.processed.total -backstage.plugin.scaffolder.tasks.completed.total -backstage.plugin.techdocs.builds.active -backstage.plugin.auth.sessions.active.total # todo: technically a core service and a backend plugin -```` - -#### Core Metrics - -Pattern: `backstage.core.{core_service}.{category}.{metric_name}` - -```yaml -# Examples -backstage.core.database.connections.active -backstage.core.scheduler.tasks.queued.total -backstage.core.http.requests.total ``` -### References - -- [General naming considerations](https://opentelemetry.io/docs/specs/semconv/general/naming/#general-naming-considerations) -- [Acquiring a meter](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter) - ## Release Plan 1. Create a new `RootMetricsService` that initializes the OpenTelemetry SDK and creates plugin-scoped metrics services. @@ -344,6 +337,10 @@ backstage.core.http.requests.total 1. Create follow-up action items to integrate the new metrics service into the core system. 1. Fully deprecate all existing metrics implementations like the existing Prometheus one-off implementations. +## Deprecation Plan + +TBD + ## Dependencies 1. The root metrics service MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters From e178acc8321fc7cca539ca43eae2142263d61294 Mon Sep 17 00:00:00 2001 From: Kurt King Date: Tue, 5 Aug 2025 17:22:26 -0600 Subject: [PATCH 12/16] clean up based on cr Signed-off-by: Kurt King --- beps/0012-metrics-service/README.md | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md index 9948f22cba..f456126246 100644 --- a/beps/0012-metrics-service/README.md +++ b/beps/0012-metrics-service/README.md @@ -7,7 +7,6 @@ owners: - '@kurtaking' project-areas: - core-framework - - observability creation-date: 2025-06-23 --- @@ -33,13 +32,11 @@ creation-date: 2025-06-23 ## Summary -Add a core `MetricsService` to Backstage's framework to provide a unified interface for metrics instrumentation. The service offers OpenTelemetry-based capabilities with support for app configuration (e.g. `app-config.yaml`). - -This approach leverages industry standards while focusing the `MetricsService` on distinct Backstage concerns, following the same pattern as other core services (DatabaseService builds on Knex, LoggerService builds on Winston, HttpRouterService builds on Express, etc.). +Add a core `MetricsService` to Backstage's framework to provide a unified interface for metrics instrumentation. The service offers industry standards while focusing the `MetricsService` on distinct Backstage concerns, following the same pattern as other core services (DatabaseService builds on Knex, LoggerService builds on Winston, HttpRouterService builds on Express, etc.). ## Motivation -There is no guidance when it comes to metrics instrumentation. While individual plugins may implement their own metrics, there's no standardized approach for metrics collection, naming, or configuration. +There is no guidance when it comes to metrics instrumentation. While individual plugins may implement their own metrics, there's no standardized approach leading to inconsistent metrics patterns across the ecosystem. By providing a core metrics service: @@ -52,19 +49,18 @@ By providing a core metrics service: - Consistent metrics patterns across all plugins - Aligned with OpenTelemetry industry standards - Provide a familiar interface as other core services -- Standardized initialization and lifecycle management The catalog and scaffolder plugins will be updated to use the new metrics service in the initial alpha release. ### Non-Goals -- Adding metrics to plugins that don't currently have it (outside of catalog and scaffolder) +- Adding metrics to plugins missing existing metrics (outside of catalog and scaffolder) - Tracing and other telemetry concerns are out of scope for this BEP. - Refactoring the existing `LoggerService`. Future work to unify observability related concerns would be ideal, but not a goal. ## Proposal -Following similar patterns to other core services, create a new `RootMetricsService` responsible for initializing the OpenTelemetry SDK, root-level concerns, and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `service.name` and optional `service.version` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter). +Following similar patterns to other core services, create a new `RootMetricsService` responsible for root-level concerns and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `service.name` and optional `service.version` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter). ## Naming Conventions @@ -75,8 +71,8 @@ All Backstage metrics follow this hierarchical pattern: **Where:** - `backstage` is the root namespace for all Backstage metrics -- `{scope}` is the system scope (either **plugin** or **core**) -- `{scope_name}` is the name of the plugin or core service (e.g., `catalog`, `scaffolder`, `database`) +- `{scope}` is the system scope (either **plugin** or **service**) +- `{scope_name}` is the name of the plugin or service (e.g., `catalog`, `scaffolder`, `database`, `scheduler`) - `{metric_name}` is the hierarchical metric name as provided by the plugin author (e.g., `entities.processed.total`, `tasks.completed.total`) ### Scope @@ -84,7 +80,7 @@ All Backstage metrics follow this hierarchical pattern: The `scope` represents where it belongs in the Backstage ecosystem. - `plugin` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) -- `core` - A core service metric (e.g. `backstage.core.database.connections.active`) +- `service` - A core service metric (e.g. `backstage.service.database.connections.active`) ### Plugin-Scoped Metrics @@ -100,13 +96,13 @@ backstage.plugin.auth.sessions.active.total # todo: technically a core service a ### Core Metrics -Pattern: `backstage.core.{core_service}.{metric_name}` +Pattern: `backstage.service.{core_service}.{metric_name}` ```yaml # Examples -backstage.core.database.connections.active -backstage.core.scheduler.tasks.queued.total -backstage.core.httpRouter.requests.total +backstage.service.database.connections.active +backstage.service.scheduler.tasks.queued.total +backstage.service.httpRouter.requests.total ``` ## Design Details From e41028cc6b5bff25abcda93396ef8425bbe2066c Mon Sep 17 00:00:00 2001 From: Kurt King Date: Wed, 6 Aug 2025 09:32:06 -0600 Subject: [PATCH 13/16] refactor: mark sdk handling out of scope Signed-off-by: Kurt King --- beps/0012-metrics-service/README.md | 119 ++++++---------------------- 1 file changed, 25 insertions(+), 94 deletions(-) diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md index f456126246..2b9d9db99c 100644 --- a/beps/0012-metrics-service/README.md +++ b/beps/0012-metrics-service/README.md @@ -32,11 +32,11 @@ creation-date: 2025-06-23 ## Summary -Add a core `MetricsService` to Backstage's framework to provide a unified interface for metrics instrumentation. The service offers industry standards while focusing the `MetricsService` on distinct Backstage concerns, following the same pattern as other core services (DatabaseService builds on Knex, LoggerService builds on Winston, HttpRouterService builds on Express, etc.). +Add a core `MetricsService` to Backstage's framework to provide a unified interface for metrics instrumentation. The service offers industry standards (OTEL) while focusing the `MetricsService` on distinct Backstage concerns, following the same pattern as other core services (DatabaseService builds on Knex, LoggerService builds on Winston, HttpRouterService builds on Express, etc.). ## Motivation -There is no guidance when it comes to metrics instrumentation. While individual plugins may implement their own metrics, there's no standardized approach leading to inconsistent metrics patterns across the ecosystem. +While individual plugins may implement their own metrics, there's no standardized approach leading to inconsistent metrics patterns across the ecosystem. For example, both `catalog_entities_count` and `catalog.processed.entities.count` are examples of existing metric patterns. Ideally, these would be standardized to `backstage.plugin.catalog.entities.count` and `backstage.plugin.catalog.entities.processed.total` respectively. By providing a core metrics service: @@ -54,6 +54,7 @@ The catalog and scaffolder plugins will be updated to use the new metrics servic ### Non-Goals +- Providing a way to configure the OpenTelemetry SDK. This is out of scope for this BEP. - Adding metrics to plugins missing existing metrics (outside of catalog and scaffolder) - Tracing and other telemetry concerns are out of scope for this BEP. - Refactoring the existing `LoggerService`. Future work to unify observability related concerns would be ideal, but not a goal. @@ -114,13 +115,6 @@ backstage.service.httpRouter.requests.total ### Integration with OpenTelemetry Auto-Instrumentation -The `RootMetricsService` will automatically enable instrumentation for known libraries leveraged by the Backstage framework. Configuration will be provided to enable or disable auto-instrumentation via inclusion or exclusion lists. - -- Express -- Knex -- Winston -- etc. - The `MetricsService` **complements** rather than duplicates auto-instrumentation by focusing on **application-level metrics** that only Backstage can provide. For example, the catalog plugin may want to track the number of entities processed by the `refresh` operation and the kind of entity being processed. ```ts @@ -135,60 +129,9 @@ entityMetrics.add(entities.length, { operation: 'refresh', kind: 'Component' }); ### Configuration -```ts -// Not final, but this is the general idea... -interface MetricsConfig { - enabled: boolean; +A challenging factor of only introducing a `MetricsService` is the need to collect other OTEL-related configuration such as resources, tracing providers, views, and more prior to starting the SDK. This means that in order to introduce a `MetricsService`, we must support all OTEL Node SDK configuration along with it. Along with this, the official recommendation from the OTEL team is to not initialize and start the SDK on behalf of the user. - resource: { - serviceName?: string; - serviceVersion?: string; - environment?: string; - }; - - collection?: { - exportIntervalMillis?: number; - // ... - }; - - exporters: Array<{ - type: 'prometheus' | 'otlp' | 'console' | '...'; - config?: Record; - }>; - - autoInstrumentation: { - enabled: boolean; - include?: string[]; - exclude?: string[]; - }; -} -``` - -```yaml -backend: - metrics: - enabled: true - - resource: - serviceName: backstage - serviceVersion: 0.0.1 - environment: production - - # Collection settings - collection: - exportIntervalMillis: 15000 - - exporters: - - type: prometheus - config: - port: 9464 - # ... - - type: console - - autoInstrumentation: - enabled: true - exclude: ['express'] -``` +With this, we will not include any configuration as part of this BEP. Users will be responsible for initializing the SDK based on the current guidance ### Interface @@ -219,21 +162,22 @@ interface MetricsService { #### Root Metrics Service -The `RootMetricsService` is responsible for initializing the OpenTelemetry SDK and creating plugin-scoped metrics services. If the end user wants to initialize their own SDK, they are responsible for initializing the OpenTelemetry SDK with their own configuration. The `RootMetricsService` is responsible for providing metrics to other root services and creating plugin-scoped metrics services. +The `RootMetricsService` is responsible for providing metrics to other root services and creating both plugin-scoped and service-scoped `MetricsService` instances. ```ts interface RootMetricsService { + // note: no config is provided to the root service. + static forRoot(): RootMetricsService; forPlugin(pluginId: string): MetricsService; + forService(serviceName: string, scope: 'plugin' | 'service'): MetricsService; } export const rootMetricsServiceFactory = createServiceFactory({ // depends on as little as possible so that it can be initialized as early as possible. service: rootMetricsServiceRef, - deps: { - rootConfig: coreServices.rootConfig, - }, - factory: ({ rootConfig }) => { - return DefaultRootMetricsService.fromConfig(rootConfig); + deps: {}, + factory: () => { + return DefaultRootMetricsService.forRoot(); }, }); ``` @@ -242,32 +186,19 @@ export const rootMetricsServiceFactory = createServiceFactory({ class DefaultRootMetricsService implements RootMetricsService { private sdk: NodeSDK; - static fromConfig(config: Config): RootMetricsService { - const metricsConfig = config.getOptionalConfig('backend.metrics'); - - const sdk = new NodeSDK({ - resource: createResourceFromConfig(metricsConfig), - instrumentations: [ - getNodeAutoInstrumentations({ - ...getAutoInstrumentationConfig(metricsConfig), - }), - ], - metricReader: createMetricReadersFromConfig(metricsConfig), - }); - - sdk.start(); - - return new DefaultRootMetricsService(sdk); + static forRoot(): RootMetricsService { + /** + * By this point, the user should have already initialized the SDK with their own configuration. + */ + return new DefaultRootMetricsService(); } - constructor(private sdk: NodeSDK) {} - forPlugin(pluginId: string): MetricsService { return new PluginMetricsService(pluginId); } - async shutdown(): Promise { - await this.sdk.shutdown(); + forService(serviceName: string, scope: 'plugin' | 'service'): MetricsService { + return new ServiceMetricsService(serviceName, scope); } } ``` @@ -321,15 +252,14 @@ entitiesProcessed.add(100); ## Release Plan -1. Create a new `RootMetricsService` that initializes the OpenTelemetry SDK and creates plugin-scoped metrics services. -1. Create the plugin-scoped `MetricsService` that provides a metrics service for plugins. +1. Create the new metrics-related services. 1. Create alpha-related documentation to add to existing core service [docs](https://backstage.io/docs/backend-system/core-services/index). 1. Release the metrics service under `@alpha`. 1. Mark all existing metrics implementations as deprecated. 1. Refactor catalog and scaffolder plugins to use the new (alpha) `MetricsService`. 1. Offer a migration path for existing adopters to migrate to the new metrics service. -1. Release the metrics service -1. Update all documentation to reference the new metrics service. +1. Release the metrics service under `@public` +1. Update remaining documentation to reference the new metrics service. 1. Create follow-up action items to integrate the new metrics service into the core system. 1. Fully deprecate all existing metrics implementations like the existing Prometheus one-off implementations. @@ -339,9 +269,10 @@ TBD ## Dependencies -1. The root metrics service MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters +1. The otel sdk MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters - we will not change the current guidance on this 1. There are one-off implementations of metrics in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. ## Alternatives -Plugin authors continue to implement their own metrics as they see fit. +- Plugin authors continue to implement their own metrics as they see fit. +- A combined TelemetryService that provides both metrics and tracing. From 1beca9052e315f375d08a9d0e84a87e272d9b368 Mon Sep 17 00:00:00 2001 From: Kurt King Date: Fri, 8 Aug 2025 08:52:15 -0600 Subject: [PATCH 14/16] chore: clean up out of scope portion Signed-off-by: Kurt King --- beps/0012-metrics-service/README.md | 66 ++++++++--------------------- 1 file changed, 18 insertions(+), 48 deletions(-) diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md index 2b9d9db99c..4ffead16b0 100644 --- a/beps/0012-metrics-service/README.md +++ b/beps/0012-metrics-service/README.md @@ -61,7 +61,7 @@ The catalog and scaffolder plugins will be updated to use the new metrics servic ## Proposal -Following similar patterns to other core services, create a new `RootMetricsService` responsible for root-level concerns and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `service.name` and optional `service.version` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter). +Following similar patterns to other core services, create a new `RootMetricsService` responsible for root-level concerns and the creation of plugin-specific `MetricsService` instances. ## Naming Conventions @@ -72,8 +72,8 @@ All Backstage metrics follow this hierarchical pattern: **Where:** - `backstage` is the root namespace for all Backstage metrics -- `{scope}` is the system scope (either **plugin** or **service**) -- `{scope_name}` is the name of the plugin or service (e.g., `catalog`, `scaffolder`, `database`, `scheduler`) +- `{scope}` is the system scope (either **plugin** or **framework**) +- `{scope_name}` is the name of the plugin or framework service (e.g., `catalog`, `scaffolder`, `database`, `scheduler`) - `{metric_name}` is the hierarchical metric name as provided by the plugin author (e.g., `entities.processed.total`, `tasks.completed.total`) ### Scope @@ -81,7 +81,7 @@ All Backstage metrics follow this hierarchical pattern: The `scope` represents where it belongs in the Backstage ecosystem. - `plugin` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) -- `service` - A core service metric (e.g. `backstage.service.database.connections.active`) +- `framework` - A metric provided by the framework (e.g. `backstage.framework.database.connections.active`) ### Plugin-Scoped Metrics @@ -95,15 +95,15 @@ backstage.plugin.techdocs.builds.active backstage.plugin.auth.sessions.active.total # todo: technically a core service and a backend plugin ``` -### Core Metrics +### Framework-Scoped Metrics -Pattern: `backstage.service.{core_service}.{metric_name}` +Pattern: `backstage.framework.{service}.{metric_name}` ```yaml # Examples -backstage.service.database.connections.active -backstage.service.scheduler.tasks.queued.total -backstage.service.httpRouter.requests.total +backstage.framework.database.connections.active +backstage.framework.scheduler.tasks.queued.total +backstage.framework.httpRouter.requests.total ``` ## Design Details @@ -125,17 +125,19 @@ The `MetricsService` **complements** rather than duplicates auto-instrumentation // MetricsService provides (manually): const entityMetrics = metricsService.createCounter('entities.processed.total'); entityMetrics.add(entities.length, { operation: 'refresh', kind: 'Component' }); + +// Metric is now available as `backstage.plugin.catalog.entities.processed.total` ``` ### Configuration A challenging factor of only introducing a `MetricsService` is the need to collect other OTEL-related configuration such as resources, tracing providers, views, and more prior to starting the SDK. This means that in order to introduce a `MetricsService`, we must support all OTEL Node SDK configuration along with it. Along with this, the official recommendation from the OTEL team is to not initialize and start the SDK on behalf of the user. -With this, we will not include any configuration as part of this BEP. Users will be responsible for initializing the SDK based on the current guidance +With this, **we will not include any configuration** as part of this BEP. Users will be responsible for initializing the SDK based on the current guidance ### Interface -Provide a wrapper around OpenTelemetry's API while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. +Provide a wrapper around OpenTelemetry's API while leveraging the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. ```ts interface MetricsService { @@ -162,7 +164,7 @@ interface MetricsService { #### Root Metrics Service -The `RootMetricsService` is responsible for providing metrics to other root services and creating both plugin-scoped and service-scoped `MetricsService` instances. +The `RootMetricsService` is responsible for providing metrics to other root services and creating both plugin-scoped and framework-scoped `MetricsService` instances. ```ts interface RootMetricsService { @@ -182,27 +184,6 @@ export const rootMetricsServiceFactory = createServiceFactory({ }); ``` -```ts -class DefaultRootMetricsService implements RootMetricsService { - private sdk: NodeSDK; - - static forRoot(): RootMetricsService { - /** - * By this point, the user should have already initialized the SDK with their own configuration. - */ - return new DefaultRootMetricsService(); - } - - forPlugin(pluginId: string): MetricsService { - return new PluginMetricsService(pluginId); - } - - forService(serviceName: string, scope: 'plugin' | 'service'): MetricsService { - return new ServiceMetricsService(serviceName, scope); - } -} -``` - #### Plugin Metrics Service Each plugin receives a metrics service that automatically namespaces all metrics to match the naming conventions. @@ -218,22 +199,9 @@ const metricsServiceFactory = createServiceFactory({ return rootMetrics.forPlugin(pluginMetadata.getId()); }, }); - -class PluginMetricsService implements MetricsService { - // ... - constructor(private pluginId: string) { - this.meter = metrics.getMeter(`backstage.plugin.${pluginId}`); - } - - // ... other interface methods - - private prefixMetricName(name: string): string { - return `backstage.plugin.${this.pluginId}.${name}`; - } -} ``` -##### Example +#### Example ```ts const entitiesProcessed = metricsService.createCounter( @@ -265,7 +233,9 @@ entitiesProcessed.add(100); ## Deprecation Plan -TBD +1. Deprecation warning are added to all existing metrics +2. New metrics will run in parallel with the deprecated ones for a period of time +3. All existing metrics are removed from the codebase in the next major version ## Dependencies From 747a180277c001dfb469304587dcd6e4a02022d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fredrik=20Adel=C3=B6w?= Date: Tue, 19 Aug 2025 09:39:14 +0200 Subject: [PATCH 15/16] Update beps/0012-metrics-service/README.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Fredrik Adelöw --- beps/0012-metrics-service/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md index 4ffead16b0..6e9d990bfb 100644 --- a/beps/0012-metrics-service/README.md +++ b/beps/0012-metrics-service/README.md @@ -239,7 +239,7 @@ entitiesProcessed.add(100); ## Dependencies -1. The otel sdk MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters - we will not change the current guidance on this +1. The `otel` SDK MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters - we will not change the current guidance on this 1. There are one-off implementations of metrics in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. ## Alternatives From 83cdd0893148a7c4e8497fbf31c5631c72db4003 Mon Sep 17 00:00:00 2001 From: Kurt King Date: Tue, 19 Aug 2025 20:18:33 -0600 Subject: [PATCH 16/16] Use core in place of framework Signed-off-by: Kurt King --- beps/0012-metrics-service/README.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md index 4ffead16b0..1909297eaf 100644 --- a/beps/0012-metrics-service/README.md +++ b/beps/0012-metrics-service/README.md @@ -72,16 +72,16 @@ All Backstage metrics follow this hierarchical pattern: **Where:** - `backstage` is the root namespace for all Backstage metrics -- `{scope}` is the system scope (either **plugin** or **framework**) -- `{scope_name}` is the name of the plugin or framework service (e.g., `catalog`, `scaffolder`, `database`, `scheduler`) -- `{metric_name}` is the hierarchical metric name as provided by the plugin author (e.g., `entities.processed.total`, `tasks.completed.total`) +- `{scope}` is the system scope (either **plugin** or **core**) +- `{scope_name}` is the name of the plugin or core service (e.g., `catalog`, `scaffolder`, `database`, `scheduler`) +- `{metric_name}` is the hierarchical metric name as provided by the plugin author (e.g., `entity.count`, `tasks.completed.total`) ### Scope The `scope` represents where it belongs in the Backstage ecosystem. -- `plugin` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) -- `framework` - A metric provided by the framework (e.g. `backstage.framework.database.connections.active`) +- `plugin` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entity.count`) +- `core` - A metric provided by the core system (e.g. `backstage.core.database.connections.active`) ### Plugin-Scoped Metrics @@ -95,15 +95,15 @@ backstage.plugin.techdocs.builds.active backstage.plugin.auth.sessions.active.total # todo: technically a core service and a backend plugin ``` -### Framework-Scoped Metrics +### Core-Scoped Metrics -Pattern: `backstage.framework.{service}.{metric_name}` +Pattern: `backstage.core.{service}.{metric_name}` ```yaml # Examples -backstage.framework.database.connections.active -backstage.framework.scheduler.tasks.queued.total -backstage.framework.httpRouter.requests.total +backstage.core.database.connections.active +backstage.core.scheduler.tasks.queued.total +backstage.core.httpRouter.requests.total ``` ## Design Details @@ -164,14 +164,16 @@ interface MetricsService { #### Root Metrics Service -The `RootMetricsService` is responsible for providing metrics to other root services and creating both plugin-scoped and framework-scoped `MetricsService` instances. +The `RootMetricsService` is responsible for providing metrics to other root services and creating both plugin-scoped and core-scoped `MetricsService` instances. ```ts interface RootMetricsService { // note: no config is provided to the root service. static forRoot(): RootMetricsService; forPlugin(pluginId: string): MetricsService; - forService(serviceName: string, scope: 'plugin' | 'service'): MetricsService; + + // final implementation will be similar to + forService(serviceName: string, scope: 'plugin' | 'core'): MetricsService; } export const rootMetricsServiceFactory = createServiceFactory({