From a292cf3c0a86775b089894abc70a44aab362ab69 Mon Sep 17 00:00:00 2001 From: Kurt King Date: Thu, 3 Jul 2025 08:16:04 -0600 Subject: [PATCH] refactor: only focus on metrics service Signed-off-by: Kurt King --- beps/0012-metrics-service/README.md | 255 ++++++++++++++++++ beps/0012-telemetry-service/README.md | 208 -------------- .../README.md | 113 -------- 3 files changed, 255 insertions(+), 321 deletions(-) create mode 100644 beps/0012-metrics-service/README.md delete mode 100644 beps/0012-telemetry-service/README.md delete mode 100644 beps/0013-telemetry-metrics-naming-conventions/README.md diff --git a/beps/0012-metrics-service/README.md b/beps/0012-metrics-service/README.md new file mode 100644 index 0000000000..8bb290d6d0 --- /dev/null +++ b/beps/0012-metrics-service/README.md @@ -0,0 +1,255 @@ +--- +title: Backstage Metrics Service +status: implementable +authors: + - '@kurtaking' +owners: + - '@kurtaking' +project-areas: + - core-framework + - observability +creation-date: 2025-06-23 +--- + +# BEP: Backstage Metrics Service + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) +- [Design Details](#design-details) + - [Configuration](#configuration) + - [Interface](#interface) + - [Naming Conventions](#naming-conventions) +- [Release Plan](#release-plan) +- [Dependencies](#dependencies) +- [Alternatives](#alternatives) + +## Summary + +Add a core `MetricsService` to Backstage's framework that provides a unified interface for metrics instrumentation. The service offers OpenTelemetry-based capabilities with support for app configuration (e.g. `app-config.yaml`). + +## Motivation + +There is currently no guidance for plugin authors when it comes to metrics instrumentation. While individual plugins may implement their own metrics, there's no standardized approach for metrics collection, naming, or configuration. + +By providing a core metrics service: + +- **Plugin Authors** and the **Community** gain a straightforward way to address metrics instrumentation and can focus on business logic instead of needing to reimplement metrics plumbing. +- **Backstage Admins** receive a reliable stream of metrics from the core system for monitoring, alerting, and troubleshooting. + +### Goals + +- Consistent metrics patterns across all plugins +- Aligned with OpenTelemetry industry standards +- Provide a familiar interface as other core services + +The catalog and scaffolder plugins will be updated to use the new metrics service in the initial release. + +### Non-Goals + +- Adding metrics to plugins that don't currently have it (outside of catalog and scaffolder) +- Providing a full Backstage telemetry SDK (aka recreate the OTEL Node SDK). +- Tracing and other telemetry concerns are out of scope for this BEP. +- Refactoring the existing `LoggerService`. Future work to unify observability related concerns would be ideal, but not a goal. + +## Proposal + +Following similar patterns to other core services, create a new `RootMetricsService` responsible for initializing metrics, root-level concerns, and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `serviceName` and optional `serviceVersion` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter). + +## Design Details + +### Configuration + + + +```yaml +backend: + metrics: + serviceName: backstage + # todo: this is optional - should we just default it to the current Backstage release and allow override? + serviceVersion: 0.0.1 +``` + +### Interface + +Provide a familiar interface as other core services while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry. + +```ts +interface MetricsService { + // Synchronous instrumentation + createCounter(name: string, options?: MetricOptions): Counter; + createUpDownCounter(name: string, options?: MetricOptions): UpDownCounter; + createHistogram(name: string, options?: MetricOptions): Histogram; + createGauge(name: string, options?: MetricOptions): Gauge; + + // Asynchronous instrumentation + createObservableCounter( + name: string, + options?: MetricOptions, + ): ObservableCounter; + createObservableUpDownCounter( + name: string, + options?: MetricOptions, + ): ObservableUpDownCounter; + createObservableGauge(name: string, options?: MetricOptions): ObservableGauge; + + // Provide access to the underlying meter for advanced configuration. + getMeter(): Meter; + + // Additional convenience methods tbd... +} +``` + +The `RootMetricsService` is responsible for the global meter, namespaced to `backstage` or the `serviceName` and `serviceVersion` provided in the app's `app-config.yaml` file. The root metrics service will depend on as little as possible so that it can be initialized as early as possible. + +```ts +export const rootMetricsServiceFactory = createServiceFactory({ + service: rootMetricsServiceRef, + deps: { + rootConfig: coreServices.rootConfig, + }, + factory: ({ rootConfig }) => { + return DefaultRootMetricsService.fromConfig(rootConfig); + }, +}); +``` + +It also provides a method for creating a plugin-scoped `MetricsService` for each registered plugin. + +```ts +interface RootMetricsService extends MetricsService { + forPlugin(pluginId: string): MetricsService; +} + +class DefaultRootMetricsService { + // ... + + static fromConfig(config: Config): RootMetricsService { + return new DefaultRootMetricsService(config); + } + + forPlugin(pluginId: string): MetricsService { + return new PluginMetricsService({ + pluginId, + serviceName: this.serviceName, + serviceVersion: this.serviceVersion, + }); + } +} +``` + +Each plugin-scoped `MetricsService` acquires a meter when it is created by calling `metrics.getMeter` from the `@opentelemetry/api` package. The internals of the service ensure the metric name is namespaced to match the naming conventions outlined in the [naming conventions](#naming-conventions) section. + +```ts +class PluginMetricsService implements MetricsService { + // ... + constructor({ + pluginId, + serviceName, + serviceVersion, + }: PluginMetricsServiceOptions) { + this.pluginId = pluginId; + this.serviceName = serviceName; + this.meter = metrics.getMeter(serviceName, serviceVersion); + } + + // Ensures consistency across all metrics + private prefixMetricName(name: string): string { + return `${this.serviceName}.plugin.${this.pluginId}.${name}`; + } +} +``` + +### Naming Conventions + +All Backstage metrics MUST follow this hierarchical pattern: + +`backstage.{scope}.{scope_name}.{category}.{metric_name}` + +**Where:** + +- `backstage` is the root namespace for all Backstage metrics +- `{scope}` is the system scope (`pluginId`, `core` for core services) +- `{scope_name}` is the name of the scope (e.g. `my-plugin`, `catalog`, `scaffolder`, etc.) +- `{category}` is the logical grouping within the `{scope}` +- `{metric_name}` is the name of the metric + +The metric name will be the name as provided by the plugin author in the form `{category}.{name}`. + +```ts +const rootMetrics = DefaultRootMetricsService.fromConfig(config); +const pluginMetricsService = rootMetrics.forPlugin('my-plugin'); + +const metric = pluginMetricsService.createCounter('my_category.my_metric'); + +metric.add(1); + +// ... +// metric is now available as `backstage.plugin.my-plugin.my_category.my_metric` +``` + +#### Scope + +The `scope` represents where it belongs in the Backstage ecosystem. + +- `pluginId` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) +- `core` - A core service metric (e.g. `backstage.core.database.connections.active`) + +##### Plugin-Scoped Metrics + +Pattern: `backstage.plugin.{pluginId}.{category}.{metric_name}` + +```yaml +# Examples +backstage.plugin.catalog.entities.processed.total +backstage.plugin.scaffolder.tasks.completed.total +backstage.plugin.techdocs.builds.active +backstage.plugin.auth.sessions.active.total # todo: technically a core service and a backend plugin +``` + +##### Core Metrics + +Pattern: `backstage.core.{core_service}.{category}.{metric_name}` + +```yaml +# Examples +backstage.core.database.connections.active +backstage.core.scheduler.tasks.queued.total +backstage.core.http.requests.total +``` + +### References + +- [General naming considerations](https://opentelemetry.io/docs/specs/semconv/general/naming/#general-naming-considerations) +- [Acquiring a meter](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter) + +## Release Plan + +1. Create a new `RootMetricsService` that initializes metrics and app-wide concerns. +1. Create the plugin-scoped `MetricsService` that provides a metrics service for plugins. +1. Create alpha-related documentation to add to existing core service [docs](https://backstage.io/docs/backend-system/core-services/index). +1. Release the metrics service as alpha. +1. Refactor catalog and scaffolder implementations to use the new (alpha) `MetricsService`. +1. Release the metrics service +1. Offer a migration path for existing adopters to migrate to the new metrics service. +1. Update all documentation to reference the new metrics service. +1. Create follow-up action items to integrate the new metrics service into the core system. +1. Fully deprecate all existing metrics implementations like the existing Prometheus one-off implementations. + +## Dependencies + +1. The root metrics service MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters, while still allowing for configuration. +1. There are one-off implementations of metrics in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. + +## Alternatives + +Plugin authors continue to implement their own metrics as they see fit. diff --git a/beps/0012-telemetry-service/README.md b/beps/0012-telemetry-service/README.md deleted file mode 100644 index f63e5c2cf4..0000000000 --- a/beps/0012-telemetry-service/README.md +++ /dev/null @@ -1,208 +0,0 @@ ---- -title: Backstage Telemetry Service -status: implementable -authors: - - '@kurtaking' -owners: - - '@kurtaking' -project-areas: - - backend - - core - - framework -creation-date: 2025-06-23 ---- - -# BEP: Backstage Telemetry Service - -- [Summary](#summary) -- [Motivation](#motivation) - - [Goals](#goals) - - [Non-Goals](#non-goals) -- [Proposal](#proposal) -- [Design Details](#design-details) - - [Configuration](#configuration) -- [Release Plan](#release-plan) -- [Dependencies](#dependencies) -- [Alternatives](#alternatives) - -## Summary - -Add a core telemetry service to Backstage's framework that provides a unified interface for metrics, tracing, and logging instrumentation. The service offers OpenTelemetry-based capabilities with zero configuration defaults while allowing extensibility through app configuration (e.g. `app-config.yaml`) and plugin modules. - -## Considerations - -1. It's possible a single telemetry service is too encompassing and that more granular services are needed. For example, a `MetricsService` and `TracingService` which would work along side the existing `LoggerService` to solve for telemetry needs. - -## Motivation - -There is currently no guidance for plugin authors when it comes to telemetry and while individual plugins may implement their own metrics or tracing, there's no unified service that provides a standardized approach for telemetry and observability. - -By providing a core telemetry service: - -- **Plugin Authors** gain a straightforward, documented way to address telemetry, and can focus on business logic instead of needing to reimplement telemetry plumbing. -- The **community** receives a standard approach to telemetry. -- **Backstage Operators/SRE/Platform Engineers** receive a consistent, reliable stream of metrics from the core system, simplifying monitoring, alerting, and troubleshooting. They can enforce metrics standards, aggregate data, and build dashboards more easily. -**Organizations** with their own observability or compliance requirements can override the core service to integrate with proprietary or custom telemetry solutions, without forking or modifying plugins. - -### Goals - -Provide a unified service that provides a standardized approach for telemetry where setup docs like [these](https://backstage.io/docs/tutorials/setup-opentelemetry/) are not needed. - -- Consistent telemetry patterns across all plugins -- Zero-configuration defaults that work out of the box -- Extensible architecture for custom telemetry backends -- OpenTelemetry integration following industry standards -- Performance monitoring capabilities built into the platform - -We will include the following two plugins in the initial release to demonstrate the new telemetry service: - -- Replace existing catalog implementation with `TelemetryService` -- Replace existing scaffolder implementation with `TelemetryService` - -### Non-Goals - -- Adding telemetry instrumentation to existing plugins that don't currently have it -- Only the catalog and scaffolder implementations will be refactored to use the new telemetry service. -- We will not start by providing a full Backstage SDK (aka recreate the OTEL Node SDK). -- We will not refactor the existing LoggerService. Future work to unify telemetry related concerns would be ideal, but not a goal. - -## Proposal - -Following similar patterns to other core services, we will create a new `RootTelemetryService` responsible for initializing telemetry and app-wide concerns, and the creation of plugin-specific telemetry services. - -A core concern is not initializing the telemetry service early enough in the backend lifecycle. Services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping functionality that will be called as early as possible in the backend lifecycle. - -## Configuration - -All OTEL configuration will be supported via the app's `app-config.yaml` file. The config schema should be based on existing resources such as the [OTEL collector configuration](https://opentelemetry.io/docs/collector/configuration/). - -```yaml -# this is not a complete example, but it shows the general idea -telemetry: - enabled: true - - resource: - serviceName: backstage-telemetry - serviceVersion: 0.0.1 - - metrics: - enabled: true - collection: - exportIntervalMillis: 15000 - - exporters: - - type: prometheus - enabled: true - config: - port: 9464 - - tracing: - enabled: false - - instrumentations: - http: false - knex: false - express: true - # ... -``` - -## Design Details - -We will provide a thin wrapper around the OpenTelemetry Node SDK to provide a consistent interface while re-exporting the types from the SDK. This lets us provide a consistent telemetry interface while leveraging types and concepts already familiar to the community. Additionally, we will rely on the auto instrumentation features of the SDK as much as possible. - -The decision to leverage the SDK comes from the fact that OpenTelemetry is already the fundamental abstraction for metrics and tracing. - -```ts -interface TelemetryService { - createCounter(name: string, options?: MetricOptions): Counter; - // - - // provide escape hatch for advanced use cases (???) - todo: I don't like this if we don't have a good reason for it - getMeter(): Meter; -} - -interface RootTelemetryService extends TelemetryService { - forPlugin(pluginId: string): TelemetryService; - - // provide escape hatches for advanced use cases (???) - getClient(): NodeSDK; // give access to the SDK -} -``` - -### Naming Conventions - -Visit [0013-telemetry-metrics-naming-conventions](../0013-telemetry-metrics-naming-conventions/README.md) for details on the naming conventions. - -The root telemetry service is responsible for initializing the global meter, namespaced to `backstage`. The root service will initialize a meter for each plugin that is registered based on the `pluginId`. The metric name will be the name of the metric as provided by the plugin author in the form `{category}.{name}`. - -```ts -const rootTelemetry = DefaultRootTelemetryService.fromConfig(config); -const pluginTelemetryService = rootTelemetry.forPlugin('my-plugin'); - -const metric = pluginTelemetryService.createCounter('my_category.my_metric'); - -metric.add(1); - -// ... -// metric is now available as `backstage.my-plugin.my_category.my_metric` -``` - -### References - -- [General naming considerations](https://opentelemetry.io/docs/specs/semconv/general/naming/#general-naming-considerations) - -### Bootstrapping - -A core concern is not initializing the telemetry service early enough in the backend lifecycle and services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping mechanism that will be called as early as possible in the backend lifecycle. - -Ideally, this is called at the start of the `createBackend` function [[ref](https://github.com/backstage/backstage/blob/ab539cc72bb503618df55815f24c4ba933607460/packages/backend-defaults/src/CreateBackend.ts#L73-L75)], but we need to gain access to the config file at minimum. Ideally the logger as well, but not as crucial. - -```ts -export function createBackend(): Backend { - // ... - // call the bootstrap function - Telemetry.bootstrap(); - // ... - return createSpecializedBackend({ defaultServiceFactories }); -} -``` - -### Plugins & Modules - -WIP - - - -## Release Plan - -1. Create a new `RootTelemetryService` that initializes telemetry and app-wide concerns. -1. Create the plugin-scoped `TelemetryService` that provides a telemetry service for plugins. -1. Release the telemetry service as alpha. -1. Refactor catalog and scaffolder implementations to use the new (alpha) `TelemetryService`. -1. Replace the existing setup docs by integrating the new `RootTelemetryService` to `createBackend` -1. Release the telemetry service -1. Offer a migration path for existing adopters to migrate to the new telemetry service. -1. Update all documentation to reference the new telemetry service. -1. Create follow-up action items to integrate the new telemetry service into the core system. -1. Fully deprecate all existing telemetry implementations like the existing Prometheus one-off implementations. - -## Dependencies - -1. The root telemetry service MUST BE initialized as EARLY as possible to prevent dependents from receiving no op meters, while still allowing for app-wide configuration. -1. There are one-off implementations of telemetry in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up. - -## Alternatives - -1. Plugin authors continue to implement their own telemetry as they see fit. -1. Adopters continue using the [setup-opentelemetry](https://backstage.io/docs/tutorials/setup-opentelemetry/) docs as a guide. - -## Outstanding Questions - -1. Are there enough Backstage-isms to build an additional layer of abstraction for metrics and tracing on top of the OpenTelemetry SDK? Do we re-export the types from the OpenTelemetry SDK or do we create our own? How do we handle breaking changes? -1. How do we guarantee that the telemetry service is initialized early enough in the backend lifecycle, while still allowing for app-wide configuration? -1. Should we provide an escape hatch for advanced use cases? -1. Should there be a single telemetry service or should we provide more granular services (e.g. adding separate `MetricsService` and `TracingService`)? -1. What core services could leverage the auto instrumentation features of the OpenTelemetry SDK? (e.g. `DatabaseService` could leverage the `knex` auto instrumentation, `HttpService` could leverage the `express` auto instrumentation, etc.) diff --git a/beps/0013-telemetry-metrics-naming-conventions/README.md b/beps/0013-telemetry-metrics-naming-conventions/README.md deleted file mode 100644 index c6cbc85d24..0000000000 --- a/beps/0013-telemetry-metrics-naming-conventions/README.md +++ /dev/null @@ -1,113 +0,0 @@ ---- -title: OpenTelemetry Metrics Naming Conventions for Backstage -status: implementable -authors: - - '@kurtaking' -owners: - - '@kurtaking' -project-areas: - - framework - - telemetry -creation-date: 2025-06-26 ---- - -# BEP: OpenTelemetry Metrics Naming Conventions for Backstage - -- [Summary](#summary) -- [Motivation](#motivation) - - [Goals](#goals) - - [Non-Goals](#non-goals) -- [Proposal](#proposal) -- [Design Details](#design-details) -- [Release Plan](#release-plan) -- [Dependencies](#dependencies) -- [Alternatives](#alternatives) - -## Summary - -This proposal establishes standardized naming conventions for OpenTelemetry metrics across the Backstage ecosystem. It defines a hierarchical naming pattern that ensures consistency, discoverability, and compliance with OpenTelemetry semantic conventions while providing clear guidance for plugin and module authors. - -## Motivation - -Currently, Backstage has: - -- Mixed metric naming patterns: Legacy Prometheus metrics (`catalog_entities_count`) alongside newer OpenTelemetry metrics (`catalog.processing.queue.delay`) -- No standard pattern for plugin-specific metrics -- Difficulty correlating metrics across plugins and providers -- Some existing metrics don't follow OpenTelemetry semantic conventions - -We need to decided on a pattern that we can use for the telemetry service to obfuscate the underlying implementation details. - -### Goals - -- Establish consistent naming patterns across all Backstage metrics -- Ensure OpenTelemetry compliance with semantic conventions -- Provide clear guidance for plugin and service authors - -### Non-Goals - -- Breaking existing metrics immediately (backward compatibility maintained) -- Forcing all plugins to emit the same metrics -- Defining specific metric values or thresholds - -## Proposal - -Provide guidance for naming metrics for Backstage. - -## Design Details - -### Core Naming Pattern - -All Backstage metrics MUST follow this hierarchical pattern: - -`backstage.{scope}.{category}.{metric_name}` - -**Where:** - -- `backstage` is the root namespace for all Backstage metrics -- `{scope}` is the system scope (`pluginId`, `core` for core services) -- `{category}` is the logical grouping within the `{scope}` -- `{metric_name}` is the name of the metric - -### Scope - -The `scope` represents where it belongs in the Backstage ecosystem. - -- `pluginId` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`) -- `core` - A core service metric (e.g. `backstage.core.database.connections.active`) - -#### Plugin Metrics - -Pattern: `backstage.{pluginId}.{category}.{metric_name}` - -```yaml -# Examples -backstage.catalog.entities.processed.total -backstage.scaffolder.tasks.completed.total -backstage.techdocs.builds.active -backstage.auth.sessions.active.total -``` - -#### Core Metrics - -Pattern: `backstage.core.{category}.{metric_name}` - -```yaml -# Examples -backstage.core.database.connections.active -backstage.core.scheduler.tasks.queued.total -backstage.core.http.requests.total -``` - -## Release Plan - -- Provide documentation for the naming conventions -- Release as part of the larger telemetry service proposal - -## Dependencies - -- [0012-telemetry-service](../0012-telemetry-service/README.md) - -## Alternatives - -Adopters implement their own metrics and naming conventions.