paulononato/backstage
1
0
Fork 0
Code Issues Pull Requests Actions 27 Packages Projects Releases Wiki Activity

refactor: only focus on metrics service

Browse Source 
Signed-off-by: Kurt King <kurtaking@gmail.com>
This commit is contained in:
Kurt King
2025-07-03 08:16:04 -06:00
parent aba73ed311
commit a292cf3c0a
3 changed files with 255 additions and 321 deletions
Download Patch File Download Diff File Expand all files Collapse all files
beps/0012-metrics-service/README.md
+255
View File
@@ -0,0 +1,255 @@
---
title: Backstage Metrics Service
status: implementable
authors:
- '@kurtaking'
owners:
- '@kurtaking'
project-areas:
- core-framework
- observability
creation-date: 2025-06-23
---
# BEP: Backstage Metrics Service
- [Summary](#summary)
- [Motivation](#motivation)
- [Goals](#goals)
- [Non-Goals](#non-goals)
- [Proposal](#proposal)
- [Design Details](#design-details)
- [Configuration](#configuration)
- [Interface](#interface)
- [Naming Conventions](#naming-conventions)
- [Release Plan](#release-plan)
- [Dependencies](#dependencies)
- [Alternatives](#alternatives)
## Summary
Add a core `MetricsService` to Backstage's framework that provides a unified interface for metrics instrumentation. The service offers OpenTelemetry-based capabilities with support for app configuration (e.g. `app-config.yaml`).
## Motivation
There is currently no guidance for plugin authors when it comes to metrics instrumentation. While individual plugins may implement their own metrics, there's no standardized approach for metrics collection, naming, or configuration.
By providing a core metrics service:
- **Plugin Authors** and the **Community** gain a straightforward way to address metrics instrumentation and can focus on business logic instead of needing to reimplement metrics plumbing.
- **Backstage Admins** receive a reliable stream of metrics from the core system for monitoring, alerting, and troubleshooting.
### Goals
- Consistent metrics patterns across all plugins
- Aligned with OpenTelemetry industry standards
- Provide a familiar interface as other core services
The catalog and scaffolder plugins will be updated to use the new metrics service in the initial release.
### Non-Goals
- Adding metrics to plugins that don't currently have it (outside of catalog and scaffolder)
- Providing a full Backstage telemetry SDK (aka recreate the OTEL Node SDK).
- Tracing and other telemetry concerns are out of scope for this BEP.
- Refactoring the existing `LoggerService`. Future work to unify observability related concerns would be ideal, but not a goal.
## Proposal
Following similar patterns to other core services, create a new `RootMetricsService` responsible for initializing metrics, root-level concerns, and the creation of plugin-specific `MetricsService` instances. The root service delegates to the plugin-scoped `MetricsService` to initialize a meter for each registered plugin based on the `serviceName` and optional `serviceVersion` provided in the app's `app-config.yaml` file. This is based on the recommendation in the OpenTelemetry [documentation](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter).
## Design Details
### Configuration
<!--
todo: I want to provide full configuration via the app-config, but that requires us wrapping the entire SDK and handling the `instrumentation.js`
instructions. I'm not sure we want to start there.
Metrics configuration will continue to be defined by the adopter in their `instrumentation.js` file as outlined in the [setup-opentelemetry](https://backstage.io/docs/tutorials/setup-opentelemetry/) docs.
The initial alpha release will include a simple configuration for the root metrics service.
-->
```yaml
backend:
metrics:
serviceName: backstage
# todo: this is optional - should we just default it to the current Backstage release and allow override?
serviceVersion: 0.0.1
```
### Interface
Provide a familiar interface as other core services while re-exporting the types from the `@opentelemetry/api` package. This introduces concepts already familiar to both the Backstage community and those familiar with OpenTelemetry.
```ts
interface MetricsService {
// Synchronous instrumentation
createCounter(name: string, options?: MetricOptions): Counter;
createUpDownCounter(name: string, options?: MetricOptions): UpDownCounter;
createHistogram(name: string, options?: MetricOptions): Histogram;
createGauge(name: string, options?: MetricOptions): Gauge;
// Asynchronous instrumentation
createObservableCounter(
name: string,
options?: MetricOptions,
): ObservableCounter;
createObservableUpDownCounter(
name: string,
options?: MetricOptions,
): ObservableUpDownCounter;
createObservableGauge(name: string, options?: MetricOptions): ObservableGauge;
// Provide access to the underlying meter for advanced configuration.
getMeter(): Meter;
// Additional convenience methods tbd...
}
```
The `RootMetricsService` is responsible for the global meter, namespaced to `backstage` or the `serviceName` and `serviceVersion` provided in the app's `app-config.yaml` file. The root metrics service will depend on as little as possible so that it can be initialized as early as possible.
```ts
export const rootMetricsServiceFactory = createServiceFactory({
service: rootMetricsServiceRef,
deps: {
rootConfig: coreServices.rootConfig,
},
factory: ({ rootConfig }) => {
return DefaultRootMetricsService.fromConfig(rootConfig);
},
});
```
It also provides a method for creating a plugin-scoped `MetricsService` for each registered plugin.
```ts
interface RootMetricsService extends MetricsService {
forPlugin(pluginId: string): MetricsService;
}
class DefaultRootMetricsService {
// ...
static fromConfig(config: Config): RootMetricsService {
return new DefaultRootMetricsService(config);
}
forPlugin(pluginId: string): MetricsService {
return new PluginMetricsService({
pluginId,
serviceName: this.serviceName,
serviceVersion: this.serviceVersion,
});
}
}
```
Each plugin-scoped `MetricsService` acquires a meter when it is created by calling `metrics.getMeter` from the `@opentelemetry/api` package. The internals of the service ensure the metric name is namespaced to match the naming conventions outlined in the [naming conventions](#naming-conventions) section.
```ts
class PluginMetricsService implements MetricsService {
// ...
constructor({
pluginId,
serviceName,
serviceVersion,
}: PluginMetricsServiceOptions) {
this.pluginId = pluginId;
this.serviceName = serviceName;
this.meter = metrics.getMeter(serviceName, serviceVersion);
}
// Ensures consistency across all metrics
private prefixMetricName(name: string): string {
return `${this.serviceName}.plugin.${this.pluginId}.${name}`;
}
}
```
### Naming Conventions
All Backstage metrics MUST follow this hierarchical pattern:
`backstage.{scope}.{scope_name}.{category}.{metric_name}`
**Where:**
- `backstage` is the root namespace for all Backstage metrics
- `{scope}` is the system scope (`pluginId`, `core` for core services)
- `{scope_name}` is the name of the scope (e.g. `my-plugin`, `catalog`, `scaffolder`, etc.)
- `{category}` is the logical grouping within the `{scope}`
- `{metric_name}` is the name of the metric
The metric name will be the name as provided by the plugin author in the form `{category}.{name}`.
```ts
const rootMetrics = DefaultRootMetricsService.fromConfig(config);
const pluginMetricsService = rootMetrics.forPlugin('my-plugin');
const metric = pluginMetricsService.createCounter('my_category.my_metric');
metric.add(1);
// ...
// metric is now available as `backstage.plugin.my-plugin.my_category.my_metric`
```
#### Scope
The `scope` represents where it belongs in the Backstage ecosystem.
- `pluginId` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`)
- `core` - A core service metric (e.g. `backstage.core.database.connections.active`)
##### Plugin-Scoped Metrics
Pattern: `backstage.plugin.{pluginId}.{category}.{metric_name}`
```yaml
# Examples
backstage.plugin.catalog.entities.processed.total
backstage.plugin.scaffolder.tasks.completed.total
backstage.plugin.techdocs.builds.active
backstage.plugin.auth.sessions.active.total # todo: technically a core service and a backend plugin
```
##### Core Metrics
Pattern: `backstage.core.{core_service}.{category}.{metric_name}`
```yaml
# Examples
backstage.core.database.connections.active
backstage.core.scheduler.tasks.queued.total
backstage.core.http.requests.total
```
### References
- [General naming considerations](https://opentelemetry.io/docs/specs/semconv/general/naming/#general-naming-considerations)
- [Acquiring a meter](https://opentelemetry.io/docs/languages/js/instrumentation/#acquiring-a-meter)
## Release Plan
1. Create a new `RootMetricsService` that initializes metrics and app-wide concerns.
1. Create the plugin-scoped `MetricsService` that provides a metrics service for plugins.
1. Create alpha-related documentation to add to existing core service [docs](https://backstage.io/docs/backend-system/core-services/index).
1. Release the metrics service as alpha.
1. Refactor catalog and scaffolder implementations to use the new (alpha) `MetricsService`.
1. Release the metrics service
1. Offer a migration path for existing adopters to migrate to the new metrics service.
1. Update all documentation to reference the new metrics service.
1. Create follow-up action items to integrate the new metrics service into the core system.
1. Fully deprecate all existing metrics implementations like the existing Prometheus one-off implementations.
## Dependencies
1. The root metrics service MUST BE initialized as EARLY as possible to prevent dependents from receiving no-op meters, while still allowing for configuration.
1. There are one-off implementations of metrics in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up.
## Alternatives
Plugin authors continue to implement their own metrics as they see fit.
beps/0012-telemetry-service/README.md
-208
View File
@@ -1,208 +0,0 @@
---
title: Backstage Telemetry Service
status: implementable
authors:
- '@kurtaking'
owners:
- '@kurtaking'
project-areas:
- backend
- core
- framework
creation-date: 2025-06-23
---
# BEP: Backstage Telemetry Service
- [Summary](#summary)
- [Motivation](#motivation)
- [Goals](#goals)
- [Non-Goals](#non-goals)
- [Proposal](#proposal)
- [Design Details](#design-details)
- [Configuration](#configuration)
- [Release Plan](#release-plan)
- [Dependencies](#dependencies)
- [Alternatives](#alternatives)
## Summary
Add a core telemetry service to Backstage's framework that provides a unified interface for metrics, tracing, and logging instrumentation. The service offers OpenTelemetry-based capabilities with zero configuration defaults while allowing extensibility through app configuration (e.g. `app-config.yaml`) and plugin modules.
## Considerations
1. It's possible a single telemetry service is too encompassing and that more granular services are needed. For example, a `MetricsService` and `TracingService` which would work along side the existing `LoggerService` to solve for telemetry needs.
## Motivation
There is currently no guidance for plugin authors when it comes to telemetry and while individual plugins may implement their own metrics or tracing, there's no unified service that provides a standardized approach for telemetry and observability.
By providing a core telemetry service:
- **Plugin Authors** gain a straightforward, documented way to address telemetry, and can focus on business logic instead of needing to reimplement telemetry plumbing.
- The **community** receives a standard approach to telemetry.
- **Backstage Operators/SRE/Platform Engineers** receive a consistent, reliable stream of metrics from the core system, simplifying monitoring, alerting, and troubleshooting. They can enforce metrics standards, aggregate data, and build dashboards more easily. -**Organizations** with their own observability or compliance requirements can override the core service to integrate with proprietary or custom telemetry solutions, without forking or modifying plugins.
### Goals
Provide a unified service that provides a standardized approach for telemetry where setup docs like [these](https://backstage.io/docs/tutorials/setup-opentelemetry/) are not needed.
- Consistent telemetry patterns across all plugins
- Zero-configuration defaults that work out of the box
- Extensible architecture for custom telemetry backends
- OpenTelemetry integration following industry standards
- Performance monitoring capabilities built into the platform
We will include the following two plugins in the initial release to demonstrate the new telemetry service:
- Replace existing catalog implementation with `TelemetryService`
- Replace existing scaffolder implementation with `TelemetryService`
### Non-Goals
- Adding telemetry instrumentation to existing plugins that don't currently have it
- Only the catalog and scaffolder implementations will be refactored to use the new telemetry service.
- We will not start by providing a full Backstage SDK (aka recreate the OTEL Node SDK).
- We will not refactor the existing LoggerService. Future work to unify telemetry related concerns would be ideal, but not a goal.
## Proposal
Following similar patterns to other core services, we will create a new `RootTelemetryService` responsible for initializing telemetry and app-wide concerns, and the creation of plugin-specific telemetry services.
A core concern is not initializing the telemetry service early enough in the backend lifecycle. Services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping functionality that will be called as early as possible in the backend lifecycle.
## Configuration
All OTEL configuration will be supported via the app's `app-config.yaml` file. The config schema should be based on existing resources such as the [OTEL collector configuration](https://opentelemetry.io/docs/collector/configuration/).
```yaml
# this is not a complete example, but it shows the general idea
telemetry:
enabled: true
resource:
serviceName: backstage-telemetry
serviceVersion: 0.0.1
metrics:
enabled: true
collection:
exportIntervalMillis: 15000
exporters:
- type: prometheus
enabled: true
config:
port: 9464
tracing:
enabled: false
instrumentations:
http: false
knex: false
express: true
# ...
```
## Design Details
We will provide a thin wrapper around the OpenTelemetry Node SDK to provide a consistent interface while re-exporting the types from the SDK. This lets us provide a consistent telemetry interface while leveraging types and concepts already familiar to the community. Additionally, we will rely on the auto instrumentation features of the SDK as much as possible.
The decision to leverage the SDK comes from the fact that OpenTelemetry is already the fundamental abstraction for metrics and tracing.
```ts
interface TelemetryService {
createCounter(name: string, options?: MetricOptions): Counter;
//
// provide escape hatch for advanced use cases (???) - todo: I don't like this if we don't have a good reason for it
getMeter(): Meter;
}
interface RootTelemetryService extends TelemetryService {
forPlugin(pluginId: string): TelemetryService;
// provide escape hatches for advanced use cases (???)
getClient(): NodeSDK; // give access to the SDK
}
```
### Naming Conventions
Visit [0013-telemetry-metrics-naming-conventions](../0013-telemetry-metrics-naming-conventions/README.md) for details on the naming conventions.
The root telemetry service is responsible for initializing the global meter, namespaced to `backstage`. The root service will initialize a meter for each plugin that is registered based on the `pluginId`. The metric name will be the name of the metric as provided by the plugin author in the form `{category}.{name}`.
```ts
const rootTelemetry = DefaultRootTelemetryService.fromConfig(config);
const pluginTelemetryService = rootTelemetry.forPlugin('my-plugin');
const metric = pluginTelemetryService.createCounter('my_category.my_metric');
metric.add(1);
// ...
// metric is now available as `backstage.my-plugin.my_category.my_metric`
```
### References
- [General naming considerations](https://opentelemetry.io/docs/specs/semconv/general/naming/#general-naming-considerations)
### Bootstrapping
A core concern is not initializing the telemetry service early enough in the backend lifecycle and services that initialize before the telemetry service risk receiving no-op meters. To mitigate this, the root service will include a bootstrapping mechanism that will be called as early as possible in the backend lifecycle.
Ideally, this is called at the start of the `createBackend` function [[ref](https://github.com/backstage/backstage/blob/ab539cc72bb503618df55815f24c4ba933607460/packages/backend-defaults/src/CreateBackend.ts#L73-L75)], but we need to gain access to the config file at minimum. Ideally the logger as well, but not as crucial.
```ts
export function createBackend(): Backend {
// ...
// call the bootstrap function
Telemetry.bootstrap();
// ...
return createSpecializedBackend({ defaultServiceFactories });
}
```
### Plugins & Modules
WIP
<!--
Plugins leverage the plugin-scoped `TelemetryService` to attach and expose the public api for their instrumentation. This is how modules will be able to attach additional instrumentation to the plugin-scoped telemetry service.
Modules need to be able to attach additional instrumentation to the plugin-scoped telemetry service. How do we handle this? How do we prevent conflicts?
-->
## Release Plan
1. Create a new `RootTelemetryService` that initializes telemetry and app-wide concerns.
1. Create the plugin-scoped `TelemetryService` that provides a telemetry service for plugins.
1. Release the telemetry service as alpha.
1. Refactor catalog and scaffolder implementations to use the new (alpha) `TelemetryService`.
1. Replace the existing setup docs by integrating the new `RootTelemetryService` to `createBackend`
1. Release the telemetry service
1. Offer a migration path for existing adopters to migrate to the new telemetry service.
1. Update all documentation to reference the new telemetry service.
1. Create follow-up action items to integrate the new telemetry service into the core system.
1. Fully deprecate all existing telemetry implementations like the existing Prometheus one-off implementations.
## Dependencies
1. The root telemetry service MUST BE initialized as EARLY as possible to prevent dependents from receiving no op meters, while still allowing for app-wide configuration.
1. There are one-off implementations of telemetry in the wild that may conflict with the proposed service. However, this is unlikely to be a problem as the SDK should continue to pick things up.
## Alternatives
1. Plugin authors continue to implement their own telemetry as they see fit.
1. Adopters continue using the [setup-opentelemetry](https://backstage.io/docs/tutorials/setup-opentelemetry/) docs as a guide.
## Outstanding Questions
1. Are there enough Backstage-isms to build an additional layer of abstraction for metrics and tracing on top of the OpenTelemetry SDK? Do we re-export the types from the OpenTelemetry SDK or do we create our own? How do we handle breaking changes?
1. How do we guarantee that the telemetry service is initialized early enough in the backend lifecycle, while still allowing for app-wide configuration?
1. Should we provide an escape hatch for advanced use cases?
1. Should there be a single telemetry service or should we provide more granular services (e.g. adding separate `MetricsService` and `TracingService`)?
1. What core services could leverage the auto instrumentation features of the OpenTelemetry SDK? (e.g. `DatabaseService` could leverage the `knex` auto instrumentation, `HttpService` could leverage the `express` auto instrumentation, etc.)
beps/0013-telemetry-metrics-naming-conventions/README.md
-113
View File
@@ -1,113 +0,0 @@
---
title: OpenTelemetry Metrics Naming Conventions for Backstage
status: implementable
authors:
- '@kurtaking'
owners:
- '@kurtaking'
project-areas:
- framework
- telemetry
creation-date: 2025-06-26
---
# BEP: OpenTelemetry Metrics Naming Conventions for Backstage
- [Summary](#summary)
- [Motivation](#motivation)
- [Goals](#goals)
- [Non-Goals](#non-goals)
- [Proposal](#proposal)
- [Design Details](#design-details)
- [Release Plan](#release-plan)
- [Dependencies](#dependencies)
- [Alternatives](#alternatives)
## Summary
This proposal establishes standardized naming conventions for OpenTelemetry metrics across the Backstage ecosystem. It defines a hierarchical naming pattern that ensures consistency, discoverability, and compliance with OpenTelemetry semantic conventions while providing clear guidance for plugin and module authors.
## Motivation
Currently, Backstage has:
- Mixed metric naming patterns: Legacy Prometheus metrics (`catalog_entities_count`) alongside newer OpenTelemetry metrics (`catalog.processing.queue.delay`)
- No standard pattern for plugin-specific metrics
- Difficulty correlating metrics across plugins and providers
- Some existing metrics don't follow OpenTelemetry semantic conventions
We need to decided on a pattern that we can use for the telemetry service to obfuscate the underlying implementation details.
### Goals
- Establish consistent naming patterns across all Backstage metrics
- Ensure OpenTelemetry compliance with semantic conventions
- Provide clear guidance for plugin and service authors
### Non-Goals
- Breaking existing metrics immediately (backward compatibility maintained)
- Forcing all plugins to emit the same metrics
- Defining specific metric values or thresholds
## Proposal
Provide guidance for naming metrics for Backstage.
## Design Details
### Core Naming Pattern
All Backstage metrics MUST follow this hierarchical pattern:
`backstage.{scope}.{category}.{metric_name}`
**Where:**
- `backstage` is the root namespace for all Backstage metrics
- `{scope}` is the system scope (`pluginId`, `core` for core services)
- `{category}` is the logical grouping within the `{scope}`
- `{metric_name}` is the name of the metric
### Scope
The `scope` represents where it belongs in the Backstage ecosystem.
- `pluginId` - A plugin-specific metric (e.g. `backstage.plugin.catalog.entities.count`)
- `core` - A core service metric (e.g. `backstage.core.database.connections.active`)
#### Plugin Metrics
Pattern: `backstage.{pluginId}.{category}.{metric_name}`
```yaml
# Examples
backstage.catalog.entities.processed.total
backstage.scaffolder.tasks.completed.total
backstage.techdocs.builds.active
backstage.auth.sessions.active.total
```
#### Core Metrics
Pattern: `backstage.core.{category}.{metric_name}`
```yaml
# Examples
backstage.core.database.connections.active
backstage.core.scheduler.tasks.queued.total
backstage.core.http.requests.total
```
## Release Plan
- Provide documentation for the naming conventions
- Release as part of the larger telemetry service proposal
## Dependencies
- [0012-telemetry-service](../0012-telemetry-service/README.md)
## Alternatives
Adopters implement their own metrics and naming conventions.