diff --git a/docs/README.md b/docs/README.md index 9c25505d10..2bda6a5aaa 100644 --- a/docs/README.md +++ b/docs/README.md @@ -64,6 +64,11 @@ better yet, a pull request. - Publishing - [Open source and NPM](plugins/publishing.md) - [Private/internal (non-open source)](plugins/publish-private.md) + - Configuration + - [Overview](conf/index.md) + - [Reading Configuration](conf/reading.md) + - [Writing Configuration](conf/writing.md) + - [Defining Configuration](conf/defining.md) - Authentication and identity - [Overview](auth/index.md) - [Add auth provider](auth/add-auth-provider.md) diff --git a/docs/conf/defining.md b/docs/conf/defining.md new file mode 100644 index 0000000000..0b15108f43 --- /dev/null +++ b/docs/conf/defining.md @@ -0,0 +1,15 @@ +# Defining Configuration for your Plugin + +There is currently no tooling support or helpers for defining plugin +configuration. But it's on the roadmap. + +Meanwhile, document the config values that you are reading in your plugin +README. + +## Format + +When defining configuration for your plugin, keep keys camelCased and stick to +existing casing conventions such as `baseUrl`. + +It is also usually best to prefer objects over arrays, as it makes it possible +to override individual values using separate files or environment variables. diff --git a/docs/conf/index.md b/docs/conf/index.md new file mode 100644 index 0000000000..af1635d627 --- /dev/null +++ b/docs/conf/index.md @@ -0,0 +1,48 @@ +# Static Configuration in Backstage + +## Summary + +Backstage ships with a flexible configuration system that provides a simple way +to configure Backstage apps and plugins for both local development and +production deployments. It helps get you up and running fast while adapting +Backstage for your specific environment. It also serves as a tool for plugin +authors to use to make it simple to pick up and install a plugin, while still +allowing for customization. + +## Supplying Configuration + +Configuration is stored in `app-config.yaml` files, with support for suffixes +such as `app-config.production.yaml` to override values for specific +environments. The configuration files themselves contain plain YAML, but with +support for loading in secrets from various sources using a `$secret` key. + +It is also possible to supply configuration through environment variables, for +example `APP_CONFIG_app_baseUrl=https://staging.example.com`. However these +should be used sparingly, usually just for temporary overrides during +development or small tweaks to be able to reuse deployment artifacts in +different environments. + +The configuration is shared between the frontend and backend, meaning that +values that are common between the two only needs to be defined once. Such as +the `backend.baseUrl`. + +For more details, see [Writing Configuration](./writing.md). + +## Reading Configuration + +As a plugin developer, you likely end up wanting to define configuration that +you want users of your plugin to supply, as well as reading that configuration +in frontend and backend plugins. For more details, see +[Reading Configuration](./reading.md) and +[Defining Configuration](./defining.md). + +## Further Reading + +More details are provided in dedicated sections of the documentation. + +- [Reading Configuration](./reading.md): How to read configuration in your + plugin. +- [Writing Configuration](./writing.md): How to provide configuration for your + Backstage deployment. +- [Defining Configuration](./defining.md): How to define configuration for users + of your plugin. diff --git a/docs/conf/reading.md b/docs/conf/reading.md new file mode 100644 index 0000000000..9979f0360e --- /dev/null +++ b/docs/conf/reading.md @@ -0,0 +1,128 @@ +# Reading Backstage Configuration + +## Config API + +There's a common configuration API for by both frontend and backend plugins. An +API reference can be found [here](../reference/utility-apis/Config.md). + +The configuration API is tailored towards failing fast in case of missing or bad +config. That's because configuration errors can always be considered programming +mistakes, and will fail deterministically. + +### Type Safety + +The methods for reading primitive values are typed, and validate that type at +runtime. For example `getNumber()` requires the underlying value to be a number, +and there will be no attempt to coerce other types into the desired one. If +`getNumber()` receives a string value, it will throw an error, explaining where +the bad config came from, and what the desired and actual types where. + +### Reading Nested Configuration + +The backing configuration data is a nested JSON structure, meaning there will be +object, within objects, arrays within objects, and so on. There are a couple of +different ways to access nested values when reading configuration, but the +primary one is to use dot-separated paths. + +For example, given the following configuration: + +```yaml +app: + baseUrl: http://localhost:3000 +``` + +We can access the `baseUrl` using `config.getString('app.baseUrl')`. Because of +this syntax, configuration keys are not allowed to contain dots. In fact, +configuration keys are validated using the following RegEx: +`/^[a-z][a-z0-9]*(?:[-_][a-z][a-z0-9]*)*$/i`. + +Another option of accessing the `baseUrl` value is to create a sub-view of the +configuration, `config.getConfig('app').getString('baseUrl')`. When reading out +single values the dot-path pattern is preferred, but creating sub-views can be +useful for when you want to pass on parts of configuration to be read out by a +separate function. For example, given something like + +```yaml +my-plugin: + items: + a: + title: Item A + path: /a + b: + title: Item B + path: /b +``` + +You can get the list of all items using the `.keys()` method, and then pass on +each sub-view to be handled individually. + +```ts +for (const itemKey of config.keys('my-plugin.items')) { + const itemConfig = config.getConfig(`my-plugin.items`).getConfig(key); + const item = createItemFromConfig(itemConfig); +} +``` + +Another option for iterating through configuration keys is to call +`config.get('my-plugin.items')`, which simply returns the JSON structure for +that position without any validation. This can be handy to use sometimes, +especially if you're passing on config to an external library. There's a clear +benefit to the sub-view approach though, which is that the user will receive +much more detailed and relevant error messages. For example, if +`itemConfig.getString('title')` fails in the above example because a boolean was +supplied, the user will receive an error message with the full path, e.g. +`my-plugin.items.b.title`, as well as the name of the config file with the bad +value. + +Note that no matter what method is used for reading out nested config, the same +merging rules apply. You will always get the same value for any way of accessing +nested config: + +```ts +// Equivalent as long as a.b.c exists and is a string +config.getString('a.b.c'); +config.getConfig('a.b').getString('c'); +config.get('a').b.c; +``` + +### Required vs Optional Configuration + +Reading configuration can be divided into two categories: required, and +optional. When reading optional configuration you use the optional methods such +as `getOptionalString`. These methods will simply return `undefined` if +configuration values are missing, allowing the called to fall back to default +values. The optional methods still validate types however, so receiving a string +in a call to `config.getOptionalNumber` will still throw an error. + +A good pattern for reading optional configuration values is to use the `??` +operator. For example: + +```ts +const title = config.getString('my-plugin.title') ?? 'My Plugin'; +``` + +To read required configuration, simply use the methods without `Optional`, for +example `getString`. These will throw an error if there is no value available. + +## Accessing ConfigApi in Frontend Plugins + +The [ConfigApi](../reference/utility-apis/Config.md) in the frontend is a +[UtilityApi](../api/utility-apis.md). It's accessible as usual via the +`configApiRef` exported from `@backstage/core`. + +Depending on the config api in another API is slightly different though, as the +`ConfigApi` implementation is supplied via the App itself and not instantiated +like other APIs. See +[packages/app/src/apis.ts](https://github.com/spotify/backstage/blob/244eef851f5aa19f91c7c9b5c12d5df95cf482ca/packages/app/src/apis.ts#L66) +for an example of how this wiring is done. + +For standalone plugin setups in `dev/index.ts`, register a factory with a +statically mocked implementation of the config API. Use the `ConfigReader` from +`@backstage/config` to create an instance and register it for the `configApiRef` +from `@backstage/core`. + +## Accessing ConfigApi in Backend Plugins + +In backend plugins the configuration is passed in via options from the main +backend package. See for example +[packages/backend/src/plugins/auth.ts](https://github.com/spotify/backstage/blob/244eef851f5aa19f91c7c9b5c12d5df95cf482ca/packages/backend/src/plugins/auth.ts#L23). diff --git a/docs/conf/writing.md b/docs/conf/writing.md new file mode 100644 index 0000000000..8ab397846d --- /dev/null +++ b/docs/conf/writing.md @@ -0,0 +1,157 @@ +# Writing Backstage Configuration Files + +## File Format + +Configuration is stored in YAML format in `app-config.yaml` files, looking +something like this: + +```yaml +app: + title: Backstage Example App + baseUrl: http://localhost:3000 + +backend: + listen: 0.0.0.0:7000 + baseUrl: http://localhost:7000 + +organization: + name: Spotify + +proxy: + /my/api: + target: https://example.com/api/ + changeOrigin: true + pathRewrite: + ^/proxy/my/api/: / +``` + +Configuration files are typically checked in and stored in the repo that houses +the rest of the Backstage application. + +## Environment Variable Overrides + +Individual configuration values can be overridden using environment variables +prefixed with `APP_CONFIG_`. Everything following that prefix in the environment +variable name will be used as the config key, with `_` replaced by `.`. For +example, to override the `app.baseUrl` value, set the `APP_CONFIG_app_baseUrl` +environment variable to the desired value. + +The value of the environment variable is parsed as JSON, but it will fall back +to being interpreted as a string if it fails to parse. Note that if you for +example want to pass on the string `"false"`, you need to wrap it in double +quotes, e.g. `export APP_CONFIG_example='"false"'`. + +While it may be tempting to use environment variable overrides for supplying a +lot of configuration values, we recommend using them sparingly. Try to stick to +using configuration files, and only use environment variables for things like +reusing deployment artifacts across staging and production environments. + +Note that environment variables work for frontend configuration too. They are +picked up by the serve tasks of `@backstage/cli` for local development, and are +injected by the entrypoint of the nginx container serving the frontend in a +production build. + +## File Resolution + +It is possible to have multiple configuration files, both to support different +environments, but also to define configuration that is local to specific +packages. + +All `app-config.yaml` files inside the monorepo root and package root are +considered, as are files with additional `local` and environment affixes such as +`development`, for example `app-config.local.yaml`, +`app-config.production.yaml`, and `app-config.development.local.yaml`. Which +environment config files are loaded is determined by the `NODE_ENV` environment +variable. Local configuration files are always loaded, but are meant for local +development overrides and should typically be `.gitignore`'d. + +All loaded configuration files are merged together using the following rules: + +- Configurations have different priority, higher priority means you replace + values from configurations with lower priority. +- Primitive values are completely replaced, as are arrays and all of their + contents. +- Objects are merged together deeply, meaning that if any of the included + configs contain a value for a given path, it will be found. + +The priority of the configurations is determined by the following rules, in +order: + +- Configuration from the `APP_CONFIG_` environment variables has the highest + priority, followed by files. +- Files inside package directories have higher priority than those in the root + directory. +- Files with environment affixes have higher priority than ones without. +- Files with the `local` affix have higher priority than ones without. + +## Secrets + +Secrets are supported via a special `$secret` key, which in turn provides a +number of different ways to read in secrets. To load a configuration value as a +secret, supply an object with a single `$secret` key, and within that supply an +object that describes how the secret is loaded. For example, the following will +read the config key `backend.mySecretKey` from the environment variable +`MY_SECRET_KEY`: + +```yaml +backend: + mySecretKey: + $secret: + env: MY_SECRET_KEY +``` + +With the above configuration, calling `config.getString('backend.mySecretKey')` +will return the value of the environment variable `MY_SECRET_KEY` when the +backend started up. All secrets are loaded at startup, so changing the contents +of secret files or environment variables will not be reflected at runtime. + +Note that secrets will never be included in the frontend bundle or development +builds. When loading configuration you have to explicitly enable reading of +secrets, which is only done for the backend configuration. + +As hinted at, secrets can be loaded from a bunch of different sources, and can +be extended with more. Below is a list of the currently supported methods for +loading secrets. + +### Env Secrets + +This reads a secret from an environment variable. For example, the following +config loads the secret from the `MY_SECRET` env var. + +```yaml +$secret: + env: MY_SECRET +``` + +### File Secrets + +This reads a secret from the entire contents of a file. The file path is +relative to the `app-config.yaml` the defines the secrets. For example, the +following reads the contents of `my-secret.txt` relative to the config file +itself: + +```yaml +$secret: + file: ./my-secret.txt +``` + +### Data File Secrets + +This reads secrets from a path within a JSON-like data file. The file path +behaves similar to file secrets, but in addition a `path` is used to point to a +specific value inside the file. Supported file extensions are `.json`, `.yaml`, +and `.yml`. For example, the following would read out `my-secret-key` from +`my-secrets.json`: + +```yaml +$secret: + data: ./my-secrets.json + path: deployment.key + +# my-secrets.json +{ + "deployment": { + "key": "my-secret-key" + } +} +``` diff --git a/mkdocs.yml b/mkdocs.yml index 43e05580d1..2380001203 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -62,6 +62,11 @@ nav: - Publishing: - Open source and NPM: 'plugins/publishing.md' - Private/internal (non-open source): 'plugins/publish-private.md' + - Configuration: + - Overview: 'conf/index.md' + - Reading Configuration: 'conf/reading.md' + - Writing Configuration: 'conf/writing.md' + - Defining Configuration: 'conf/defining.md' - Authentication and identity: - Overview: 'auth/index.md' - Add auth provider: 'auth/add-auth-provider.md'