Rework catalog configuration; add integrations documentation
Signed-off-by: Tim Hansen <timbonicus@gmail.com>
This commit is contained in:
Tim Hansen
@@ -0,0 +1,30 @@
|
||||
---
|
||||
id: locations
|
||||
title: Azure DevOps Locations
|
||||
sidebar_label: Locations
|
||||
description: Documentation on Azure DevOps location integration
|
||||
---
|
||||
|
||||
The Azure integration supports loading catalog entities from Azure DevOps.
|
||||
Components can be added to
|
||||
[static catalog configuration](../../features/software-catalog/configuration.md),
|
||||
or registered with the
|
||||
[catalog-import](https://github.com/backstage/backstage/tree/master/plugins/catalog-import)
|
||||
plugin.
|
||||
|
||||
```yaml
|
||||
integrations:
|
||||
azure:
|
||||
- host: dev.azure.com
|
||||
token:
|
||||
$env: AZURE_TOKEN
|
||||
```
|
||||
|
||||
> Note: An Azure DevOps provider is added automatically at startup for
|
||||
> convenience, so you only need to list it if you want to supply a
|
||||
> [token](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate).
|
||||
|
||||
The configuration is a structure with two elements:
|
||||
|
||||
- `host`: The DevOps host; only `dev.azure.com` is supported.
|
||||
- `token` (optional): An personal access token as expected by Azure DevOps.
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
id: locations
|
||||
title: BitBucket Locations
|
||||
sidebar_label: Locations
|
||||
description: Documentation on BitBucket location integration
|
||||
---
|
||||
|
||||
The BitBucket integration supports loading catalog entities from bitbucket.com
|
||||
or a self-hosted BitBucket. Components can be added to
|
||||
[static catalog configuration](../../features/software-catalog/configuration.md),
|
||||
or registered with the
|
||||
[catalog-import](https://github.com/backstage/backstage/tree/master/plugins/catalog-import)
|
||||
plugin.
|
||||
|
||||
```yaml
|
||||
integrations:
|
||||
bitbucket:
|
||||
- host: bitbucket.org
|
||||
username:
|
||||
$env: BITBUCKET_USERNAME
|
||||
token:
|
||||
$env: BITBUCKET_TOKEN
|
||||
```
|
||||
|
||||
> Note: A public BitBucket provider is added automatically at startup for
|
||||
> convenience, so you only need to list it if you want to supply a
|
||||
> [token](https://confluence.atlassian.com/bitbucketserver/personal-access-tokens-939515499.html).
|
||||
|
||||
Directly under the `bitbucket` key is a list of provider configurations, where
|
||||
you can list the BitBucket providers you want to fetch data from. Each entry is
|
||||
a structure with up to four elements:
|
||||
|
||||
- `host`: The host of the BitBucket instance, e.g. `bitbucket.company.com`.
|
||||
- `username`: The BitBucket username to use in API requests. If a username is
|
||||
not supplied, anonymous access will be used.
|
||||
- `token` (optional): An personal access token as expected by BitBucket. Either
|
||||
an access token **or** a password may be supplied.
|
||||
- `appPassword` (optional): The password for the BitBucket user. Either an
|
||||
appPassword **or** an access token may be supplied.
|
||||
- `apiBaseUrl` (optional): The URL of the GitLab API. For self-hosted
|
||||
installations, it is commonly at `https://<host>/api/v4`. For gitlab.com, this
|
||||
configuration is not needed as it can be inferred.
|
||||
@@ -0,0 +1,52 @@
|
||||
---
|
||||
id: discovery
|
||||
title: GitHub Discovery
|
||||
sidebar_label: Discovery
|
||||
description: Documentation on GitHub organization discovery
|
||||
---
|
||||
|
||||
The GitHub integration has a special discovery processor for discovering catalog
|
||||
entities within a GitHub organization. The processor will crawl the GitHub
|
||||
organization and register entities matching the configured path. This can be
|
||||
useful as an alternative to static locations or manually adding things to the
|
||||
catalog.
|
||||
|
||||
To use the discovery processor, you'll need a GitHub integration
|
||||
[set up](locations.md) with a `GITHUB_TOKEN`. Then you can add a location target
|
||||
to the catalog configuration:
|
||||
|
||||
```yaml
|
||||
catalog:
|
||||
locations:
|
||||
- type: github-discovery
|
||||
target: https://github.com/myorg/service-*/blob/main/catalog-info.yaml
|
||||
```
|
||||
|
||||
Note the `github-discovery` type, as this is not a regular `url` processor.
|
||||
|
||||
The target is composed of three parts:
|
||||
|
||||
- The base organization URL, `https://github.com/myorg` in this case
|
||||
- The repository blob to scan, which accepts \* wildcard tokens. This can simply
|
||||
be `*` to scan all repositories in the organization. This example only looks
|
||||
for repositories prefixed with `service-`.
|
||||
- The path within each repository to find the catalog YAML file. This will
|
||||
usually be `/blob/main/catalog-info.yaml`, `/blob/master/catalog-info.yaml` or
|
||||
a similar variation for catalog files stored in the root directory of each
|
||||
repository.
|
||||
|
||||
## GitHub API Rate Limits
|
||||
|
||||
GitHub
|
||||
[rate limits](https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting)
|
||||
API requests to 5,000 per hour (or more for Enterprise accounts). The default
|
||||
Backstage catalog backend refreshes data every 100 seconds, which issues an API
|
||||
request for each discovered location.
|
||||
|
||||
This means if you have more than ~140 catalog entities, you may get throttled by
|
||||
rate limiting. This will soon be resolved once catalog refreshes make use of
|
||||
ETags; to work around this in the meantime, you can change the refresh rate of
|
||||
the catalog in your `packages/backend/src/plugins/catalog.ts` file.
|
||||
|
||||
This is true for any method of adding GitHub entities to the catalog, but
|
||||
especially easy to hit with automatic discovery.
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
id: locations
|
||||
title: GitHub Locations
|
||||
sidebar_label: Locations
|
||||
description: Documentation on GitHub location integration
|
||||
---
|
||||
|
||||
The GitHub integration supports loading catalog entities from github.com or
|
||||
GitHub Enterprise. Components can be added to
|
||||
[static catalog configuration](../../features/software-catalog/configuration.md),
|
||||
registered with the
|
||||
[catalog-import](https://github.com/backstage/backstage/tree/master/plugins/catalog-import)
|
||||
plugin, or [discovered](discovery.md) from a GitHub organization. Users and
|
||||
Groups can also be [loaded from an organization](org.md).
|
||||
|
||||
## Configuration
|
||||
|
||||
To use this integration, add configuration to your root `app-config.yaml`:
|
||||
|
||||
```yaml
|
||||
integrations:
|
||||
github:
|
||||
- host: github.com
|
||||
token:
|
||||
$env: GITHUB_TOKEN
|
||||
- host: ghe.example.net
|
||||
apiBaseUrl: https://ghe.example.net/api/v3
|
||||
rawBaseUrl: https://ghe.example.net/raw
|
||||
token:
|
||||
$env: GHE_TOKEN
|
||||
```
|
||||
|
||||
> Note: A public GitHub provider is added automatically at startup for
|
||||
> convenience, so you only need to list it if you want to supply a
|
||||
> [token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token).
|
||||
|
||||
Directly under the `github` key is a list of provider configurations, where you
|
||||
can list the various GitHub-compatible providers you want to be able to fetch
|
||||
data from. Each entry is a structure with up to four elements:
|
||||
|
||||
- `host` (optional): The host of the location target that you want to match on.
|
||||
The default host is `github.com`.
|
||||
- `token` (optional): An authentication token as expected by GitHub. If
|
||||
supplied, it will be passed along with all calls to this provider, both API
|
||||
and raw. If it is not supplied, anonymous access will be used.
|
||||
- `apiBaseUrl` (optional): If you want to communicate using the APIv3 method
|
||||
with this provider, specify the base URL for its endpoint here, with no
|
||||
trailing slash. Specifically when the target is GitHub, you can leave it out
|
||||
to be inferred automatically. For a GitHub Enterprise installation, it is
|
||||
commonly at `https://api.<host>` or `https://<host>/api/v3`.
|
||||
- `rawBaseUrl` (optional): If you want to communicate using the raw HTTP method
|
||||
with this provider, specify the base URL for its endpoint here, with no
|
||||
trailing slash. Specifically when the target is public GitHub, you can leave
|
||||
it out to be inferred automatically. For a GitHub Enterprise installation, it
|
||||
is commonly at `https://api.<host>` or `https://<host>/api/v3`.
|
||||
|
||||
You need to supply either `apiBaseUrl` or `rawBaseUrl` or both (except for
|
||||
public GitHub, for which we can infer them). The `apiBaseUrl` will always be
|
||||
preferred over the other if a `token` is given, otherwise `rawBaseUrl` will be
|
||||
preferred.
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
id: locations
|
||||
title: GitLab Locations
|
||||
sidebar_label: Locations
|
||||
description: Documentation on GitLab location integration
|
||||
---
|
||||
|
||||
The GitLab integration supports loading catalog entities from gitlab.com or a
|
||||
self-hosted GitLab. Components can be added to
|
||||
[static catalog configuration](../../features/software-catalog/configuration.md),
|
||||
or registered with the
|
||||
[catalog-import](https://github.com/backstage/backstage/tree/master/plugins/catalog-import)
|
||||
plugin.
|
||||
|
||||
```yaml
|
||||
integrations:
|
||||
gitlab:
|
||||
- host: gitlab.com
|
||||
token:
|
||||
$env: GITLAB_TOKEN
|
||||
```
|
||||
|
||||
> Note: A public GitLab provider is added automatically at startup for
|
||||
> convenience, so you only need to list it if you want to supply a
|
||||
> [token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html).
|
||||
|
||||
Directly under the `gitlab` key is a list of provider configurations, where you
|
||||
can list the GitLab providers you want to fetch data from. Each entry is a
|
||||
structure with up to four elements:
|
||||
|
||||
- `host`: The host of the GitLab instance, e.g. `gitlab.company.com`.
|
||||
- `token` (optional): An authentication token as expected by GitLab. If
|
||||
supplied, it will be passed along with all calls to this provider, both API
|
||||
and raw. If it is not supplied, anonymous access will be used.
|
||||
- `apiBaseUrl` (optional): The URL of the GitLab API. For self-hosted
|
||||
installations, it is commonly at `https://<host>/api/v4`. For gitlab.com, this
|
||||
configuration is not needed as it can be inferred.
|
||||
- `baseUrl` (optional): The base URL for this provider, e.g.
|
||||
`https://gitlab.com`. If this is not provided, it is assumed to be
|
||||
`https://{host}`.
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
id: index
|
||||
title: Integrations
|
||||
sidebar_label: Overview
|
||||
description: Documentation on Backstage Integrations
|
||||
---
|
||||
|
||||
Integrations allow Backstage to read or publish data using external providers
|
||||
|
||||
- such as GitHub, GitLab, BitBucket, LDAP, or cloud providers.
|
||||
|
||||
## Configuration
|
||||
|
||||
Integration are configured at the root level of `app-config.yaml` since
|
||||
integrations are used by many Backstage core features and other plugins.
|
||||
|
||||
Each key under `integrations` is a separate configuration for a single external
|
||||
provider. Providers each have different configuration; here's an example of
|
||||
configuration to use both GitHub and BitBucket:
|
||||
|
||||
```yaml
|
||||
integrations:
|
||||
github:
|
||||
- host: github.com
|
||||
token:
|
||||
$env: GITHUB_TOKEN
|
||||
bitbucket:
|
||||
- host: bitbucket.org
|
||||
username:
|
||||
$env: BITBUCKET_USERNAME
|
||||
appPassword:
|
||||
$env: BITBUCKET_APP_PASSWORD
|
||||
```
|
||||
|
||||
See documentation for each type of integration for full details on
|
||||
configuration.
|
||||
Reference in New Issue
Block a user