Merge branch 'master' into feature/bitbucket-cloud-discovery
This commit is contained in:
barticus
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 17 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 22 KiB |
@@ -244,9 +244,7 @@ export interface OAuthProviderHandlers {
|
||||
req: express.Request,
|
||||
options: Record<string, string>,
|
||||
): Promise<RedirectInfo>;
|
||||
handler(
|
||||
req: express.Request,
|
||||
): Promise<{
|
||||
handler(req: express.Request): Promise<{
|
||||
response: AuthResponse<OAuthProviderInfo>;
|
||||
refreshToken?: string;
|
||||
}>;
|
||||
|
||||
@@ -10,11 +10,16 @@ that can authenticate users using GitHub or GitHub Enterprise OAuth.
|
||||
|
||||
## Create an OAuth App on GitHub
|
||||
|
||||
To add GitHub authentication, you must create an OAuth App from the GitHub
|
||||
To add GitHub authentication, you must create either a GitHub App, or an OAuth
|
||||
App from the GitHub
|
||||
[developer settings](https://github.com/settings/developers). The `Homepage URL`
|
||||
should point to Backstage's frontend, while the `Authorization callback URL`
|
||||
will point to the auth backend.
|
||||
|
||||
Note that if you're using a GitHub App, the allowed scopes are configured as
|
||||
part of that app. This means you need to verify what scopes the plugins you use
|
||||
require, so be sure to check the plugin READMEs for that information.
|
||||
|
||||
Settings for local development:
|
||||
|
||||
- Application name: Backstage (or your custom app name)
|
||||
@@ -51,3 +56,11 @@ The GitHub provider is a structure with three configuration keys:
|
||||
To add the provider to the frontend, add the `githubAuthApi` reference and
|
||||
`SignInPage` component as shown in
|
||||
[Adding the provider to the sign-in page](../index.md#adding-the-provider-to-the-sign-in-page).
|
||||
|
||||
## Difference between GitHub Apps and GitHub OAuth Apps
|
||||
|
||||
GitHub Apps handle OAuth scope at the app installation level, meaning that the
|
||||
`scope` parameter for the call to `getAccessToken` in the frontend has no
|
||||
effect. When calling `getAccessToken` in open source plugins, one should still
|
||||
include the appropriate scope, but also document in the plugin README what
|
||||
scopes are required for GitHub Apps.
|
||||
|
||||
@@ -111,6 +111,48 @@ export default async function createPlugin({
|
||||
...
|
||||
```
|
||||
|
||||
## Resolving membership through the catalog
|
||||
|
||||
If you want to provide additional claims through Sign-In resolvers but still
|
||||
have the software catalog handle group (and transitive group) membership, you
|
||||
can do this using the `CatalogIdentityClient` provided as context to Sign-In
|
||||
resolvers:
|
||||
|
||||
```ts
|
||||
export default async function createPlugin({
|
||||
...
|
||||
}: PluginEnvironment): Promise<Router> {
|
||||
return await createRouter({
|
||||
...
|
||||
providerFactories: {
|
||||
google: createGoogleProvider({
|
||||
signIn: {
|
||||
resolver: async ({ profile: { email } }, ctx) => {
|
||||
const [id] = email?.split('@') ?? '';
|
||||
// Fetch from an external system that returns entity claims like:
|
||||
// ['user:default/breanna.davison', ...]
|
||||
const ent = await externalSystemClient.getUsernames(email);
|
||||
|
||||
// Resolve group membership from the Backstage catalog
|
||||
const fullEnt = await ctx.catalogIdentityClient.resolveCatalogMembership({
|
||||
entityRefs: [id].concat(ent),
|
||||
logger: ctx.logger,
|
||||
});
|
||||
const token = await ctx.tokenIssuer.issueToken({
|
||||
claims: { sub: id, ent: fullEnt },
|
||||
});
|
||||
return { id, token };
|
||||
},
|
||||
},
|
||||
}),
|
||||
...
|
||||
```
|
||||
|
||||
The `resolveCatalogMembership` method will retrieve the referenced entities from
|
||||
the catalog, if possible, and check for
|
||||
[memberOf](../features/software-catalog/well-known-relations.md#memberof-and-hasmember)
|
||||
relations to add additional entity claims.
|
||||
|
||||
## AuthHandler
|
||||
|
||||
Similar to a custom sign-in resolver, you can also write a custom auth handler
|
||||
|
||||
@@ -44,7 +44,7 @@ The Microsoft provider is a structure with three configuration keys:
|
||||
|
||||
- `clientId`: Application (client) ID, found on App Registration > Overview
|
||||
- `clientSecret`: Secret, found on App Registration > Certificates & secrets
|
||||
- `tenentId`: Directory (tenant) ID, found on App Registration > Overview
|
||||
- `tenantId`: Directory (tenant) ID, found on App Registration > Overview
|
||||
|
||||
## Adding the provider to the Backstage frontend
|
||||
|
||||
|
||||
@@ -4,8 +4,6 @@ title: Search Architecture
|
||||
description: Documentation on Search Architecture
|
||||
---
|
||||
|
||||
# Search Architecture
|
||||
|
||||
> _This architecture has not been fully implemented yet. Find our milestones to
|
||||
> follow our progress and help contribute on the
|
||||
> [Search Roadmap](./README.md#project-roadmap)._
|
||||
|
||||
@@ -4,8 +4,6 @@ title: Search Concepts
|
||||
description: Documentation on Backstage Search Concepts
|
||||
---
|
||||
|
||||
# Search Concepts
|
||||
|
||||
Backstage Search lets you find the right information you are looking for in the
|
||||
Backstage ecosystem.
|
||||
|
||||
|
||||
@@ -4,8 +4,6 @@ title: Getting Started with Search
|
||||
description: How to set up and install Backstage Search
|
||||
---
|
||||
|
||||
# Getting Started
|
||||
|
||||
Search functions as a plugin to Backstage, so you will need to use Backstage to
|
||||
use Search.
|
||||
|
||||
@@ -252,13 +250,9 @@ an example:
|
||||
Backstage Search isn't a search engine itself, rather, it provides an interface
|
||||
between your Backstage instance and a
|
||||
[Search Engine](./concepts.md#search-engines) of your choice. Currently, we only
|
||||
support one, an in-memory search Engine called Lunr. It can be instantiated like
|
||||
this:
|
||||
|
||||
```typescript
|
||||
const searchEngine = new LunrSearchEngine({ logger });
|
||||
const indexBuilder = new IndexBuilder({ logger, searchEngine });
|
||||
```
|
||||
support two engines, an in-memory search Engine called Lunr and ElasticSearch.
|
||||
See [Search Engines](./search-engines.md) documentation for more information how
|
||||
to configure these in your Backstage instance.
|
||||
|
||||
Backstage Search can be used to power search of anything! Plugins like the
|
||||
Catalog offer default [collators](./concepts.md#collators) (e.g.
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
---
|
||||
id: search-engines
|
||||
title: Search Engines
|
||||
description: Choosing and configuring your search engine for Backstage
|
||||
---
|
||||
|
||||
Backstage supports 2 search engines by default, an in-memory engine called Lunr
|
||||
and ElasticSearch. You can configure your own search engines by implementing the
|
||||
provided interface as mentioned in the
|
||||
[search backend documentation.](./getting-started.md#Backend)
|
||||
|
||||
Provided search engine implementations have their own way of constructing
|
||||
queries, which may be something you want to modify. Alterations to the querying
|
||||
logic of a search engine can be made by providing your own implementation of a
|
||||
QueryTranslator interface. This modification can be done without touching
|
||||
provided search engines by using the exposed setter to set the modified query
|
||||
translator into the instance.
|
||||
|
||||
```typescript
|
||||
const searchEngine = new LunrSearchEngine({ logger });
|
||||
searchEngine.setTranslator(new MyNewAndBetterQueryTranslator());
|
||||
```
|
||||
|
||||
## Lunr
|
||||
|
||||
Lunr search engine is enabled by default for your backstage instance if you have
|
||||
not done additional changes to the scaffolded app.
|
||||
|
||||
Lunr can be instantiated like this:
|
||||
|
||||
```typescript
|
||||
// app/backend/src/plugins/search.ts
|
||||
const searchEngine = new LunrSearchEngine({ logger });
|
||||
const indexBuilder = new IndexBuilder({ logger, searchEngine });
|
||||
```
|
||||
|
||||
## Postgres
|
||||
|
||||
The Postgres based search engine only requires that postgres being configured as
|
||||
the database engine for Backstage. Therefore it targets setups that want to
|
||||
avoid maintaining another external service like elastic search. The search
|
||||
provides decent results and performs well with ten thousands of indexed
|
||||
documents. The connection to postgres is established via the database manager
|
||||
also used by other plugins.
|
||||
|
||||
> **Important**: The search plugin requires at least Postgres 12!
|
||||
|
||||
To use the `PgSearchEngine`, make sure that you have a Postgres database
|
||||
configured and make the following changes to your backend:
|
||||
|
||||
1. Add a dependency on `@backstage/plugin-search-backend-module-pg` to your
|
||||
backend's `package.json`.
|
||||
2. Initialize the search engine. It is recommended to initialize it with a
|
||||
fallback to the lunr search engine if you are running Backstage for
|
||||
development locally with SQLite:
|
||||
|
||||
```typescript
|
||||
// In packages/backend/src/plugins/search.ts
|
||||
|
||||
// Initialize a connection to a search engine.
|
||||
const searchEngine = (await PgSearchEngine.supported(database))
|
||||
? await PgSearchEngine.from({ database })
|
||||
: new LunrSearchEngine({ logger });
|
||||
```
|
||||
|
||||
## ElasticSearch
|
||||
|
||||
Backstage supports ElasticSearch search engine connections, indexing and
|
||||
querying out of the box. Available configuration options enable usage of either
|
||||
AWS or Elastic.co hosted solutions, or a custom self-hosted solution.
|
||||
|
||||
Similarly to Lunr above, ElasticSearch can be set up like this:
|
||||
|
||||
```typescript
|
||||
// app/backend/src/plugins/search.ts
|
||||
const searchEngine = await ElasticSearchSearchEngine.initialize({
|
||||
logger,
|
||||
config,
|
||||
});
|
||||
const indexBuilder = new IndexBuilder({ logger, searchEngine });
|
||||
```
|
||||
|
||||
For the engine to be available, your backend package needs a dependency into
|
||||
package `@backstage/plugin-search-backend-module-elasticsearch`.
|
||||
|
||||
ElasticSearch needs some additional configuration before it is ready to use
|
||||
within your instance. The configuration options are documented in the
|
||||
[configuration schema definition file.](https://github.com/backstage/backstage/blob/master/plugins/search-backend-module-elasticsearch/config.d.ts)
|
||||
|
||||
The underlying functionality is using official ElasticSearch client version 7.x,
|
||||
meaning that ElasticSearch version 7 is the only one confirmed to be supported.
|
||||
|
||||
## Example configurations
|
||||
|
||||
### AWS
|
||||
|
||||
Using AWS hosted ElasticSearch the only configuration option needed is the URL
|
||||
to the ElasticSearch service. The implementation assumes that environment
|
||||
variables for AWS access key id and secret access key are defined in accordance
|
||||
to the
|
||||
[default AWS credential chain.](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html).
|
||||
|
||||
```yaml
|
||||
search:
|
||||
elasticsearch:
|
||||
provider: aws
|
||||
node: https://my-backstage-search-asdfqwerty.eu-west-1.es.amazonaws.com
|
||||
```
|
||||
|
||||
### Elastic.co
|
||||
|
||||
Elastic Cloud hosted ElasticSearch uses a Cloud ID to determine the instance of
|
||||
hosted ElasticSearch to connect to. Additionally, username and password needs to
|
||||
be provided either directly or using environment variables like defined in
|
||||
[Backstage documentation.](https://backstage.io/docs/conf/writing#includes-and-dynamic-data)
|
||||
|
||||
```yaml
|
||||
search:
|
||||
elasticsearch:
|
||||
provider: elastic
|
||||
cloudId: backstage-elastic:asdfqwertyasdfqwertyasdfqwertyasdfqwerty==
|
||||
auth:
|
||||
username: elastic
|
||||
password: changeme
|
||||
```
|
||||
|
||||
### Others
|
||||
|
||||
Other ElasticSearch instances can be connected to by using standard
|
||||
ElasticSearch authentication methods and exposed URL, provided that the cluster
|
||||
supports that. The configuration options needed are the URL to the node and
|
||||
authentication information. Authentication can be handled by either providing
|
||||
username/password or an API key. For more information how to create an API key,
|
||||
see
|
||||
[Elastic documentation on API keys](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html).
|
||||
|
||||
#### Configuration examples
|
||||
|
||||
##### With username and password
|
||||
|
||||
```yaml
|
||||
search:
|
||||
elasticsearch:
|
||||
node: http://localhost:9200
|
||||
auth:
|
||||
username: elastic
|
||||
password: changeme
|
||||
```
|
||||
|
||||
##### With API key
|
||||
|
||||
```yaml
|
||||
search:
|
||||
elasticsearch:
|
||||
node: http://localhost:9200
|
||||
auth:
|
||||
apiKey: base64EncodedKey
|
||||
```
|
||||
@@ -34,15 +34,14 @@ export const CustomCatalogPage = ({
|
||||
}: CatalogPageProps) => {
|
||||
return (
|
||||
<PageWithHeader title={`${orgName} Catalog`} themeId="home">
|
||||
<Content>
|
||||
<ContentHeader title="Components">
|
||||
<CreateComponentButton />
|
||||
<SupportButton>All your software catalog entities</SupportButton>
|
||||
</ContentHeader>
|
||||
<EntityListProvider>
|
||||
<EntityListProvider>
|
||||
<Content>
|
||||
<ContentHeader titleComponent={<CatalogKindHeader />}>
|
||||
<CreateButton title="Create Component" to={link} />
|
||||
<SupportButton>All your software catalog entities</SupportButton>
|
||||
</ContentHeader>
|
||||
<FilteredEntityLayout>
|
||||
<FilterContainer>
|
||||
<EntityKindPicker initialFilter="component" hidden />
|
||||
<EntityTypePicker />
|
||||
<UserListPicker initialFilter={initiallySelectedFilter} />
|
||||
<EntityTagPicker />
|
||||
@@ -51,8 +50,8 @@ export const CustomCatalogPage = ({
|
||||
<CatalogTable columns={columns} actions={actions} />
|
||||
</EntityListContainer>
|
||||
</FilteredEntityLayout>
|
||||
</EntityListProvider>
|
||||
</Content>
|
||||
</Content>
|
||||
</EntityListProvider>
|
||||
</PageWithHeader>
|
||||
);
|
||||
};
|
||||
@@ -187,7 +186,7 @@ new `CustomCatalogIndexPage`.
|
||||
# packages/app/src/App.tsx
|
||||
const routes = (
|
||||
<FlatRoutes>
|
||||
<Navigate key="/" to="/catalog" />
|
||||
<Navigate key="/" to="catalog" />
|
||||
- <Route path="/catalog" element={<CatalogIndexPage />} />
|
||||
+ <Route path="/catalog" element={<CustomCatalogIndexPage />} />
|
||||
```
|
||||
|
||||
@@ -245,6 +245,22 @@ the entity belongs to the `"default"` namespace.
|
||||
Namespaces may also be part of the catalog, and are `v1` / `Namespace` entities,
|
||||
i.e. not Backstage specific but the same as in Kubernetes.
|
||||
|
||||
### `title` [optional]
|
||||
|
||||
A display name of the entity, to be presented in user interfaces instead of the
|
||||
`name` property above, when available.
|
||||
|
||||
This field is sometimes useful when the `name` is cumbersome or ends up being
|
||||
perceived as overly technical. The title generally does not have as stringent
|
||||
format requirements on it, so it may contain special characters and be more
|
||||
explanatory. Do keep it very short though, and avoid situations where a title
|
||||
can be confused with the name of another entity, or where two entities share a
|
||||
title.
|
||||
|
||||
Note that this is only for display purposes, and may be ignored by some parts of
|
||||
the code. [Entity references](references.md) still always make use of the `name`
|
||||
property for example, not the title.
|
||||
|
||||
### `description` [optional]
|
||||
|
||||
A human readable description of the entity, to be shown in Backstage. Should be
|
||||
@@ -531,8 +547,8 @@ The current set of well-known and common values for this field is:
|
||||
|
||||
### `spec.owner` [required]
|
||||
|
||||
An [entity reference](#string-references) to the owner of the component, e.g.
|
||||
`artist-relations-team`. This field is required.
|
||||
An [entity reference](references.md#string-references) to the owner of the
|
||||
component, e.g. `artist-relations-team`. This field is required.
|
||||
|
||||
In Backstage, the owner of a component is the singular entity (commonly a team)
|
||||
that bears ultimate responsibility for the component, and has the authority and
|
||||
@@ -550,8 +566,8 @@ component, but there will always be one ultimate owner.
|
||||
|
||||
### `spec.system` [optional]
|
||||
|
||||
An [entity reference](#string-references) to the system that the component
|
||||
belongs to, e.g. `artist-engagement-portal`. This field is optional.
|
||||
An [entity reference](references.md#string-references) to the system that the
|
||||
component belongs to, e.g. `artist-engagement-portal`. This field is optional.
|
||||
|
||||
| [`kind`](#apiversion-and-kind-required) | Default [`namespace`](#namespace-optional) | Generated [relation](well-known-relations.md) type |
|
||||
| --------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------- |
|
||||
@@ -559,8 +575,8 @@ belongs to, e.g. `artist-engagement-portal`. This field is optional.
|
||||
|
||||
### `spec.subcomponentOf` [optional]
|
||||
|
||||
An [entity reference](#string-references) to another component of which the
|
||||
component is a part, e.g. `spotify-ios-app`. This field is optional.
|
||||
An [entity reference](references.md#string-references) to another component of
|
||||
which the component is a part, e.g. `spotify-ios-app`. This field is optional.
|
||||
|
||||
| [`kind`](#apiversion-and-kind-required) | Default [`namespace`](#namespace-optional) | Generated [relation](well-known-relations.md) type |
|
||||
| ---------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------- |
|
||||
@@ -568,8 +584,8 @@ component is a part, e.g. `spotify-ios-app`. This field is optional.
|
||||
|
||||
### `spec.providesApis` [optional]
|
||||
|
||||
An array of [entity references](#string-references) to the APIs that are
|
||||
provided by the component, e.g. `artist-api`. This field is optional.
|
||||
An array of [entity references](references.md#string-references) to the APIs
|
||||
that are provided by the component, e.g. `artist-api`. This field is optional.
|
||||
|
||||
| [`kind`](#apiversion-and-kind-required) | Default [`namespace`](#namespace-optional) | Generated [relation](well-known-relations.md) type |
|
||||
| --------------------------------------- | ------------------------------------------ | --------------------------------------------------------------------------------------------------- |
|
||||
@@ -577,8 +593,8 @@ provided by the component, e.g. `artist-api`. This field is optional.
|
||||
|
||||
### `spec.consumesApis` [optional]
|
||||
|
||||
An array of [entity references](#string-references) to the APIs that are
|
||||
consumed by the component, e.g. `artist-api`. This field is optional.
|
||||
An array of [entity references](references.md#string-references) to the APIs
|
||||
that are consumed by the component, e.g. `artist-api`. This field is optional.
|
||||
|
||||
| [`kind`](#apiversion-and-kind-required) | Default [`namespace`](#namespace-optional) | Generated [relation](well-known-relations.md) type |
|
||||
| --------------------------------------- | ------------------------------------------ | --------------------------------------------------------------------------------------------------- |
|
||||
@@ -586,9 +602,9 @@ consumed by the component, e.g. `artist-api`. This field is optional.
|
||||
|
||||
### `spec.dependsOn` [optional]
|
||||
|
||||
An array of [entity references](#string-references) to the components and
|
||||
resources that the component depends on, e.g. `artists-db`. This field is
|
||||
optional.
|
||||
An array of [entity references](references.md#string-references) to the
|
||||
components and resources that the component depends on, e.g. `artists-db`. This
|
||||
field is optional.
|
||||
|
||||
| [`kind`](#apiversion-and-kind-required) | Default [`namespace`](#namespace-optional) | Generated [relation](well-known-relations.md) type |
|
||||
| --------------------------------------- | ------------------------------------------ | --------------------------------------------------------------------------------------------- |
|
||||
@@ -690,12 +706,6 @@ shape, this kind has the following structure.
|
||||
|
||||
Exactly equal to `backstage.io/v1beta2` and `Template`, respectively.
|
||||
|
||||
### `metadata.title` [required]
|
||||
|
||||
The nice display name for the template as a string, e.g. `React SSR Template`.
|
||||
This field is required as is used to reference the template to the user instead
|
||||
of the `metadata.name` field.
|
||||
|
||||
### `metadata.tags` [optional]
|
||||
|
||||
A list of strings that can be associated with the template, e.g.
|
||||
@@ -722,8 +732,8 @@ You can find out more about the `steps` key
|
||||
|
||||
### `spec.owner` [optional]
|
||||
|
||||
An [entity reference](#string-references) to the owner of the template, e.g.
|
||||
`artist-relations-team`. This field is required.
|
||||
An [entity reference](references.md#string-references) to the owner of the
|
||||
template, e.g. `artist-relations-team`. This field is required.
|
||||
|
||||
In Backstage, the owner of a Template is the singular entity (commonly a team)
|
||||
that bears ultimate responsibility for the Template, and has the authority and
|
||||
@@ -832,8 +842,8 @@ The current set of well-known and common values for this field is:
|
||||
|
||||
### `spec.owner` [required]
|
||||
|
||||
An [entity reference](#string-references) to the owner of the component, e.g.
|
||||
`artist-relations-team`. This field is required.
|
||||
An [entity reference](references.md#string-references) to the owner of the
|
||||
component, e.g. `artist-relations-team`. This field is required.
|
||||
|
||||
In Backstage, the owner of an API is the singular entity (commonly a team) that
|
||||
bears ultimate responsibility for the API, and has the authority and capability
|
||||
@@ -851,8 +861,8 @@ one ultimate owner.
|
||||
|
||||
### `spec.system` [optional]
|
||||
|
||||
An [entity reference](#string-references) to the system that the API belongs to,
|
||||
e.g. `artist-engagement-portal`. This field is optional.
|
||||
An [entity reference](references.md#string-references) to the system that the
|
||||
API belongs to, e.g. `artist-engagement-portal`. This field is optional.
|
||||
|
||||
| [`kind`](#apiversion-and-kind-required) | Default [`namespace`](#namespace-optional) | Generated [relation](well-known-relations.md) type |
|
||||
| --------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------- |
|
||||
@@ -1059,8 +1069,8 @@ Exactly equal to `backstage.io/v1alpha1` and `Resource`, respectively.
|
||||
|
||||
### `spec.owner` [required]
|
||||
|
||||
An [entity reference](#string-references) to the owner of the resource, e.g.
|
||||
`artist-relations-team`. This field is required.
|
||||
An [entity reference](references.md#string-references) to the owner of the
|
||||
resource, e.g. `artist-relations-team`. This field is required.
|
||||
|
||||
In Backstage, the owner of a resource is the singular entity (commonly a team)
|
||||
that bears ultimate responsibility for the resource, and has the authority and
|
||||
@@ -1091,8 +1101,8 @@ Some common values for this field could be:
|
||||
|
||||
### `spec.system` [optional]
|
||||
|
||||
An [entity reference](#string-references) to the system that the resource
|
||||
belongs to, e.g. `artist-engagement-portal`. This field is optional.
|
||||
An [entity reference](references.md#string-references) to the system that the
|
||||
resource belongs to, e.g. `artist-engagement-portal`. This field is optional.
|
||||
|
||||
| [`kind`](#apiversion-and-kind-required) | Default [`namespace`](#namespace-optional) | Generated [relation](well-known-relations.md) type |
|
||||
| --------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------- |
|
||||
@@ -1100,9 +1110,9 @@ belongs to, e.g. `artist-engagement-portal`. This field is optional.
|
||||
|
||||
### `spec.dependsOn` [optional]
|
||||
|
||||
An array of [entity references](#string-references) to the components and
|
||||
resources that the resource depends on, e.g. `artist-lookup`. This field is
|
||||
optional.
|
||||
An array of [entity references](references.md#string-references) to the
|
||||
components and resources that the resource depends on, e.g. `artist-lookup`.
|
||||
This field is optional.
|
||||
|
||||
| [`kind`](#apiversion-and-kind-required) | Default [`namespace`](#namespace-optional) | Generated [relation](well-known-relations.md) type |
|
||||
| --------------------------------------- | ------------------------------------------ | --------------------------------------------------------------------------------------------- |
|
||||
@@ -1146,8 +1156,8 @@ Exactly equal to `backstage.io/v1alpha1` and `System`, respectively.
|
||||
|
||||
### `spec.owner` [required]
|
||||
|
||||
An [entity reference](#string-references) to the owner of the system, e.g.
|
||||
`artist-relations-team`. This field is required.
|
||||
An [entity reference](references.md#string-references) to the owner of the
|
||||
system, e.g. `artist-relations-team`. This field is required.
|
||||
|
||||
In Backstage, the owner of a system is the singular entity (commonly a team)
|
||||
that bears ultimate responsibility for the system, and has the authority and
|
||||
@@ -1165,8 +1175,8 @@ but there will always be one ultimate owner.
|
||||
|
||||
### `spec.domain` [optional]
|
||||
|
||||
An [entity reference](#string-references) to the domain that the system belongs
|
||||
to, e.g. `artists`. This field is optional.
|
||||
An [entity reference](references.md#string-references) to the domain that the
|
||||
system belongs to, e.g. `artists`. This field is optional.
|
||||
|
||||
| [`kind`](#apiversion-and-kind-required) | Default [`namespace`](#namespace-optional) | Generated [relation](well-known-relations.md) type |
|
||||
| --------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------- |
|
||||
@@ -1205,8 +1215,8 @@ Exactly equal to `backstage.io/v1alpha1` and `Domain`, respectively.
|
||||
|
||||
### `spec.owner` [required]
|
||||
|
||||
An [entity reference](#string-references) to the owner of the domain, e.g.
|
||||
`artist-relations-team`. This field is required.
|
||||
An [entity reference](references.md#string-references) to the owner of the
|
||||
domain, e.g. `artist-relations-team`. This field is required.
|
||||
|
||||
In Backstage, the owner of a domain is the singular entity (commonly a team)
|
||||
that bears ultimate responsibility for the domain, and has the authority and
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
---
|
||||
id: life-of-an-entity
|
||||
title: The Life of an Entity
|
||||
sidebar_label: The Life of an Entity
|
||||
# prettier-ignore
|
||||
description: The life cycle of entities, from being introduced into the catalog, through processing, to being removed again
|
||||
---
|
||||
|
||||
This document gives a high level overview of the catalog backend, and the
|
||||
technical processes involved in making entities flow through it. It is mainly
|
||||
aimed at developers who want to understand the internals while installing or
|
||||
extending the catalog. However, it can be informative for other personas too.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
The catalog forms a hub of sorts, where entities are ingested from various
|
||||
authoritative sources and held in a database, subject to automated processing,
|
||||
and then presented through an API for quick and easy access by Backstage and
|
||||
others. The most common source is [YAML files](descriptor-format.md) on a
|
||||
standard format, living in version control systems near the source code of
|
||||
systems that they describe. Those files are registered with the catalog and
|
||||
maintained by the respective owners. The catalog makes sure to keep itself up to
|
||||
date with changes to those files.
|
||||
|
||||
The main extension points where developers can customize the catalog are:
|
||||
|
||||
- _Entity providers_, that feed initial raw entity data into the catalog,
|
||||
- _Policies_, that establish baseline rules about the shape of entities,
|
||||
- _Processors_, that validate, analyze, and mutate the raw entity data into its
|
||||
final form.
|
||||
|
||||
The high level processes involved are:
|
||||
|
||||
- _Ingestion_, where entity providers fetch raw entity data from external
|
||||
sources and seed it into the database,
|
||||
- _Processing_, where the policies and processors continually treat the ingested
|
||||
data and may emit both other raw entities (that are also subject to
|
||||
processing), errors, relations to other entities, etc.,
|
||||
- _Stitching_, where all of the data emitted by various processors are assembled
|
||||
together into the final output entity.
|
||||
|
||||
An entity is not visible to the outside world (through the catalog API), until
|
||||
it has passed through the last process and landed among the final entities.
|
||||
|
||||
|
||||
|
||||
The details of these processes are described below.
|
||||
|
||||
## Ingestion
|
||||
|
||||
Each catalog deployment has a number of entity providers installed. They are
|
||||
responsible for fetching data from external authoritative sources in any way
|
||||
that they see fit, to translate those into entity objects, and to notify the
|
||||
database when those entities are added or removed. These are the _unprocessed
|
||||
entities_ that will be subject to later processing (see below), and they form
|
||||
the very basis of existence for entities. If there were no entity providers, no
|
||||
entities would ever enter the system.
|
||||
|
||||
The database always keeps track of the set of entities that belong to each
|
||||
provider; no two providers can try to output the same entity. And when a
|
||||
provider signals the removal of an entity, then that leads to an _eager
|
||||
deletion_: the entity and all auxiliary data that it has led to in the database
|
||||
is immediately purged.
|
||||
|
||||
|
||||
|
||||
There are two providers installed by default: the one that deals with user
|
||||
registered locations (e.g. URLs to YAML files), and the one that deals with
|
||||
static locations in the app-config. You can add more third party providers by
|
||||
passing them to the catalog builder in your backend initialization code, and you
|
||||
can easily write your own.
|
||||
|
||||
An entity provider is a class that implements the `EntityProvider` interface. It
|
||||
has three main parts:
|
||||
|
||||
- The identity: Each provider instance has a unique, stable identifier that the
|
||||
database can use to keep track of the originator of each unprocessed entity.
|
||||
- The connection: During backend startup, each provider is attached to the
|
||||
catalog runtime.
|
||||
- The stream of events: During its lifetime, the provider can issue change
|
||||
events to the runtime at any point in time, to modify its set of unprocessed
|
||||
entities.
|
||||
|
||||
It is entirely up to the provider to choose how and when it produces these
|
||||
change events. For example, the app-config provider only fires off an update at
|
||||
startup and then lies dormant. The location database provider does an initial
|
||||
update at startup, and then small delta updates every time a location database
|
||||
change is detected. The LDAP provider is driven externally by a timer loop that
|
||||
occasionally triggers a full update. Some future provider may be entirely event
|
||||
driven, feeding off an event bus or web hook. There is no magic coordination
|
||||
among providers; if they need to arrange synchronization or locking among
|
||||
themselves for example to avoid duplicate work across multiple catalog service
|
||||
machines, they need to handle that out-of-band.
|
||||
|
||||
The entities that are emitted get some coarse validation applied to them, to
|
||||
ensure that they at least adhere to the most basic schema rules about how an
|
||||
entity should be shaped. For example, they need to have a `kind`, a
|
||||
`metadata.name`, and optionally a `metadata.namespace`, among others. Apart from
|
||||
that, the ingestion stage considers its work done, and stores the unprocessed
|
||||
entities to be picked up at a later time by the processing system. This means
|
||||
that the more precise validation rules that you put in place on entities are
|
||||
_not_ yet applied at this stage.
|
||||
|
||||
## Processing
|
||||
|
||||
Every unprocessed entity comes with a timestamp, which tells at what time that
|
||||
the processing loop should next try to process it. When the entity first
|
||||
appears, this timestamp is set to "now" - asking for it to be picked up as soon
|
||||
as possible.
|
||||
|
||||
Each catalog deployment has a number of processors installed. They are
|
||||
responsible for receiving unprocessed entities that the catalog decided are due
|
||||
for processing, and then running that data through a number of processing
|
||||
stages. mutating the entity and emitting auxiliary data about it. When all of
|
||||
that is done, the catalog takes all of that information and stores it as the
|
||||
processed entity, and errors and relations to other entities separately. Then,
|
||||
the catalog checks to see what entities are touched by that output, and triggers
|
||||
the final assembly of those (see Stitching below).
|
||||
|
||||
There are several stages involved in the processing.
|
||||
|
||||
> TODO: More info here
|
||||
|
||||
## Stitching
|
||||
|
||||
The stitching is currently a fixed process, that cannot be modified or extended.
|
||||
This means that any modifications you want to make on the final result, has to
|
||||
happen during ingestion or processing.
|
||||
|
||||
> TODO: More info here
|
||||
@@ -57,18 +57,48 @@ if the original location delegates to another location. A common case is, that a
|
||||
location is registered as `bootstrap:bootstrap` which means that it is part of
|
||||
the `app-config.yaml` of a Backstage installation.
|
||||
|
||||
### backstage.io/orphan
|
||||
|
||||
This annotation is either absent, or present with the exact _string_ value
|
||||
`"true"`. It should never be added manually. Instead, the catalog itself injects
|
||||
the annotation as part of its processing loops, on entities that are found to
|
||||
have no registered locations or config locations that keep them "active" /
|
||||
"alive".
|
||||
|
||||
For example, suppose that the user first registers a location URL pointing to a
|
||||
`Location` kind entity, which in turn refers to two `Component` kind entities in
|
||||
two other files nearby. The end result is that the catalog contains those three
|
||||
entities. Now suppose that the user edits the original `Location` entity to only
|
||||
refer to the first of the `Component` kind entities. This will intentionally
|
||||
_not_ lead to the other `Component` entity to be removed from the catalog (for
|
||||
safety reasons). Instead, it gains this orphan marker annotation, to make it
|
||||
clear that user action is required to completely remove it, if desired.
|
||||
|
||||
```yaml
|
||||
# Example:
|
||||
metadata:
|
||||
annotations:
|
||||
backstage.io/orphan: 'true'
|
||||
```
|
||||
|
||||
### backstage.io/techdocs-ref
|
||||
|
||||
```yaml
|
||||
# Example:
|
||||
metadata:
|
||||
annotations:
|
||||
backstage.io/techdocs-ref: url:https://github.com/backstage/backstage/tree/master
|
||||
backstage.io/techdocs-ref: dir:.
|
||||
```
|
||||
|
||||
The value of this annotation is a location reference string (see above). If this
|
||||
annotation is specified, it is expected to point to a repository that the
|
||||
TechDocs system can read and generate docs from.
|
||||
The value of this annotation informs _where_ TechDocs source content is stored
|
||||
so that it can be read and docs can be generated from it. Most commonly, it's
|
||||
written as a path, relative to the location of the `catalog-info.yaml` itself,
|
||||
where the associated `mkdocs.yml` file can be found.
|
||||
|
||||
In unusual situations where the documentation for a catalog entity does not live
|
||||
alongside the entity's source code, the value of this annotation can point to an
|
||||
absolute URL, matching the location reference string format outlined above, for
|
||||
example: `url:https://github.com/backstage/backstage/tree/master`
|
||||
|
||||
### backstage.io/view-url, backstage.io/edit-url
|
||||
|
||||
|
||||
@@ -325,7 +325,7 @@ spec:
|
||||
output:
|
||||
links:
|
||||
- url: '{{steps.publish.output.remoteUrl}}'
|
||||
text: 'Go to Repo'
|
||||
title: 'Go to Repo'
|
||||
```
|
||||
|
||||
## Questions?
|
||||
|
||||
@@ -131,6 +131,9 @@ want to have those as well as your new one, you'll need to do the following:
|
||||
|
||||
```ts
|
||||
import { createBuiltinActions } from '@backstage/plugin-scaffolder-backend';
|
||||
import { ScmIntegrations } from '@backstage/integration';
|
||||
|
||||
const integrations = ScmIntegrations.fromConfig(config);
|
||||
|
||||
const builtInActions = createBuiltinActions({
|
||||
containerRunner,
|
||||
|
||||
@@ -29,7 +29,7 @@ about TechDocs and the philosophy in its
|
||||
- Explore and take advantage of the large ecosystem of
|
||||
[MkDocs plugins](https://www.mkdocs.org/user-guide/plugins/) to create a rich
|
||||
reading experience.
|
||||
- Search for and find docs (coming soon).
|
||||
- Search for and find docs.
|
||||
- Highlight text and raise an Issue to create feedback loop to drive quality
|
||||
documentation (future).
|
||||
- Contribute to and deploy from a marketplace of TechDocs widgets (future).
|
||||
@@ -54,23 +54,12 @@ providers are used.
|
||||
| Google Cloud Storage (GCS) | Yes ✅ |
|
||||
| Amazon Web Services (AWS) S3 | Yes ✅ |
|
||||
| Azure Blob Storage | Yes ✅ |
|
||||
| OpenStack Swift | Yes ✅ |
|
||||
| OpenStack Swift | Community ✅ |
|
||||
|
||||
[Reach out to us](#feedback) if you want to request more platforms.
|
||||
|
||||
## Project roadmap
|
||||
|
||||
### **Ongoing work 🚧**
|
||||
|
||||
**Beta release** -
|
||||
[Milestone](https://github.com/backstage/backstage/milestone/29)
|
||||
|
||||
- It should be possible and easy to use TechDocs in most environments across
|
||||
organizations.
|
||||
- Minimal bugs, better error handling and scalable backend and frontend.
|
||||
- Documentation Search
|
||||
- TechDocs Homepage with basic features
|
||||
|
||||
### **Future work 🔮**
|
||||
|
||||
**General Availability (GA) release** -
|
||||
|
||||
@@ -24,13 +24,13 @@ techdocs:
|
||||
|
||||
runIn: 'docker'
|
||||
|
||||
# techdocs.generator.dockerImage can be used to control the docker image used during documentation generation. This can be useful
|
||||
# (Optional) techdocs.generator.dockerImage can be used to control the docker image used during documentation generation. This can be useful
|
||||
# if you want to use MkDocs plugins or other packages that are not included in the default techdocs-container (spotify/techdocs).
|
||||
# NOTE: This setting is only used when techdocs.generator.runIn is set to 'docker'.
|
||||
|
||||
dockerImage: 'spotify/techdocs'
|
||||
|
||||
# techdocs.generator.pullImage can be used to disable pulling the latest docker image by default. This can be useful when you are
|
||||
# (Optional) techdocs.generator.pullImage can be used to disable pulling the latest docker image by default. This can be useful when you are
|
||||
# using a custom techdocs.generator.dockerImage and you have a custom docker login requirement. For example, you need to login to
|
||||
# AWS ECR to pull the docker image.
|
||||
# NOTE: Disabling this requires the docker image was pulled by other means before running the techdocs generator.
|
||||
@@ -113,6 +113,14 @@ techdocs:
|
||||
# https://docs.microsoft.com/en-us/azure/storage/common/storage-auth?toc=/azure/storage/blobs/toc.json
|
||||
accountKey: ${TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_KEY}
|
||||
|
||||
# (Optional and not recommended) Prior to version [0.x.y] of TechDocs, docs
|
||||
# sites could only be accessed over paths with case-sensitive entity triplets
|
||||
# e.g. (namespace/Kind/name). If you are upgrading from an older version of
|
||||
# TechDocs and are unable to perform the necessary migration of files in your
|
||||
# external storage, you can set this value to `true` to temporarily revert to
|
||||
# the old, case-sensitive entity triplet behavior.
|
||||
legacyUseCaseSensitiveTripletPaths: false
|
||||
|
||||
# (Optional and Legacy) TechDocs makes API calls to techdocs-backend using this URL. e.g. get docs of an entity, get metadata, etc.
|
||||
# You don't have to specify this anymore.
|
||||
|
||||
|
||||
@@ -66,9 +66,7 @@ Update your component's entity description by adding the following lines to its
|
||||
```yaml
|
||||
metadata:
|
||||
annotations:
|
||||
backstage.io/techdocs-ref: url:https://github.com/org/repo
|
||||
# Or
|
||||
# backstage.io/techdocs-ref: url:https://github.com/org/repo/tree/branchName/subFolder
|
||||
backstage.io/techdocs-ref: dir:.
|
||||
```
|
||||
|
||||
The
|
||||
|
||||
@@ -33,14 +33,24 @@ In `packages/app/src/App.tsx`, import `TechDocsPage` and add the following to
|
||||
`FlatRoutes`:
|
||||
|
||||
```tsx
|
||||
import { TechDocsPage } from '@backstage/plugin-techdocs';
|
||||
import {
|
||||
DefaultTechDocsHome,
|
||||
TechDocsIndexPage,
|
||||
TechDocsReaderPage,
|
||||
} from '@backstage/plugin-techdocs';
|
||||
|
||||
// ...
|
||||
|
||||
const AppRoutes = () => {
|
||||
<FlatRoutes>
|
||||
// ... other plugin routes
|
||||
<Route path="/docs" element={<TechdocsPage />} />
|
||||
<Route path="/docs" element={<TechDocsIndexPage />}>
|
||||
<DefaultTechDocsHome />
|
||||
</Route>
|
||||
<Route
|
||||
path="/docs/:namespace/:kind/:name/*"
|
||||
element={<TechDocsReaderPage />}
|
||||
/>
|
||||
</FlatRoutes>;
|
||||
};
|
||||
```
|
||||
@@ -204,7 +214,7 @@ techdocs:
|
||||
builder: 'local'
|
||||
publisher:
|
||||
type: 'local'
|
||||
generators:
|
||||
generator:
|
||||
techdocs: local
|
||||
```
|
||||
|
||||
|
||||
@@ -35,38 +35,64 @@ In your Backstage instance's `app-config.yaml`, set `techdocs.builder` from
|
||||
`'local'` to `'external'`. By doing this, TechDocs will not try to generate
|
||||
docs. Look at [TechDocs configuration](configuration.md) for reference.
|
||||
|
||||
## How to use URL Reader in TechDocs Prepare step?
|
||||
## How to understand techdocs-ref annotation values
|
||||
|
||||
If TechDocs is configured to generate docs, it will first download the
|
||||
repository associated with the `backstage.io/techdocs-ref` annotation defined in
|
||||
the Entity's `catalog-info.yaml` file. This is also called the
|
||||
If TechDocs is configured to generate docs, it will first download source files
|
||||
based on the value of the `backstage.io/techdocs-ref` annotation defined in the
|
||||
Entity's `catalog-info.yaml` file. This is also called the
|
||||
[Prepare](./concepts.md#techdocs-preparer) step.
|
||||
|
||||
There are two kinds of preparers or two ways of downloading these source files
|
||||
We strongly recommend that the `backstage.io/techdocs-ref` annotation in each
|
||||
documented catalog entity's `catalog-info.yaml` be set to `dir:.` in almost all
|
||||
situations. This is because TechDocs is aligned with the "docs like code"
|
||||
philosophy, whereby documentation should be authored and managed alongside the
|
||||
source code of the underlying software itself.
|
||||
|
||||
- Preparer 1: Doing a `git clone` of the repository (also known as Common Git
|
||||
Preparer)
|
||||
- Preparer 2: Downloading an archive.zip or equivalent of the repository (also
|
||||
known as URL Reader)
|
||||
When you see `dir:.`, you can translate it to mean:
|
||||
|
||||
If `backstage.io/techdocs-ref` is equal to any of these -
|
||||
- That the documentation source code lives in the same location as the
|
||||
`catalog-info.yaml` file.
|
||||
- That, in particular, the `mkdocs.yml` file is a sibling of `catalog-info.yaml`
|
||||
(meaning, it is in the same directory)
|
||||
- And that all of the source content of the documentation would be available if
|
||||
one were to download the directory containing those two files (as well as all
|
||||
sub-directories).
|
||||
|
||||
1. `github:https://githubhost.com/org/repo`
|
||||
2. `gitlab:https://gitlabhost.com/org/repo`
|
||||
3. `bitbucket:https://bitbuckethost.com/project/repo`
|
||||
4. `azure/api:https://azurehost.com/org/project`
|
||||
The directory tree of the entity would look something like this:
|
||||
|
||||
Then Common Git Preparer will be used i.e. a `git clone`. But the URL Reader is
|
||||
a much faster way to do this step. Convert the `backstage.io/techdocs-ref`
|
||||
values to the following -
|
||||
```
|
||||
├── catalog-info.yaml
|
||||
├── mkdocs.yml
|
||||
└── docs
|
||||
└── index.md
|
||||
```
|
||||
|
||||
1. `url:https://githubhost.com/org/repo/tree/<branch_name>`
|
||||
2. `url:https://gitlabhost.com/org/repo/tree/<branch_name>`
|
||||
3. `url:https://bitbuckethost.com/project/repo/src/<branch_name>`
|
||||
4. `url:https://azurehost.com/organization/project/_git/repository`
|
||||
If, for example, you wanted to keep a lean root directory, you could place your
|
||||
`mkdocs.yml` file in a subdirectory and update the `backstage.io/techdocs-ref`
|
||||
annotation value accordingly, e.g. to `dir:./sub-folder`:
|
||||
|
||||
Note that you can also provide a path to a non-root directory inside the
|
||||
repository which contains the `docs/` directory.
|
||||
```
|
||||
├── catalog-info.yaml
|
||||
└── sub-folder
|
||||
├── mkdocs.yml
|
||||
└── docs
|
||||
└── index.md
|
||||
```
|
||||
|
||||
In rare situations where your TechDocs source content is managed and stored in a
|
||||
location completely separate from your `catalog-info.yaml`, you can instead
|
||||
specify a URL location reference, the exact value of which will vary based on
|
||||
the source code hosting provider. Notice that instead of the `dir:` prefix, the
|
||||
`url:` prefix is used instead. For example:
|
||||
|
||||
- **GitHub**: `url:https://githubhost.com/org/repo/tree/<branch_name>`
|
||||
- **GitLab**: `url:https://gitlabhost.com/org/repo/tree/<branch_name>`
|
||||
- **Bitbucket**: `url:https://bitbuckethost.com/project/repo/src/<branch_name>`
|
||||
- **Azure**: `url:https://azurehost.com/organization/project/_git/repository`
|
||||
|
||||
Note, just as it's possible to specify a subdirectory with the `dir:` prefix,
|
||||
you can also provide a path to a non-root directory inside the repository which
|
||||
contains the `mkdocs.yml` file and `docs/` directory.
|
||||
|
||||
e.g.
|
||||
`url:https://github.com/backstage/backstage/tree/master/plugins/techdocs-backend/examples/documented-component`
|
||||
@@ -78,83 +104,116 @@ the repository. The archive does not have any git history attached to it. Also
|
||||
it is a compressed file. Hence the file size is significantly smaller than how
|
||||
much data git clone has to transfer.
|
||||
|
||||
## How to use a custom TechDocs home page?
|
||||
## How to customize the TechDocs home page?
|
||||
|
||||
### 1st way: TechDocsCustomHome with a custom configuration
|
||||
TechDocs uses a composability pattern similar to the Search and Catalog plugins
|
||||
in Backstage. While a default table experience, similar to the one provided by
|
||||
the Catalog plugin, is made available for ease-of-use, it's possible for you to
|
||||
provide a completely custom experience, tailored to the needs of your
|
||||
organization.
|
||||
|
||||
As an example, in your main App.tsx:
|
||||
This is done in your `app` package. By default, you might see something like
|
||||
this in your `App.tsx`:
|
||||
|
||||
```tsx
|
||||
import {
|
||||
TechDocsCustomHome,
|
||||
PanelType,
|
||||
TechDocsReaderPage,
|
||||
} from '@backstage/plugin-techdocs';
|
||||
import { Entity } from '@backstage/catalog-model';
|
||||
|
||||
const tabsConfig = [
|
||||
{
|
||||
label: 'Custom Tab',
|
||||
panels: [
|
||||
{
|
||||
title: 'Custom Documents Cards 1',
|
||||
description:
|
||||
'Explore your internal technical ecosystem through documentation.',
|
||||
panelType: 'DocsCardGrid' as PanelType,
|
||||
// optional, is applied to a container of the panel (excludes header of panel)
|
||||
panelCSS: { maxHeight: '400px', overflow:'auto' },
|
||||
filterPredicate: (entity: Entity) => !!entity.metadata.annotations?.['customCardAnnotationOne'];
|
||||
},
|
||||
{
|
||||
title: 'Custom Documents Cards 2',
|
||||
description:
|
||||
'Explore your internal technical ecosystem through documentation.',
|
||||
panelType: 'DocsCardGrid' as PanelType,
|
||||
panelCSS: { maxHeight: '400px', overflow:'auto' },
|
||||
filterPredicate: (entity: Entity) => !!entity.metadata.annotations?.['customCardAnnotationTwo'];
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
label: 'Overview',
|
||||
panels: [
|
||||
{
|
||||
title: 'Overview',
|
||||
description:
|
||||
'Explore your internal technical ecosystem through documentation.',
|
||||
panelType: 'DocsTable' as PanelType,
|
||||
filterPredicate: () => true,
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
const routes = (
|
||||
const AppRoutes = () => {
|
||||
<FlatRoutes>
|
||||
<Route
|
||||
path="/docs"
|
||||
element={<TechDocsCustomHome tabsConfig={tabsConfig} />}
|
||||
/>
|
||||
<Route
|
||||
path="/docs/:namespace/:kind/:name/*"
|
||||
element={<TechDocsReaderPage />}
|
||||
/>
|
||||
</FlatRoutes>
|
||||
<Route path="/docs" element={<TechDocsIndexPage />}>
|
||||
<DefaultTechDocsHome />
|
||||
</Route>
|
||||
</FlatRoutes>;
|
||||
};
|
||||
```
|
||||
|
||||
An example of tabsConfig that corresponds to the default documentation home page
|
||||
can be found at `plugins/techdocs/src/home/components/TechDocsHome.tsx`.
|
||||
But you can replace `<DefaultTechDocsHome />` with any React component, which
|
||||
will be rendered in its place. Most likely, you would want to create and
|
||||
maintain such a component in a new directory at
|
||||
`packages/app/src/components/techdocs`, and import and use it in `App.tsx`:
|
||||
|
||||
Currently `panelType` has DocsCardGrid and DocsTable available. We currently
|
||||
recommend that DocsCardGrid can be optionally vertically stacked by setting a
|
||||
maxHeight using `panelCSS`, and DocsTable to be in a tab by itself.
|
||||
```tsx
|
||||
import { CustomTechDocsHome } from './components/techdocs/CustomTechDocsHome';
|
||||
// ...
|
||||
const AppRoutes = () => {
|
||||
<FlatRoutes>
|
||||
<Route path="/docs" element={<TechDocsIndexPage />}>
|
||||
<CustomTechDocsHome />
|
||||
</Route>
|
||||
</FlatRoutes>;
|
||||
};
|
||||
```
|
||||
|
||||
### 2nd way: Custom home page plugin
|
||||
## How to migrate from TechDocs Alpha to Beta
|
||||
|
||||
A custom home page plugin can be built that uses the components extensions
|
||||
DocsCardGrid and DocsTable, exported from @backstage/techdocs. They both take a
|
||||
array of documentation entities ( i.e.have a 'backstage.io/techdocs-ref'
|
||||
annotation ) as an 'entities' attribute.
|
||||
> This guide only applies to the "recommended" TechDocs deployment method (where
|
||||
> an external storage provider and external CI/CD is used). If you use the
|
||||
> "basic" or "out-of-the-box" setup, you can stop here! No action needed.
|
||||
|
||||
For a reference to the React structure of the default home page, please refer to
|
||||
`plugins/techdocs/src/home/components/TechDocsCustomHome.tsx`.
|
||||
For the purposes of this guide, TechDocs Beta version is defined as:
|
||||
|
||||
- **TechDocs Plugin**: At least `v0.11.0`
|
||||
- **TechDocs Backend Plugin**: At least `v0.10.0`
|
||||
- **TechDocs CLI**: At least `v0.7.0`
|
||||
|
||||
The beta version of TechDocs made a breaking change to the way TechDocs content
|
||||
was accessed and stored, allowing pages to be accessed with case-insensitive
|
||||
entity triplet paths (e.g. `/docs/namespace/kind/name` whereas in prior
|
||||
versions, they could only be accessed at `/docs/namespace/Kind/name`). In order
|
||||
to enable this change, documentation has to be stored in an external storage
|
||||
provider using an object key whose entity triplet is lower-cased.
|
||||
|
||||
New installations of TechDocs since the beta version will work fine with no
|
||||
action, but for those who were running TechDocs prior to this version, a
|
||||
migration will need to be performed so that all existing content in your storage
|
||||
bucket matches this lower-case entity triplet expectation.
|
||||
|
||||
1. **Ensure you have the right permissions on your storage provider**: In order
|
||||
to migrate files in your storage provider, the `techdocs-cli` needs to be
|
||||
able to read/copy/rename/move/delete files. The exact instructions vary by
|
||||
storage provider, but check the [using cloud storage][using-cloud-storage]
|
||||
page for details.
|
||||
|
||||
2. **Run a non-destructive migration of files**: Ensure you have the latest
|
||||
version of `techdocs-cli` installed. Then run the following command, using
|
||||
the details relevant for your provider / configuration. This will copy all
|
||||
files from, e.g. `namespace/Kind/name/index.html` to
|
||||
`namespace/kind/name/index.html`, without removing the original files.
|
||||
|
||||
```sh
|
||||
techdocs-cli migrate --publisher-type <awsS3|googleGcs|azureBlobStorage> --storage-name <bucket/container name> --verbose
|
||||
```
|
||||
|
||||
3. **Deploy the updated versions of the TechDocs plugins**: Once the migration
|
||||
above has been run, you can deploy the beta versions of the TechDocs backend
|
||||
and frontend plugins to your Backstage instance.
|
||||
|
||||
4. **Verify that your TechDocs sites are still loading/accessible**: Try
|
||||
accessing a TechDocs site using different entity-triplet case variants, e.g.
|
||||
`/docs/namespace/KIND/name` or `/docs/namespace/kind/name`. Your TechDocs
|
||||
site should load regardless of the URL path casing you use.
|
||||
|
||||
5. **Clean up the old objects from storage**: Once you've verified that your
|
||||
TechDocs site is accessible, you can clean up your storage bucket by
|
||||
re-running the `migrate` command on the TechDocs CLI, but with an additional
|
||||
`removeOriginal` flag passed:
|
||||
|
||||
```sh
|
||||
techdocs-cli migrate --publisher-type <awsS3|googleGcs|azureBlobStorage> --storage-name <bucket/container name> --removeOriginal --verbose
|
||||
```
|
||||
|
||||
6. **Update your CI/CD pipelines to use the beta version of the TechDocs CLI**:
|
||||
Finally, you can update all of your CI/CD pipelines to use at least v0.x.y of
|
||||
the TechDocs CLI, ensuring that all sites are published to the new,
|
||||
lower-cased entity triplet paths going forward.
|
||||
|
||||
If you encounter problems running this migration, please [report the
|
||||
issue][beta-migrate-bug]. You can temporarily revert to pre-beta storage
|
||||
expectations with a configuration change:
|
||||
|
||||
```yaml
|
||||
techdocs:
|
||||
legacyUseCaseSensitiveTripletPaths: true
|
||||
```
|
||||
|
||||
[beta-migrate-bug]:
|
||||
https://github.com/backstage/backstage/issues/new?assignees=&labels=bug&template=bug_template.md&title=[TechDocs]%20Unable%20to%20run%20beta%20migration
|
||||
[using-cloud-storage]: ./using-cloud-storage.md
|
||||
|
||||
@@ -61,7 +61,7 @@ If you do not prefer (3a) and optionally like to use a service account, you can
|
||||
follow these steps.
|
||||
|
||||
Create a new Service Account and a key associated with it. In roles of the
|
||||
service account, use "Storage Admin".
|
||||
service account, use "Storage Object Admin".
|
||||
|
||||
If you want to create a custom role, make sure to include both `get` and
|
||||
`create` permissions for both "Objects" and "Buckets". See
|
||||
@@ -143,6 +143,8 @@ permissions to:
|
||||
|
||||
- `s3:ListBucket` to retrieve bucket metadata
|
||||
- `s3:PutObject` to upload files to the bucket
|
||||
- `s3:DeleteObject` and `s3:DeleteObjectVersion` to delete stale content during
|
||||
re-publishing
|
||||
|
||||
To _read_ TechDocs from the S3 bucket the IAM policy needs to have at a minimum
|
||||
permissions to:
|
||||
@@ -345,6 +347,10 @@ techdocs:
|
||||
accountKey: ${TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_KEY}
|
||||
```
|
||||
|
||||
In either case, the account or credentials used to access your container and all
|
||||
TechDocs objects underneath it should have the `Storage Blog Data Owner` role
|
||||
applied, in order to read, write, and delete objects as needed.
|
||||
|
||||
**4. That's it!**
|
||||
|
||||
Your Backstage app is now ready to use Azure Blob Storage for TechDocs, to store
|
||||
@@ -393,9 +399,34 @@ techdocs:
|
||||
|
||||
Set the configs in your `app-config.yaml` to point to your container name.
|
||||
|
||||
https://docs.openstack.org/api-ref/identity/v3/?expanded=password-authentication-with-unscoped-authorization-detail#password-authentication-with-unscoped-authorization
|
||||
https://docs.openstack.org/api-ref/identity/v3/?expanded=password-authentication-with-unscoped-authorization-detail,authenticating-with-an-application-credential-detail#authenticating-with-an-application-credential
|
||||
for more details.
|
||||
|
||||
```yaml
|
||||
techdocs:
|
||||
publisher:
|
||||
type: 'openStackSwift'
|
||||
openStackSwift:
|
||||
containerName: 'name-of-techdocs-storage-bucket'
|
||||
credentials:
|
||||
id: ${OPENSTACK_SWIFT_STORAGE_APPLICATION_CREDENTIALS_ID}
|
||||
secret: ${OPENSTACK_SWIFT_STORAGE_APPLICATION_CREDENTIALS_SECRET}
|
||||
authUrl: ${OPENSTACK_SWIFT_STORAGE_AUTH_URL}
|
||||
swiftUrl: ${OPENSTACK_SWIFT_STORAGE_SWIFT_URL}
|
||||
```
|
||||
|
||||
**4. That's it!**
|
||||
|
||||
Your Backstage app is now ready to use OpenStack Swift Storage for TechDocs, to
|
||||
store and read the static generated documentation files. When you start the
|
||||
backend of the app, you should be able to see
|
||||
`techdocs info Successfully connected to the OpenStack Swift Storage container`
|
||||
in the logs.
|
||||
|
||||
## Bonus: Migration from old OpenStack Swift Configuration
|
||||
|
||||
Let's assume we have the old OpenStack Swift configuration here.
|
||||
|
||||
```yaml
|
||||
techdocs:
|
||||
publisher:
|
||||
@@ -412,10 +443,40 @@ techdocs:
|
||||
region: ${OPENSTACK_SWIFT_STORAGE_REGION}
|
||||
```
|
||||
|
||||
**4. That's it!**
|
||||
##### Step 1: Change the credential keys
|
||||
|
||||
Your Backstage app is now ready to use OpenStack Swift Storage for TechDocs, to
|
||||
store and read the static generated documentation files. When you start the
|
||||
backend of the app, you should be able to see
|
||||
`techdocs info Successfully connected to the OpenStack Swift Storage container`
|
||||
in the logs.
|
||||
Since the new SDK uses _Application Credentials_ to authenticate OpenStack, we
|
||||
need to change the keys `credentials.username` to `credentials.id`,
|
||||
`credentials.password` to `credentials.secret` and use Application Credential ID
|
||||
and secret here. For more detail about credentials look
|
||||
[here](https://docs.openstack.org/api-ref/identity/v3/?expanded=password-authentication-with-unscoped-authorization-detail,authenticating-with-an-application-credential-detail#authenticating-with-an-application-credential).
|
||||
|
||||
##### Step 2: Remove the unused keys
|
||||
|
||||
Since the new SDK doesn't use the old way authentication, we don't need the keys
|
||||
`openStackSwift.keystoneAuthVersion`, `openStackSwift.domainId`,
|
||||
`openStackSwift.domainName` and `openStackSwift.region`. So you can remove them.
|
||||
|
||||
##### Step 3: Add Swift URL
|
||||
|
||||
The new SDK needs the OpenStack Swift connection URL for connecting the Swift.
|
||||
So you need to add a new key called `openStackSwift.swiftUrl` and give the
|
||||
OpenStack Swift url here. Example url should look like that:
|
||||
`https://example.com:6780/swift/v1`
|
||||
|
||||
##### That's it!
|
||||
|
||||
Your new configuration should look like that!
|
||||
|
||||
```yaml
|
||||
techdocs:
|
||||
publisher:
|
||||
type: 'openStackSwift'
|
||||
openStackSwift:
|
||||
containerName: 'name-of-techdocs-storage-bucket'
|
||||
credentials:
|
||||
id: ${OPENSTACK_SWIFT_STORAGE_APPLICATION_CREDENTIALS_ID}
|
||||
secret: ${OPENSTACK_SWIFT_STORAGE_APPLICATION_CREDENTIALS_SECRET}
|
||||
authUrl: ${OPENSTACK_SWIFT_STORAGE_AUTH_URL}
|
||||
swiftUrl: ${OPENSTACK_SWIFT_STORAGE_SWIFT_URL}
|
||||
```
|
||||
|
||||
@@ -35,7 +35,7 @@ If you want more control over the theme, and for example customize font sizes
|
||||
and margins, you can use the lower-level `createThemeOverrides` function
|
||||
exported by [@backstage/theme](https://www.npmjs.com/package/@backstage/theme)
|
||||
in combination with
|
||||
[createMuiTheme](https://material-ui.com/customization/theming/#createmuitheme-options-args-theme)
|
||||
[createTheme](https://material-ui.com/customization/theming/#createmuitheme-options-args-theme)
|
||||
from [@material-ui/core](https://www.npmjs.com/package/@material-ui/core). See
|
||||
the
|
||||
[@backstage/theme source](https://github.com/backstage/backstage/tree/master/packages/theme/src)
|
||||
@@ -128,7 +128,7 @@ const themeOptions = createThemeOptions({
|
||||
tool: genPageTheme(['#123456','#123456'], shapes.round),
|
||||
service: genPageTheme(['#123456','#123456'], shapes.wave),
|
||||
website: genPageTheme(['#123456','#123456'], shapes.wave),
|
||||
library: genPageTheme(['#123456','#123456'] shapes.wave),
|
||||
library: genPageTheme(['#123456','#123456'], shapes.wave),
|
||||
other: genPageTheme(['#123456','#123456'], shapes.wave),
|
||||
app: genPageTheme(['#123456','#123456'], shapes.wave),
|
||||
apis: genPageTheme(['#123456','#123456'], shapes.wave),
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
id: locations
|
||||
sidebar_label: Locations
|
||||
title: Amazon Web Services S3 Locations
|
||||
# prettier-ignore
|
||||
description: Setting up an integration with Amazon Web Services S3
|
||||
---
|
||||
|
||||
The AWS S3 integration supports loading catalog entities from an S3 Bucket.
|
||||
Entities can be added to
|
||||
[static catalog configuration](../../features/software-catalog/configuration.md),
|
||||
or registered with the
|
||||
[catalog-import](https://github.com/backstage/backstage/tree/master/plugins/catalog-import)
|
||||
plugin.
|
||||
|
||||
## Configuration
|
||||
|
||||
To use this integration, add configuration to your `app-config.yaml`:
|
||||
|
||||
```yaml
|
||||
integrations:
|
||||
awsS3:
|
||||
- host: amazonaws.com
|
||||
accessKeyId: ${AWS_ACCESS_KEY_ID}
|
||||
secretAccessKey: ${AWS_SECRET_ACCESS_KEY}
|
||||
```
|
||||
|
||||
Then make sure the environment variables `AWS_ACCESS_KEY_ID` and
|
||||
`AWS_SECRET_ACCESS_KEY` are set when you run Backstage.
|
||||
|
||||
Users with multiple AWS accounts may want to use a role for S3 storage that is
|
||||
in a different AWS account. Using the `roleArn` parameter as seen below, you can
|
||||
instruct the AWS S3 reader to assume a role before accessing S3:
|
||||
|
||||
```yaml
|
||||
integrations:
|
||||
awsS3:
|
||||
- host: amazonaws.com
|
||||
accessKeyId: ${AWS_ACCESS_KEY_ID}
|
||||
secretAccessKey: ${AWS_SECRET_ACCESS_KEY}
|
||||
roleArn: 'arn:aws:iam::xxxxxxxxxxxx:role/example-role'
|
||||
```
|
||||
@@ -108,13 +108,11 @@ matching repository is processed.
|
||||
repository.
|
||||
|
||||
```typescript
|
||||
const customRepositoryParser: BitbucketRepositoryParser = async function* customRepositoryParser({
|
||||
client,
|
||||
repository,
|
||||
}) {
|
||||
// Custom logic for interpret the matching repository.
|
||||
// See defaultRepositoryParser for an example
|
||||
};
|
||||
const customRepositoryParser: BitbucketRepositoryParser =
|
||||
async function* customRepositoryParser({ client, repository }) {
|
||||
// Custom logic for interpret the matching repository.
|
||||
// See defaultRepositoryParser for an example
|
||||
};
|
||||
|
||||
const processor = BitbucketDiscoveryProcessor.fromConfig(env.config, {
|
||||
parser: customRepositoryParser,
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
id: discovery
|
||||
title: GitLab Discovery
|
||||
sidebar_label: Discovery
|
||||
# prettier-ignore
|
||||
description: Automatically discovering catalog entities from repositories in GitLab
|
||||
---
|
||||
|
||||
The GitLab integration has a special discovery processor for discovering catalog
|
||||
entities from GitLab. The processor will crawl the GitLab instance and register
|
||||
entities matching the configured path. This can be useful as an alternative to
|
||||
static locations or manually adding things to the catalog.
|
||||
|
||||
To use the discovery processor, you'll need a GitLab integration
|
||||
[set up](locations.md) with a `token`. Then you can add a location target to the
|
||||
catalog configuration:
|
||||
|
||||
```yaml
|
||||
catalog:
|
||||
locations:
|
||||
- type: gitlab-discovery
|
||||
target: https://gitlab.com/group/subgroup/blob/main/catalog-info.yaml
|
||||
```
|
||||
|
||||
Note the `gitlab-discovery` type, as this is not a regular `url` processor.
|
||||
|
||||
The target is composed of three parts:
|
||||
|
||||
- The base URL, `https://gitlab.com` in this case
|
||||
- The group path, `group/subgroup` in this case. This is optional: If you omit
|
||||
this path the processor will scan the entire GitLab instance instead.
|
||||
- The path within each repository to find the catalog YAML file. This will
|
||||
usually be `/blob/main/catalog-info.yaml`, `/blob/master/catalog-info.yaml` or
|
||||
a similar variation for catalog files stored in the root directory of each
|
||||
repository. If you want to use the repository's default branch use the `*`
|
||||
wildcard, e.g.: `/blob/*/catalog-info.yaml`
|
||||
@@ -312,14 +312,6 @@ configuration.
|
||||
|
||||
Stability: `1`
|
||||
|
||||
### `register-component` [GitHub](https://github.com/backstage/backstage/tree/master/plugins/register-component/)
|
||||
|
||||
A frontend plugin that allows the user to register entity locations in the
|
||||
catalog.
|
||||
|
||||
Stability: `0`. This plugin is likely to be replaced by a generic entity import
|
||||
plugin instead.
|
||||
|
||||
### `scaffolder` [GitHub](https://github.com/backstage/backstage/tree/master/plugins/scaffolder/)
|
||||
|
||||
The frontend scaffolder plugin where one can browse templates and initiate
|
||||
|
||||
@@ -89,6 +89,26 @@ integrations:
|
||||
- $include: example-backstage-app-credentials.yaml
|
||||
```
|
||||
|
||||
### Limiting the GitHub App installations
|
||||
|
||||
If you want to limit the GitHub app installations visible to backstage you may
|
||||
optionally include the `allowedInstallationOwners` option.
|
||||
|
||||
```yaml
|
||||
appId: 1
|
||||
allowedInstallationOwners: ['GlobexCorp']
|
||||
clientId: client id
|
||||
clientSecret: client secret
|
||||
webhookSecret: webhook secret
|
||||
privateKey: |
|
||||
-----BEGIN RSA PRIVATE KEY-----
|
||||
...Key content...
|
||||
-----END RSA PRIVATE KEY-----
|
||||
```
|
||||
|
||||
This will result in backstage preventing the use of any installation that is not
|
||||
within the allow list.
|
||||
|
||||
### Permissions for pull requests
|
||||
|
||||
These are the minimum permissions required for creating a pull request with
|
||||
|
||||
Reference in New Issue
Block a user