Merge remote-tracking branch 'origin' into github-installations-limit
This commit is contained in:
@@ -4,8 +4,6 @@ title: Search Architecture
|
||||
description: Documentation on Search Architecture
|
||||
---
|
||||
|
||||
# Search Architecture
|
||||
|
||||
> _This architecture has not been fully implemented yet. Find our milestones to
|
||||
> follow our progress and help contribute on the
|
||||
> [Search Roadmap](./README.md#project-roadmap)._
|
||||
|
||||
@@ -4,8 +4,6 @@ title: Search Concepts
|
||||
description: Documentation on Backstage Search Concepts
|
||||
---
|
||||
|
||||
# Search Concepts
|
||||
|
||||
Backstage Search lets you find the right information you are looking for in the
|
||||
Backstage ecosystem.
|
||||
|
||||
|
||||
@@ -4,8 +4,6 @@ title: Getting Started with Search
|
||||
description: How to set up and install Backstage Search
|
||||
---
|
||||
|
||||
# Getting Started
|
||||
|
||||
Search functions as a plugin to Backstage, so you will need to use Backstage to
|
||||
use Search.
|
||||
|
||||
@@ -252,13 +250,9 @@ an example:
|
||||
Backstage Search isn't a search engine itself, rather, it provides an interface
|
||||
between your Backstage instance and a
|
||||
[Search Engine](./concepts.md#search-engines) of your choice. Currently, we only
|
||||
support one, an in-memory search Engine called Lunr. It can be instantiated like
|
||||
this:
|
||||
|
||||
```typescript
|
||||
const searchEngine = new LunrSearchEngine({ logger });
|
||||
const indexBuilder = new IndexBuilder({ logger, searchEngine });
|
||||
```
|
||||
support two engines, an in-memory search Engine called Lunr and ElasticSearch.
|
||||
See [Search Engines](./search-engines.md) documentation for more information how
|
||||
to configure these in your Backstage instance.
|
||||
|
||||
Backstage Search can be used to power search of anything! Plugins like the
|
||||
Catalog offer default [collators](./concepts.md#collators) (e.g.
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
---
|
||||
id: search-engines
|
||||
title: Search Engines
|
||||
description: Choosing and configuring your search engine for Backstage
|
||||
---
|
||||
|
||||
Backstage supports 2 search engines by default, an in-memory engine called Lunr
|
||||
and ElasticSearch. You can configure your own search engines by implementing the
|
||||
provided interface as mentioned in the
|
||||
[search backend documentation.](./getting-started.md#Backend)
|
||||
|
||||
Provided search engine implementations have their own way of constructing
|
||||
queries, which may be something you want to modify. Alterations to the querying
|
||||
logic of a search engine can be made by providing your own implementation of a
|
||||
QueryTranslator interface. This modification can be done without touching
|
||||
provided search engines by using the exposed setter to set the modified query
|
||||
translator into the instance.
|
||||
|
||||
```typescript
|
||||
const searchEngine = new LunrSearchEngine({ logger });
|
||||
searchEngine.setTranslator(new MyNewAndBetterQueryTranslator());
|
||||
```
|
||||
|
||||
## Lunr
|
||||
|
||||
Lunr search engine is enabled by default for your backstage instance if you have
|
||||
not done additional changes to the scaffolded app.
|
||||
|
||||
Lunr can be instantiated like this:
|
||||
|
||||
```typescript
|
||||
// app/backend/src/plugins/search.ts
|
||||
const searchEngine = new LunrSearchEngine({ logger });
|
||||
const indexBuilder = new IndexBuilder({ logger, searchEngine });
|
||||
```
|
||||
|
||||
## Postgres
|
||||
|
||||
The Postgres based search engine only requires that postgres being configured as
|
||||
the database engine for Backstage. Therefore it targets setups that want to
|
||||
avoid maintaining another external service like elastic search. The search
|
||||
provides decent results and performs well with ten thousands of indexed
|
||||
documents. The connection to postgres is established via the database manager
|
||||
also used by other plugins.
|
||||
|
||||
> **Important**: The search plugin requires at least Postgres 11!
|
||||
|
||||
To use the `PgSearchEngine`, make sure that you have a Postgres database
|
||||
configured and make the following changes to your backend:
|
||||
|
||||
1. Add a dependency on `@backstage/plugin-search-backend-module-pg` to your
|
||||
backend's `package.json`.
|
||||
2. Initialize the search engine. It is recommended to initialize it with a
|
||||
fallback to the lunr search engine if you are running Backstage for
|
||||
development locally with SQLite:
|
||||
|
||||
```typescript
|
||||
// In packages/backend/src/plugins/search.ts
|
||||
|
||||
// Initialize a connection to a search engine.
|
||||
const searchEngine = (await PgSearchEngine.supported(database))
|
||||
? await PgSearchEngine.from({ database })
|
||||
: new LunrSearchEngine({ logger });
|
||||
```
|
||||
|
||||
## ElasticSearch
|
||||
|
||||
Backstage supports ElasticSearch search engine connections, indexing and
|
||||
querying out of the box. Available configuration options enable usage of either
|
||||
AWS or Elastic.co hosted solutions, or a custom self-hosted solution.
|
||||
|
||||
Similarly to Lunr above, ElasticSearch can be set up like this:
|
||||
|
||||
```typescript
|
||||
// app/backend/src/plugins/search.ts
|
||||
const searchEngine = await ElasticSearchSearchEngine.initialize({
|
||||
logger,
|
||||
config,
|
||||
});
|
||||
const indexBuilder = new IndexBuilder({ logger, searchEngine });
|
||||
```
|
||||
|
||||
For the engine to be available, your backend package needs a dependency into
|
||||
package `@backstage/plugin-search-backend-module-elasticsearch`.
|
||||
|
||||
ElasticSearch needs some additional configuration before it is ready to use
|
||||
within your instance. The configuration options are documented in the
|
||||
[configuration schema definition file.](https://github.com/backstage/backstage/blob/master/plugins/search-backend-module-elasticsearch/config.d.ts)
|
||||
|
||||
The underlying functionality is using official ElasticSearch client version 7.x,
|
||||
meaning that ElasticSearch version 7 is the only one confirmed to be supported.
|
||||
|
||||
## Example configurations
|
||||
|
||||
### AWS
|
||||
|
||||
Using AWS hosted ElasticSearch the only configuration option needed is the URL
|
||||
to the ElasticSearch service. The implementation assumes that environment
|
||||
variables for AWS access key id and secret access key are defined in accordance
|
||||
to the
|
||||
[default AWS credential chain.](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html).
|
||||
|
||||
```yaml
|
||||
search:
|
||||
elasticsearch:
|
||||
provider: aws
|
||||
node: https://my-backstage-search-asdfqwerty.eu-west-1.es.amazonaws.com
|
||||
```
|
||||
|
||||
### Elastic.co
|
||||
|
||||
Elastic Cloud hosted ElasticSearch uses a Cloud ID to determine the instance of
|
||||
hosted ElasticSearch to connect to. Additionally, username and password needs to
|
||||
be provided either directly or using environment variables like defined in
|
||||
[Backstage documentation.](https://backstage.io/docs/conf/writing#includes-and-dynamic-data)
|
||||
|
||||
```yaml
|
||||
search:
|
||||
elasticsearch:
|
||||
provider: elastic
|
||||
cloudId: backstage-elastic:asdfqwertyasdfqwertyasdfqwertyasdfqwerty==
|
||||
auth:
|
||||
username: elastic
|
||||
password: changeme
|
||||
```
|
||||
|
||||
### Others
|
||||
|
||||
Other ElasticSearch instances can be connected to by using standard
|
||||
ElasticSearch authentication methods and exposed URL, provided that the cluster
|
||||
supports that. The configuration options needed are the URL to the node and
|
||||
authentication information. Authentication can be handled by either providing
|
||||
username/password or an API key or a bearer token. In case both
|
||||
username/password combination and one of the tokens are provided, token takes
|
||||
precedence. For more information how to create an API key, see
|
||||
[Elastic documentation on API keys](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html),
|
||||
and how to create a bearer token see
|
||||
[Elastic documentation on tokens.](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-service-token.html)
|
||||
|
||||
#### Configuration examples
|
||||
|
||||
##### With username and password
|
||||
|
||||
```yaml
|
||||
search:
|
||||
elasticsearch:
|
||||
node: http://localhost:9200
|
||||
auth:
|
||||
username: elastic
|
||||
password: changeme
|
||||
```
|
||||
|
||||
##### With bearer token
|
||||
|
||||
```yaml
|
||||
search:
|
||||
elasticsearch:
|
||||
node: http://localhost:9200
|
||||
auth:
|
||||
bearer: token
|
||||
```
|
||||
|
||||
##### With API key
|
||||
|
||||
```yaml
|
||||
search:
|
||||
elasticsearch:
|
||||
node: http://localhost:9200
|
||||
auth:
|
||||
apiKey: base64EncodedKey
|
||||
```
|
||||
@@ -27,27 +27,33 @@ default catalog page and create a component in a
|
||||
```tsx
|
||||
// imports, etc omitted for brevity. for full source see:
|
||||
// https://github.com/backstage/backstage/blob/master/plugins/catalog/src/components/CatalogPage/CatalogPage.tsx
|
||||
export const CustomCatalogPage = () => {
|
||||
export const CustomCatalogPage = ({
|
||||
columns,
|
||||
actions,
|
||||
initiallySelectedFilter = 'owned',
|
||||
}: CatalogPageProps) => {
|
||||
return (
|
||||
<CatalogLayout>
|
||||
<PageWithHeader title={`${orgName} Catalog`} themeId="home">
|
||||
<Content>
|
||||
<ContentHeader title="Components">
|
||||
<CreateComponentButton />
|
||||
<CreateButton title="Create Component" to={link} />
|
||||
<SupportButton>All your software catalog entities</SupportButton>
|
||||
</ContentHeader>
|
||||
<div className={styles.contentWrapper}>
|
||||
<EntityListProvider>
|
||||
<div>
|
||||
<EntityListProvider>
|
||||
<FilteredEntityLayout>
|
||||
<FilterContainer>
|
||||
<EntityKindPicker initialFilter="component" hidden />
|
||||
<EntityTypePicker />
|
||||
<UserListPicker />
|
||||
<UserListPicker initialFilter={initiallySelectedFilter} />
|
||||
<EntityTagPicker />
|
||||
</div>
|
||||
<CatalogTable />
|
||||
</EntityListProvider>
|
||||
</div>
|
||||
</FilterContainer>
|
||||
<EntityListContainer>
|
||||
<CatalogTable columns={columns} actions={actions} />
|
||||
</EntityListContainer>
|
||||
</FilteredEntityLayout>
|
||||
</EntityListProvider>
|
||||
</Content>
|
||||
</CatalogLayout>
|
||||
</PageWithHeader>
|
||||
);
|
||||
};
|
||||
```
|
||||
@@ -137,19 +143,27 @@ export const EntitySecurityTierPicker = () => {
|
||||
Now we can add the component to `CustomCatalogPage`:
|
||||
|
||||
```diff
|
||||
export const CustomCatalogPage = () => {
|
||||
export const CustomCatalogPage = ({
|
||||
columns,
|
||||
actions,
|
||||
initiallySelectedFilter = 'owned',
|
||||
}: CatalogPageProps) => {
|
||||
return (
|
||||
...
|
||||
<EntityListProvider>
|
||||
<div>
|
||||
<EntityListProvider>
|
||||
<FilteredEntityLayout>
|
||||
<FilterContainer>
|
||||
<EntityKindPicker initialFilter="component" hidden />
|
||||
<EntityTypePicker />
|
||||
<UserListPicker />
|
||||
+ <EntitySecurityTierPicker />
|
||||
<UserListPicker initialFilter={initiallySelectedFilter} />
|
||||
+ <EntitySecurityTierPicker />
|
||||
<EntityTagPicker />
|
||||
</div>
|
||||
<CatalogTable />
|
||||
</EntityListProvider>
|
||||
<FilterContainer>
|
||||
<EntityListContainer>
|
||||
<CatalogTable columns={columns} actions={actions} />
|
||||
</EntityListContainer>
|
||||
</FilteredEntityLayout>
|
||||
</EntityListProvider>
|
||||
...
|
||||
};
|
||||
```
|
||||
|
||||
@@ -8,12 +8,11 @@ description: The Backstage Software Catalog
|
||||
|
||||
## What is a Software Catalog?
|
||||
|
||||
The Backstage Software Catalog — actually, a software catalog, since it includes
|
||||
more than just services — is a centralized system that keeps track of ownership
|
||||
and metadata for all the software in your ecosystem (services, websites,
|
||||
libraries, data pipelines, etc). The catalog is built around the concept of
|
||||
[metadata YAML files](descriptor-format.md) stored together with the code, which
|
||||
are then harvested and visualized in Backstage.
|
||||
The Backstage Software Catalog is a centralized system that keeps track of
|
||||
ownership and metadata for all the software in your ecosystem (services,
|
||||
websites, libraries, data pipelines, etc). The catalog is built around the
|
||||
concept of [metadata YAML files](descriptor-format.md) stored together with the
|
||||
code, which are then harvested and visualized in Backstage.
|
||||
|
||||

|
||||
|
||||
|
||||
@@ -63,12 +63,18 @@ the `app-config.yaml` of a Backstage installation.
|
||||
# Example:
|
||||
metadata:
|
||||
annotations:
|
||||
backstage.io/techdocs-ref: url:https://github.com/backstage/backstage/tree/master
|
||||
backstage.io/techdocs-ref: dir:.
|
||||
```
|
||||
|
||||
The value of this annotation is a location reference string (see above). If this
|
||||
annotation is specified, it is expected to point to a repository that the
|
||||
TechDocs system can read and generate docs from.
|
||||
The value of this annotation informs _where_ TechDocs source content is stored
|
||||
so that it can be read and docs can be generated from it. Most commonly, it's
|
||||
written as a path, relative to the location of the `catalog-info.yaml` itself,
|
||||
where the associated `mkdocs.yml` file can be found.
|
||||
|
||||
In unusual situations where the documentation for a catalog entity does not live
|
||||
alongside the entity's source code, the value of this annotation can point to an
|
||||
absolute URL, matching the location reference string format outlined above, for
|
||||
example: `url:https://github.com/backstage/backstage/tree/master`
|
||||
|
||||
### backstage.io/view-url, backstage.io/edit-url
|
||||
|
||||
@@ -101,18 +107,22 @@ repository itself. If the URL points to a folder, it is important that it is
|
||||
suffixed with a `'/'` in order for relative path resolution to work
|
||||
consistently.
|
||||
|
||||
### jenkins.io/github-folder
|
||||
### jenkins.io/job-full-name
|
||||
|
||||
```yaml
|
||||
# Example:
|
||||
metadata:
|
||||
annotations:
|
||||
jenkins.io/github-folder: folder-name/job-name
|
||||
jenkins.io/job-full-name: folder-name/job-name
|
||||
```
|
||||
|
||||
The value of this annotation is the path to a job on Jenkins, that builds this
|
||||
entity.
|
||||
|
||||
The value can be the format of just `[folder-path]` or
|
||||
`[instanceName]:[folder-path]`, if multiple instances are configured in
|
||||
`app-config.yaml`
|
||||
|
||||
Specifying this annotation may enable Jenkins related features in Backstage for
|
||||
that entity.
|
||||
|
||||
@@ -304,6 +314,10 @@ This annotation allowed to load the API definition from another location. Use
|
||||
[substitution](./descriptor-format.md#substitutions-in-the-descriptor-format)
|
||||
instead.
|
||||
|
||||
### jenkins.io/github-folder
|
||||
|
||||
Use the `jenkins.io/job-full-name` instead.
|
||||
|
||||
## Links
|
||||
|
||||
- [Descriptor Format: annotations](descriptor-format.md#annotations-optional)
|
||||
|
||||
@@ -33,9 +33,12 @@ scaffolder:
|
||||
|
||||
### Disabling Docker in Docker situation (Optional)
|
||||
|
||||
Software Templates use
|
||||
[Cookiecutter](https://github.com/cookiecutter/cookiecutter) as a templating
|
||||
library. By default it will use the
|
||||
Software templates use the `fetch:template` action by default, which requires no
|
||||
external dependencies and offers a
|
||||
[Cookiecutter-compatible mode](https://backstage.io/docs/features/software-templates/builtin-actions#using-cookiecuttercompat-mode).
|
||||
There is also a `fetch:cookiecutter` action, which uses
|
||||
[Cookiecutter](https://github.com/cookiecutter/cookiecutter) directly for
|
||||
templating. By default, the `fetch:cookiecutter` action will use the
|
||||
[scaffolder-backend/Cookiecutter](https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/scripts/Cookiecutter.dockerfile)
|
||||
docker image.
|
||||
|
||||
|
||||
@@ -227,6 +227,33 @@ spec:
|
||||
inputType: tel
|
||||
```
|
||||
|
||||
#### Hide or mask sensitive data on Review step
|
||||
|
||||
Sometimes, specially in custom fields, you collect some data on Create form that
|
||||
must not be shown to the user on Review step. To hide or mask this data, you can
|
||||
use `ui:widget: password` or set some properties of `ui:backstage`:
|
||||
|
||||
```yaml
|
||||
- title: Hide or mask values
|
||||
properties:
|
||||
password:
|
||||
title: Password
|
||||
type: string
|
||||
ui:widget: password # will print '******' as value for property 'password' on Review Step
|
||||
masked:
|
||||
title: Masked
|
||||
type: string
|
||||
ui:backstage:
|
||||
review:
|
||||
mask: '<some-value-to-show>' # will print '<some-value-to-show>' as value for property 'Masked' on Review Step
|
||||
hidden:
|
||||
title: Hidden
|
||||
type: string
|
||||
ui:backstage:
|
||||
review:
|
||||
show: false # wont print any info about 'hidden' property on Review Step
|
||||
```
|
||||
|
||||
#### The Repository Picker
|
||||
|
||||
So in order to make working with repository providers easier, we've built a
|
||||
|
||||
@@ -13,14 +13,29 @@ configuration options for TechDocs.
|
||||
# File: app-config.yaml
|
||||
|
||||
techdocs:
|
||||
# generators.techdocs can have two values: 'docker' or 'local'. This is to determine how to run the generator - whether to
|
||||
# spin up the techdocs-container docker image or to run mkdocs locally (assuming all the dependencies are taken care of).
|
||||
# You want to change this to 'local' if you are running Backstage using your own custom Docker setup and want to avoid running
|
||||
# into Docker in Docker situation. Read more here
|
||||
# https://backstage.io/docs/features/techdocs/getting-started#disable-docker-in-docker-situation-optional
|
||||
# techdocs.generator is used to configure how documentation sites are generated using MkDocs.
|
||||
|
||||
generators:
|
||||
techdocs: 'docker'
|
||||
generator:
|
||||
# techdocs.generator.runIn can be either 'docker' or 'local'. This is to determine how to run the generator - whether to
|
||||
# spin up the techdocs-container docker image or to run mkdocs locally (assuming all the dependencies are taken care of).
|
||||
# You want to change this to 'local' if you are running Backstage using your own custom Docker setup and want to avoid running
|
||||
# into Docker in Docker situation. Read more here
|
||||
# https://backstage.io/docs/features/techdocs/getting-started#disable-docker-in-docker-situation-optional
|
||||
|
||||
runIn: 'docker'
|
||||
|
||||
# (Optional) techdocs.generator.dockerImage can be used to control the docker image used during documentation generation. This can be useful
|
||||
# if you want to use MkDocs plugins or other packages that are not included in the default techdocs-container (spotify/techdocs).
|
||||
# NOTE: This setting is only used when techdocs.generator.runIn is set to 'docker'.
|
||||
|
||||
dockerImage: 'spotify/techdocs'
|
||||
|
||||
# (Optional) techdocs.generator.pullImage can be used to disable pulling the latest docker image by default. This can be useful when you are
|
||||
# using a custom techdocs.generator.dockerImage and you have a custom docker login requirement. For example, you need to login to
|
||||
# AWS ECR to pull the docker image.
|
||||
# NOTE: Disabling this requires the docker image was pulled by other means before running the techdocs generator.
|
||||
|
||||
pullImage: true
|
||||
|
||||
# techdocs.builder can be either 'local' or 'external.
|
||||
# If builder is set to 'local' and you open a TechDocs page, techdocs-backend will try to generate the docs, publish to storage
|
||||
|
||||
@@ -66,9 +66,7 @@ Update your component's entity description by adding the following lines to its
|
||||
```yaml
|
||||
metadata:
|
||||
annotations:
|
||||
backstage.io/techdocs-ref: url:https://github.com/org/repo
|
||||
# Or
|
||||
# backstage.io/techdocs-ref: url:https://github.com/org/repo/tree/branchName/subFolder
|
||||
backstage.io/techdocs-ref: dir:.
|
||||
```
|
||||
|
||||
The
|
||||
|
||||
@@ -35,38 +35,64 @@ In your Backstage instance's `app-config.yaml`, set `techdocs.builder` from
|
||||
`'local'` to `'external'`. By doing this, TechDocs will not try to generate
|
||||
docs. Look at [TechDocs configuration](configuration.md) for reference.
|
||||
|
||||
## How to use URL Reader in TechDocs Prepare step?
|
||||
## How to understand techdocs-ref annotation values
|
||||
|
||||
If TechDocs is configured to generate docs, it will first download the
|
||||
repository associated with the `backstage.io/techdocs-ref` annotation defined in
|
||||
the Entity's `catalog-info.yaml` file. This is also called the
|
||||
If TechDocs is configured to generate docs, it will first download source files
|
||||
based on the value of the `backstage.io/techdocs-ref` annotation defined in the
|
||||
Entity's `catalog-info.yaml` file. This is also called the
|
||||
[Prepare](./concepts.md#techdocs-preparer) step.
|
||||
|
||||
There are two kinds of preparers or two ways of downloading these source files
|
||||
We strongly recommend that the `backstage.io/techdocs-ref` annotation in each
|
||||
documented catalog entity's `catalog-info.yaml` be set to `dir:.` in almost all
|
||||
situations. This is because TechDocs is aligned with the "docs like code"
|
||||
philosophy, whereby documentation should be authored and managed alongside the
|
||||
source code of the underlying software itself.
|
||||
|
||||
- Preparer 1: Doing a `git clone` of the repository (also known as Common Git
|
||||
Preparer)
|
||||
- Preparer 2: Downloading an archive.zip or equivalent of the repository (also
|
||||
known as URL Reader)
|
||||
When you see `dir:.`, you can translate it to mean:
|
||||
|
||||
If `backstage.io/techdocs-ref` is equal to any of these -
|
||||
- That the documentation source code lives in the same location as the
|
||||
`catalog-info.yaml` file.
|
||||
- That, in particular, the `mkdocs.yml` file is a sibling of `catalog-info.yaml`
|
||||
(meaning, it is in the same directory)
|
||||
- And that all of the source content of the documentation would be available if
|
||||
one were to download the directory containing those two files (as well as all
|
||||
sub-directories).
|
||||
|
||||
1. `github:https://githubhost.com/org/repo`
|
||||
2. `gitlab:https://gitlabhost.com/org/repo`
|
||||
3. `bitbucket:https://bitbuckethost.com/project/repo`
|
||||
4. `azure/api:https://azurehost.com/org/project`
|
||||
The directory tree of the entity would look something like this:
|
||||
|
||||
Then Common Git Preparer will be used i.e. a `git clone`. But the URL Reader is
|
||||
a much faster way to do this step. Convert the `backstage.io/techdocs-ref`
|
||||
values to the following -
|
||||
```
|
||||
├── catalog-info.yaml
|
||||
├── mkdocs.yml
|
||||
└── docs
|
||||
└── index.md
|
||||
```
|
||||
|
||||
1. `url:https://githubhost.com/org/repo/tree/<branch_name>`
|
||||
2. `url:https://gitlabhost.com/org/repo/tree/<branch_name>`
|
||||
3. `url:https://bitbuckethost.com/project/repo/src/<branch_name>`
|
||||
4. `url:https://azurehost.com/organization/project/_git/repository`
|
||||
If, for example, you wanted to keep a lean root directory, you could place your
|
||||
`mkdocs.yml` file in a subdirectory and update the `backstage.io/techdocs-ref`
|
||||
annotation value accordingly, e.g. to `dir:./sub-folder`:
|
||||
|
||||
Note that you can also provide a path to a non-root directory inside the
|
||||
repository which contains the `docs/` directory.
|
||||
```
|
||||
├── catalog-info.yaml
|
||||
└── sub-folder
|
||||
├── mkdocs.yml
|
||||
└── docs
|
||||
└── index.md
|
||||
```
|
||||
|
||||
In rare situations where your TechDocs source content is managed and stored in a
|
||||
location completely separate from your `catalog-info.yaml`, you can instead
|
||||
specify a URL location reference, the exact value of which will vary based on
|
||||
the source code hosting provider. Notice that instead of the `dir:` prefix, the
|
||||
`url:` prefix is used instead. For example:
|
||||
|
||||
- **GitHub**: `url:https://githubhost.com/org/repo/tree/<branch_name>`
|
||||
- **GitLab**: `url:https://gitlabhost.com/org/repo/tree/<branch_name>`
|
||||
- **Bitbucket**: `url:https://bitbuckethost.com/project/repo/src/<branch_name>`
|
||||
- **Azure**: `url:https://azurehost.com/organization/project/_git/repository`
|
||||
|
||||
Note, just as it's possible to specify a subdirectory with the `dir:` prefix,
|
||||
you can also provide a path to a non-root directory inside the repository which
|
||||
contains the `mkdocs.yml` file and `docs/` directory.
|
||||
|
||||
e.g.
|
||||
`url:https://github.com/backstage/backstage/tree/master/plugins/techdocs-backend/examples/documented-component`
|
||||
|
||||
@@ -150,6 +150,39 @@ permissions to:
|
||||
- `s3:ListBucket` - To retrieve bucket metadata
|
||||
- `s3:GetObject` - To retrieve files from the bucket
|
||||
|
||||
> Note: If you need to migrate documentation objects from an older-style path
|
||||
> format including case-sensitive entity metadata, you will need to add some
|
||||
> additional permissions to be able to perform the migration, including:
|
||||
>
|
||||
> - `s3:PutBucketAcl` (for copying files,
|
||||
> [more info here](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectAcl.html))
|
||||
> - `s3:DeleteObject` and `s3:DeleteObjectVersion` (for deleting migrated files,
|
||||
> [more info here](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html))
|
||||
>
|
||||
> ...And you will need to ensure the permissions apply to the bucket itself, as
|
||||
> well as all resources under the bucket. See the example policy below.
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "TechDocsWithMigration",
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"s3:PutObject",
|
||||
"s3:GetObject",
|
||||
"s3:DeleteObjectVersion",
|
||||
"s3:ListBucket",
|
||||
"s3:DeleteObject",
|
||||
"s3:PutObjectAcl"
|
||||
],
|
||||
"Resource": ["arn:aws:s3:::your-bucket", "arn:aws:s3:::your-bucket/*"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**4a. (Recommended) Setup authentication the AWS way, using environment
|
||||
variables**
|
||||
|
||||
|
||||
@@ -35,7 +35,7 @@ If you want more control over the theme, and for example customize font sizes
|
||||
and margins, you can use the lower-level `createThemeOverrides` function
|
||||
exported by [@backstage/theme](https://www.npmjs.com/package/@backstage/theme)
|
||||
in combination with
|
||||
[createMuiTheme](https://material-ui.com/customization/theming/#createmuitheme-options-args-theme)
|
||||
[createTheme](https://material-ui.com/customization/theming/#createmuitheme-options-args-theme)
|
||||
from [@material-ui/core](https://www.npmjs.com/package/@material-ui/core). See
|
||||
the
|
||||
[@backstage/theme source](https://github.com/backstage/backstage/tree/master/packages/theme/src)
|
||||
|
||||
@@ -24,11 +24,9 @@ an easier path to make Pull Requests.
|
||||
Backstage provides the `@backstage/create-app` package to scaffold standalone
|
||||
instances of Backstage. You will need to have
|
||||
[Node.js](https://nodejs.org/en/download/) Active LTS Release installed
|
||||
(currently v14), [Yarn](https://classic.yarnpkg.com/en/docs/install) and
|
||||
[Python](https://www.python.org/downloads/) (although you likely have it
|
||||
already). You will also need to have
|
||||
[Docker](https://docs.docker.com/engine/install/) installed to use some features
|
||||
like Software Templates and TechDocs.
|
||||
(currently v14) and [Yarn](https://classic.yarnpkg.com/en/docs/install). You
|
||||
will also need to have [Docker](https://docs.docker.com/engine/install/)
|
||||
installed to use some features like Software Templates and TechDocs.
|
||||
|
||||
Using `npx` you can then run the following to create an app in a chosen
|
||||
subdirectory of your current working directory:
|
||||
|
||||
@@ -38,7 +38,11 @@ The target is composed of four parts:
|
||||
repositories prefixed with `service-`.
|
||||
- The path within each repository to find the catalog YAML file. This will
|
||||
usually be `/catalog-info.yaml` or a similar variation for catalog files
|
||||
stored in the root directory of each repository.
|
||||
stored in the root directory of each repository. If omitted, the default value
|
||||
`catalog-info.yaml` will be used. E.g. given that `my-project`and `service-a`
|
||||
exists, `https://bitbucket.mycompany.com/projects/my-project/repos/service-*/`
|
||||
will result in:
|
||||
`https://bitbucket.mycompany.com/projects/my-project/repos/service-a/catalog-info.yaml`.
|
||||
|
||||
## Custom repository processing
|
||||
|
||||
|
||||
+151
-82
@@ -1,116 +1,185 @@
|
||||
---
|
||||
id: roadmap
|
||||
title: Project roadmap
|
||||
description: Roadmap of Backstage Project
|
||||
title: Roadmap
|
||||
description: Roadmap of Backstage
|
||||
---
|
||||
|
||||
## Current status
|
||||
## The Backstage Roadmap
|
||||
|
||||
> Backstage is currently under rapid development. This means that you can expect
|
||||
> APIs and features to evolve. It is also recommended that teams who adopt
|
||||
> Backstage today [upgrade their installation](../cli/commands.md#versionsbump)
|
||||
> as new [releases](https://github.com/backstage/backstage/releases) become
|
||||
> available, as Backwards compatibility is not yet guaranteed.
|
||||
Backstage is currently under rapid development. This page details the project’s
|
||||
public roadmap, the result of ongoing collaboration between the core maintainers
|
||||
and the broader Backstage community. Treat the roadmap as an ever-evolving guide
|
||||
to keep us aligned as a community on:
|
||||
|
||||
## Phases
|
||||
- Upcoming enhancements and benefits,
|
||||
- Planning contributions and support,
|
||||
- Planning the project’s adoption,
|
||||
- Understanding what things are coming soon,
|
||||
- Avoiding duplication of work
|
||||
|
||||
We have divided the project into three high-level _phases_:
|
||||
### How to influence the roadmap
|
||||
|
||||
- 🐣 **Phase 1:** Extensible frontend platform (Done ✅) - You will be able to
|
||||
easily create a single consistent UI layer for your internal infrastructure
|
||||
and tools. A set of reusable
|
||||
[UX patterns and components](https://backstage.io/storybook) help ensure a
|
||||
consistent experience between tools.
|
||||
As we evolve Backstage, we want you to contribute actively in the journey to
|
||||
define the most effective developer experience in the world.
|
||||
|
||||
- 🐢 **Phase 2:** Software Catalog
|
||||
([alpha released](https://backstage.io/blog/2020/06/22/backstage-service-catalog-alpha)) -
|
||||
With a single catalog, Backstage makes it easy for a team to manage ten
|
||||
services — and makes it possible for your company to manage thousands of them.
|
||||
A roadmap is only useful if it captures real needs. If you have success stories,
|
||||
feedback, or ideas, we want to hear from you! If you plan to work (or are
|
||||
already working) on a new or existing feature, please let us know, so that we
|
||||
can update the roadmap accordingly. We are also happy to share knowledge and
|
||||
context that will help your feature land successfully.
|
||||
|
||||
- 🐇 **Phase 3:** Ecosystem (ongoing, see
|
||||
[Plugin Marketplace](https://backstage.io/plugins)) - Everyone's
|
||||
infrastructure stack is different. By fostering a vibrant community of
|
||||
contributors we hope to provide an ecosystem of Open Source
|
||||
plugins/integrations that allows you to pick the tools that match your stack.
|
||||
|
||||
## Detailed roadmap
|
||||
|
||||
If you have questions about the roadmap or want to provide feedback, we would
|
||||
love to hear from you! Please create an
|
||||
[Issue](https://github.com/backstage/backstage/issues/new/choose), ping us on
|
||||
[Discord](https://discord.gg/EBHEGzX) or reach out directly at
|
||||
[backstage-interest@spotify.com](mailto:backstage-interest@spotify.com).
|
||||
|
||||
Want to help out? Awesome ❤️ Head over to
|
||||
You can also head over to the
|
||||
[CONTRIBUTING](https://github.com/backstage/backstage/blob/master/CONTRIBUTING.md)
|
||||
guidelines to get started.
|
||||
|
||||
### Ongoing work 🚧
|
||||
If you have specific questions about the roadmap, please create an
|
||||
[issue](https://github.com/backstage/backstage/issues/new/choose), ping us on
|
||||
[Discord](https://discord.gg/awD6SxgQ), or
|
||||
[book time](http://calendly.com/spotify-backstage) with the Spotify team.
|
||||
|
||||
- **[Platform stabilization](https://github.com/backstage/backstage/milestone/19)** -
|
||||
Stabilize the core of Backstage, including its core features, so that the
|
||||
platform can be depended on for production use. After this, plugins will
|
||||
require little-to-no maintenance.
|
||||
### How to read the roadmap
|
||||
|
||||
- **[Kubernetes plugin for service owners](https://github.com/backstage/backstage/issues/2857)** -
|
||||
Improve native support for Kubernetes, making it easier for service owners to
|
||||
see and manage their services running in K8s, regardless if that's locally, in
|
||||
AWS, GCS, Azure, or elsewhere.
|
||||
The Backstage roadmap lays out both [“what’s next”](#whats-next) and
|
||||
[“future work”](#future-work). With "next" we mean features planned for release
|
||||
within the ongoing quarter starting in July until September 2021 included. With
|
||||
"future" we mean features in the radar, but not yet scheduled.
|
||||
|
||||
- **[Search platform](../features/search/README.md)** - Evolve the basic search
|
||||
functionality currently available into a platform that **a)** enables search
|
||||
across the software catalog, TechDocs, and any other information exposed by
|
||||
plugins, and **b)** supports a variety of search engine technologies.
|
||||
The long-term roadmap (12 - 36 months) is not detailed in the public roadmap.
|
||||
Third-party contributions are also not currently included in the roadmap. Let us
|
||||
know about any ongoing developments and we’re happy to include it here as well.
|
||||
|
||||
- **[Software Templates V2](https://github.com/backstage/backstage/issues/2771)** -
|
||||
Expand the templates to make the steps more composable by adding the ability
|
||||
to add more steps for custom logic, including webhooks and using authorization
|
||||
from integrations.
|
||||
### Roadmap evolution
|
||||
|
||||
### Future work 🔮
|
||||
Will this roadmap change? Obviously!
|
||||
|
||||
- **Golden Path for Plugin Development** - Create an easy, standardized way for
|
||||
developers to build plugins that will encourage contributions and lead to a
|
||||
richer ecosystem for everyone.
|
||||
Roadmap are always evolving and ours is no different; you can expect to see this
|
||||
updated roughly every month.
|
||||
|
||||
- **[GraphQL API](https://github.com/backstage/backstage/milestone/13)** - A
|
||||
GraphQL API will open up the rich metadata provided by Backstage in a single
|
||||
query. Plugins can easily query this API as well as extend the model where
|
||||
needed.
|
||||
## What’s next
|
||||
|
||||
- **Inter-Plugin Communication** - **[Under consideration]** Establish more
|
||||
clearly defined patterns for plugins to communicate.
|
||||
The feature set below is planned for the ongoing quarter, and grouped by theme.
|
||||
The list order doesn’t necessarily reflect priority, and the development/release
|
||||
cycle will vary based on maintainer schedules.
|
||||
|
||||
- **Improved Access Control** - **[Under consideration]** Provide finer grained
|
||||
access controls and management for better control of the platform user
|
||||
experience.
|
||||
### Backstage Core
|
||||
|
||||
### Plugins
|
||||
The following features are planned for release:
|
||||
|
||||
Building and maintaining [plugins](https://backstage.io/plugins) is the work of
|
||||
the entire Backstage community.
|
||||
- **Composable homepage:** We’re seeing lots of interest from the community in
|
||||
reusable components to build a homepage experience where users can easily
|
||||
surface what they might find useful to start their tasks. Check out the
|
||||
[milestone](https://github.com/backstage/backstage/milestone/34) for further
|
||||
details.
|
||||
- **Improved responsiveness:** Check out the
|
||||
[RFC here](https://github.com/backstage/backstage/issues/6318) for further
|
||||
details on how to improve the responsiveness for Backstage's UI.
|
||||
|
||||
A list of plugins that are in development is
|
||||
[available here](https://github.com/backstage/backstage/issues?q=is%3Aissue+is%3Aopen+label%3Aplugin+sort%3Areactions-%2B1-desc).
|
||||
We strongly recommend to upvote 👍 plugins you are interested in. This helps us
|
||||
and the community prioritize what plugins to build.
|
||||
### Software Templates
|
||||
|
||||
Are you missing a plugin for your favorite tool? Please
|
||||
[suggest a new one](https://github.com/backstage/backstage/issues/new?labels=plugin&template=plugin_template.md&title=%5BPlugin%5D+THE+PLUGIN+NAME).
|
||||
Chances are that someone will jump in and help build it.
|
||||
The following features are planned for release::
|
||||
|
||||
### Community Initiatives 🧑🤝🧑
|
||||
- **Re-creation/resubmission in case of failure:** Speed up productivity by
|
||||
allowing developers to relaunch a project after a failure or any unexpected
|
||||
problem. In the current version, this task requires retyping and a full
|
||||
re-creation from scratch.
|
||||
- **Performance and usability improvements for contributors:** Reach a relevant
|
||||
improvement in templating's performance through the replacement of
|
||||
[handlebars](https://handlebarsjs.com/). Other replacements will be considered
|
||||
as part of this task (possibly
|
||||
[cookiecutter](https://cookiecutter.readthedocs.io/)) for easier software
|
||||
template creation, allowing more contributors to reach their goals without
|
||||
having to learn new tooling.
|
||||
- **Improved extensibility through inclusion:** Make software templates more
|
||||
maintainable and extensible by adding `$include` support for parameters.
|
||||
- **Authenticated job creation:** Created jobs will be able to run with an
|
||||
authenticated user with all actions tracked for future consumption and
|
||||
evidence. Track users creating jobs and make “jobs created by me” reporting
|
||||
available.
|
||||
|
||||
- [**Backstage Community Sessions**](https://github.com/backstage/community#meetups) -
|
||||
A monthly meetup for the community to come together to share and learn about
|
||||
the latest happenings in Backstage.
|
||||
### Software Catalog
|
||||
|
||||
- **Backstage Hackathons** - (Coming soon) Open to everyone in our Backstage
|
||||
community, a celebration of you, the project and building awesome things
|
||||
together
|
||||
The following features are planned for release:
|
||||
|
||||
### Completed milestones ✅
|
||||
- **Request For Comments (RFC) for composability improvements (routing):**
|
||||
Enable plugins to be auto-added and make plugin installation and upgrades
|
||||
easier for all Backstage users. This includes information card layouts, entity
|
||||
pages containing content and hooking the external header, considering the
|
||||
support of a separate deployment, and configuration for plugins.
|
||||
- **Removing duplicated entities in catalog:** As any adopter knows, a software
|
||||
catalog can contain thousands or more entities and it is very important to
|
||||
avoid duplications in naming to prevent failures. With this development task,
|
||||
two entities with the same name won't be allowed as described
|
||||
[here](https://github.com/backstage/backstage/issues/4760).
|
||||
- **Connecting identity to ownership to prepare for role-based access control
|
||||
([RBAC](https://en.wikipedia.org/wiki/Role-based_access_control)):** This is a
|
||||
first step to supporting RBAC for the software catalog (see the
|
||||
[future work section](#future-work) for further details). Provide each entity
|
||||
within the software catalog with a recognized owner.
|
||||
- **Catalog performance improvements through improved caching:** Fix the
|
||||
performance gaps in the catalog processor, which currently doesn’t have a
|
||||
strong caching mechanism. The current version often requires fetching a
|
||||
relevant amount of data, especially at scale.
|
||||
|
||||
### Search
|
||||
|
||||
The following features are planned for release:
|
||||
|
||||
- ElasticSearch integration: Add ElasticSearch to the Search Platform as the
|
||||
underlying search engine. Check out the
|
||||
[milestone here](https://github.com/backstage/backstage/milestone/27) for
|
||||
further details.
|
||||
|
||||
### TechDocs
|
||||
|
||||
The following features are planned for release:
|
||||
|
||||
- **TechDocs beta release:** Fix remaining bugs to get TechDocs to Beta. Check
|
||||
out the [milestone here](https://github.com/backstage/backstage/milestone/29)
|
||||
for further details.
|
||||
|
||||
## Future work
|
||||
|
||||
The following feature list doesn’t represent a commitment to develop and the
|
||||
list order doesn’t reflect any priority or importance. But these features are on
|
||||
the maintainers’ radar, with clear interest expressed by the community.
|
||||
|
||||
- **Improved UX design:** Provide a better Backstage user experience through
|
||||
visual guidelines and templates, especially navigation across plug-ins and
|
||||
portal functionalities.
|
||||
- **Catalog composability (routing):** Follow up development after the RFC
|
||||
planned for the ongoing quarter (see [what’s next](#whats-next) for further
|
||||
details).
|
||||
- **Catalog-import improvements:** Provide a faster (scalability) and better
|
||||
(more features like move/rename) way to import entities into the Software
|
||||
Catalog. Importing items in the Software Catalog is crucial for creating a
|
||||
Backstage proof-of-concept or testing/planning for broader organizational
|
||||
adoption. This enhancement better supports getting developers to use Backstage
|
||||
with less effort and customization.
|
||||
- **Catalog improvements:** Add pagination and sourcing to Software Catalog.
|
||||
- **[GraphQL](https://graphql.org/) support:** Introduce the ability to query
|
||||
Backstage backend services with a standard query language for APIs.
|
||||
- **Software templates performance improvements through decoupling a separate
|
||||
worker:** Improve performance through decoupling resource-consuming services
|
||||
and making them asynchronous. In the current version, project auto-creation
|
||||
through the Software Templating system can consume a lot of resources and
|
||||
bottleneck many concurrent projects created simultaneously.
|
||||
- **API discovery and documentation:** Add better support for the
|
||||
[gRPC](https://grpc.io/).
|
||||
- **Adding TechDocs search to the Search Platform:** Having this capability in
|
||||
place will provide a better and new major version of the Search Platform
|
||||
(v3.0). You can refer to the
|
||||
[milestone here](https://github.com/backstage/backstage/milestone/28) for
|
||||
further details.
|
||||
- **TechDocs GA release:** Work toward enhancements necessary to get TechDocs to
|
||||
general availability. Check out the
|
||||
[milestone here](https://github.com/backstage/backstage/milestone/30) for
|
||||
further details.
|
||||
|
||||
## Completed milestones
|
||||
|
||||
Read more about the completed (and released) features for reference.
|
||||
|
||||
- [[Search] Out-of-the-Box Implementation (Alpha)](https://github.com/backstage/backstage/milestone/26)
|
||||
- [Deploy a product demo at `demo.backstage.io`](https://demo.backstage.io)
|
||||
- [Kubernetes plugin - v1](https://github.com/backstage/backstage/tree/master/plugins/kubernetes)
|
||||
- [Helm charts](https://github.com/backstage/backstage/tree/master/contrib/chart/backstage)
|
||||
|
||||
@@ -99,15 +99,14 @@ You may tweak these components, rename them and/or replace them completely.
|
||||
|
||||
## Connecting the plugin to the Backstage app
|
||||
|
||||
There are three things needed for a Backstage app to start making use of a
|
||||
plugin.
|
||||
There are two things needed for a Backstage app to start making use of a plugin.
|
||||
|
||||
1. Add plugin as dependency in `app/package.json`
|
||||
2. Import and use one or more plugin extensions, for example in
|
||||
`app/src/App.tsx`.
|
||||
|
||||
Luckily these three steps happen automatically when you create a plugin with the
|
||||
Backstage CLI.
|
||||
Luckily both of these steps happen automatically when you create a plugin with
|
||||
the Backstage CLI.
|
||||
|
||||
## Talking to the outside world
|
||||
|
||||
|
||||
Reference in New Issue
Block a user