Merge branch 'backstage:master' into 7340-remote-config-fix

This commit is contained in:
Praveen Ranjan Keshri
2021-12-07 22:46:46 +05:30
committed by GitHub
593 changed files with 26063 additions and 2542 deletions
+1 -1
View File
@@ -291,7 +291,7 @@ The figure below shows the relationship between
<span style="color: #b85450">fooApiRef</span>.
<div style="text-align:center">
<img src="../assets/utility-apis-fig1.svg" alt="Figure showing the relationship between utility APIs, the apps that provide them, and the plugins that consume them">
<img src="../assets/utility-apis-fig1.svg" alt="Figure showing the relationship between utility APIs, the apps that provide them, and the plugins that consume them" />
</div>
The current method for connecting Utility API providers and consumers is via the
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 64 KiB

+2
View File
@@ -16,8 +16,10 @@ Backstage identity information in your app or plugins.
Backstage comes with many common authentication providers in the core library:
- [Atlassian](atlassian/provider.md)
- [Auth0](auth0/provider.md)
- [Azure](microsoft/provider.md)
- [Bitbucket](bitbucket/provider.md)
- [GitHub](github/provider.md)
- [GitLab](gitlab/provider.md)
- [Google](google/provider.md)
+3 -3
View File
@@ -7,9 +7,9 @@ description: How to build a Backstage Docker image for deployment
This section describes how to build a Backstage App into a deployable Docker
image. It is split into three sections, first covering the host build approach,
which is recommended due its speed and more efficient and often simpler caching.
The second section covers a full multi-stage Docker build, and the last section
covers how to deploy the frontend and backend as separate images.
which is recommended due to its speed and more efficient and often simpler
caching. The second section covers a full multi-stage Docker build, and the last
section covers how to deploy the frontend and backend as separate images.
Something that goes for all of these docker deployment strategies is that they
are stateless, so for a production deployment you will want to set up and
+36 -4
View File
@@ -26,6 +26,7 @@ kubernetes:
name: minikube
authProvider: 'serviceAccount'
skipTLSVerify: false
skipMetricsLookup: true
serviceAccountToken: ${K8S_MINIKUBE_TOKEN}
dashboardUrl: http://127.0.0.1:64713 # url copied from running the command: minikube service kubernetes-dashboard -n kubernetes-dashboard
dashboardApp: standard
@@ -37,6 +38,7 @@ kubernetes:
projectId: 'gke-clusters'
region: 'europe-west1'
skipTLSVerify: true
skipMetricsLookup: true
```
### `serviceLocatorMethod`
@@ -86,8 +88,13 @@ cluster. Valid values are:
##### `clusters.\*.skipTLSVerify`
This determines whether or not the Kubernetes client verifies the TLS
certificate presented by the API server. Defaults to `false`.
This determines whether the Kubernetes client verifies the TLS certificate
presented by the API server. Defaults to `false`.
##### `clusters.\*.skipMetricsLookup`
This determines whether the Kubernetes client looks up resource metrics
CPU/Memory for pods returned by the API server. Defaults to `false`.
##### `clusters.\*.serviceAccountToken` (optional)
@@ -188,8 +195,13 @@ regions.
##### `skipTLSVerify`
This determines whether or not the Kubernetes client verifies the TLS
certificate presented by the API server. Defaults to `false`.
This determines whether the Kubernetes client verifies the TLS certificate
presented by the API server. Defaults to `false`.
##### `skipMetricsLookup`
This determines whether the Kubernetes client looks up resource metrics
CPU/Memory for pods returned by the API server. Defaults to `false`.
### `customResources` (optional)
@@ -219,6 +231,26 @@ The custom resource's apiVersion.
The plural representing the custom resource.
### `apiVersionOverrides` (optional)
Overrides for the API versions used to make requests for the corresponding
objects. If using a legacy Kubernetes version, you may use this config to
override the default API versions to ones that are supported by your cluster.
Example:
```yaml
---
kubernetes:
apiVersionOverrides:
cronjobs: 'v1beta1'
```
For more information on which API versions are supported by your cluster, please
view the Kubernetes API docs for your Kubernetes version (e.g.
[API Groups for v1.22](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#-strong-api-groups-strong-)
)
### Role Based Access Control
The current RBAC permissions required are read-only cluster wide, for the
+17 -5
View File
@@ -114,13 +114,15 @@ const routes = (
In `Root.tsx`, add the `SidebarSearchModal` component:
```bash
import { SidebarSearchModal } from '@backstage/plugin-search';
import { SidebarSearchModal, SearchContextProvider } from '@backstage/plugin-search';
export const Root = ({ children }: PropsWithChildren<{}>) => (
<SidebarPage>
<Sidebar>
<SidebarLogo />
<SidebarSearchModal />
<SearchContextProvider>
<SidebarSearchModal />
</SearchContextProvider>
<SidebarDivider />
...
```
@@ -154,13 +156,17 @@ import { DefaultCatalogCollator } from '@backstage/plugin-catalog-backend';
export default async function createPlugin({
logger,
discovery,
tokenManager,
}: PluginEnvironment) {
const searchEngine = new LunrSearchEngine({ logger });
const indexBuilder = new IndexBuilder({ logger, searchEngine });
indexBuilder.addCollator({
defaultRefreshIntervalSeconds: 600,
collator: new DefaultCatalogCollator({ discovery }),
collator: new DefaultCatalogCollator({
discovery,
tokenManager,
}),
});
const { scheduler } = await indexBuilder.build();
@@ -285,7 +291,10 @@ const indexBuilder = new IndexBuilder({ logger, searchEngine });
indexBuilder.addCollator({
defaultRefreshIntervalSeconds: 600,
collator: new DefaultCatalogCollator({ discovery }),
collator: new DefaultCatalogCollator({
discovery,
tokenManager,
}),
});
indexBuilder.addCollator({
@@ -303,6 +312,9 @@ its `defaultRefreshIntervalSeconds` value, like this:
```typescript {3}
indexBuilder.addCollator({
defaultRefreshIntervalSeconds: 600,
collator: new DefaultCatalogCollator({ discovery }),
collator: new DefaultCatalogCollator({
discovery,
tokenManager,
}),
});
```
+2 -1
View File
@@ -2,7 +2,7 @@
id: how-to-guides
title: Search "HOW TO" guides
sidebar_label: "HOW TO" guides
description: Search "HOW TO" guides
description: Search "HOW TO" guides
---
## How to implement your own Search API
@@ -74,6 +74,7 @@ indexBuilder.addCollator({
collator: DefaultTechDocsCollator.fromConfig(config, {
discovery,
logger,
tokenManager,
}),
});
```
+3 -3
View File
@@ -4,9 +4,9 @@ title: Search Engines
description: Choosing and configuring your search engine for Backstage
---
Backstage supports 2 search engines by default, an in-memory engine called Lunr
and ElasticSearch. You can configure your own search engines by implementing the
provided interface as mentioned in the
Backstage supports 3 search engines by default, an in-memory engine called Lunr,
ElasticSearch and Postgres. You can configure your own search engines by
implementing the provided interface as mentioned in the
[search backend documentation.](./getting-started.md#Backend)
Provided search engine implementations have their own way of constructing
@@ -18,17 +18,31 @@ The next step is to add
[add templates](http://backstage.io/docs/features/software-templates/adding-templates)
to your Backstage app.
### GitHub
### Publishing defaults
For GitHub, you can configure who can see the new repositories that are created
by specifying `visibility` option. Valid options are `public`, `private` and
`internal`. The `internal` option is for GitHub Enterprise clients, which means
public within the enterprise.
Software templates can define _publish_ actions, such as `publish:github`, to
create new repositories or submit pull / merge requests to existing
repositories. You can configure the author and commit message through the
`scaffolder` configuration in `app-config.yaml`:
```yaml
scaffolder:
github:
visibility: public # or 'internal' or 'private'
defaultAuthor:
name: M.C. Hammer # Defaults to `Scaffolder`
email: hammer@donthurtem.com # Defaults to `scaffolder@backstage.io`
defaultCommitMessage: "U can't touch this" # Defaults to 'Initial commit'
```
To configure who can see the new repositories created from software templates,
add the `repoVisibility` key within a software template:
```yaml
- id: publish
name: Publish
action: publish:github
input:
repoUrl: '{{ parameters.repoUrl }}'
repoVisibility: public # or 'internal' or 'private'
```
### Disabling Docker in Docker situation (Optional)
@@ -53,3 +67,30 @@ You can do so by including the following lines in the last step of your
RUN apt-get update && apt-get install -y python3 python3-pip
RUN pip3 install cookiecutter
```
### Customizing the ScaffolderPage with Grouping and Filtering
Once you have more than a few software templates you may want to customize your
`ScaffolderPage` by grouping and surfacing certain templates together. You can
accomplish this by creating `groups` and passing them to your `ScaffolderPage`
like below
```
<ScaffolderPage
groups={[
{
title: "Recommended",
filter: entity =>
entity?.metadata?.tags?.includes('recommended') ?? false,
},
]}
/>
```
This code will group all templates with the 'recommended' tag together at the
top of the page above any other templates not filtered by this group or others.
You can also further customize groups by passing in a `titleComponent` instead
of a `title` which will be a component to use as the header instead of just the
default `ContentHeader` with the `title` set as it's value.
![Grouped Templates](../../assets/software-templates/grouped-templates.png)
+1 -1
View File
@@ -12,7 +12,7 @@ code, template in some variables, and then publish the template to some
locations like GitHub or GitLab.
<video width="100%" height="100%" controls>
<source src="https://backstage.io/blog/assets/2020-08-05/feature.mp4" type="video/mp4">
<source src="https://backstage.io/blog/assets/2020-08-05/feature.mp4" type="video/mp4" />
</video>
### Getting Started
+7 -7
View File
@@ -40,7 +40,7 @@ storage system (e.g. AWS S3, GCS or Azure Blob Storage). Read more in
## Recommended deployment
This is how we recommend deploying TechDocs in production environment.
This is how we recommend deploying TechDocs in a production environment.
<img data-zoomable src="../../assets/techdocs/architecture-recommended.drawio.svg" alt="TechDocs Architecture diagram" />
@@ -58,12 +58,12 @@ Similar to how it is done in the Basic setup, the TechDocs Reader requests
your configured storage solution for the necessary files and returns them to
TechDocs Reader.
Note about caching: We have noticed internally that some storage providers can
be quite slow, which is why we are recommending a cache that sits between the
TechDocs Reader and the Storage.
_Feel free to suggest better ideas to us in #docs-like-code channel in Discord
or via a GitHub issue._
Depending on your chosen cloud storage provider and its real-world proximity to
your backend server, there may be a comparably high amount of latency when
loading TechDocs sites using this deployment approach. If you encounter this,
you can optionally configure the `techdocs-backend` to cache responses in a
cache store
[supported by Backstage](../../overview/architecture-overview.md#cache).
### Security consideration
+16
View File
@@ -135,6 +135,22 @@ techdocs:
# the old, case-sensitive entity triplet behavior.
legacyUseCaseSensitiveTripletPaths: false
# techdocs.cache is optional, and is only recommended when you've configured
# an external techdocs.publisher.type above. Also requires backend.cache to
# be configured with a valid cache store.
cache:
# Represents the number of milliseconds a statically built asset should
# stay cached. Cache invalidation is handled automatically by the frontend,
# which compares the build times in cached metadata vs. canonical storage,
# allowing long TTLs (e.g. 1 month/year)
ttl: 3600000
# (Optional) The time (in milliseconds) that the TechDocs backend will wait
# for a cache service to respond before continuing on as though the cached
# object was not found (e.g. when the cache sercice is unavailable). The
# default value is 1000
readTimeout: 500
# (Optional and Legacy) TechDocs makes API calls to techdocs-backend using this URL. e.g. get docs of an entity, get metadata, etc.
# You don't have to specify this anymore.
+1 -1
View File
@@ -36,7 +36,7 @@ This will create a new Backstage App inside the current folder. The name of the
app-folder is the name that was provided when prompted.
<p align='center'>
<img src='../assets/getting-started/create-app_output.png' width='600' alt='create app'>
<img src='../assets/getting-started/create-app_output.png' width='600' alt='create app' />
</p>
Inside that directory, it will generate all the files and folder structure
+20 -17
View File
@@ -23,7 +23,7 @@ guide to do a repository-based installation.
- Access to a Linux-based operating system, such as Linux, MacOS or
[Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/)
- An account with elevated rights
- An account with elevated rights to install the dependencies
- `curl` or `wget` installed
- Node.js Active LTS Release installed (currently v14) using one of these
methods:
@@ -36,15 +36,16 @@ guide to do a repository-based installation.
- `yarn` [Installation](https://classic.yarnpkg.com/en/docs/install)
- `docker` [installation](https://docs.docker.com/engine/install/)
- `git` [installation](https://github.com/git-guides/install-git)
- If the system is not directly accessible over your network, the following
ports need to be opened: 3000, 7007
- If the system is not directly accessible over your network the following ports
need to be opened: 3000, 7007. This is quite uncommon, unless when you're
installing in a container, VM or remote system.
### Create your Backstage App
To install the Backstage Standalone app, we make use of `npx`, a tool to run
Node executables straight from the registry. Running the command below will
install Backstage. The wizard will create a subdirectory inside your current
working directory.
Node executables straight from the registry. This tool is part of your Node.js
installation. Running the command below will install Backstage. The wizard will
create a subdirectory inside your current working directory.
```bash
npx @backstage/create-app
@@ -57,7 +58,7 @@ The wizard will ask you
SQLite option.
<p align='center'>
<img src='../assets/getting-started/wizard.png' alt='Screenshot of the wizard asking for a name for the app, and a selection menu for the database.'>
<img src='../assets/getting-started/wizard.png' alt='Screenshot of the wizard asking for a name for the app, and a selection menu for the database.' />
</p>
### Run the Backstage app
@@ -72,18 +73,27 @@ yarn dev
```
<p align='center'>
<img src='../assets/getting-started/startup.png' alt='Screenshot of the command output, with the message web pack compiled successfully.'>
<img src='../assets/getting-started/startup.png' alt='Screenshot of the command output, with the message web pack compiled successfully.'/>
</p>
It might take a little while, but as soon as the message
`[0] webpack compiled successfully` appears, you can open a browser and directly
navigate to your freshly installed Backstage portal at `http://localhost:3000`.
You can start exploring the demo immediately.
You can start exploring the demo immediately. Please note that the in-memory
database will be cleared when you restart the app, so you'll most likely want to
carry on with the database steps.
<p align='center'>
<img src='../assets/getting-started/portal.png' alt='Screenshot of the Backstage portal.'>
<img src='../assets/getting-started/portal.png' alt='Screenshot of the Backstage portal.'/>
</p>
The most common next steps are to move to a persistent database, configure
authentication, and add a plugin:
- [Switching from SQLite to PostgresQL](https://backstage.io/docs/tutorials/switching-sqlite-postgres)
- [Setting up Authentication](https://backstage.io/docs/auth/)
- [Adding a plugin](https://backstage.io/docs/getting-started/configure-app-with-plugins)
Congratulations! That should be it. Let us know how it went:
[on discord](https://discord.gg/EBHEGzX), file issues for any
[feature](https://github.com/backstage/backstage/issues/new?labels=help+wanted&template=feature_template.md)
@@ -93,10 +103,3 @@ or
[bugs](https://github.com/backstage/backstage/issues/new?labels=bug&template=bug_template.md)
you have, and feel free to
[contribute](https://github.com/backstage/backstage/blob/master/CONTRIBUTING.md)!
The most common next steps are to configure Backstage, add a plugin and moving
to a more persistent database:
- [Setting up Authentication](https://backstage.io/docs/auth/)
- [Switching from SQLite to PostgresQL](https://backstage.io/docs/tutorials/switching-sqlite-postgres)
- [Adding a plugin](https://backstage.io/docs/getting-started/configure-app-with-plugins)
+1 -5
View File
@@ -166,13 +166,9 @@ are separated out into their own folder, see further down.
plugin and [techdocs-cli](https://github.com/backstage/techdocs-cli).
- [`test-utils/`](https://github.com/backstage/backstage/tree/master/packages/test-utils) -
This package contains more general purpose testing facilities for testing a
This package contains general purpose testing facilities for testing a
Backstage App or its plugins.
- [`test-utils-core/`](https://github.com/backstage/backstage/tree/master/packages/test-utils-core) -
This package contains specific testing facilities used when testing Backstage
core internals.
- [`theme/`](https://github.com/backstage/backstage/tree/master/packages/theme) -
Holds the Backstage Theme.
+2
View File
@@ -165,6 +165,7 @@ common example being the `migrations` directory.
Usage: backstage-cli backend:build [options]
Options:
--minify Minify the generated code
-h, --help display help for command
```
@@ -371,6 +372,7 @@ the monorepo.
Usage: backstage-cli plugin:build [options]
Options:
--minify Minify the generated code
-h, --help display help for command
```
+1 -10
View File
@@ -27,7 +27,7 @@ point building on top of the previous one:
and the new APIs can be used in parallel. This deprecation must have been
released for at least two weeks before the deprecated API is removed in a
minor version bump.
- **3** - The time limit for the deprecation is 3 months instead of two days.
- **3** - The time limit for the deprecation is 3 months instead of two weeks.
TL;DR:
@@ -175,15 +175,6 @@ Utilities for writing tests for Backstage plugins and apps.
Stability: `2`
### `test-utils-core` [GitHub](https://github.com/backstage/backstage/tree/master/packages/test-utils-core/)
Internal testing utilities that are separated out for usage in
@backstage/core-app-api and @backstage/core-plugin-api. All exports are
re-exported by @backstage/test-utils. This package should not be depended on
directly.
Stability: See @backstage/test-utils
### `theme` [GitHub](https://github.com/backstage/backstage/tree/master/packages/theme/)
The core Backstage MUI theme along with customization utilities.
+68
View File
@@ -0,0 +1,68 @@
---
id: backend-to-backend-auth
title: Backend-to-Backend Authentication
description:
Guide for authenticating API requests between Backstage plugin backends
---
This tutorial describes the steps needed to handle _backend-to-backend
authentication_, which allows plugin backends to determine whether a given
request originates from a legitimate Backstage backend by verifying a token
signed with a shared secret. This system has limited use for now, but will be
needed to support the upcoming framework for permissions and authorization (see
[the PRFC on the topic](https://github.com/backstage/backstage/pull/7761) for
more details).
Backends have no concept of a Backstage identity, so instead they use a token
generated using a shared key stored in config. You can generate a unique key for
your app in a terminal, and set the `BACKEND_SECRET` environment variable to the
resulting value.
```bash
node -p 'require("crypto").randomBytes(24).toString("base64")'
```
Requests originating from a backend plugin can be authenticated by decorating
them with a backend token. Backend tokens can be generated using a
`TokenManager`, which can be passed to plugin backends via the
`PluginEnvironment`. The `TokenManager` provided in new Backstage instances
generated by `create-app` is a stub, which returns empty tokens and accepts any
input string as valid. To enable backend-to-backend authentication, you'll need
to instantiate a new one using the secret from your config instead:
```diff
// packages/backend/src/index.ts
function makeCreateEnv(config: Config) {
const root = getRootLogger();
const reader = UrlReaders.default({ logger: root, config });
const discovery = SingleHostDiscovery.fromConfig(config);
root.info(`Created UrlReader ${reader}`);
const cacheManager = CacheManager.fromConfig(config);
const databaseManager = DatabaseManager.fromConfig(config);
- const tokenManager = ServerTokenManager.noop();
+ const tokenManager = ServerTokenManager.fromConfig(config);
```
With this `tokenManager`, you can then generate a server token for requests:
```typescript
const { token } = await this.tokenManager.getToken();
const response = await fetch(pluginBackendApiUrl, {
method: 'GET',
headers: {
...headers,
Authorization: `Bearer ${token}`,
},
});
```
You can use the same `tokenManager` to authenticate tokens supplied on incoming
requests:
```typescript
await tokenManager.authenticate(token); // throws if token is invalid
```