Merge remote-tracking branch 'upstream/master' into kaos/cli-config-schema

This commit is contained in:
Andreas Stenius
2021-02-24 08:56:58 +01:00
377 changed files with 10032 additions and 3155 deletions
+19 -1
View File
@@ -181,7 +181,25 @@ The `IgnoringErrorApi` would then be imported in the app, and wired up like
this:
```ts
builder.add(errorApiRef, new IgnoringErrorApi());
const app = createApp({
apis: [
/* ApiFactories */
createApiFactory(errorApiRef, new IgnoringErrorApi()),
// OR
// If your API has dependencies, you use the object form
createApiFactory({
api: errorApiRef,
deps: { configApi: configApiRef },
factory({ configApi }) {
return new IgnoringErrorApi({
reportingUrl: configApi.getString('error.reportingUrl'),
});
},
}),
],
// ... other options
});
```
Note that the above line will cause an error if `IgnoreErrorApi` does not fully
+3 -23
View File
@@ -24,7 +24,6 @@ Below is a cleaned up output of `yarn backstage-cli --help`.
```text
app:build Build an app for a production release
app:diff Diff an existing app with the creation template
app:serve Serve an app for local development
backend:build Build a backend plugin
@@ -113,25 +112,6 @@ Options:
-h, --help display help for command
```
## app:diff
Scope: `app`
Diff an existing app with the template used in `@backstage/create-app`. This
will verify that your app package has not diverged from the template, and can be
useful to run after updating the version of `@backstage/cli` in your app.
This command is experimental and may be removed in the future.
```text
Usage: backstage-cli app:diff
Options:
--check Fail if changes are required
--yes Apply all changes
-h, --help display help for command
```
## app:serve
Scope: `app`
@@ -207,15 +187,15 @@ The following is an example of a `Dockerfile` that can be used to package the
output of `backstage-cli backend:bundle` into an image:
```Dockerfile
FROM node:14-buster
FROM node:14-buster-slim
WORKDIR /app
ADD yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./
RUN yarn install --production --network-timeout 600000 && rm -rf "$(yarn cache dir)"
RUN yarn install --frozen-lockfile --production --network-timeout 300000 && rm -rf "$(yarn cache dir)"
ADD packages/backend/dist/bundle.tar.gz app-config.yaml ./
CMD node packages/backend
CMD ["node", "packages/backend"]
```
```text
+9 -6
View File
@@ -33,15 +33,18 @@ our users.
### Transparent
There are a lot of exciting things coming up and we want to keep you in the
loop! Keep an eye on our Milestones in GitHub to see where were headed. Well
also be posting updates in the _#design_ channel on
[Discord](https://discord.gg/EBHEGzX). Not only that, we want to keep you
informed on the decisions weve made and why weve made them.
loop! Keep an eye on our
[Milestones in GitHub](https://github.com/backstage/backstage/milestones) to see
where we're headed and review the
[open design issues](https://github.com/backstage/backstage/issues?q=is%3Aopen+is%3Aissue+label%3Adesign),
to see if you can help. We'll also be posting updates in the _#design_ channel
on [Discord](https://discord.gg/EBHEGzX). Not only that, we want to keep you
informed on the decisions we've made and why we've made them.
## 🛠 Our Practice
The chart below details how we work. **_Stay tuned_**: We are currently in the
process of securing a Figma workspace for Backstage Open Source, and we plan on
The chart below details how we work. We have a
[Figma workspace for Backstage Open Source](figma.md), and we plan on
referencing Figma documents to share specs and prototypes with the community.
### Creating a New Design Component
+45 -19
View File
@@ -30,33 +30,59 @@ it doesn't.
Add the following entry to the head of your `packages/app/src/plugins.ts`:
```ts
export { plugin as CatalogPlugin } from '@backstage/plugin-catalog';
export { catalogPlugin } from '@backstage/plugin-catalog';
```
Add the following to your `packages/app/src/apis.ts`:
Next we need to install the two pages that the catalog plugin provides. You can
choose any name for these routes, but we recommend the following:
```tsx
import {
catalogPlugin,
CatalogIndexPage,
CatalogEntityPage,
} from '@backstage/plugin-catalog';
// Add to the top-level routes, directly within <FlatRoutes>
<Route path="/catalog" element={<CatalogIndexPage />} />
<Route path="/catalog/:namespace/:kind/:name" element={<CatalogEntityPage />}>
{/*
This is the root of the custom entity pages for your app, refer to the example app
in the main repo or the output of @backstage/create-app for an example
*/}
<EntityPage />
</Route>
```
The catalog plugin also has one external route that needs to be bound for it to
function: the `createComponent` route which should link to the page where the
user can create components. In a typical setup the create component route will
be linked to the Scaffolder plugin's template index page:
```ts
import { catalogApiRef, CatalogClient } from '@backstage/plugin-catalog';
import { catalogPlugin } from '@backstage/plugin-catalog';
import { scaffolderPlugin } from '@backstage/plugin-scaffolder';
// Inside the ApiRegistry builder function ...
builder.add(
catalogApiRef,
new CatalogClient({
apiOrigin: backendUrl,
basePath: '/catalog',
}),
);
const app = createApp({
// ...
bindRoutes({ bind }) {
bind(catalogPlugin.externalRoutes, {
createComponent: scaffolderPlugin.routes.root,
});
},
});
```
Where `backendUrl` is the `backend.baseUrl` from config, i.e.
`const backendUrl = config.getString('backend.baseUrl')`.
You may also want to add a link to the catalog index page to your sidebar:
The catalog components depend on a number of other
[Utility APIs](../../api/utility-apis.md) to function, including at least the
`ErrorApi` and `StorageApi`. You can find an example of how to install these in
your app
[here](https://github.com/backstage/backstage/blob/61c3a7e5b750dc7c059ef16b188594d31b2c04c2/packages/app/src/apis.ts#L80).
```tsx
import HomeIcon from '@material-ui/icons/Home';
// Somewhere within the <Sidebar>
<SidebarItem icon={HomeIcon} to="/catalog" text="Home" />;
```
This is all that is needed for the frontend part of the Catalog plugin to work!
## Gotchas that we will fix
@@ -70,6 +70,35 @@ The value of this annotation is a location reference string (see above). If this
annotation is specified, it is expected to point to a repository that the
TechDocs system can read and generate docs from.
### backstage.io/view-url, backstage.io/edit-url
```yaml
# Example:
metadata:
annotations:
backstage.io/view-url: https://some.website/catalog-info.yaml
backstage.io/edit-url: https://github.com/my-org/catalog/edit/master/my-service.jsonnet
```
These annotations allow customising links from the catalog pages. The view URL
should point to the canonical metadata YAML that governs this entity. The edit
URL should point to the source file for the metadata. In the example above,
`my-org` generates its catalog data from Jsonnet files in a monorepo, so the
view and edit links need changing.
### backstage.io/source-location
```yaml
# Example:
metadata:
annotations:
backstage.io/source-location: github:https://github.com/my-org/my-service
```
A `Location` reference that points to the source code of the entity (typically a
`Component`). Useful when catalog files do not get ingested from the source code
repository itself.
### jenkins.io/github-folder
```yaml
@@ -33,27 +33,27 @@ it doesn't.
Add the following entry to the head of your `packages/app/src/plugins.ts`:
```ts
export { plugin as ScaffolderPlugin } from '@backstage/plugin-scaffolder';
export { scaffolderPlugin } from '@backstage/plugin-scaffolder';
```
Add the following to your `packages/app/src/apis.ts`:
Next we need to install the root page that the Scaffolder plugin provides. You
can choose any path for the route, but we recommend the following:
```ts
import { scaffolderApiRef, ScaffolderApi } from '@backstage/plugin-scaffolder';
```tsx
import { ScaffolderPage } from '@backstage/plugin-scaffolder';
// Inside the ApiRegistry builder function ...
builder.add(
scaffolderApiRef,
new ScaffolderApi({
apiOrigin: backendUrl,
basePath: '/scaffolder/v1',
}),
);
// Add to the top-level routes, directly within <FlatRoutes>
<Route path="/create" element={<ScaffolderPage />} />;
```
Where `backendUrl` is the `backend.baseUrl` from config, i.e.
`const backendUrl = config.getString('backend.baseUrl')`.
You may also want to add a link to the template index page to your sidebar:
```tsx
import CreateComponentIcon from '@material-ui/icons/AddCircleOutline';
// Somewhere within the <Sidebar>
<SidebarItem icon={CreateComponentIcon} to="create" text="Create..." />;
```
This is all that is needed for the frontend part of the Scaffolder plugin to
work!
@@ -85,29 +85,25 @@ following contents to get you up and running quickly.
import {
CookieCutter,
createRouter,
FilePreparer,
GithubPreparer,
GitlabPreparer,
Preparers,
Publishers,
GithubPublisher,
GitlabPublisher,
CreateReactAppTemplater,
Templaters,
RepoVisibilityOptions,
} from '@backstage/plugin-scaffolder-backend';
import { Octokit } from '@octokit/rest';
import { Gitlab } from '@gitbeaker/node';
import { SingleHostDiscovery } from '@backstage/backend-common';
import type { PluginEnvironment } from '../types';
import Docker from 'dockerode';
import { CatalogClient } from '@backstage/catalog-client';
export default async function createPlugin({
logger,
config,
database,
}: PluginEnvironment) {
const cookiecutterTemplater = new CookieCutter();
const craTemplater = new CreateReactAppTemplater();
const templaters = new Templaters();
templaters.register('cookiecutter', cookiecutterTemplater);
templaters.register('cra', craTemplater);
@@ -115,12 +111,19 @@ export default async function createPlugin({
const publishers = await Publishers.fromConfig(config, { logger });
const dockerClient = new Docker();
const discovery = SingleHostDiscovery.fromConfig(config);
const catalogClient = new CatalogClient({ discoveryApi: discovery });
return await createRouter({
preparers,
templaters,
publishers,
logger,
config,
dockerClient,
database,
catalogClient,
});
}
```
+34 -11
View File
@@ -142,6 +142,8 @@ variables**
You should follow the
[AWS security best practices guide for authentication](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html).
TechDocs needs access to read files and metadata of the S3 bucket.
If the environment variables
- `AWS_ACCESS_KEY_ID`
@@ -149,15 +151,21 @@ If the environment variables
- `AWS_REGION`
are set and can be used to access the bucket you created in step 2, they will be
used by the AWS SDK v3 Node.js client for authentication.
[Refer to the official documentation.](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-environment.html)
used by the AWS SDK V2 Node.js client for authentication.
[Refer to the official documentation for loading credentials in Node.js from environment variables](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-environment.html).
If the environment variables are missing, the AWS SDK tries to read the
`~/.aws/credentials` file for credentials.
[Refer to the official documentation.](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-shared.html)
[Refer to the official documentation.](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-shared.html)
Note that the region of the bucket has to be set for the AWS SDK to work.
[See this](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-region.html).
If you are using Amazon EC2 instance to deploy Backstage, you do not need to
obtain the access keys separately. They can be made available in the environment
automatically by defining appropriate IAM role with access to the bucket. Read
more in
[official AWS documentation for using IAM roles.](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html#use-roles).
The AWS Region of the bucket is optional since TechDocs uses AWS SDK V2 and not
V3.
**3b. Authentication using app-config.yaml**
@@ -181,13 +189,28 @@ techdocs:
```
Refer to the
[official AWS documentation for obtaining the credentials](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/getting-your-credentials.html).
[official AWS documentation for obtaining the credentials](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/getting-your-credentials.html).
Note: If you are using Amazon EC2 instance to deploy Backstage, you do not need
to obtain the access keys separately. They can be made available in the
environment automatically by defining appropriate IAM role with access to the
bucket. Read more
[here](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html#use-roles).
**3c. Authentication using an assumed role** Users with multiple AWS accounts
may want to use a role for S3 storage that is in a different AWS account. Using
the `roleArn` parameter as seen below, you can instruct the TechDocs publisher
to assume a role before accessing S3.
```yaml
techdocs:
publisher:
type: 'awsS3'
awsS3:
bucketName: 'name-of-techdocs-storage-bucket'
region:
$env: AWS_REGION
credentials:
roleArn: arn:aws:iam::123456789012:role/my-backstage-role
```
Note: Assuming a role requires that primary credentials are already configured
at `AWS.config.credentials`. Read more about
[assuming roles in AWS](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html).
**4. That's it!**
@@ -1,6 +1,6 @@
---
id: development-environment
title: Development Environment
id: contributors
title: Contributors
# prettier-ignore
description: Documentation on how to get set up for doing development on the Backstage repository
---
@@ -10,6 +10,21 @@ repository.
## Cloning the Repository
Ok. So you're gonna want some code right? Go ahead and fork the repository into
your own GitHub account and clone that code to your local machine or you can
grab the one for the origin like so:
```bash
git clone git@github.com/backstage/backstage --depth 1
```
If you cloned a fork, you can add the upstream dependency like so:
```bash
git remote add upstream git@github.com:backstage/backstage
git pull upstream master
```
After you have cloned the Backstage repository, you should run the following
commands once to set things up for development:
@@ -25,9 +40,11 @@ Open a terminal window and start the web app by using the following command from
the project root. Make sure you have run the above mentioned commands first.
```bash
$ yarn start
$ yarn dev
```
This is going to start two things, the frontend (:3000) and the backend (:7000).
This should open a local instance of Backstage in your browser, otherwise open
one of the URLs printed in the terminal.
@@ -38,12 +55,27 @@ setting an environment variable `PORT` on your local machine. e.g.
Once successfully started, you should see the following message in your terminal
window:
```sh
$ concurrently "yarn start" "yarn start-backend"
$ yarn workspace example-app start
$ yarn workspace example-backend start
$ backstage-cli app:serve
$ backstage-cli backend:dev
[0] Loaded config from app-config.yaml
[1] Build succeeded
[0] 「wds」: Project is running at http://localhost:3000/
[0] 「wds」: webpack output is served from /
[0] 「wds」: Content not from webpack is served from $BACKSTAGE_DIR/packages/app/public
[0] 「wds」: 404s will fallback to /index.html
[0] 「wdm」: wait until bundle finished: /
[1] 2021-02-12T20:58:17.614Z backstage info Loaded config from app-config.yaml
```
You can now view example-app in the browser.
Local: http://localhost:8080
On Your Network: http://192.168.1.224:8080
```
You'll see how you get both logs for the frontend `webpack-dev-server` which
serves the react app ([0]) and the backend ([1]);
Visit http://localhost:3000 and you should see the bleeding edge of Backstage
ready for contributions!
## Editor
+2 -16
View File
@@ -177,20 +177,6 @@ the root directory:
yarn workspace backend start
```
Now you're free to hack away on your own Backstage installation!
### Troubleshooting
#### Cannot find module
You may encounter an error similar to below:
```
internal/modules/cjs/loader.js:968
throw err;
^
Error: Cannot find module '../build/Debug/nodegit.node'
```
This can occur if an npm dependency is not completely installed. Because some
dependencies run a post-install script, ensure that both your npm and yarn
configs have the `ignore-scripts` flag set to `false`.
+238
View File
@@ -0,0 +1,238 @@
---
id: deployment-docker
title: Docker
description: Documentation on how to deploy Backstage as a Docker image
---
This section describes how to build a Backstage App into a deployable Docker
image. It is split into three sections, first covering the host build approach,
which is recommended due its speed and more efficient and often simpler caching.
The second section covers a full multi-stage Docker build, and the last section
covers how to split frontend content into a separate image.
Something that goes for all of these docker deployment strategies is that they
are stateless, so for a production deployment you will want to set up and
connect to an external PostgreSQL instance where the backend plugins can store
their state, rather than using SQLite.
### Host Build
This section describes how to build a Docker image from a Backstage repo with
most of the build happening outside of Docker. This is almost always the faster
approach, as the build steps tend to execute faster, and it's possible to have
more efficient caching of dependencies on the host, where a single change won't
bust the entire cache.
The required steps in the host build are to install dependencies with
`yarn install`, generate type definitions using `yarn tsc`, and build all
packages with `yarn build`.
> NOTE: Using `yarn build` to build packages and bundle the backend assumes that
> you have migrated to using `backstage-cli backend:bundle` as your build script
> in the backend package.
In a CI workflow it might look something like this:
```bash
yarn install --frozen-lockfile
# tsc outputs type definitions to dist-types/ in the repo root, which are then consumed by the build
yarn tsc
# Build all packages and in the end bundle them all up into the packages/backend/dist folder.
yarn build
```
Once the host build is complete, we are ready to build our image. We use the
following `Dockerfile`, which is also included when creating a new app with
`@backstage/create-app`:
```Dockerfile
# This dockerfile builds an image for the backend package.
# It should be executed with the root of the repo as docker context.
#
# Before building this image, be sure to have run the following commands in the repo root:
#
# yarn install
# yarn tsc
# yarn build
#
# Once the commands have been run, you can build the image using `yarn build-image`
FROM node:14-buster-slim
WORKDIR /app
# Copy repo skeleton first, to avoid unnecessary docker cache invalidation.
# The skeleton contains the package.json of each package in the monorepo,
# and along with yarn.lock and the root package.json, that's enough to run yarn install.
ADD yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./
RUN yarn install --frozen-lockfile --production --network-timeout 300000 && rm -rf "$(yarn cache dir)"
# Then copy the rest of the backend bundle, along with any other files we might want.
ADD packages/backend/dist/bundle.tar.gz app-config.yaml ./
CMD ["node", "packages/backend", "--config", "app-config.yaml"]
```
For more details on how the `backend:bundle` command and the `skeleton.tar.gz`
file works, see the
[`backend:bundle` command docs](../cli/commands.md#backendbundle)
The `Dockerfile` is typically placed at `packages/backend/Dockerfile`, but needs
to be executed with the root of the repo as the build context, in order to get
access to the root `yarn.lock` and `package.json`, along with any other files
that might be needed, such as `.npmrc`.
In order to speed up the build we can significantly reduce the build context
size using the following `.dockerignore` in the root of the repo:
```text
.git
node_modules
packages
!packages/backend/dist
plugins
```
With the project build and the `.dockerignore` and `Dockerfile` in place, we are
now ready to build the final image. Assuming we're at the root of the repo, we
execute the build like this:
```bash
docker image build . -f packages/backend/Dockerfile --tag backstage
```
To try out the image locally you can run the following:
```sh
docker run -it -p 7000:7000 backstage
```
You should then start to get logs in your terminal, and then you can open your
browser at `http://localhost:7000`
### Multistage Build
This section describes how to set up a multi-stage Docker build that builds the
entire project within Docker. This is typically slower than a host build, but is
sometimes desired because Docker in Docker is not available in the build
environment, or due to other requirements.
The build is split into three different stages, where the first stage finds all
of the `package.json`s that are relevant for the initial install step enabling
us to cache the initial `yarn install` that installs all dependencies. The
second stage executes the build itself, and is similar to the steps we execute
on the host in the host build. The third and final stage then packages it all
together into the final image, and is similar to the `Dockerfile` of the host
build.
The following `Dockerfile` executes the multi-stage build and should be added to
the repo root:
```Dockerfile
# Stage 1 - Create yarn install skeleton layer
FROM node:14-buster-slim AS packages
WORKDIR /app
COPY package.json yarn.lock ./
COPY packages packages
COPY plugins plugins
RUN find packages \! -name "package.json" -mindepth 2 -maxdepth 2 -exec rm -rf {} \+
# Stage 2 - Install dependencies and build packages
FROM node:14-buster-slim AS build
WORKDIR /app
COPY --from=packages /app .
RUN yarn install --frozen-lockfile --network-timeout 600000 && rm -rf "$(yarn cache dir)"
COPY . .
RUN yarn tsc
RUN yarn --cwd packages/backend backstage-cli backend:bundle --build-dependencies
# Stage 3 - Build the actual backend image and install production dependencies
FROM node:14-buster-slim
WORKDIR /app
# Copy the install dependencies from the build stage and context
COPY --from=build /app/yarn.lock /app/package.json /app/packages/backend/dist/skeleton.tar.gz ./
RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz
RUN yarn install --frozen-lockfile --production --network-timeout 600000 && rm -rf "$(yarn cache dir)"
# Copy the built packages from the build stage
COPY --from=build /app/packages/backend/dist/bundle.tar.gz .
RUN tar xzf bundle.tar.gz && rm bundle.tar.gz
# Copy any other files that we need at runtime
COPY app-config.yaml ./
CMD ["node", "packages/backend", "--config", "app-config.yaml"]
```
Note that a newly created Backstage app will typically not have a `plugins/`
folder, so you will want to comment that line out. This build also does not work
in the main repo, since the `backstage-cli` which is used for the build doesn't
end up being properly installed.
To speed up the build when not running in a fresh clone of the repo you should
set up a `.dockerignore`. This one is different than the host build one, because
we want to have access to the source code of all packages for the build, but can
ignore any existing build output or dependencies:
```text
node_modules
packages/*/dist
packages/*/node_modules
plugins/*/dist
plugins/*/node_modules
```
Once you have added both the `Dockerfile` and `.dockerignore` to the root of
your project, run the following to build the container under a specified tag.
```sh
docker image build -t backstage .
```
To try out the image locally you can run the following:
```sh
docker run -it -p 7000:7000 backstage
```
You should then start to get logs in your terminal, and then you can open your
browser at `http://localhost:7000`
### Separate Frontend
It is sometimes desirable to serve the frontend separately from the backend,
either from a separate image or for example a static file serving provider. The
first step in doing so is to remove the `app-backend` plugin from the backend
package, which is done as follows:
1. Delete `packages/backend/src/plugins/app.ts`
2. Remove the following lines from `packages/backend/src/index.ts`:
```tsx
import app from './plugins/app';
// ...
const appEnv = useHotMemoize(module, () => createEnv('app'));
// ...
.addRouter('', await app(appEnv));
```
3. Remove the `@backstage/plugin-app-backend` and the app package dependency
(e.g. `app`) from `packages/backend/packages.json`. If you don't remove the
app package dependency the app will still be built and bundled with the
backend.
Once the `app-backend` is removed from the backend, you can use your favorite
static file serving method for serving the frontend. An example of how to set up
an NGINX image is available in the
[contrib folder in the main repo](https://github.com/backstage/backstage/blob/master/contrib/docker/frontend-with-nginx/Dockerfile)
-92
View File
@@ -4,98 +4,6 @@ title: Other
description: Documentation on different ways of Deployment
---
## Docker
Here we have an example Dockerfile that you can use to build everything together
in one container. This Dockerfile uses multi-stage builds, and a
`backend:bundle` command from the CLI.
It also provides caching on the `yarn install`'s so that you don't have to do it
unless absolutely necessary.
> Note: This Dockerfile assumes that you're running SQLite, or your
> configuration is setup to connect to an external PostgreSQL Database.
```Dockerfile
# Stage 1 - Create yarn install skeleton layer
FROM node:14-buster AS packages
WORKDIR /app
COPY package.json yarn.lock ./
COPY packages packages
# Uncomment this line if you have a local plugins folder
# COPY plugins plugins
RUN find packages \! -name "package.json" -mindepth 2 -maxdepth 2 -print | xargs rm -rf
# Stage 2 - Install dependencies and build packages
FROM node:14-buster AS build
WORKDIR /app
COPY --from=packages /app .
RUN yarn install --network-timeout 600000 && rm -rf "$(yarn cache dir)"
COPY . .
RUN yarn tsc
RUN yarn --cwd packages/backend backstage-cli backend:bundle --build-dependencies
# Stage 3 - Build the actual backend image and install production dependencies
FROM node:14-buster
WORKDIR /app
# Copy from build stage
COPY --from=build /app/yarn.lock /app/package.json /app/packages/backend/dist/skeleton.tar.gz ./
RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz
RUN yarn install --production --network-timeout 600000 && rm -rf "$(yarn cache dir)"
COPY --from=build /app/packages/backend/dist/bundle.tar.gz .
RUN tar xzf bundle.tar.gz && rm bundle.tar.gz
COPY app-config.yaml app-config.production.yaml ./
CMD ["node", "packages/backend", "--config", "app-config.yaml", "--config", "app-config.production.yaml"]
```
Before building you should also include a `.dockerignore`. This will greatly
improve the context boot up time of Docker as we are no longer sending all of
the `node_modules` into the context. It also helps us avoid some limitations and
errors that may occur when trying to share the `node_modules` folder to inside
the build.
You can add the following contents to the root of your repository at
`.dockerignore` and it might look something like the following:
```dockerignore
.git
node_modules
packages/*/node_modules
plugins/*/node_modules
plugins/*/dist
```
Once you have added both the `Dockerfile` and `.dockerignore` to the root of
your project, and run the following to build the container under a specified
tag.
```sh
$ docker build -t example-deployment .
```
To run the image locally you can run:
```sh
$ docker run -it -p 7000:7000 example-deployment
```
You should then start to get logs in your terminal, and then you can open your
browser at `http://localhost:7000`
## Heroku
Deploying to Heroku is relatively easy following these steps.
+1 -6
View File
@@ -19,7 +19,7 @@ general, it's easier to fork and clone this project. That will let you stay up
to date with the latest changes, and gives you an easier path to make Pull
Requests towards this repo.
### Creating a Standalone App
### Create your Backstage App
Backstage provides the `@backstage/create-app` package to scaffold standalone
instances of Backstage. You will need to have
@@ -46,8 +46,3 @@ look something like this. You can read more about this process in
You can read more in our
[CONTRIBUTING](https://github.com/backstage/backstage/blob/master/CONTRIBUTING.md)
guide, which can help you get setup with a Backstage development environment.
### Next steps
Take a look at the [Running Backstage Locally](./running-backstage-locally.md)
guide to learn how to set up Backstage, and how to develop on the platform.
@@ -104,7 +104,6 @@ The value of Backstage grows with every new plugin that gets added. Here is a
collection of tutorials that will guide you through setting up and extending an
instance of Backstage with your own plugins.
- [Development Environment](development-environment.md)
- [Create a Backstage Plugin](../plugins/create-a-plugin.md)
- [Structure of a Plugin](../plugins/structure-of-a-plugin.md)
- [Utility APIs](../api/utility-apis.md)
+71
View File
@@ -0,0 +1,71 @@
---
id: org
title: GitHub Organizational Data
sidebar_label: Org Data
# prettier-ignore
description: Setting up ingestion of organizational data from GitHub
---
The Backstage catalog can be set up to ingest organizational data - teams and
users - directly from an organization in GitHub or GitHub Enterprise. The result
is a hierarchy of
[`User`](../../features/software-catalog/descriptor-format.md#kind-user) and
[`Group`](../../features/software-catalog/descriptor-format.md#kind-group) kind
entities that mirror your org setup.
## Installation
The processor that performs the import, `GithubOrgReaderProcessor`, comes
installed with the default setup of Backstage.
If you replace the set of processors in your installation using that facility of
the catalog builder class, you can import and add it as follows.
```ts
// Typically in packages/backend/src/plugins/catalog.ts
import { GithubOrgReaderProcessor } from '@backstage/plugin-catalog-backend';
builder.replaceProcessors(
GithubOrgReaderProcessor.fromConfig(config, { logger }),
// ...
);
```
## Configuration
The following configuration enables an import of the teams and users under the
org `https://github.com/my-org-name` on public GitHub.
```yaml
catalog:
locations:
- type: github-org
target: https://github.com/my-org-name
processors:
githubOrg:
providers:
- target: https://github.com
apiBaseUrl: https://api.github.com
token:
$env: GITHUB_TOKEN
```
Locations point out the specific org(s) you want to import. The `type` of these
locations must be `github-org`, and the `target` must point to the exact URL of
some organization. You can have several such location entries if you want, but
typically you will have just one.
The processor itself is configured in the other block, under
`catalog.processors.githubOrg`. There may be many providers, each targeting a
specific `target` which is supposed to be the address of the home page of GitHub
or your GitHub Enterprise installation.
The example above assumes that the backend is started with an environment
variable called `GITHUB_TOKEN` that contains a Personal Access Token. The token
needs to have at least the scopes `read:org`, `read:user`, and `user:email` in
the given `target`.
If you want to address your own GitHub Enterprise instance, replace occurrences
of `https://github.com` in the configuration above with the address of your
GitHub Enterprise home page, and the `apiBaseUrl` to where your API endpoint
lives - commonly on the form `https://<host>/api/v3`.
@@ -0,0 +1,24 @@
---
id: installation
title: Google Analytics Installation
sidebar_label: Installation
# prettier-ignore
description: Adding Google Analytics to Your App
---
There is a basic Google Analytics integration built into Backstage. You can
enable it by adding the following to your app configuration:
```yaml
app:
googleAnalyticsTrackingId: UA-000000-0
```
Replace the tracking ID with your own.
For more information, learn about Google Analytics
[here](https://marketingplatform.google.com/about/analytics/).
The default behavior is only to send a pageview hit to Google Analytics. To
record more, look at the developer documentation
[here](https://developers.google.com/analytics/devguides/collection/gtagjs).
+280
View File
@@ -0,0 +1,280 @@
---
id: org
title: LDAP Organizational Data
sidebar_label: Org Data
# prettier-ignore
description: Setting up ingestion of organizational data from LDAP
---
The Backstage catalog can be set up to ingest organizational data - groups and
users - directly from an LDAP compatible service. The result is a hierarchy of
[`User`](../../features/software-catalog/descriptor-format.md#kind-user) and
[`Group`](../../features/software-catalog/descriptor-format.md#kind-group) kind
entities that mirror your org setup.
## Installation
The processor that performs the import, `LdapOrgReaderProcessor`, comes
installed with the default setup of Backstage.
If you replace the set of processors in your installation using that facility of
the catalog builder class, you can import and add it as follows.
```ts
// Typically in packages/backend/src/plugins/catalog.ts
import { LdapOrgReaderProcessor } from '@backstage/plugin-catalog-backend';
builder.replaceProcessors(
LdapOrgReaderProcessor.fromConfig(config, { logger }),
// ...
);
```
## Configuration
The following configuration is a small example of how a setup could look for
importing groups and users from a corporate LDAP server.
```yaml
catalog:
locations:
- type: ldap-org
target: ldaps://ds.example.net
processors:
ldapOrg:
providers:
- target: ldaps://ds.example.net
bind:
dn: uid=ldap-reader-user,ou=people,ou=example,dc=example,dc=net
secret:
$env: LDAP_SECRET
users:
dn: ou=people,ou=example,dc=example,dc=net
options:
filter: (uid=*)
map:
description: l
set:
metadata.customField: 'hello'
groups:
dn: ou=access,ou=groups,ou=example,dc=example,dc=net
options:
filter: (&(objectClass=some-group-class)(!(groupType=email)))
map:
description: l
set:
metadata.customField: 'hello'
```
Locations point out the specific org(s) you want to import. The `type` of these
locations must be `ldap-org`, and the `target` must point to the exact URL
(starting with `ldap://` or `ldaps://`) of the targeted LDAP server. You can
have several such location entries if you want, but typically you will have just
one.
The processor itself is configured in the other block, under
`catalog.processors.ldapOrg`. There may be many providers, each targeting a
specific `target` which is supposed to be on the same form as the location
`target`.
These config blocks have a lot of options in them, so we will describe each
"root" key within the block separately.
### target
This is the URL of the targeted server, typically on the form
`ldaps://ds.example.net` for SSL enabled servers or `ldap://ds.example.net`
without SSL.
### bind
The bind block specifies how the plugin should bind (essentially, to
authenticate) towards the server. It has the following fields.
```yaml
dn: uid=ldap-reader-user,ou=people,ou=example,dc=example,dc=net
secret:
$env: LDAP_SECRET
```
The `dn` is the full LDAP DN (distinguished name) for the user that the plugin
authenticates itself as. At this point, only regular user based authentication
is supported.
The `secret` is the password of the same user. In this example, it is given in
the form of an environment variable `LDAP_SECRET`, that has to be set when the
backend starts.
### users
The `users` block defines the settings that govern the reading and
interpretation of users. Its fields are explained in separate sections below.
### users.dn
The DN under which users are stored, e.g.
`ou=people,ou=example,dc=example,dc=net`.
### users.options
The search options to use when sending the query to the server, when reading all
users. All of the options are shown below, with their default values, but they
are all optional.
```yaml
options:
# One of 'base', 'one', or 'sub'.
scope: one
# The filter is the one that you commonly will want to specify explicitly. It
# is a string on the standard LDAP query format. Use it to select out the set
# of users that are of actual interest to ingest. For example, you may want
# to filter out disabled users.
filter: (uid=*)
# The attribute selectors for each item, as passed to the LDAP server.
attributes: ['*', '+']
# This field is either 'false' to disable paging when reading from the
# server, or an object on the form '{ pageSize: 100, pagePause: true }' that
# specifies the details of how the paging shall work.
paged: false
```
### users.set
This optional piece lets you specify a number of JSON paths (on a.b.c form) and
hard coded values to set on those paths. This can be useful for example if you
want to hard code a namespace or similar on the generated entities.
```yaml
set:
# Just an example; the key and value can be anything
metadata.namespace: 'ldap'
```
### users.map
Mappings from well known entity fields, to LDAP attribute names. This is where
you are able to define how to interpret the attributes of each LDAP result item,
and to move them into the corresponding entity fields. All of the options are
shown below, with their default values, but they are all optional.
If you leave out an optional mapping, it will still be copied using that default
value. For example, even if you do not put in the field `displayName` in your
config, the processor will still copy the attribute `cn` into the entity field
`spec.profile.displayName`.
```yaml
map:
# The name of the attribute that holds the relative
# distinguished name of each entry.
rdn: uid
# The name of the attribute that shall be used for the value of
# the metadata.name field of the entity.
name: uid
# The name of the attribute that shall be used for the value of
# the metadata.description field of the entity.
description: description
# The name of the attribute that shall be used for the value of
# the spec.profile.displayName field of the entity.
displayName: cn
# The name of the attribute that shall be used for the value of
# the spec.profile.email field of the entity.
email: mail
# The name of the attribute that shall be used for the value of
# the spec.profile.picture field of the entity.
picture: <nothing, left out>
# The name of the attribute that shall be used for the values of
# the spec.memberOf field of the entity.
memberOf: memberOf
```
### groups
The `groups` block defines the settings that govern the reading and
interpretation of groups. Its fields are explained in separate sections below.
### groups.dn
The DN under which groups are stored, e.g.
`ou=people,ou=example,dc=example,dc=net`.
### groups.options
The search options to use when sending the query to the server, when reading all
groups. All of the options are shown below, with their default values, but they
are all optional.
```yaml
options:
# One of 'base', 'one', or 'sub'.
scope: one
# The filter is the one that you commonly will want to specify explicitly. It
# is a string on the standard LDAP query format. Use it to select out the set
# of groups that are of actual interest to ingest. For example, you may want
# to filter out disabled groups.
filter: (&(objectClass=some-group-class)(!(groupType=email)))
# The attribute selectors for each item, as passed to the LDAP server.
attributes: ['*', '+']
# This field is either 'false' to disable paging when reading from the
# server, or an object on the form '{ pageSize: 100, pagePause: true }' that
# specifies the details of how the paging shall work.
paged: false
```
### groups.set
This optional piece lets you specify a number of JSON paths (on a.b.c form) and
hard coded values to set on those paths. This can be useful for example if you
want to hard code a namespace or similar on the generated entities.
```yaml
set:
# Just an example; the key and value can be anything
metadata.namespace: 'ldap'
```
### groups.map
Mappings from well known entity fields, to LDAP attribute names. This is where
you are able to define how to interpret the attributes of each LDAP result item,
and to move them into the corresponding entity fields. All of the options are
shown below, with their default values, but they are all optional.
If you leave out an optional mapping, it will still be copied using that default
value. For example, even if you do not put in the field `displayName` in your
config, the processor will still copy the attribute `cn` into the entity field
`spec.profile.displayName`. If the target field is optional, such as the display
name, the importer will accept missing attributes and just leave the target
field unset. If the target field is mandatory, such as the name of the entity,
validation will fail if the source attribute is missing.
```yaml
map:
# The name of the attribute that holds the relative
# distinguished name of each entry. This value is copied into a
# well known annotation to be able to query by it later.
rdn: cn
# The name of the attribute that shall be used for the value of
# the metadata.name field of the entity.
name: cn
# The name of the attribute that shall be used for the value of
# the metadata.description field of the entity.
description: description
# The name of the attribute that shall be used for the value of
# the spec.type field of the entity.
type: groupType
# The name of the attribute that shall be used for the value of
# the spec.profile.displayName field of the entity.
displayName: cn
# The name of the attribute that shall be used for the value of
# the spec.profile.email field of the entity.
email: <nothing, left out>
# The name of the attribute that shall be used for the value of
# the spec.profile.picture field of the entity.
picture: <nothing, left out>
# The name of the attribute that shall be used for the values of
# the spec.parent field of the entity.
memberOf: memberOf
# The name of the attribute that shall be used for the values of
# the spec.children field of the entity.
members: member
```
+2 -12
View File
@@ -10,18 +10,8 @@ Backstage integrator.
## Google Analytics
There is a basic Google Analytics integration built into Backstage. You can
enable it by adding the following to your app configuration:
```yaml
app:
googleAnalyticsTrackingId: UA-000000-0
```
Replace the tracking ID with your own.
For more information, learn about Google Analytics
[here](https://marketingplatform.google.com/about/analytics/).
See how to install Google Analytics in your app
[here](../integrations/google-analytics/installation.md)
## Logging
+1 -1
View File
@@ -274,7 +274,7 @@ export {
};
```
**`./\_\_mocks\_\_/MyApi.js`**
**`./__mocks__/MyApi.js`**
```js
export {
+27 -30
View File
@@ -6,8 +6,8 @@ description: Introduction to files and folders in the Backstage Project reposito
---
Backstage is a complex project, and the GitHub repository contains many
different files and folders. This document aims to clarify what purpose of those
files and folders are.
different files and folders. This document aims to clarify the purpose of those
files and folders.
## General purpose files and folders
@@ -25,7 +25,7 @@ the code.
Standard GitHub folder. It contains - amongst other things - our workflow
definitions and templates. Worth noting is the
[styles](https://github.com/backstage/backstage/tree/master/.github/styles)
folder which is used for a markdown spellchecker.
sub-folder which is used for a markdown spellchecker.
- [`.yarn/`](https://github.com/backstage/backstage/tree/master/.yarn) -
Backstage ships with it's own `yarn` implementation. This allows us to have
@@ -33,7 +33,7 @@ the code.
yarn versioning differences.
- [`contrib/`](https://github.com/backstage/backstage/tree/master/contrib) -
Collection of examples or resources provided by the community. We really
Collection of examples or resources contributed by the community. We really
appreciate contributions in here and encourage them being kept up to date.
- [`docs/`](https://github.com/backstage/backstage/tree/master/docs) - This is
@@ -43,10 +43,11 @@ the code.
file may be needed as sections are added/removed.
- [`.editorconfig`](https://github.com/backstage/backstage/tree/master/.editorconfig) -
A configuration file used by most common code editors.
A configuration file used by most common code editors. Learn more at
[EditorConfig.org](https://editorconfig.org/).
- [`.imgbotconfig`](https://github.com/backstage/backstage/tree/master/.imgbotconfig) -
Configuration for a [bot](https://imgbot.net/)
Configuration for a [bot](https://imgbot.net/) which helps reduce image sizes.
## Monorepo packages
@@ -103,16 +104,16 @@ are separated out into their own folder, see further down.
diff, create-plugins and more. In the early days of this project, we started
out with calling tools directly - such as `eslint` - through `package.json`.
But as it was tricky to have a good development experience around that when we
change named tooling, we opted for wrapping those in our own cli. That way
change named tooling, we opted for wrapping those in our own CLI. That way
everything looks the same in `package.json`. Much like
[react-scripts](https://github.com/facebook/create-react-app/tree/master/packages/react-scripts).
- [`cli-common/`](https://github.com/backstage/backstage/tree/master/packages/cli-common) -
This package mainly handles path resolving. It is a separate package to reduce
bugs in
[cli](https://github.com/backstage/backstage/tree/master/packages/cli). We
[CLI](https://github.com/backstage/backstage/tree/master/packages/cli). We
also want as few dependencies as possible to reduce download time when running
the cli which is another reason this is a separate package.
the CLI which is another reason this is a separate package.
- [`config/`](https://github.com/backstage/backstage/tree/master/packages/config) -
The way we read configuration data. This package can take a bunch of config
@@ -139,14 +140,6 @@ are separated out into their own folder, see further down.
detail that we try to hide from our users, and no one should have to depend on
it directly.
- [`test-utils/`](https://github.com/backstage/backstage/tree/master/packages/test-utils) -
This package contains specific testing facilities used when testing
`core-api`.
- [`test-utils-core/`](https://github.com/backstage/backstage/tree/master/packages/test-utils-core) -
This package contains more general purpose testing facilities for testing a
Backstage App.
- [`create-app/`](https://github.com/backstage/backstage/tree/master/packages/create-app) -
An CLI to specifically scaffold a new Backstage App. It does so by using a
[template](https://github.com/backstage/backstage/tree/master/packages/create-app/templates/default-app).
@@ -161,26 +154,30 @@ are separated out into their own folder, see further down.
to read out definitions and generate documentation for it.
- [`e2e-test/`](https://github.com/backstage/backstage/tree/master/packages/e2e-test) -
Another CLI that can be run to try out what would happen if you built all the
packages, publish them, created a new app, and the run it. CI uses this for
Another CLI that can be run to try out what would happen if you build all the
packages, publish them, create a new app, and then run them. CI uses this for
e2e-tests.
- [`integration/`](https://github.com/backstage/backstage/tree/master/packages/integration) -
Common functionalities of integrations like GitHub, GitLab, etc.
- [`storybook/`](https://github.com/backstage/backstage/tree/master/packages/storybook) -
This folder contains only the storybook config. Stories are within the core
package. The Backstage Storybook is found
[here](https://backstage.io/storybook)
This folder contains only the Storybook config which helps visualize our
reusable React components. Stories are within the core package, and are
published in the [Backstage Storybook](https://backstage.io/storybook).
- [`techdocs-common/`](https://github.com/backstage/backstage/tree/master/packages/techdocs-common) -
Common functionalities for TechDocs, to be shared between
[techdocs-backend](https://github.com/backstage/backstage/tree/master/plugins/techdocs-backend)
plugin and [techdocs-cli](https://github.com/backstage/techdocs-cli).
- [`test-utils-core/`](https://github.com/backstage/backstage/tree/master/packages/test-utils-core)
- [`test-utils/`](https://github.com/backstage/backstage/tree/master/packages/test-utils) -
This package contains specific testing facilities used when testing
`core-api`.
- [`test-utils/`](https://github.com/backstage/backstage/tree/master/packages/test-utils)
- [`test-utils-core/`](https://github.com/backstage/backstage/tree/master/packages/test-utils-core) -
This package contains more general purpose testing facilities for testing a
Backstage App.
- [`theme/`](https://github.com/backstage/backstage/tree/master/packages/theme) -
Holds the Backstage Theme.
@@ -196,10 +193,10 @@ We can categorize plugins into three different types; **Frontend**, **Backend**
and **GraphQL**. We differentiate these types of plugins when we name them, with
a dash-suffix. `-backend` means its a backend plugin and so on.
One reason for splitting a plugin is because of to it's dependencies. Another
reason is for clear separation of concerns.
One reason for splitting a plugin is because of its dependencies. Another reason
is for clear separation of concerns.
Take a look at our [Plugin Gallery](https://backstage.io/plugins) or browse
Take a look at our [Plugin Marketplace](https://backstage.io/plugins) or browse
through the
[`plugins/`](https://github.com/backstage/backstage/tree/master/plugins) folder.
@@ -212,7 +209,7 @@ monorepo setup.
This folder contains the source code for backstage.io. It is built with
[Docusaurus](https://docusaurus.io/). This folder is not part of the monorepo
due to dependency reasons. Look at the
[README](https://github.com/backstage/backstage/blob/master/microsite/README.md)
[microsite README](https://github.com/backstage/backstage/blob/master/microsite/README.md)
for instructions on how to run it locally.
## Root files specifically used by the `app`
@@ -222,8 +219,8 @@ Some of these files may be subject to be moved out of the root sometime in the
future.
- [`.npmrc`](https://github.com/backstage/backstage/tree/master/.npmrc) - It's
common for companies to have their own npm registry, this files makes sure
that this folder use the public registry.
common for companies to have their own npm registry, and this file makes sure
that this folder always uses the public registry.
- [`.vale.ini`](https://github.com/backstage/backstage/tree/master/.vale.ini) -
[Spell checker](https://github.com/errata-ai/vale) for Markdown files.