Merge branch 'master' of github.com:backstage/backstage into mob/scaffolder-frontend
* 'master' of github.com:backstage/backstage: (118 commits) cli: Fix handling of dynamic imports in esm.js files minor typo in migration chore(deps): bump archiver from 5.1.0 to 5.2.0 dockerfile: mention build-image command Apply suggestions from code review update backend Dockerfile to use config example and fix comment docs: add full docker deployment docs chore: fix code review chore: fixing syntax docs: fixing custom implementations of utitiy apis a small start to the integrations section of the config TechDocs: Add changeset about Docker permission fix Updated unit tests for the new UI TechDocs: Pass user and group ID when invoking docker container Replace logging erro and return undefined for a throw new Error @types/react 16 not 17 Use a more strict type for `variant` of cards docs(TechDocs): Add more context with AWS docs hyperlinks Added missing dep on @types/react Removed unused import ...
This commit is contained in:
blam
@@ -181,7 +181,25 @@ The `IgnoringErrorApi` would then be imported in the app, and wired up like
|
||||
this:
|
||||
|
||||
```ts
|
||||
builder.add(errorApiRef, new IgnoringErrorApi());
|
||||
const app = createApp({
|
||||
apis: [
|
||||
/* ApiFactories */
|
||||
createApiFactory(errorApiRef, new IgnoringErrorApi()),
|
||||
|
||||
// OR
|
||||
// If your API has dependencies, you use the object form
|
||||
createApiFactory({
|
||||
api: errorApiRef,
|
||||
deps: { configApi: configApiRef },
|
||||
factory({ configApi }) {
|
||||
return new IgnoringErrorApi({
|
||||
reportingUrl: configApi.getString('error.reportingUrl'),
|
||||
});
|
||||
},
|
||||
}),
|
||||
],
|
||||
// ... other options
|
||||
});
|
||||
```
|
||||
|
||||
Note that the above line will cause an error if `IgnoreErrorApi` does not fully
|
||||
|
||||
@@ -206,15 +206,15 @@ The following is an example of a `Dockerfile` that can be used to package the
|
||||
output of `backstage-cli backend:bundle` into an image:
|
||||
|
||||
```Dockerfile
|
||||
FROM node:14-buster
|
||||
FROM node:14-buster-slim
|
||||
WORKDIR /app
|
||||
|
||||
ADD yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./
|
||||
RUN yarn install --production --network-timeout 600000 && rm -rf "$(yarn cache dir)"
|
||||
RUN yarn install --frozen-lockfile --production --network-timeout 300000 && rm -rf "$(yarn cache dir)"
|
||||
|
||||
ADD packages/backend/dist/bundle.tar.gz app-config.yaml ./
|
||||
|
||||
CMD node packages/backend
|
||||
CMD ["node", "packages/backend"]
|
||||
```
|
||||
|
||||
```text
|
||||
|
||||
@@ -142,6 +142,8 @@ variables**
|
||||
You should follow the
|
||||
[AWS security best practices guide for authentication](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html).
|
||||
|
||||
TechDocs needs access to read files and metadata of the S3 bucket.
|
||||
|
||||
If the environment variables
|
||||
|
||||
- `AWS_ACCESS_KEY_ID`
|
||||
@@ -149,15 +151,21 @@ If the environment variables
|
||||
- `AWS_REGION`
|
||||
|
||||
are set and can be used to access the bucket you created in step 2, they will be
|
||||
used by the AWS SDK v3 Node.js client for authentication.
|
||||
[Refer to the official documentation.](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-environment.html)
|
||||
used by the AWS SDK V2 Node.js client for authentication.
|
||||
[Refer to the official documentation for loading credentials in Node.js from environment variables](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-environment.html).
|
||||
|
||||
If the environment variables are missing, the AWS SDK tries to read the
|
||||
`~/.aws/credentials` file for credentials.
|
||||
[Refer to the official documentation.](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-shared.html)
|
||||
[Refer to the official documentation.](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-shared.html)
|
||||
|
||||
Note that the region of the bucket has to be set for the AWS SDK to work.
|
||||
[See this](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-region.html).
|
||||
If you are using Amazon EC2 instance to deploy Backstage, you do not need to
|
||||
obtain the access keys separately. They can be made available in the environment
|
||||
automatically by defining appropriate IAM role with access to the bucket. Read
|
||||
more in
|
||||
[official AWS documentation for using IAM roles.](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html#use-roles).
|
||||
|
||||
The AWS Region of the bucket is optional since TechDocs uses AWS SDK V2 and not
|
||||
V3.
|
||||
|
||||
**3b. Authentication using app-config.yaml**
|
||||
|
||||
@@ -181,13 +189,7 @@ techdocs:
|
||||
```
|
||||
|
||||
Refer to the
|
||||
[official AWS documentation for obtaining the credentials](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/getting-your-credentials.html).
|
||||
|
||||
Note: If you are using Amazon EC2 instance to deploy Backstage, you do not need
|
||||
to obtain the access keys separately. They can be made available in the
|
||||
environment automatically by defining appropriate IAM role with access to the
|
||||
bucket. Read more
|
||||
[here](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html#use-roles).
|
||||
[official AWS documentation for obtaining the credentials](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/getting-your-credentials.html).
|
||||
|
||||
**4. That's it!**
|
||||
|
||||
|
||||
+39
-7
@@ -1,6 +1,6 @@
|
||||
---
|
||||
id: development-environment
|
||||
title: Development Environment
|
||||
id: contributors
|
||||
title: Contributors
|
||||
# prettier-ignore
|
||||
description: Documentation on how to get set up for doing development on the Backstage repository
|
||||
---
|
||||
@@ -10,6 +10,21 @@ repository.
|
||||
|
||||
## Cloning the Repository
|
||||
|
||||
Ok. So you're gonna want some code right? Go ahead and fork the repository into
|
||||
your own GitHub account and clone that code to your local machine or you can
|
||||
grab the one for the origin like so:
|
||||
|
||||
```bash
|
||||
git clone git@github.com/backstage/backstage --depth 1
|
||||
```
|
||||
|
||||
If you cloned a fork, you can add the upstream dependency like so:
|
||||
|
||||
```bash
|
||||
git remote add upstream git@github.com:backstage/backstage
|
||||
git pull upstream master
|
||||
```
|
||||
|
||||
After you have cloned the Backstage repository, you should run the following
|
||||
commands once to set things up for development:
|
||||
|
||||
@@ -25,9 +40,11 @@ Open a terminal window and start the web app by using the following command from
|
||||
the project root. Make sure you have run the above mentioned commands first.
|
||||
|
||||
```bash
|
||||
$ yarn start
|
||||
$ yarn dev
|
||||
```
|
||||
|
||||
This is going to start two things, the frontend (:3000) and the backend (:7000).
|
||||
|
||||
This should open a local instance of Backstage in your browser, otherwise open
|
||||
one of the URLs printed in the terminal.
|
||||
|
||||
@@ -38,12 +55,27 @@ setting an environment variable `PORT` on your local machine. e.g.
|
||||
Once successfully started, you should see the following message in your terminal
|
||||
window:
|
||||
|
||||
```sh
|
||||
$ concurrently "yarn start" "yarn start-backend"
|
||||
$ yarn workspace example-app start
|
||||
$ yarn workspace example-backend start
|
||||
$ backstage-cli app:serve
|
||||
$ backstage-cli backend:dev
|
||||
[0] Loaded config from app-config.yaml
|
||||
[1] Build succeeded
|
||||
[0] ℹ ｢wds｣: Project is running at http://localhost:3000/
|
||||
[0] ℹ ｢wds｣: webpack output is served from /
|
||||
[0] ℹ ｢wds｣: Content not from webpack is served from $BACKSTAGE_DIR/packages/app/public
|
||||
[0] ℹ ｢wds｣: 404s will fallback to /index.html
|
||||
[0] ℹ ｢wdm｣: wait until bundle finished: /
|
||||
[1] 2021-02-12T20:58:17.614Z backstage info Loaded config from app-config.yaml
|
||||
```
|
||||
You can now view example-app in the browser.
|
||||
|
||||
Local: http://localhost:8080
|
||||
On Your Network: http://192.168.1.224:8080
|
||||
```
|
||||
You'll see how you get both logs for the frontend `webpack-dev-server` which
|
||||
serves the react app ([0]) and the backend ([1]);
|
||||
|
||||
Visit http://localhost:3000 and you should see the bleeding edge of Backstage
|
||||
ready for contributions!
|
||||
|
||||
## Editor
|
||||
|
||||
@@ -177,20 +177,6 @@ the root directory:
|
||||
yarn workspace backend start
|
||||
```
|
||||
|
||||
Now you're free to hack away on your own Backstage installation!
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
#### Cannot find module
|
||||
|
||||
You may encounter an error similar to below:
|
||||
|
||||
```
|
||||
internal/modules/cjs/loader.js:968
|
||||
throw err;
|
||||
^
|
||||
|
||||
Error: Cannot find module '../build/Debug/nodegit.node'
|
||||
```
|
||||
|
||||
This can occur if an npm dependency is not completely installed. Because some
|
||||
dependencies run a post-install script, ensure that both your npm and yarn
|
||||
configs have the `ignore-scripts` flag set to `false`.
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
---
|
||||
id: deployment-docker
|
||||
title: Docker
|
||||
description: Documentation on how to deploy Backstage as a Docker image
|
||||
---
|
||||
|
||||
This section describes how to build a Backstage App into a deployable Docker
|
||||
image. It is split into three sections, first covering the host build approach,
|
||||
which is recommended due its speed and more efficient and often simpler caching.
|
||||
The second section covers a full multi-stage Docker build, and the last section
|
||||
covers how to split frontend content into a separate image.
|
||||
|
||||
Something that goes for all of these docker deployment strategies is that they
|
||||
are stateless, so for a production deployment you will want to set up and
|
||||
connect to an external PostgreSQL instance where the backend plugins can store
|
||||
their state, rather than using SQLite.
|
||||
|
||||
### Host Build
|
||||
|
||||
This section describes how to build a Docker image from a Backstage repo with
|
||||
most of the build happening outside of Docker. This is almost always the faster
|
||||
approach, as the build steps tend to execute faster, and it's possible to have
|
||||
more efficient caching of dependencies on the host, where a single change won't
|
||||
bust the entire cache.
|
||||
|
||||
The required steps in the host build are to install dependencies with
|
||||
`yarn install`, generate type definitions using `yarn tsc`, and build all
|
||||
packages with `yarn build`.
|
||||
|
||||
> NOTE: Using `yarn build` to build packages and bundle the backend assumes that
|
||||
> you have migrated to using `backstage-cli backend:bundle` as your build script
|
||||
> in the backend package.
|
||||
|
||||
In a CI workflow it might look something like this:
|
||||
|
||||
```bash
|
||||
yarn install --frozen-lockfile
|
||||
|
||||
# tsc outputs type definitions to dist-types/ in the repo root, which are then consumed by the build
|
||||
yarn tsc
|
||||
|
||||
# Build all packages and in the end bundle them all up into the packages/backend/dist folder.
|
||||
yarn build
|
||||
```
|
||||
|
||||
Once the host build is complete, we are ready to build our image. We use the
|
||||
following `Dockerfile`, which is also included when creating a new app with
|
||||
`@backstage/create-app`:
|
||||
|
||||
```Dockerfile
|
||||
# This dockerfile builds an image for the backend package.
|
||||
# It should be executed with the root of the repo as docker context.
|
||||
#
|
||||
# Before building this image, be sure to have run the following commands in the repo root:
|
||||
#
|
||||
# yarn install
|
||||
# yarn tsc
|
||||
# yarn build
|
||||
#
|
||||
# Once the commands have been run, you can build the image using `yarn build-image`
|
||||
|
||||
FROM node:14-buster-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy repo skeleton first, to avoid unnecessary docker cache invalidation.
|
||||
# The skeleton contains the package.json of each package in the monorepo,
|
||||
# and along with yarn.lock and the root package.json, that's enough to run yarn install.
|
||||
ADD yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./
|
||||
|
||||
RUN yarn install --frozen-lockfile --production --network-timeout 300000 && rm -rf "$(yarn cache dir)"
|
||||
|
||||
# Then copy the rest of the backend bundle, along with any other files we might want.
|
||||
ADD packages/backend/dist/bundle.tar.gz app-config.yaml ./
|
||||
|
||||
CMD ["node", "packages/backend", "--config", "app-config.yaml"]
|
||||
```
|
||||
|
||||
For more details on how the `backend:bundle` command and the `skeleton.tar.gz`
|
||||
file works, see the
|
||||
[`backend:bundle` command docs](../cli/commands.md#backendbundle)
|
||||
|
||||
The `Dockerfile` is typically placed at `packages/backend/Dockerfile`, but needs
|
||||
to be executed with the root of the repo as the build context, in order to get
|
||||
access to the root `yarn.lock` and `package.json`, along with any other files
|
||||
that might be needed, such as `.npmrc`.
|
||||
|
||||
In order to speed up the build we can significantly reduce the build context
|
||||
size using the following `.dockerignore` in the root of the repo:
|
||||
|
||||
```text
|
||||
.git
|
||||
node_modules
|
||||
packages
|
||||
!packages/backend/dist
|
||||
plugins
|
||||
```
|
||||
|
||||
With the project build and the `.dockerignore` and `Dockerfile` in place, we are
|
||||
now ready to build the final image. Assuming we're at the root of the repo, we
|
||||
execute the build like this:
|
||||
|
||||
```bash
|
||||
docker image build . -f packages/backend/Dockerfile --tag backstage
|
||||
```
|
||||
|
||||
To try out the image locally you can run the following:
|
||||
|
||||
```sh
|
||||
docker run -it -p 7000:7000 backstage
|
||||
```
|
||||
|
||||
You should then start to get logs in your terminal, and then you can open your
|
||||
browser at `http://localhost:7000`
|
||||
|
||||
### Multistage Build
|
||||
|
||||
This section describes how to set up a multi-stage Docker build that builds the
|
||||
entire project within Docker. This is typically slower than a host build, but is
|
||||
sometimes desired because Docker in Docker is not available in the build
|
||||
environment, or due to other requirements.
|
||||
|
||||
The build is split into three different stages, where the first stage finds all
|
||||
of the `package.json`s that are relevant for the initial install step enabling
|
||||
us to cache the initial `yarn install` that installs all dependencies. The
|
||||
second stage executes the build itself, and is similar to the steps we execute
|
||||
on the host in the host build. The third and final stage then packages it all
|
||||
together into the final image, and is similar to the `Dockerfile` of the host
|
||||
build.
|
||||
|
||||
The following `Dockerfile` executes the multi-stage build and should be added to
|
||||
the repo root:
|
||||
|
||||
```Dockerfile
|
||||
# Stage 1 - Create yarn install skeleton layer
|
||||
FROM node:14-buster-slim AS packages
|
||||
|
||||
WORKDIR /app
|
||||
COPY package.json yarn.lock ./
|
||||
|
||||
COPY packages packages
|
||||
COPY plugins plugins
|
||||
|
||||
RUN find packages \! -name "package.json" -mindepth 2 -maxdepth 2 -print | xargs rm -rf
|
||||
|
||||
# Stage 2 - Install dependencies and build packages
|
||||
FROM node:14-buster-slim AS build
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=packages /app .
|
||||
|
||||
RUN yarn install --frozen-lockfile --network-timeout 600000 && rm -rf "$(yarn cache dir)"
|
||||
|
||||
COPY . .
|
||||
|
||||
RUN yarn tsc
|
||||
RUN yarn --cwd packages/backend backstage-cli backend:bundle --build-dependencies
|
||||
|
||||
# Stage 3 - Build the actual backend image and install production dependencies
|
||||
FROM node:14-buster-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy the install dependencies from the build stage and context
|
||||
COPY --from=build /app/yarn.lock /app/package.json /app/packages/backend/dist/skeleton.tar.gz ./
|
||||
RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz
|
||||
|
||||
RUN yarn install --frozen-lockfile --production --network-timeout 600000 && rm -rf "$(yarn cache dir)"
|
||||
|
||||
# Copy the built packages from the build stage
|
||||
COPY --from=build /app/packages/backend/dist/bundle.tar.gz .
|
||||
RUN tar xzf bundle.tar.gz && rm bundle.tar.gz
|
||||
|
||||
# Copy any other files that we need at runtime
|
||||
COPY app-config.yaml ./
|
||||
|
||||
CMD ["node", "packages/backend", "--config", "app-config.yaml"]
|
||||
```
|
||||
|
||||
Note that a newly created Backstage app will typically not have a `plugins/`
|
||||
folder, so you will want to comment that line out. This build also does not work
|
||||
in the main repo, since the `backstage-cli` which is used for the build doesn't
|
||||
end up being properly installed.
|
||||
|
||||
To speed up the build when not running in a fresh clone of the repo you should
|
||||
set up a `.dockerignore`. This one is different than the host build one, because
|
||||
we want to have access to the source code of all packages for the build, but can
|
||||
ignore any existing build output or dependencies:
|
||||
|
||||
```text
|
||||
node_modules
|
||||
packages/*/dist
|
||||
packages/*/node_modules
|
||||
plugins/*/dist
|
||||
plugins/*/node_modules
|
||||
```
|
||||
|
||||
Once you have added both the `Dockerfile` and `.dockerignore` to the root of
|
||||
your project, run the following to build the container under a specified tag.
|
||||
|
||||
```sh
|
||||
docker image build -t backstage .
|
||||
```
|
||||
|
||||
To try out the image locally you can run the following:
|
||||
|
||||
```sh
|
||||
docker run -it -p 7000:7000 backstage
|
||||
```
|
||||
|
||||
You should then start to get logs in your terminal, and then you can open your
|
||||
browser at `http://localhost:7000`
|
||||
|
||||
### Separate Frontend
|
||||
|
||||
It is sometimes desirable to serve the frontend separately from the backend,
|
||||
either from a separate image or for example a static file serving provider. The
|
||||
first step in doing so is to remove the `app-backend` plugin from the backend
|
||||
package, which is done as follows:
|
||||
|
||||
1. Delete `packages/backend/src/plugins/app.ts`
|
||||
2. Remove the following lines from `packages/backend/src/index.ts`:
|
||||
```tsx
|
||||
import app from './plugins/app';
|
||||
// ...
|
||||
const appEnv = useHotMemoize(module, () => createEnv('app'));
|
||||
// ...
|
||||
.addRouter('', await app(appEnv));
|
||||
```
|
||||
3. Remove the `@backstage/plugin-app-backend` and the app package dependency
|
||||
(e.g. `app`) from `packages/backend/packages.json`. If you don't remove the
|
||||
app package dependency the app will still be built and bundled with the
|
||||
backend.
|
||||
|
||||
Once the `app-backend` is removed from the backend, you can use your favorite
|
||||
static file serving method for serving the frontend. An example of how to set up
|
||||
an NGINX image is available in the
|
||||
[contrib folder in the main repo](https://github.com/backstage/backstage/blob/master/contrib/docker/frontend-with-nginx/Dockerfile)
|
||||
@@ -4,98 +4,6 @@ title: Other
|
||||
description: Documentation on different ways of Deployment
|
||||
---
|
||||
|
||||
## Docker
|
||||
|
||||
Here we have an example Dockerfile that you can use to build everything together
|
||||
in one container. This Dockerfile uses multi-stage builds, and a
|
||||
`backend:bundle` command from the CLI.
|
||||
|
||||
It also provides caching on the `yarn install`'s so that you don't have to do it
|
||||
unless absolutely necessary.
|
||||
|
||||
> Note: This Dockerfile assumes that you're running SQLite, or your
|
||||
> configuration is setup to connect to an external PostgreSQL Database.
|
||||
|
||||
```Dockerfile
|
||||
# Stage 1 - Create yarn install skeleton layer
|
||||
FROM node:14-buster AS packages
|
||||
|
||||
WORKDIR /app
|
||||
COPY package.json yarn.lock ./
|
||||
|
||||
COPY packages packages
|
||||
|
||||
# Uncomment this line if you have a local plugins folder
|
||||
# COPY plugins plugins
|
||||
|
||||
RUN find packages \! -name "package.json" -mindepth 2 -maxdepth 2 -print | xargs rm -rf
|
||||
|
||||
# Stage 2 - Install dependencies and build packages
|
||||
FROM node:14-buster AS build
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=packages /app .
|
||||
|
||||
RUN yarn install --network-timeout 600000 && rm -rf "$(yarn cache dir)"
|
||||
|
||||
COPY . .
|
||||
|
||||
RUN yarn tsc
|
||||
RUN yarn --cwd packages/backend backstage-cli backend:bundle --build-dependencies
|
||||
|
||||
# Stage 3 - Build the actual backend image and install production dependencies
|
||||
FROM node:14-buster
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy from build stage
|
||||
COPY --from=build /app/yarn.lock /app/package.json /app/packages/backend/dist/skeleton.tar.gz ./
|
||||
RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz
|
||||
|
||||
RUN yarn install --production --network-timeout 600000 && rm -rf "$(yarn cache dir)"
|
||||
|
||||
COPY --from=build /app/packages/backend/dist/bundle.tar.gz .
|
||||
RUN tar xzf bundle.tar.gz && rm bundle.tar.gz
|
||||
|
||||
COPY app-config.yaml app-config.production.yaml ./
|
||||
|
||||
CMD ["node", "packages/backend", "--config", "app-config.yaml", "--config", "app-config.production.yaml"]
|
||||
```
|
||||
|
||||
Before building you should also include a `.dockerignore`. This will greatly
|
||||
improve the context boot up time of Docker as we are no longer sending all of
|
||||
the `node_modules` into the context. It also helps us avoid some limitations and
|
||||
errors that may occur when trying to share the `node_modules` folder to inside
|
||||
the build.
|
||||
|
||||
You can add the following contents to the root of your repository at
|
||||
`.dockerignore` and it might look something like the following:
|
||||
|
||||
```dockerignore
|
||||
.git
|
||||
node_modules
|
||||
packages/*/node_modules
|
||||
plugins/*/node_modules
|
||||
plugins/*/dist
|
||||
```
|
||||
|
||||
Once you have added both the `Dockerfile` and `.dockerignore` to the root of
|
||||
your project, and run the following to build the container under a specified
|
||||
tag.
|
||||
|
||||
```sh
|
||||
$ docker build -t example-deployment .
|
||||
```
|
||||
|
||||
To run the image locally you can run:
|
||||
|
||||
```sh
|
||||
$ docker run -it -p 7000:7000 example-deployment
|
||||
```
|
||||
|
||||
You should then start to get logs in your terminal, and then you can open your
|
||||
browser at `http://localhost:7000`
|
||||
|
||||
## Heroku
|
||||
|
||||
Deploying to Heroku is relatively easy following these steps.
|
||||
|
||||
@@ -19,7 +19,7 @@ general, it's easier to fork and clone this project. That will let you stay up
|
||||
to date with the latest changes, and gives you an easier path to make Pull
|
||||
Requests towards this repo.
|
||||
|
||||
### Creating a Standalone App
|
||||
### Create your Backstage App
|
||||
|
||||
Backstage provides the `@backstage/create-app` package to scaffold standalone
|
||||
instances of Backstage. You will need to have
|
||||
@@ -46,8 +46,3 @@ look something like this. You can read more about this process in
|
||||
You can read more in our
|
||||
[CONTRIBUTING](https://github.com/backstage/backstage/blob/master/CONTRIBUTING.md)
|
||||
guide, which can help you get setup with a Backstage development environment.
|
||||
|
||||
### Next steps
|
||||
|
||||
Take a look at the [Running Backstage Locally](./running-backstage-locally.md)
|
||||
guide to learn how to set up Backstage, and how to develop on the platform.
|
||||
|
||||
@@ -104,7 +104,6 @@ The value of Backstage grows with every new plugin that gets added. Here is a
|
||||
collection of tutorials that will guide you through setting up and extending an
|
||||
instance of Backstage with your own plugins.
|
||||
|
||||
- [Development Environment](development-environment.md)
|
||||
- [Create a Backstage Plugin](../plugins/create-a-plugin.md)
|
||||
- [Structure of a Plugin](../plugins/structure-of-a-plugin.md)
|
||||
- [Utility APIs](../api/utility-apis.md)
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
---
|
||||
id: org
|
||||
title: GitHub Organizational Data
|
||||
sidebar_label: Org Data
|
||||
# prettier-ignore
|
||||
description: Setting up ingestion of organizational data from GitHub
|
||||
---
|
||||
|
||||
The Backstage catalog can be set up to ingest organizational data - teams and
|
||||
users - directly from an organization in GitHub or GitHub Enterprise. The result
|
||||
is a hierarchy of
|
||||
[`User`](../../features/software-catalog/descriptor-format.md#kind-user) and
|
||||
[`Group`](../../features/software-catalog/descriptor-format.md#kind-group) kind
|
||||
entities that mirror your org setup.
|
||||
|
||||
## Installation
|
||||
|
||||
The processor that performs the import, `GithubOrgReaderProcessor`, comes
|
||||
installed with the default setup of Backstage.
|
||||
|
||||
If you replace the set of processors in your installation using that facility of
|
||||
the catalog builder class, you can import and add it as follows.
|
||||
|
||||
```ts
|
||||
// Typically in packages/backend/src/plugins/catalog.ts
|
||||
import { GithubOrgReaderProcessor } from '@backstage/plugin-catalog-backend';
|
||||
|
||||
builder.replaceProcessors(
|
||||
GithubOrgReaderProcessor.fromConfig(config, { logger }),
|
||||
// ...
|
||||
);
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The following configuration enables an import of the teams and users under the
|
||||
org `https://github.com/my-org-name` on public GitHub.
|
||||
|
||||
```yaml
|
||||
catalog:
|
||||
locations:
|
||||
- type: github-org
|
||||
target: https://github.com/my-org-name
|
||||
processors:
|
||||
githubOrg:
|
||||
providers:
|
||||
- target: https://github.com
|
||||
apiBaseUrl: https://api.github.com
|
||||
token:
|
||||
$env: GITHUB_TOKEN
|
||||
```
|
||||
|
||||
Locations point out the specific org(s) you want to import. The `type` of these
|
||||
locations must be `github-org`, and the `target` must point to the exact URL of
|
||||
some organization. You can have several such location entries if you want, but
|
||||
typically you will have just one.
|
||||
|
||||
The processor itself is configured in the other block, under
|
||||
`catalog.processors.githubOrg`. There may be many providers, each targeting a
|
||||
specific `target` which is supposed to be the address of the home page of GitHub
|
||||
or your GitHub Enterprise installation.
|
||||
|
||||
The example above assumes that the backend is started with an environment
|
||||
variable called `GITHUB_TOKEN` that contains a Personal Access Token. The token
|
||||
needs to have at least the scopes `read:org`, `read:user`, and `user:email` in
|
||||
the given `target`.
|
||||
|
||||
If you want to address your own GitHub Enterprise instance, replace occurrences
|
||||
of `https://github.com` in the configuration above with the address of your
|
||||
GitHub Enterprise home page, and the `apiBaseUrl` to where your API endpoint
|
||||
lives - commonly on the form `https://<host>/api/v3`.
|
||||
@@ -0,0 +1,280 @@
|
||||
---
|
||||
id: org
|
||||
title: LDAP Organizational Data
|
||||
sidebar_label: Org Data
|
||||
# prettier-ignore
|
||||
description: Setting up ingestion of organizational data from LDAP
|
||||
---
|
||||
|
||||
The Backstage catalog can be set up to ingest organizational data - groups and
|
||||
users - directly from an LDAP compatible service. The result is a hierarchy of
|
||||
[`User`](../../features/software-catalog/descriptor-format.md#kind-user) and
|
||||
[`Group`](../../features/software-catalog/descriptor-format.md#kind-group) kind
|
||||
entities that mirror your org setup.
|
||||
|
||||
## Installation
|
||||
|
||||
The processor that performs the import, `LdapOrgReaderProcessor`, comes
|
||||
installed with the default setup of Backstage.
|
||||
|
||||
If you replace the set of processors in your installation using that facility of
|
||||
the catalog builder class, you can import and add it as follows.
|
||||
|
||||
```ts
|
||||
// Typically in packages/backend/src/plugins/catalog.ts
|
||||
import { LdapOrgReaderProcessor } from '@backstage/plugin-catalog-backend';
|
||||
|
||||
builder.replaceProcessors(
|
||||
LdapOrgReaderProcessor.fromConfig(config, { logger }),
|
||||
// ...
|
||||
);
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The following configuration is a small example of how a setup could look for
|
||||
importing groups and users from a corporate LDAP server.
|
||||
|
||||
```yaml
|
||||
catalog:
|
||||
locations:
|
||||
- type: ldap-org
|
||||
target: ldaps://ds.example.net
|
||||
processors:
|
||||
ldapOrg:
|
||||
providers:
|
||||
- target: ldaps://ds.example.net
|
||||
bind:
|
||||
dn: uid=ldap-reader-user,ou=people,ou=example,dc=example,dc=net
|
||||
secret:
|
||||
$env: LDAP_SECRET
|
||||
users:
|
||||
dn: ou=people,ou=example,dc=example,dc=net
|
||||
options:
|
||||
filter: (uid=*)
|
||||
map:
|
||||
description: l
|
||||
set:
|
||||
metadata.customField: 'hello'
|
||||
groups:
|
||||
dn: ou=access,ou=groups,ou=example,dc=example,dc=net
|
||||
options:
|
||||
filter: (&(objectClass=some-group-class)(!(groupType=email)))
|
||||
map:
|
||||
description: l
|
||||
set:
|
||||
metadata.customField: 'hello'
|
||||
```
|
||||
|
||||
Locations point out the specific org(s) you want to import. The `type` of these
|
||||
locations must be `ldap-org`, and the `target` must point to the exact URL
|
||||
(starting with `ldap://` or `ldaps://`) of the targeted LDAP server. You can
|
||||
have several such location entries if you want, but typically you will have just
|
||||
one.
|
||||
|
||||
The processor itself is configured in the other block, under
|
||||
`catalog.processors.ldapOrg`. There may be many providers, each targeting a
|
||||
specific `target` which is supposed to be on the same form as the location
|
||||
`target`.
|
||||
|
||||
These config blocks have a lot of options in them, so we will describe each
|
||||
"root" key within the block separately.
|
||||
|
||||
### target
|
||||
|
||||
This is the URL of the targeted server, typically on the form
|
||||
`ldaps://ds.example.net` for SSL enabled servers or `ldap://ds.example.net`
|
||||
without SSL.
|
||||
|
||||
### bind
|
||||
|
||||
The bind block specifies how the plugin should bind (essentially, to
|
||||
authenticate) towards the server. It has the following fields.
|
||||
|
||||
```yaml
|
||||
dn: uid=ldap-reader-user,ou=people,ou=example,dc=example,dc=net
|
||||
secret:
|
||||
$env: LDAP_SECRET
|
||||
```
|
||||
|
||||
The `dn` is the full LDAP DN (distinguished name) for the user that the plugin
|
||||
authenticates itself as. At this point, only regular user based authentication
|
||||
is supported.
|
||||
|
||||
The `secret` is the password of the same user. In this example, it is given in
|
||||
the form of an environment variable `LDAP_SECRET`, that has to be set when the
|
||||
backend starts.
|
||||
|
||||
### users
|
||||
|
||||
The `users` block defines the settings that govern the reading and
|
||||
interpretation of users. Its fields are explained in separate sections below.
|
||||
|
||||
### users.dn
|
||||
|
||||
The DN under which users are stored, e.g.
|
||||
`ou=people,ou=example,dc=example,dc=net`.
|
||||
|
||||
### users.options
|
||||
|
||||
The search options to use when sending the query to the server, when reading all
|
||||
users. All of the options are shown below, with their default values, but they
|
||||
are all optional.
|
||||
|
||||
```yaml
|
||||
options:
|
||||
# One of 'base', 'one', or 'sub'.
|
||||
scope: one
|
||||
# The filter is the one that you commonly will want to specify explicitly. It
|
||||
# is a string on the standard LDAP query format. Use it to select out the set
|
||||
# of users that are of actual interest to ingest. For example, you may want
|
||||
# to filter out disabled users.
|
||||
filter: (uid=*)
|
||||
# The attribute selectors for each item, as passed to the LDAP server.
|
||||
attributes: ['*', '+']
|
||||
# This field is either 'false' to disable paging when reading from the
|
||||
# server, or an object on the form '{ pageSize: 100, pagePause: true }' that
|
||||
# specifies the details of how the paging shall work.
|
||||
paged: false
|
||||
```
|
||||
|
||||
### users.set
|
||||
|
||||
This optional piece lets you specify a number of JSON paths (on a.b.c form) and
|
||||
hard coded values to set on those paths. This can be useful for example if you
|
||||
want to hard code a namespace or similar on the generated entities.
|
||||
|
||||
```yaml
|
||||
set:
|
||||
# Just an example; the key and value can be anything
|
||||
metadata.namespace: 'ldap'
|
||||
```
|
||||
|
||||
### users.map
|
||||
|
||||
Mappings from well known entity fields, to LDAP attribute names. This is where
|
||||
you are able to define how to interpret the attributes of each LDAP result item,
|
||||
and to move them into the corresponding entity fields. All of the options are
|
||||
shown below, with their default values, but they are all optional.
|
||||
|
||||
If you leave out an optional mapping, it will still be copied using that default
|
||||
value. For example, even if you do not put in the field `displayName` in your
|
||||
config, the processor will still copy the attribute `cn` into the entity field
|
||||
`spec.profile.displayName`.
|
||||
|
||||
```yaml
|
||||
map:
|
||||
# The name of the attribute that holds the relative
|
||||
# distinguished name of each entry.
|
||||
rdn: uid
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the metadata.name field of the entity.
|
||||
name: uid
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the metadata.description field of the entity.
|
||||
description: description
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the spec.profile.displayName field of the entity.
|
||||
displayName: cn
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the spec.profile.email field of the entity.
|
||||
email: mail
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the spec.profile.picture field of the entity.
|
||||
picture: <nothing, left out>
|
||||
# The name of the attribute that shall be used for the values of
|
||||
# the spec.memberOf field of the entity.
|
||||
memberOf: memberOf
|
||||
```
|
||||
|
||||
### groups
|
||||
|
||||
The `groups` block defines the settings that govern the reading and
|
||||
interpretation of groups. Its fields are explained in separate sections below.
|
||||
|
||||
### groups.dn
|
||||
|
||||
The DN under which groups are stored, e.g.
|
||||
`ou=people,ou=example,dc=example,dc=net`.
|
||||
|
||||
### groups.options
|
||||
|
||||
The search options to use when sending the query to the server, when reading all
|
||||
groups. All of the options are shown below, with their default values, but they
|
||||
are all optional.
|
||||
|
||||
```yaml
|
||||
options:
|
||||
# One of 'base', 'one', or 'sub'.
|
||||
scope: one
|
||||
# The filter is the one that you commonly will want to specify explicitly. It
|
||||
# is a string on the standard LDAP query format. Use it to select out the set
|
||||
# of groups that are of actual interest to ingest. For example, you may want
|
||||
# to filter out disabled groups.
|
||||
filter: (&(objectClass=some-group-class)(!(groupType=email)))
|
||||
# The attribute selectors for each item, as passed to the LDAP server.
|
||||
attributes: ['*', '+']
|
||||
# This field is either 'false' to disable paging when reading from the
|
||||
# server, or an object on the form '{ pageSize: 100, pagePause: true }' that
|
||||
# specifies the details of how the paging shall work.
|
||||
paged: false
|
||||
```
|
||||
|
||||
### groups.set
|
||||
|
||||
This optional piece lets you specify a number of JSON paths (on a.b.c form) and
|
||||
hard coded values to set on those paths. This can be useful for example if you
|
||||
want to hard code a namespace or similar on the generated entities.
|
||||
|
||||
```yaml
|
||||
set:
|
||||
# Just an example; the key and value can be anything
|
||||
metadata.namespace: 'ldap'
|
||||
```
|
||||
|
||||
### groups.map
|
||||
|
||||
Mappings from well known entity fields, to LDAP attribute names. This is where
|
||||
you are able to define how to interpret the attributes of each LDAP result item,
|
||||
and to move them into the corresponding entity fields. All of the options are
|
||||
shown below, with their default values, but they are all optional.
|
||||
|
||||
If you leave out an optional mapping, it will still be copied using that default
|
||||
value. For example, even if you do not put in the field `displayName` in your
|
||||
config, the processor will still copy the attribute `cn` into the entity field
|
||||
`spec.profile.displayName`. If the target field is optional, such as the display
|
||||
name, the importer will accept missing attributes and just leave the target
|
||||
field unset. If the target field is mandatory, such as the name of the entity,
|
||||
validation will fail if the source attribute is missing.
|
||||
|
||||
```yaml
|
||||
map:
|
||||
# The name of the attribute that holds the relative
|
||||
# distinguished name of each entry. This value is copied into a
|
||||
# well known annotation to be able to query by it later.
|
||||
rdn: cn
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the metadata.name field of the entity.
|
||||
name: cn
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the metadata.description field of the entity.
|
||||
description: description
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the spec.type field of the entity.
|
||||
type: groupType
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the spec.profile.displayName field of the entity.
|
||||
displayName: cn
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the spec.profile.email field of the entity.
|
||||
email: <nothing, left out>
|
||||
# The name of the attribute that shall be used for the value of
|
||||
# the spec.profile.picture field of the entity.
|
||||
picture: <nothing, left out>
|
||||
# The name of the attribute that shall be used for the values of
|
||||
# the spec.parent field of the entity.
|
||||
memberOf: memberOf
|
||||
# The name of the attribute that shall be used for the values of
|
||||
# the spec.children field of the entity.
|
||||
members: member
|
||||
```
|
||||
+11
-11
@@ -27,17 +27,17 @@ to grant the app more permissions if needed.
|
||||
|
||||
### Using the CLI (public GitHub only)
|
||||
|
||||
You can use the `backstage-cli` to create GitHub App' using a manifest file that
|
||||
we provide. This gives us a way to automate some of the work required to create
|
||||
a GitHub app.
|
||||
You can use the `backstage-cli` to create a GitHub App using a manifest file
|
||||
that we provide. This gives us a way to automate some of the work required to
|
||||
create a GitHub app.
|
||||
|
||||
You can read more about the `backstage-cli create-github-app` method
|
||||
[here](../cli/commands.md#create-github-app)
|
||||
You can read more about the
|
||||
[`backstage-cli create-github-app` method](../cli/commands.md#create-github-app).
|
||||
|
||||
Once you've gone through the CLI command, it should produce a `yaml` file in the
|
||||
Once you've gone through the CLI command, it should produce a YAML file in the
|
||||
root of the project which you can then use as an `include` in your
|
||||
`app-config.yaml`. You can go ahead and skip to
|
||||
[here](#including-in-integrations-config) if you've got to this part.
|
||||
`app-config.yaml`. You can go ahead and
|
||||
[skip ahead](#including-in-integrations-config) if you've already got an app.
|
||||
|
||||
### GitHub Enterprise
|
||||
|
||||
@@ -46,9 +46,9 @@ You have to create the GitHub Application manually using these
|
||||
as GitHub Enterprise does not support creation of apps from manifests.
|
||||
|
||||
Once the application is created you have to generate a private key for the
|
||||
application it in a `yaml` file.
|
||||
application and place it in a YAML file.
|
||||
|
||||
The yaml file must include the following information. Please note that the
|
||||
The YAML file must include the following information. Please note that the
|
||||
indentation for the `privateKey` is required.
|
||||
|
||||
```yaml
|
||||
@@ -64,7 +64,7 @@ privateKey: |
|
||||
|
||||
### Including in Integrations Config
|
||||
|
||||
Once the credentials are stored in a yaml file generated by `create-github-app`
|
||||
Once the credentials are stored in a YAML file generated by `create-github-app`,
|
||||
or manually by following the [GitHub Enterprise](#gitHub-enterprise)
|
||||
instructions, they can be included in the `app-config.yaml` under the
|
||||
`integrations` section.
|
||||
|
||||
Reference in New Issue
Block a user