From d13aea9cd29f604571261758becd75d680df23ca Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Wed, 17 Feb 2021 18:01:59 +0100 Subject: [PATCH] docs: add full docker deployment docs --- docs/cli/commands.md | 6 +- docs/getting-started/deployment-docker.md | 237 ++++++++++++++++++++++ docs/getting-started/deployment-other.md | 92 --------- microsite/sidebars.json | 1 + mkdocs.yml | 1 + 5 files changed, 242 insertions(+), 95 deletions(-) create mode 100644 docs/getting-started/deployment-docker.md diff --git a/docs/cli/commands.md b/docs/cli/commands.md index 726f2a922c..ea72f7e050 100644 --- a/docs/cli/commands.md +++ b/docs/cli/commands.md @@ -206,15 +206,15 @@ The following is an example of a `Dockerfile` that can be used to package the output of `backstage-cli backend:bundle` into an image: ```Dockerfile -FROM node:14-buster +FROM node:14-buster-slim WORKDIR /app ADD yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./ -RUN yarn install --production --network-timeout 600000 && rm -rf "$(yarn cache dir)" +RUN yarn install --frozen-lockfile --production --network-timeout 300000 && rm -rf "$(yarn cache dir)" ADD packages/backend/dist/bundle.tar.gz app-config.yaml ./ -CMD node packages/backend +CMD ["node", "packages/backend"] ``` ```text diff --git a/docs/getting-started/deployment-docker.md b/docs/getting-started/deployment-docker.md new file mode 100644 index 0000000000..dc7e33c60d --- /dev/null +++ b/docs/getting-started/deployment-docker.md @@ -0,0 +1,237 @@ +--- +id: deployment-docker +title: Docker +description: Documentation on how to deploy Backstage as a Docker image +--- + +This section describes how to build a Backstage App into a deployable Docker +image. It is split into three sections, first covering the host build approach, +which is recommended due its speed and more efficient and often simpler caching. +The second section covers a full multi-stage Docker build, and the last section +covers how to split frontend content into a separate image. + +Something that goes for all of these docker deployment strategies is that they +are stateless, so for a production deployment you will want to set up and +connect to an external PostgreSQL instance where the backend plugins can store +their state, rather than using SQLite. + +### Host Build + +This section describes how to build a Docker image from a Backstage repo with +most of the build happening outside of Docker. This is almost always the faster +approach, as the build steps tend to execute faster, and it's possible to have +more efficient caching of dependencies on the host, where a single change won't +bust the entire cache. + +The required steps in the host build are to install dependencies with +`yarn install`, generate type definitions using `yarn tsc`, and build all +packages with `yarn build`. + +> NOTE: Using `yarn build` to build packages and bundle the backend assumes that +> you have migrated to using `backstage-cli backend:bundle` as your build script +> in the backend package. + +In a CI workflow it might look something like this: + +```bash +yarn install --frozen-lockfile + +# tsc outputs type definitions to dist-types/ in the repo root, which are then consumed by the build +yarn tsc + +# Build all packages and in the end bundle them all up into the packages/backend/dist folder. +yarn build +``` + +Once the host build is complete, we are ready to build our image. We use the +following `Dockerfile`, which is also included when creating a new app with +`@backstage/create-app`: + +```Dockerfile +# This dockerfile builds an image for the backend package. +# It should be executed with the root of the repo as docker context. +# +# Before building this image, be sure to have run the following commands in the repo root: +# +# yarn install +# yarn tsc +# yarn build + +FROM node:14-buster-slim + +WORKDIR /app + +# Copy repo skeleton first, to avoid unnecessary docker cache invalidation. +# The skeleton contains the package.json of each package in the monorepo, +# and along with yarn.lock and the root package.json, that's enough to run yarn install. +ADD yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./ + +RUN yarn install --frozen-lockfile --production --network-timeout 300000 && rm -rf "$(yarn cache dir)" + +# This will copy the contents of the dist-workspace when running the build-image command. +# Do not use this Dockerfile outside of that command, as it will copy in the source code instead. +ADD packages/backend/dist/bundle.tar.gz app-config.yaml ./ + +CMD ["node", "packages/backend"] +``` + +For more details on how the `backend:bundle` command and the `skeleton.tar.gz` +file works, see the +[`backend:bundle` command docs](../cli/commands.md#backendbundle) + +The `Dockerfile` is typically placed at `packages/backend/Dockerfile`, but needs +to be executed with the root of the repo as the build context, in order to get +access to the root `yarn.lock` and `package.json`, along with any other files +that might be needed, such as `.npmrc`. + +In order to speed up the build we can significantly reduce the build context +size using the following `.dockerignore` in the root of the repo: + +```text +.git +node_modules +packages +!packages/backend/dist +plugins +``` + +With the project build and the `.dockerignore` and `Dockerfile` in place, we are +now ready to build the final image. Assuming we're at the root of the repo, we +execute the build like this: + +```bash +docker image build . -f packages/backend/Dockerfile --tag backstage +``` + +To try out the image locally you can run the following: + +```sh +docker run -it -p 7000:7000 backstage +``` + +You should then start to get logs in your terminal, and then you can open your +browser at `http://localhost:7000` + +### Multistage Build + +This section describes how to set up a multi-stage Docker build that builds the +entire project within Docker. This is typically slower than a host build, but is +sometimes desired because Docker in Docker is not available in the build +environment, or due to other requirements. + +The build is split into three different stages, where the first stage finds all +of the `package.json`s that are relevant for the initial install step enabling +us to cache the initial `yarn install` that installs all dependencies. The +second stage executes the build itself, and is similar to the steps we execute +on the host in the host build. The third and final stage then packages it all +together into the final image, and is similar to the `Dockerfile` of the host +build. + +The following `Dockerfile` executes the multi-stage build and should be added to +the repo root: + +```Dockerfile +# Stage 1 - Create yarn install skeleton layer +FROM node:14-buster-slim AS packages + +WORKDIR /app +COPY package.json yarn.lock ./ + +COPY packages packages +COPY plugins plugins + +RUN find packages \! -name "package.json" -mindepth 2 -maxdepth 2 -print | xargs rm -rf + +# Stage 2 - Install dependencies and build packages +FROM node:14-buster-slim AS build + +WORKDIR /app +COPY --from=packages /app . + +RUN yarn install --frozen-lockfile --network-timeout 600000 && rm -rf "$(yarn cache dir)" + +COPY . . + +RUN yarn tsc +RUN yarn --cwd packages/backend backstage-cli backend:bundle --build-dependencies + +# Stage 3 - Build the actual backend image and install production dependencies +FROM node:14-buster-slim + +WORKDIR /app + +# Copy the install dependencies from the build stage and context +COPY --from=build /app/yarn.lock /app/package.json /app/packages/backend/dist/skeleton.tar.gz ./ +RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz + +RUN yarn install --frozen-lockfile --production --network-timeout 600000 && rm -rf "$(yarn cache dir)" + +# Copy the built packages from the build stage +COPY --from=build /app/packages/backend/dist/bundle.tar.gz . +RUN tar xzf bundle.tar.gz && rm bundle.tar.gz + +# Copy any other files that we need at runtime +COPY app-config.yaml ./ + +CMD ["node", "packages/backend", "--config", "app-config.yaml"] +``` + +Note that a newly created Backstage app will typically not have a `plugins/` +folder, so you will want to comment that line out. This build also does not work +in the main repo, since the `backstage-cli` which is used for the build doesn't +end up being properly installed. + +To speed up the build when not running in a fresh clone of the repo you should +set up a `.dockerignore`. This one is different than the host build one, because +we want to have access to the source code of all packages for the build, but can +ignore any existing build output or dependencies: + +```text +node_modules +packages/*/dist +packages/*/node_modules +plugins/*/dist +plugins/*/node_modules +``` + +Once you have added both the `Dockerfile` and `.dockerignore` to the root of +your project, run the following to build the container under a specified tag. + +```sh +docker image build -t backstage . +``` + +To try out the image locally you can run the following: + +```sh +docker run -it -p 7000:7000 backstage +``` + +You should then start to get logs in your terminal, and then you can open your +browser at `http://localhost:7000` + +### Separate Frontend + +It is sometimes desirable to serve the frontend separately from the backend, +either from a separate image or for example a static file serving provider. The +first step in doing so is to remove the `app-backend` plugin from the backend +package, which is done as follows: + +1. Delete `packages/backend/src/plugins/app.ts` +2. Remove the following lines from `packages/backend/src/index.ts`: + ```tsx + import app from './plugins/app'; + // ... + const appEnv = useHotMemoize(module, () => createEnv('app')); + // ... + .addRouter('', await app(appEnv)); + ``` +3. Remove the `@backstage/plugin-app-backend` and the app package dependency + (e.g. `app`) from `packages/backend/packages.json`. If you don't remove the + app package dependency the app will still be built and bundled with the + backend. + +Once the `app-backend` is removed from the backend, you can use your favorite +static file serving method for serving the frontend. An example of how to set up +an NGINX image is available in the +[contrib folder in the main repo](https://github.com/backstage/backstage/blob/master/contrib/docker/frontend-with-nginx/Dockerfile) diff --git a/docs/getting-started/deployment-other.md b/docs/getting-started/deployment-other.md index e92b54d08b..cc622d00f5 100644 --- a/docs/getting-started/deployment-other.md +++ b/docs/getting-started/deployment-other.md @@ -4,98 +4,6 @@ title: Other description: Documentation on different ways of Deployment --- -## Docker - -Here we have an example Dockerfile that you can use to build everything together -in one container. This Dockerfile uses multi-stage builds, and a -`backend:bundle` command from the CLI. - -It also provides caching on the `yarn install`'s so that you don't have to do it -unless absolutely necessary. - -> Note: This Dockerfile assumes that you're running SQLite, or your -> configuration is setup to connect to an external PostgreSQL Database. - -```Dockerfile -# Stage 1 - Create yarn install skeleton layer -FROM node:14-buster AS packages - -WORKDIR /app -COPY package.json yarn.lock ./ - -COPY packages packages - -# Uncomment this line if you have a local plugins folder -# COPY plugins plugins - -RUN find packages \! -name "package.json" -mindepth 2 -maxdepth 2 -print | xargs rm -rf - -# Stage 2 - Install dependencies and build packages -FROM node:14-buster AS build - -WORKDIR /app -COPY --from=packages /app . - -RUN yarn install --network-timeout 600000 && rm -rf "$(yarn cache dir)" - -COPY . . - -RUN yarn tsc -RUN yarn --cwd packages/backend backstage-cli backend:bundle --build-dependencies - -# Stage 3 - Build the actual backend image and install production dependencies -FROM node:14-buster - -WORKDIR /app - -# Copy from build stage -COPY --from=build /app/yarn.lock /app/package.json /app/packages/backend/dist/skeleton.tar.gz ./ -RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz - -RUN yarn install --production --network-timeout 600000 && rm -rf "$(yarn cache dir)" - -COPY --from=build /app/packages/backend/dist/bundle.tar.gz . -RUN tar xzf bundle.tar.gz && rm bundle.tar.gz - -COPY app-config.yaml app-config.production.yaml ./ - -CMD ["node", "packages/backend", "--config", "app-config.yaml", "--config", "app-config.production.yaml"] -``` - -Before building you should also include a `.dockerignore`. This will greatly -improve the context boot up time of Docker as we are no longer sending all of -the `node_modules` into the context. It also helps us avoid some limitations and -errors that may occur when trying to share the `node_modules` folder to inside -the build. - -You can add the following contents to the root of your repository at -`.dockerignore` and it might look something like the following: - -```dockerignore -.git -node_modules -packages/*/node_modules -plugins/*/node_modules -plugins/*/dist -``` - -Once you have added both the `Dockerfile` and `.dockerignore` to the root of -your project, and run the following to build the container under a specified -tag. - -```sh -$ docker build -t example-deployment . -``` - -To run the image locally you can run: - -```sh -$ docker run -it -p 7000:7000 example-deployment -``` - -You should then start to get logs in your terminal, and then you can open your -browser at `http://localhost:7000` - ## Heroku Deploying to Heroku is relatively easy following these steps. diff --git a/microsite/sidebars.json b/microsite/sidebars.json index ebce8d900c..0ac8f2929b 100644 --- a/microsite/sidebars.json +++ b/microsite/sidebars.json @@ -27,6 +27,7 @@ "type": "subcategory", "label": "Deployment", "ids": [ + "getting-started/deployment-docker", "getting-started/deployment-k8s", "getting-started/deployment-helm", "getting-started/deployment-other" diff --git a/mkdocs.yml b/mkdocs.yml index a303fd9978..0e3c4494e1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -24,6 +24,7 @@ nav: - Configuring App with plugins: 'getting-started/configure-app-with-plugins.md' - Customize the look-and-feel of your App: 'getting-started/app-custom-theme.md' - Deployment scenarios: + - Docker: 'getting-started/deployment-docker.md' - Kubernetes: 'getting-started/deployment-k8s.md' - Kubernetes and Helm: 'getting-started/deployment-helm.md' - Other: 'getting-started/deployment-other.md'