paulononato/backstage
1
0
Fork 0
Code Issues Pull Requests Actions 35 Packages Projects Releases Wiki Activity

docs: restructure local-dev and rename to tooling

Browse Source 
Signed-off-by: Patrik Oldsberg <poldsberg@gmail.com>
This commit is contained in:
Patrik Oldsberg
2024-05-17 11:28:17 +02:00
parent e6fb2dc950
commit a1ae9cc72c
20 changed files with 66 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tooling/cli/01-overview.md
+39
View File
@@ -0,0 +1,39 @@
---
id: overview
title: Overview
description: Overview of the Backstage CLI
---
## Introduction
A goal of Backstage is to provide a delightful developer experience in and
around the project. Creating new [apps](../../references/glossary.md#app) and
[plugins](../../references/glossary.md#plugin) should be simple, iteration
speed should be fast, and the overhead of maintaining custom tooling should be
minimal. As a part of accomplishing this goal, Backstage provides its own build
system and tooling, delivered primarily through the
[`@backstage/cli`](https://www.npmjs.com/package/@backstage/cli) [package](../../references/glossary.md#package). When
creating an app using
[`@backstage/create-app`](https://www.npmjs.com/package/@backstage/create-app),
you receive a project that's already prepared with a typical setup and package
scripts for executing the most common commands.
Under the hood the CLI uses [Webpack](https://webpack.js.org/) for bundling,
[Rollup](https://rollupjs.org/) for building packages,
[Jest](https://jestjs.io/) for testing, and [eslint](https://eslint.org/) for
linting. It also includes tooling for working within Backstage apps, for example
for keeping the app up to date and verifying static configuration. For a more
in-depth look into the tooling, see the [build system](./02-build-system.md)
page, and for a list of commands, see the [commands](./03-commands.md) page.
While the Backstage tooling is opinionated in how it works, it is also possible
to use your own tooling either partially or fully. For example, the CLI provides
a command for building a plugin package for publishing, but the output is a
quite standard combination of transpiled JavaScript and TypeScript type
declarations. The usage of the command from the CLI can therefore be augmented
or replaced with other tools if necessary.
The Backstage CLI intentionally does not provide many hooks for overriding or
customizing the build process. This is to allow for evolution of the CLI without
having to take a wide API surface into account. This allows us to iterate and
improve the tooling, as well as to more easily keep the system up to date.
docs/tooling/cli/02-build-system.md
+681
View File
@@ -0,0 +1,681 @@
---
id: build-system
title: Build System
description: A deep dive into the Backstage build system
---
The Backstage build system is a collection of build and development tools that
help you lint, test, develop and finally release your Backstage projects. The
purpose of the build system is to provide an out-of-the-box solution that works
well with Backstage and lets you focus on building an app rather than having to
spend time setting up your own tooling.
The build system setup is part of the
[@backstage/cli](https://www.npmjs.com/package/@backstage/cli), and already
included in any project that you create using
[@backstage/create-app](https://www.npmjs.com/package/@backstage/create-app). It
is similar to for example
[react-scripts](https://www.npmjs.com/package/react-scripts), which is the
tooling you get with
[create-react-app](https://github.com/facebook/create-react-app). The Backstage
build system relies heavily on existing open source tools from the JavaScript
and TypeScript ecosystem, such as [Webpack](https://webpack.js.org/),
[Rollup](https://rollupjs.org/), [Jest](https://jestjs.io/), and
[ESLint](https://eslint.org/).
## Design Considerations
There are a couple of core beliefs and constraints that guided the design of the
Backstage build system. The first and most important is that we put the
development experience first. If we need to cut corners or add complexity we do
so in other areas, but the experience of firing up an editor and iterating on
some code should always be as smooth as possible.
In addition, there are a number of hard and soft requirements:
- Monorepos - The build system should support multi-package workspaces
- Publishing - It should be possible to build and publish individual packages
- Scale - It should scale to hundreds of large packages without excessive wait
times
- Reloads - The development flow should support quick on-save hot reloads
- Simple - Usage should be simple and configuration should be kept minimal
- Universal - Development towards both web applications, isomorphic packages,
and Node.js
- Modern - The build system targets modern environments
- Editors - Type checking and linting should be available within most editors
During the design of the build system this collection of requirements was not
something that was supported by existing tools like for example `react-scripts`.
The requirements of scaling in combination of monorepo, publishing, and editor
support led us to adopting our own specialized setup.
## Structure
We can divide the development flow within Backstage into a couple of different
steps:
- **Formatting** - Applies a consistent formatting to your source code
- **Linting** - Analyzes your source code for potential problems
- **Type Checking** - Verifies that TypeScript types are valid
- **Testing** - Runs different levels of test suites towards your project
- **Building** - Compiles the source code in an individual package
- **Bundling** - Combines a package and all of its dependencies into a
production-ready bundle
These steps are generally kept isolated from each other, with each step focusing
on its specific task. For example, we do not do linting or type checking
together with the building or bundling. This is so that we can provide more
flexibility and avoid duplicate work, improving performance. It is strongly
recommended that as a part of developing within Backstage you use a code editor
or IDE that has support for formatting, linting, and type checking.
Let's dive into a detailed look at each of these steps and how they are
implemented in a typical Backstage app.
## Package Roles
> Package roles were introduced in March 2022. To migrate existing projects, see the [migration guide](../../tutorials/package-role-migration.md).
The Backstage build system uses the concept of package roles in order to help keep
configuration lean, provide utility and tooling, and enable optimizations. A package
role is a single string that identifies what the purpose of a package is, and it's
defined in the `package.json` of each package like this:
```json
{
"name": "my-package",
"backstage": {
"role": "<role>"
},
...
}
```
These are the available roles that are currently supported by the Backstage build system:
| Role | Description | Example |
| ---------------------- | -------------------------------------------- | -------------------------------------------- |
| frontend | Bundled frontend application | `packages/app` |
| backend | Bundled backend application | `packages/backend` |
| cli | Package used as a command-line interface | `@backstage/cli`, `@backstage/codemods` |
| web-library | Web library for use by other packages | `@backstage/plugin-catalog-react` |
| node-library | Node.js library for use by other packages | `@backstage/plugin-techdocs-node` |
| common-library | Isomorphic library for use by other packages | `@backstage/plugin-permission-common` |
| frontend-plugin | Backstage frontend plugin | `@backstage/plugin-scaffolder` |
| frontend-plugin-module | Backstage frontend plugin module | `@backstage/plugin-analytics-module-ga` |
| backend-plugin | Backstage backend plugin | `@backstage/plugin-auth-backend` |
| backend-plugin-module | Backstage backend plugin module | `@backstage/plugin-search-backend-module-pg` |
Most of the steps that we cover below have an accompanying command that is intended to be used as a package script. The commands are all available under the `backstage-cli package` category, and many of the commands will behave differently depending on the role of the package. The commands are intended to be used like this:
```json
{
"scripts": {
"start": "backstage-cli package start",
"build": "backstage-cli package build",
"lint": "backstage-cli package lint",
...
}
}
```
## Formatting
The formatting setup lives completely within each Backstage application and is
not part of the CLI. In an app created with `@backstage/create-app` the
formatting is handled by [prettier](https://prettier.io/), but each application
can choose their own formatting rules and switch to a different formatter if
desired.
## Linting
The Backstage CLI includes a `lint` command, which is a thin wrapper around
`eslint`. It adds a few options that can't be set through configuration, such as
including the `.ts` and `.tsx` extensions in the set of linted files. The `lint`
command simply provides a sane default and is not intended to be customizable.
If you want to supply more advanced options you can invoke `eslint` directly
instead.
In addition to the `lint` command, the Backstage CLI also includes a set of base
ESLint configurations, one for frontend and one for backend packages. These lint
configurations in turn build on top of the lint rules from
[@spotify/web-scripts](https://github.com/spotify/web-scripts).
In a standard Backstage setup, each individual package has its own lint
configuration, along with a root configuration that applies to the entire
project. The configuration in each package starts out as a standard configuration
that is determined based on the package role, but it can be customized to fit the needs of each package.
A minimal `.eslintrc.js` configuration now looks like this:
```js
module.exports = require('@backstage/cli/config/eslint-factory')(__dirname);
```
But you can provide custom overrides for each package using the optional second argument:
```js
module.exports = require('@backstage/cli/config/eslint-factory')(__dirname, {
ignorePatterns: ['templates/'],
rules: {
'jest/expect-expect': 'off',
},
});
```
The configuration factory also provides utilities for extending the configuration in ways that are otherwise very cumbersome to do with plain ESLint, particularly for rules like `no-restricted-syntax`. These are the extra keys that are available:
| Key | Description |
| -------------------------- | ------------------------------------------------------------------ |
| `tsRules` | Additional rules to apply to TypeScript files |
| `testRules` | Additional rules to apply to tests files |
| `restrictedImports` | Additional paths to add to `no-restricted-imports` |
| `restrictedImportPatterns` | Additional patterns to add to `no-restricted-imports` |
| `restrictedSrcImports` | Additional paths to add to `no-restricted-imports` in src files |
| `restrictedTestImports` | Additional paths to add to `no-restricted-imports` in test files |
| `restrictedSyntax` | Additional patterns to add to `no-restricted-syntax` |
| `restrictedSrcSyntax` | Additional patterns to add to `no-restricted-syntax` in src files |
| `restrictedTestSyntax` | Additional patterns to add to `no-restricted-syntax` in test files |
## Type Checking
Just like formatting, the Backstage CLI does not have its own command for type
checking. It does however have a base configuration with both recommended
defaults as well as some required settings for the build system to work.
Perhaps the most notable part about the TypeScript setup in Backstage projects
is that the entire project is one big compilation unit. This is due to
performance optimization as well as ease of use, since breaking projects down
into smaller pieces has proven to both lead to a more complicated setup, as well
as type checking of the entire project being an order of magnitude slower. In
order to make this setup work, the entrypoint of each package needs to point to
the TypeScript source files, which in turn causes some complications during
publishing that we'll talk about in [that section](#publishing).
The type checking is generally configured to be incremental for local
development, with the output stored in the `dist-types` folder in the repo root.
This provides a significant speedup when running `tsc` multiple times locally,
but it does make the initial run a little bit slower. Because of the slower
initial run we disable incremental type checking in the `tcs:full` Yarn script
that is included by default in any Backstage app and is intended for use in CI.
Another optimization that is used by default is to skip the checking of library
types, this means that TypeScript will not verify that types within
`node_modules` are sound. Disabling this check significantly speeds up type
checking, but in the end it is still an important check that should not be
completely omitted, it's simply unlikely to catch issues that are introduced
during local development. What we opt for instead is to include the check in CI
through the `tsc:full` script, which will run a full type check, including
`node_modules`.
For the two reasons mentioned above, it is **highly** recommended to use the
`tsc:full` script to run type checking in CI.
## Testing
As mentioned above, the Backstage CLI uses [Jest](https://jestjs.io/), which is
a JavaScript test framework that covers both test execution and assertions. Jest
executes all tests in Node.js, including frontend browser code. The trick it
uses is to execute the tests in a Node.js VM using various predefined
environments, such as one based on [`jsdom`](https://github.com/jsdom/jsdom)
that helps mimic browser APIs and behavior.
The Backstage CLI has its own command that helps execute tests,
`backstage-cli test`, as well as its own configuration at
`@backstage/cli/config/jest.js`. The command is a relatively thin wrapper around
running `jest` directly. Its main responsibility is to make sure the included
configuration is used, as well setting the `NODE_ENV` and `TZ` environment
variables, and provided some sane default flags like `--watch` if executed
within a Git repository.
The by far biggest amount of work is done by the Jest configuration included
with the Backstage CLI. It both takes care of providing a default Jest
configuration, as well as allowing for configuration overrides to be defined in
each `package.json`. How this can be done in practice is discussed in the
[Jest configuration](#jest-configuration) section.
## Building
The primary purpose of the build process is to prepare packages for publishing,
but it's also used as part of the backend bundling process. Since it's only used
in these two cases, any Backstage app that does not use the Backend parts of the
project may not need to interact with the build process at all. It can
nevertheless be useful to know how it works, since all of the published
Backstage packages are built using this process.
The build is currently using [Rollup](https://rollupjs.org/) and executes in
isolation for each individual package. The build is invoked using the `package build`
command, and applies to all packages roles except the bundled ones, `frontend` and `backend`.
There are three different possible outputs of the build process: JavaScript in
CommonJS module format, JavaScript in ECMAScript module format, and type
declarations. Each invocation of a build command will write one or more of these
outputs to the `dist` folder in the package, and in addition copy any asset
files like stylesheets or images. For more details on what syntax and file
formats are supported by the build process, see the [loaders section](#loaders).
When building CommonJS or ESM output, the build commands will always use
`src/index.ts` as the entrypoint. All non-relative modules imports are considered
external, meaning the Rollup build will only compile the source code of the package
itself. All import statements of external dependencies, even within the same
[monorepo](../../references/glossary.md#monorepo), will stay intact.
The build of the type definitions works quite differently. The entrypoint of the
type definition build is the relative location of the package within the
`dist-types` folder in the project root. This means that it is important to run
type checking before building any packages with type definitions, and that
emitting type declarations must be enabled in the TypeScript configuration. The
reason for the type definition build step is to strip out all types but the ones
that are exported from the package, leaving a much cleaner type definition file
and making sure that the type definitions are in sync with the generated
JavaScript.
## Bundling
The goal of the bundling process is to combine multiple packages together into a
single runtime unit. The way this is done varies between frontend and backend,
as well as local development versus production deployment. Because of that we
cover each combination of these cases separately.
### Frontend Development
The frontend development setup is used for all packages with a frontend role, and
is invoked using the `package start` command.
The only difference between the different roles is that packages with the `'frontend'`
role use `src/index` as the entrypoint, while other roles instead use `dev/index`.
When running the start command, a development server
will be set up that listens to the protocol, host and port set by `app.baseUrl`
in the configuration. If needed it is also possible to override the listening
options through the `app.listen` configuration.
The frontend development bundling is currently based on
[Webpack](https://webpack.js.org/) and
[Webpack Dev Server](https://webpack.js.org/configuration/dev-server/). The
Webpack configuration itself varies very little between the frontend development
and production bundling, so we'll dive more into the configuration in the
production section below. The main differences are that `process.env.NODE_ENV`
is set to `'development'`, minification is disabled, cheap source maps are used,
and [React Refresh](https://github.com/pmmmwh/react-refresh-webpack-plugin#readme)
is enabled.
If you prefer to run type checking and linting as part of the Webpack process,
you can enable usage of the
[`ForkTsCheckerWebpackPlugin`](https://www.npmjs.com/package/fork-ts-checker-webpack-plugin)
by passing the `--check` flag. Although as mentioned above, the recommended way
to handle these checks during development is to use an editor that has built-in
support for them instead.
### Frontend Production
The frontend production bundling creates your typical web content
[bundle](../../references/glossary.md#bundle), all contained within a single
folder, ready for static serving. It is used when building packages with the
`'frontend'` role, and unlike the development bundling there is no way to
build a production bundle of an individual plugin.
The output of the bundling process is written to the `dist` folder in the package.
Just like the development bundling, the production bundling is based on
[Webpack](https://webpack.js.org/). It uses the
[`HtmlWebpackPlugin`](https://webpack.js.org/plugins/html-webpack-plugin/) to
generate the `index.html` entry point, and includes a default template that's
included with the CLI. You can replace the bundled template by adding
`public/index.html` to your app package. The template has access to two global
constants, `publicPath` which is the public base path that the bundle is
intended to be served at, as well as `config` which is your regular frontend
scoped configuration from `@backstage/config`.
The Webpack configuration also includes a custom plugin for resolving packages
correctly from linked in packages, the `ModuleScopePlugin` from
[`react-dev-utils`](https://www.npmjs.com/package/react-dev-utils) which makes
sure that imports don't reach outside the package, a few fallbacks for some
Node.js modules like `'buffer'` and `'events'`, a plugin that writes the
frontend configuration to the bundle as `process.env.APP_CONFIG`, and lastly minification handled by
[esbuild](https://esbuild.github.io/) using the
[`esbuild-loader`](https://npm.im/esbuild-loader). There are of course also a
set of loaders configured, which you can read more about in the
[loaders](#loaders) and [transpilation](#transpilation) sections.
During the build, the following constants are also set:
```java
process.env.NODE_ENV = 'production';
process.env.BUILD_INFO = {
cliVersion: '0.4.0', // The version of the CLI package
gitVersion: 'v0.4.0-86-ge54815618', // output of `git describe --always`
packageVersion: '1.0.5', // The version of the app package itself
timestamp: 1678900000000, // Date.now() when the build started
commit: 'e548156182a973ed4b459e18533afc22c85ffff8', // output of `git rev-parse HEAD`
};
```
The output of the bundling process is split into two categories of files with
separate caching strategies. The first is a set of generic assets with plain
names in the root of the `dist/` folder. You will want to serve these with
short-lived caching or no caching at all. The second is a set of hashed static
assets in the `dist/static/` folder, which you can configure to be cached for a
much longer time.
The configuration of static assets is optimized for frequent changes and serving
over HTTP 2.0. The assets are aggressively split into small chunks, which means
the browser has to make a lot of small requests to load them. The upside is that
changes to individual plugins and packages will invalidate a smaller number of
files, thereby allowing for rapid development without much impact on the page
load performance.
### Backend Development
The backend development bundling is also based on Webpack, but rather than
starting up a web server, the backend is started up using the
[`RunScriptWebpackPlugin`](https://www.npmjs.com/package/run-script-webpack-plugin).
The reason for using Webpack for development of the backend is both that it is a
convenient way to handle transpilation of a large set of packages, as well us
allowing us to use hot module replacement and maintaining state while reloading
individual backend modules. This is particularly useful when running the backend
with in-memory SQLite as the database choice.
Except for executing in Node.js rather than a web server, the backend
development bundling configuration is quite similar to the frontend one. It
shares most of the Webpack configuration, including the transpilation setup.
Some differences are that it does not inject any environment variables or node
module fallbacks, and it uses
[`webpack-node-externals`](https://www.npmjs.com/package/webpack-node-externals)
to avoid bundling in dependency modules.
If you want to inspect the running Node.js process, the `--inspect` and
`--inspect-brk` flags can be used, as they will be passed through as options to
`node` execution.
### Backend Production
The backend production bundling uses a completely different setup than the other
bundling options. Rather than using Webpack, the backend production bundling
instead collects the backend packages and all of their local dependencies into a
deployment archive. The archive is written to `dist/bundle.tar.gz`, and contains
the packaged version of each of these packages. The layout of the packages in
the archive is the same as the directory layout in the monorepo, and the bundle
also contains the root `package.json` and `yarn.lock` files.
Note that before creating a production bundle you must first build all of the
backend packages. This can be done automatically when executing the
`backend:bundle` command by passing the `--build-dependencies` flag. It is an
optional flag since it is quite common that the packages are already built
earlier on in your build process, and building them again would result in
duplicate work.
In order to use the bundle, you extract it into a directory, run
`yarn install --production`, and then start the backend using your backend
package as the Node.js entry point, for example `node packages/backend`.
The `dist/bundle.tar.gz` is accompanied by a `dist/skeleton.tar.gz`, which has
the same layout, but only contains `package.json` files. This skeleton archive
can be used to run a `yarn install` in environments that will benefit from the
caching that this enables, such as Docker image builds. To use the skeleton
archive you copy it over to the target directory along with the root
`package.json` and `yarn.lock`, extract the archive, and then run
`yarn install --production`. Your target directory will then have all
dependencies installed, and as soon as you copy over and extract the contents of
the `bundle.tar.gz` archive on top of it, the backend will be ready to run.
The following is an example of a `Dockerfile` that can be used to package the
output of building a package with role `'backend'` into an image:
```Dockerfile
FROM node:18-bookworm-slim
WORKDIR /app
COPY yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./
RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz
# install sqlite3 dependencies
RUN apt-get update && \
apt-get install -y libsqlite3-dev python3 cmake g++ && \
rm -rf /var/lib/apt/lists/* && \
yarn config set python /usr/bin/python3
RUN yarn install --frozen-lockfile --production --network-timeout 300000 && rm -rf "$(yarn cache dir)"
COPY packages/backend/dist/bundle.tar.gz app-config.yaml ./
RUN tar xzf bundle.tar.gz && rm bundle.tar.gz
CMD ["node", "packages/backend"]
```
## Transpilation
The transpilers used by the Backstage CLI have been chosen according to the same
design considerations that were mentioned above. A few specific requirements are
of course support for TypeScript and JSX, but also React hot reloads or refresh,
and hoisting of Jest mocks. The Backstage CLI also only targets up to date and
modern browsers, so we actually want to keep the transpilation process as
lightweight as possible, and leave most syntax intact.
Apart from these requirements, the deciding factor for which transpiler to use
is their speed. The build process keeps the integration with the transpilers
lightweight, without additional plugins or such. This enables us to switch out
transpilers as new options and optimizations become available, and keep on using
the best options that are available.
Our current selection of transpilers are [esbuild](https://esbuild.github.io/)
and [Sucrase](https://github.com/alangpierce/sucrase). The reason we choose to
use two transpilers is that esbuild is faster than Sucrase and produces slightly
nicer output, but it does not have the same set of features, for example it does
not support React hot reloading.
The benchmarking of the various options was done in
[ts-build-bench](https://github.com/Rugvip/ts-build-bench). This benchmarking
project allows for setups of different shapes and sizes of monorepos, but the
setup we consider the most important in our case is a large monorepo with lots
of medium to large packages that are being bundled with Webpack. Some rough
findings have been that esbuild is the fastest option right now, with Sucrase
following closely after and then [SWC](https://swc.rs/) closely after that.
After those there's a pretty big gap up to the TypeScript compiler run in
transpilation only mode, and lastly another jump up to Babel, being by far the
slowest out of the transpilers we tested.
Something to note about these benchmarks is that they take the full Webpack
bundling time into account. This means that even though some transpilation
options may be orders of magnitude faster than others, the total time is not
impacted in the same way as there are lots of other things that go into the
bundling process. Still, switching from for example Babel to Sucrase is able to
make the bundling anywhere from two to five times faster.
## Loaders
The Backstage CLI is configured to support a set of loaders throughout all parts
of the build system, including the bundling, tests, builds, and type checking.
Loaders are always selected based on the file extension. The following is a list
of all supported file extensions:
| Extension | Exports | Purpose |
| --------- | ------------- | ------------------ |
| `.ts` | Script Module | TypeScript |
| `.tsx` | Script Module | TypeScript and XML |
| `.js` | Script Module | JavaScript |
| `.jsx` | Script Module | JavaScript and XML |
| `.mjs` | Script Module | ECMAScript Module |
| `.cjs` | Script Module | CommonJS Module |
| `.json` | JSON Data | JSON Data |
| `.yml` | JSON Data | YAML Data |
| `.yaml` | JSON Data | YAML Data |
| `.css` | classes | Style sheet |
| `.eot` | URL Path | Font |
| `.ttf` | URL Path | Font |
| `.woff2` | URL Path | Font |
| `.woff` | URL Path | Font |
| `.bmp` | URL Path | Image |
| `.gif` | URL Path | Image |
| `.jpeg` | URL Path | Image |
| `.jpg` | URL Path | Image |
| `.png` | URL Path | Image |
| `.svg` | URL Path | Image |
| `.md` | URL Path | Markdown File |
## Jest Configuration
The Backstage CLI bundles its own Jest configuration file, which is used
automatically when running `backstage-cli test`. It's available at
`@backstage/cli/config/jest.js` and can be inspected
[here](https://github.com/backstage/backstage/blob/master/packages/cli/config/jest.js).
Usage of this configuration can be overridden either by passing a
`--config <path>` flag to `backstage-cli test`, or placing a `jest.config.js` or
`jest.config.ts` file in your package.
The built-in configuration brings a couple of benefits and features. The most
important one being a baseline transformer and module configuration that enables
support for the listed [loaders](#loaders) within tests. It will also
automatically detect and use `src/setupTests.ts` if it exists, and provides a
coverage configuration that works well with our selected transpilers. The configuration
will also detect the appropriate Jest environment for each package role, running
`web-libraries` with the `"jsdom"` environment, `node-libraries` with `"node"`, and so on.
The configuration also takes a project-wide approach, with the expectation most
if not all packages within a monorepo will use the same base configuration. This
allows for optimizations such as sharing the Jest transform cache across all
packages in a monorepo, avoiding unnecessary transpilation. It also makes it
possible to load in all Jest configurations at once, and with that run
`yarn test <pattern>` from the root of a monorepo without having to set the
working directory to the package that the test is in.
Where small customizations are needed, such as setting coverage thresholds or
support for specific transforms, it is possible to override the Jest
configuration through the `"jest"` field in `package.json`. For a full list of
options, see the [Jest documentation](https://jestjs.io/docs/en/configuration).
These overrides will be loaded in from all `package.json` files in the directory ancestry, meaning
that you can place common configuration in the `package.json` at the root of a
monorepo. If multiple overrides are found, they will be merged together with
configuration further down in the directory tree taking precedence.
The overrides in a single `package.json` may for example look like this:
```json
"jest": {
"coverageThreshold": {
"global": {
"functions": 100,
"lines": 100,
"statements": 100
}
}
},
```
### Debugging Jest Tests
For your productivity working with unit tests it's quite essential to have your debugging configured in IDE. It will help you to identify the root cause of the issue faster.
#### IntelliJ IDEA
1. Update Jest configuration template by:
- Click on "Edit Configurations" on top panel
- In the modal dialog click on link "Edit configuration templates..." located in the bottom left corner.
- In "Jest package" you have to point to relative path of jest module (it will be suggested by IntelliJ), i.e. `~/proj/backstage/node_modules/jest`
- In "Jest config" point to your jest configuration file, use absolute path for that, i.e. `--config /Users/user/proj/backstage/packages/cli/config/jest.js --runInBand`
2. Now you can run any tests by clicking on green arrow located on `describe` or `it`.
#### VS Code
```jsonc
{
"jest.jestCommandLine": "node_modules/.bin/jest --config node_modules/@backstage/cli/config/jest.js",
// In a large repo like the Backstage main repo you likely want to disable
// watch mode and the initial test run too, leaving just manual and perhaps
// on-save test runs in place.
"jest.autoRun": {
"watch": false,
"onSave": "test-src-file"
}
}
```
A complete launch configuration for VS Code debugging may look like this:
```jsonc
{
"type": "node",
"name": "vscode-jest-tests.v2",
"request": "launch",
"args": [
"repo",
"test",
"--runInBand",
"--watchAll=false",
"--testNamePattern",
"${jest.testNamePattern}",
"--runTestsByPath",
"${jest.testFile}"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"disableOptimisticBPs": true,
"program": "${workspaceFolder}/node_modules/.bin/backstage-cli"
}
```
## Publishing
Package publishing is an optional part of the Backstage build system and not
something you will need to worry about unless you are publishing packages to a
registry. In order to publish a package, you first need to build it, which will
populate the `dist` folder. Because the Backstage build system is optimized for
local development along with our particular TypeScript and bundling setup, it is
not possible to publish the package immediately at this point. This is because
the entry points of the package will still be pointing to `src/index.ts`, but we
want them to point to `dist/` in the published package.
In order to work around this, the Backstage CLI provides `prepack` and
`postpack` commands that help prepare the package for publishing. These scripts
are automatically run by Yarn before publishing a package.
The `prepack` command will take entry point fields in `"publishConfig"`, such as
`"main"` and `"module"`, and move them to the top level of the `package.json`.
This lets you point at the desired files in the `dist` folder during publishing.
The `postpack` command will simply revert this change in order to leave your
project clean.
The following is an excerpt of a typical setup of an isomorphic library package:
```json
"main": "src/index.ts",
"types": "src/index.ts",
"publishConfig": {
"access": "public",
"main": "dist/index.cjs.js",
"module": "dist/index.esm.js",
"types": "dist/index.d.ts"
},
"scripts": {
"build": "backstage-cli package build",
"lint": "backstage-cli package lint",
"test": "backstage-cli package test",
"clean": "backstage-cli package clean",
"prepack": "backstage-cli package prepack",
"postpack": "backstage-cli package postpack"
},
"files": ["dist"],
```
## Subpath Exports
The Backstage CLI supports implementation of subpath exports through the `"exports"` field in `package.json`. It might for example look like this:
```json
"name": "@backstage/plugin-foo",
"exports": {
".": "./src/index.ts",
"./components": "./src/components.ts",
},
```
This in turn would allow you to import anything exported in `src/index.ts` via `@backstage/plugins-foo`, and `src/components.ts` via `@backstage/plugins-foo/components`. Note that patterns are not supported, meaning the exports may not contain `*` wildcards.
As with the rest of the Backstage CLI build system, the setup is optimized for local development, which is why the `"exports"` targets point directly to source files. The `package build` command will detect the `"exports"` field and automatically generate the corresponding `dist` files, and the `prepublish` command will rewrite the `"exports"` field to point to the `dist` files, as well as generating folder-based entry points for backwards compatibility.
TypeScript support is currently handled though the `typesVersions` field, as there is not yet a module resolution mode that works well with `"exports"`. You can craft the `typesVersions` yourself, but it will also be automatically generated by the `migrate package-exports` command.
To add subpath exports to an existing package, simply add the desired `"exports"` fields and then run the following command:
```bash
yarn backstage-cli migrate package-exports
```
docs/tooling/cli/03-commands.md
+390
View File
@@ -0,0 +1,390 @@
---
id: commands
title: Commands
description: Descriptions of all commands available in the CLI.
---
This page lists all commands provided by the Backstage CLI, what they're for,
and where to use them.
## help
This command displays a help summary or detailed help screens for each command.
Below is a cleaned up output of `yarn backstage-cli --help`:
```text
new [options] Open up an interactive guide to creating new things in
your app
test Run tests, forwarding args to Jest, defaulting to watch
mode [DEPRECATED]
config:docs [options] Browse the configuration reference documentation
config:print [options] Print the app configuration for the current package
config:check [options] Validate that the given configuration loads and matches
schema
config:schema [options] Print configuration schema
repo [command] Command that run across an entire Backstage project
package [command] Lifecycle scripts for individual packages
migrate [command] Migration utilities
versions:bump [options] Bump Backstage packages to the latest versions
versions:check [options] Check Backstage package versioning
clean Delete cache directories [DEPRECATED]
build-workspace <workspace-dir> [packages...] Builds a temporary dist workspace from the provided
packages
create-github-app <github-org> Create new GitHub App in your organization.
info Show helpful information for debugging and reporting bugs
help [command] display help for command
```
The `package` command category, `yarn backstage-cli package --help`:
```text
start [options] Start a package for local development
build [options] Build a package for production deployment or publishing
lint [options] [directories...] Lint a package
test Run tests, forwarding args to Jest, defaulting to watch mode
clean Delete cache directories
prepack Prepares a package for packaging before publishing
postpack Restores the changes made by the prepack command
help [command] display help for command
```
The `repo` command category, `yarn backstage-cli repo --help`:
```text
build [options] Build packages in the project, excluding bundled app and backend packages.
lint [options] Lint all packages in the project
clean Delete cache and output directories
list-deprecations [options] List deprecations
test [options] Run tests, forwarding args to Jest, defaulting to watch mode
help [command] display help for command
```
The `migrate` command category, `yarn backstage-cli migrate --help`:
```text
package-roles Add package role field to packages that don't have it
package-scripts Set package scripts according to each package role
package-exports Synchronize package subpath export definitions
package-lint-configs Migrates all packages to use @backstage/cli/config/eslint-factory
react-router-deps Migrates the react-router dependencies for all packages to be peer dependencies
help [command] display help for command
```
## repo build
Builds all packages in the project, excluding bundled packages by default, i.e. ones
with the role `'frontend'` or `'backend'`.
```text
Usage: backstage-cli repo build [options]
Build packages in the project, excluding bundled app and backend packages.
Options:
--all Build all packages, including bundled app and backend packages.
--since <ref> Only build packages and their dev dependents that changed since the specified ref
```
## repo lint
Lint all packages in the project.
```text
Usage: backstage-cli repo lint [options]
Lint all packages in the project
Options:
--format <format> Lint report output format (default: "eslint-formatter-friendly")
--since <ref> Only lint packages that changed since the specified ref
--fix Attempt to automatically fix violations
```
## package start
Starts the package for local development. See the frontend and backend development parts in the build system [bundling](./02-build-system.md#bundling) section for more details.
```text
Usage: backstage-cli package start [options]
Start a package for local development
Options:
--config <path> Config files to load instead of app-config.yaml (default: [])
--role <name> Run the command with an explicit package role
--check Enable type checking and linting if available
--inspect Enable debugger in Node.js environments
--inspect-brk Enable debugger in Node.js environments, breaking before code starts
```
## package build
Build an individual package based on its role. See the build system [building](./02-build-system.md#building) and [bundling](./02-build-system.md#bundling) sections for more details.
```text
Usage: backstage-cli package build [options]
Build a package for production deployment or publishing
Options:
--role <name> Run the command with an explicit package role
--minify Minify the generated code. Does not apply to app package (app is minified by default).
--skip-build-dependencies Skip the automatic building of local dependencies. Applies to backend packages only.
--stats If bundle stats are available, write them to the output directory. Applies to app packages only.
--config <path> Config files to load instead of app-config.yaml. Applies to app packages only. (default: [])
```
## package lint
Lint a package. In addition to the default `eslint` behavior, this command will
include TypeScript files, treat warnings as errors, and default to linting the
entire directory if no specific files are listed. For more information, see the
build system [linting](./02-build-system.md#linting) section.
```text
Usage: backstage-cli package lint [options]
Lint a package
Options:
--format <format> Lint report output format (default: "eslint-formatter-friendly")
--fix Attempt to automatically fix violations
```
## package test
Run tests, forwarding all unknown options to Jest, and defaulting to watch mode.
When executing the tests, `process.env.NODE_ENV` will be set to `"test"`.
This command uses a default Jest configuration that is included in the CLI,
which is set up with similar goals for speed, scale, and working within a
monorepo. The configuration sets the `src` as the root directory, enforces the
`.test.` infix for tests, and uses `src/setupTests.ts` as the test setup
location. The included configuration also supports test execution at the root of
a yarn workspaces monorepo by automatically creating one grouped configuration
that includes all packages that have `backstage-cli test` in their package
`test` script.
For more information about configuration overrides and editor support, see the [Jest Configuration section](./02-build-system.md#jest-configuration) in the build system documentation.
```text
Usage: backstage-cli package test [options]
Run tests, forwarding args to Jest, defaulting to watch mode
Options:
--backstage-cli-help display help for command
```
## package clean
Remove cache and output directories.
```text
Usage: backstage-cli package clean [options]
Delete cache directories
```
## package prepack
This command should be added as `scripts.prepack` in all packages. It enables
packaging- and publish-time overrides for fields inside `packages.json`.
For more details, see the build system [publishing](./02-build-system.md#publishing) section.
```text
Usage: backstage-cli package prepack [options]
Prepares a package for packaging before publishing
```
## package postpack
This should be added as `scripts.postpack` in all packages. It restores
`package.json` to what it looked like before calling the `prepack` command.
```text
Usage: backstage-cli package postpack [options]
Restores the changes made by the prepack command
```
## new
The `new` command opens up an interactive guide for you to create new things
in your app. If you do not pass in any options it is completely interactive, but
it is possible to pre-select what you want to create using the `--select` flag,
and provide options using `--option`, for example:
```bash
backstage-cli new --select plugin --option id=foo
```
This command is typically added as script in the root `package.json` to be
executed with `yarn new`, using options that are appropriate for the organization
that owns the app repo. For example you may have it set up like this:
```json
{
"scripts": {
"new": "backstage-cli new --scope internal --no-private --npm-registry https://acme.org/npm"
}
}
```
```text
Usage: backstage-cli create [options]
Options:
--select <name> Select the thing you want to be creating upfront
--option <name>=<value> Pre-fill options for the creation process (default: [])
--scope <scope> The scope to use for new packages
--npm-registry <URL> The package registry to use for new packages
--no-private Do not mark new packages as private
-h, --help display help for command
```
## config\:docs
This commands opens up the reference documentation of your apps local
configuration schema in the browser. This is useful to get an overview of what
configuration values are available to use, a description of what they do and
their format, and where they get sent.
```text
Usage: backstage-cli config:docs [options]
Browse the configuration reference documentation
Options:
--package <name> Only include the schema that applies to the given package
-h, --help display help for command
```
## config\:print
Print the static configuration, defaulting to reading `app-config.yaml` in the
repo root, using schema collected from all local packages in the repo.
For example, to validate that a given configuration value is visible in the
frontend when building the `my-app` package, you can use the following:
```bash
yarn backstage-cli config:print --frontend --package my-app
```
```text
Usage: backstage-cli config:print [options]
Options:
--package <name> Only load config schema that applies to the given package
--lax Do not require environment variables to be set
--frontend Print only the frontend configuration
--with-secrets Include secrets in the printed configuration
--format <format> Format to print the configuration in, either json or yaml [yaml]
--config <path> Config files to load instead of app-config.yaml (default: [])
-h, --help display help for command
```
## config\:check
Validate that static configuration loads and matches schema, defaulting to
reading `app-config.yaml` in the repo root and using schema collected from all
local packages in the repo.
```text
Usage: backstage-cli config:check [options]
Options:
--package <name> Only load config schema that applies to the given package
--lax Do not require environment variables to be set
--frontend Only validate the frontend configuration
--deprecated Output deprecated configuration settings
--strict Ensure that the provided config(s) has no errors and does not contain keys not in the schema.
--config <path> Config files to load instead of app-config.yaml (default: [])
-h, --help display help for command
```
## config\:schema
Dump the configuration schema that was collected from all local packages in the
repo.
Note: when run by `yarn`, supply the yarn option `--silent` if you are using the
output in a command line pipe to avoid non schema output in the pipeline.
```text
Usage: backstage-cli config:schema [options]
Print configuration schema
Options:
--package <name> Only output config schema that applies to the given package
--format <format> Format to print the schema in, either json or yaml [yaml]
-h, --help display help for command
```
## versions\:bump
Bump all `@backstage` packages to the latest versions. This checks for updates
in the package registry, and will update entries both in `yarn.lock` and
`package.json` files when necessary.
```text
Usage: backstage-cli versions:bump [options]
Options:
-h, --help display help for command
--pattern <glob> Override glob for matching packages to upgrade
--release <version|next|main> Bump to a specific Backstage release line or version (default: "main")
```
## versions\:check
Validate `@backstage` dependencies within the repo, making sure that there are
no duplicates of packages that might lead to breakages.
By supplying the `--fix` flag the command will attempt to fix any conflict that
can be resolved by editing `yarn.lock`, but will not attempt to search for
remote updates or modify any `package.json` files.
```text
Usage: backstage-cli versions:check [options]
Options:
--fix Fix any auto-fixable versioning problems
-h, --help display help for command
```
## build-workspace
Builds a mirror of the workspace using the packaged production version of each
package. This essentially calls `yarn pack` in each included package and unpacks
the resulting archive in the target `workspace-dir`.
```text
Usage: backstage-cli build-workspace [options] <workspace-dir>
```
## create-github-app
Creates a GitHub App in your GitHub organization. This is an alternative to
token-based [GitHub integration](../../integrations/github/locations.md). See
[GitHub Apps for Backstage Authentication](../../integrations/github/github-apps.md).
Launches a browser to create the App through GitHub and saves the result as a
YAML file that can be referenced in the GitHub integration configuration.
```text
Usage: backstage-cli create-github-app <github-org>
```
## info
Outputs debug information which is useful when opening an issue. Outputs system
information, node.js and npm versions, CLI version and type (inside backstage
repo or a created app), all `@backstage/*` package dependency versions.
```text
Usage: backstage-cli info
```