Merge pull request #4718 from backstage/mob/scaffolder-schema-v2

scaffolder: Introduce support for custom actions
This commit is contained in:
Johan Haals
2021-03-02 10:54:47 +01:00
committed by GitHub
65 changed files with 4333 additions and 836 deletions
@@ -522,49 +522,90 @@ consumed by the component, e.g. `artist-api`. This field is optional.
## Kind: Template
Describes the following entity kind:
The following describes the following entity kind:
| Field | Value |
| ------------ | ----------------------- |
| `apiVersion` | `backstage.io/v1alpha1` |
| `kind` | `Template` |
| Field | Value |
| ------------ | ---------------------- |
| `apiVersion` | `backstage.io/v1beta2` |
| `kind` | `Template` |
A Template describes a skeleton for use with the Scaffolder. It is used for
describing what templating library is supported, and also for documenting the
variables that the template requires using
[JSON Forms Schema](https://jsonforms.io/).
If you're looking for docs on `v1alpha1` you can find them
[here](../software-templates/legacy.md)
A template definition describes both the parameters that are rendered in the
frontend part of the scaffolding wizard, and the steps that are executed when
scaffolding that component.
Descriptor files for this kind may look as follows.
```yaml
apiVersion: backstage.io/v1alpha1
apiVersion: backstage.io/v1beta2
kind: Template
# some metadata about the template itself
metadata:
name: react-ssr-template
title: React SSR Template
description:
Next.js application skeleton for creating isomorphic web applications.
tags:
- recommended
- react
name: v1beta2-demo
title: Test Action template
description: scaffolder v1beta2 template demo
spec:
owner: web@example.com
templater: cookiecutter
type: website
path: '.'
schema:
required:
- component_id
- description
properties:
component_id:
title: Name
type: string
description: Unique name of the component
description:
title: Description
type: string
description: Description of the component
owner: backstage/techdocs-core
type: service
# these are the steps which are rendered in the frontend with the form input
parameters:
- title: Fill in some steps
required:
- name
properties:
name:
title: Name
type: string
description: Unique name of the component
ui:autofocus: true
ui:options:
rows: 5
- title: Choose a location
required:
- repoUrl
properties:
repoUrl:
title: Repository Location
type: string
ui:field: RepoUrlPicker
ui:options:
allowedHosts:
- github.com
# here's the steps that are executed in series in the scaffolder backend
steps:
- id: fetch-base
name: Fetch Base
action: fetch:cookiecutter
input:
url: ./template
values:
name: '{{ parameters.name }}'
- id: fetch-docs
name: Fetch Docs
action: fetch:plain
input:
targetPath: ./community
url: https://github.com/backstage/community/tree/main/backstage-community-sessions
- id: publish
name: Publish
action: publish:github
input:
allowedHosts: ['github.com']
description: 'This is {{ parameters.name }}'
repoUrl: '{{ parameters.repoUrl }}'
- id: register
name: Register
action: catalog:register
input:
repoContentsUrl: '{{ steps.publish.output.repoContentsUrl }}'
catalogInfoPath: '/catalog-info.yaml'
```
In addition to the [common envelope metadata](#common-to-all-kinds-the-metadata)
@@ -572,7 +613,7 @@ shape, this kind has the following structure.
### `apiVersion` and `kind` [required]
Exactly equal to `backstage.io/v1alpha1` and `Template`, respectively.
Exactly equal to `backstage.io/v1beta2` and `Template`, respectively.
### `metadata.title` [required]
@@ -605,31 +646,15 @@ The current set of well-known and common values for this field is:
- `website` - a website
- `library` - a software library, such as an npm module or a Java library
### `spec.templater` [required]
### `spec.parameters` [required]
The templating library that is supported by the template skeleton as a string,
e.g `cookiecutter`.
You can find out more about the `parameters` key
[here](../software-templates/writing-templates.md)
Different skeletons will use different templating syntax, so it's common that
the template will need to be run with a particular piece of software.
### `spec.steps` [optional]
This key will be used to identify the correct templater which is registered into
the `TemplatersBuilder`.
The values which are available by default are:
- `cookiecutter` - [cookiecutter](https://github.com/cookiecutter/cookiecutter).
### `spec.path` [optional]
The string location where the templater should be run if it is not on the same
level as the `template.yaml` definition, e.g. `./cookiecutter/skeleton`.
This will set the `cwd` when running the templater to the folder path that you
specify relative to the `template.yaml` definition.
This is also particularly useful when you have multiple template definitions in
the same repository but only a single `template.yaml` registered in backstage.
You can find out more about the `steps` key
[here](../software-templates/writing-templates.md)
## Kind: API
@@ -4,49 +4,80 @@ title: Adding your own Templates
description: Documentation on Adding your own Templates
---
Templates are stored in the **Service Catalog** under a kind `Template`. The
Templates are stored in the **Software Catalog** under a kind `Template`. The
minimum that is needed to define a template is a `template.yaml` file, but it
would be good to also have some files in there that can be templated in.
A simple `template.yaml` definition might look something like this:
```yaml
apiVersion: backstage.io/v1alpha1
apiVersion: backstage.io/v1beta2
kind: Template
# some metadata about the template itself
metadata:
# unique name per namespace for the template
name: react-ssr-template
# title of the template
title: React SSR Template
# a description of the template
description:
Next.js application skeleton for creating isomorphic web applications.
# some tags to display in the frontend
tags:
- recommended
- react
name: v1beta2-demo
title: Test Action template
description: scaffolder v1beta2 template demo
spec:
# which templater key to use in the templaters builder
templater: cookiecutter
# what does this template create
type: website
# if the template is not in the current directory where this definition is kept then specify
path: './template'
# the schema for the form which is displayed in the frontend.
# should follow JSON schema for forms: https://jsonforms.io/
schema:
required:
- component_id
- description
properties:
component_id:
title: Name
type: string
description: Unique name of the component
description:
title: Description
type: string
description: Description of the component
owner: backstage/techdocs-core
type: service
# these are the steps which are rendered in the frontend with the form input
parameters:
- title: Fill in some steps
required:
- name
properties:
name:
title: Name
type: string
description: Unique name of the component
ui:autofocus: true
ui:options:
rows: 5
- title: Choose a location
required:
- repoUrl
properties:
repoUrl:
title: Repository Location
type: string
ui:field: RepoUrlPicker
ui:options:
allowedHosts:
- github.com
# here's the steps that are executed in series in the scaffolder backend
steps:
- id: fetch-base
name: Fetch Base
action: fetch:cookiecutter
input:
url: ./template
values:
name: '{{ parameters.name }}'
- id: fetch-docs
name: Fetch Docs
action: fetch:plain
input:
targetPath: ./community
url: https://github.com/backstage/community/tree/main/backstage-community-sessions
- id: publish
name: Publish
action: publish:github
input:
allowedHosts: ['github.com']
description: 'This is {{ parameters.name }}'
repoUrl: '{{ parameters.repoUrl }}'
- id: register
name: Register
action: catalog:register
input:
repoContentsUrl: '{{ steps.publish.output.repoContentsUrl }}'
catalogInfoPath: '/catalog-info.yaml'
```
[Template Entity](../software-catalog/descriptor-format.md#kind-template)
@@ -55,14 +86,6 @@ contains more information about the required fields.
Once we have a `template.yaml` ready, we can then add it to the service catalog
for use by the scaffolder.
_NOTE_: When the `publish` step is completed, it is currently assumed by the
scaffolder that the final repository should contain a `catalog-info.yaml` in
order to register this with the Catalog in Backstage.
Currently the catalog supports loading definitions from GitHub + Local Files. To
load from other places, not only will there need to be another preparer, but the
support to load the location will also need to be added to the Catalog.
You can add the template files to the catalog through
[static location configuration](../software-catalog/configuration.md#static-location-configuration),
for example:
@@ -76,52 +99,8 @@ catalog:
- allow: [Template]
```
Templates can also be added by posting the to the catalog directly. Note that if
you're doing this, you need to configure the catalog to allow template entities
to be ingested from any source, for example:
Or you can add the template using the `catalog-import` plugin, which unless
configured differently should be running on `/catalog-import`.
```yaml
catalog:
rules:
- allow: [Component, API, Template]
```
For loading from a file, the following command should work when the backend is
running:
```sh
curl \
--location \
--request POST 'localhost:7000/api/catalog/locations' \
--header 'Content-Type: application/json' \
--data-raw "{\"type\": \"file\", \"target\": \"${YOUR PATH HERE}/template.yaml\"}"
```
If loading from a Git location, you can run the following
```sh
curl \
--location \
--request POST 'localhost:7000/api/catalog/locations' \
--header 'Content-Type: application/json' \
--data-raw "{\"type\": \"github\", \"target\": \"https://${GITHUB URL}/${YOUR GITHUB ORG/REPO}/blob/master/${PATH TO FOLDER}/template.yaml\"}"
```
This should then have been added the catalog, and be listed under the create
page at http://localhost:3000/create.
The `type` field which is chosen in the request to add the `template.yaml` to
the Service Catalog here, will become the `PreparerKey` which will be used to
select the `Preparer` when creating a job.
### Adding form values in the Scaffolder Wizard
The `spec.schema` property in the
[Template Entity](../software-catalog/descriptor-format.md#kind-template) is a
`yaml` version of the JSON Form Schema standard.
Here you can define the key/values and then the wizard will convert this to a
form for the user to fill in when your template is selected.
You can find out much more about the standard and how to use it here:
https://jsonforms.io
For information about writing your own templates, you can check out the docs
[here](./writing-templates.md)
@@ -0,0 +1,115 @@
---
id: builtin-actions
title: Builtin actions
description: Documentation describing the built-in template actions.
---
# Built-in Actions
This is the list of built-in template actions
## `fetch:plain`
Downloads content and places it in the workspacePath or optionally in a
subdirectory specified by the `targetPath` input option.
input:
- `url` (required) Relative path or absolute URL pointing to the directory tree
to fetch.
- `targetPath` Target path within the working directory to download the contents
to.
output: nothing
## `fetch:cookiecutter`
Downloads template from `url` and templates with cookiecutter
input:
- `url` (required) Relative path or absolute URL pointing to the directory tree
to fetch.
- `targetPath` Target path within the working directory to download the contents
to.
- `values` Values to pass on to cookiecutter for templating.
output: nothing
## `catalog:register`
Registers entity in the software catalog.
input:
- `catalogInfoUrl` (required) An absolute URL pointing to the catalog info file
location.
output:
- `repoContentsUrl` An absolute URL pointing to the root of a repository
directory tree.
- `catalogInfoPath` A relative path from the repo root pointing to the catalog
info file, defaults to /catalog-info.yaml
## `publish:github`
Initializes a git repository of contents in workspacePath and publishes to
GitHub.
input:
- `repoUrl` (required) Repository location
- `repoVisibility` (optional, default: 'private') Possible values: 'private',
'public' or 'internal'.
output:
- `remoteUrl` A URL to the repository with the provider
- `repoContentsUrl` A URL to the root of the repository
## `publish:gitlab`
Initializes a git repository of contents in workspacePath and publishes to
GitLab.
input:
- `repoUrl` (required) Repository location
- `repoVisibility` (optional, default: 'private') Possible values: 'private',
'public' or 'internal'.
output:
- `remoteUrl` A URL to the repository with the provider
- `repoContentsUrl` A URL to the root of the repository
## `publish:bitbucket`
Initializes a git repository of contents in workspacePath and publishes to
Bitbucket.
input:
- `repoUrl` (required) Repository location
- `repoVisibility` (optional, default: 'private') Possible values: 'private',
'public'.
output:
- `remoteUrl` A URL to the repository with the provider
- `repoContentsUrl` A URL to the root of the repository
## `publish:azure`
Initializes a git repository of contents in workspacePath and publishes to
Azure.
input:
- `repoUrl` (required) Repository location
output:
- `remoteUrl` A URL to the repository with the provider
- `repoContentsUrl` A URL to the root of the repository
@@ -1,101 +0,0 @@
---
id: extending-preparer
title: Create your own Preparer
description: Documentation on Creating your own Preparer
---
Preparers are responsible for reading the location of the definition of a
[Template Entity](../../software-catalog/descriptor-format.md#kind-template) and
making a temporary folder with the contents of the selected skeleton.
Currently, we provide two different providers that can parse two different
location protocols:
- `file://`
- `github://`
These two are added to the `PreparersBuilder` and then passed into the
`createRouter` function of the `@backstage/plugin-scaffolder-backend`.
A full example backend can be found in
[`scaffolder.ts`](https://github.com/backstage/backstage/blob/d91c10f654475a60829fa33a5c81018e517a319a/packages/backend/src/plugins/scaffolder.ts),
but it looks something like the following
```ts
import {
createRouter,
FilePreparer,
GithubPreparer,
Preparers,
} from '@backstage/plugin-scaffolder-backend';
import type { PluginEnvironment } from '../types';
export default async function createPlugin({ logger }: PluginEnvironment) {
const preparers = new Preparers();
const filePreparer = new FilePreparer();
const githubPreparer = new GithubPreparer();
preparers.register('file', filePreparer);
preparers.register('github', githubPreparer);
return await createRouter({
preparers,
templaters,
logger,
});
}
```
As you can see in the above code, a `PreparerBuilder` is created, and then two
of the `preparers` are registered with the different protocols that they accept.
The `protocol` is set on the
[Template Entity](../../software-catalog/descriptor-format.md#kind-template)
when added to the service catalog. You can see more about this `PreparerKey`
here in [Register your own template](../adding-templates.md)
**note:** Currently the catalog supports loading definitions from GitHub + Local
Files, which translate into the two `PreparerKeys`: `file` and `github`. To load
from other places, not only will there need to be another preparer, but the
support to load the location will also need to be added to the Catalog.
### Creating your own Preparer to add to the `PreparerBuilder`
All preparers need to implement the `PreparerBase` type.
That type looks like the following:
```ts
export type PreparerBase = {
prepare(
template: TemplateEntityV1alpha1,
opts: { logger: Logger },
): Promise<string>;
};
```
The `prepare` function will be given the
[Template Entity](../../software-catalog/descriptor-format.md#kind-template)
along with the source of where the `template.yaml` was loaded from under the
`metedata.annotations.managed-by-location` property.
Now it's up to you to implement a function which can go and fetch the skeleton
and put the contents into a temporary directory and return that directory path.
Some good examples exist here:
- https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/src/scaffolder/stages/prepare/file.ts
- https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/src/scaffolder/stages/prepare/github.ts
### Registering your own Preparer
You can register the preparer that you have created with the `PreparerBuilder`
by using the `PreparerKey` from the Catalog, for example like this:
```ts
const preparers = new Preparers();
preparers.register('gcs', new GoogleCloudStoragePreparer());
```
And then pass this into the `createRouter` function.
@@ -1,96 +0,0 @@
---
id: extending-publisher
title: Create your own Publisher
description: Documentation on Creating your own Publisher
---
Publishers are responsible for pushing and storing the templated skeleton after
the values have been templated by the `Templater`. See
[Create your own templater](./create-your-own-templater.md) for more info.
They receive a directory or location where the templater has successfully run
and is now ready to store somewhere. They also are given some other options
which are sent from the frontend, such as the `storePath` which is a string of
where the frontend thinks we should save this templated folder.
Currently we provide the following `publishers`:
- `github`
This publisher is passed through to the `createRouter` function of the
`@backstage/plugin-scaffolder-backend`. Currently, only one publisher is
supported, but PR's are always welcome.
An full example backend can be found
[here](https://github.com/backstage/backstage/blob/d91c10f654475a60829fa33a5c81018e517a319a/packages/backend/src/plugins/scaffolder.ts),
but it looks something like the following
```ts
import {
createRouter,
GithubPublisher,
} from '@backstage/plugin-scaffolder-backend';
import { Octokit } from '@octokit/rest';
import type { PluginEnvironment } from '../types';
export default async function createPlugin({ logger }: PluginEnvironment) {
const githubClient = new Octokit({ auth: process.env.GITHUB_TOKEN });
const publisher = new GithubPublisher({ client: githubClient });
return await createRouter({
publisher,
logger,
});
}
```
The publisher will always be called with the location from the selected
`Preparer`.
### Create your own Publisher and register it with the Scaffolder
All `publishers` need to implement the `PublisherBase` type.
That type looks like the following:
```ts
export type PublisherBase = {
publish(opts: {
entity: TemplateEntityV1alpha1;
values: TemplaterValues;
directory: string;
}): Promise<{ remoteUrl: string }>;
};
```
The `publisher` function will be called with an `options` object which contains
the following:
- `entity` - the
[Template Entity](../../software-catalog/descriptor-format.md#kind-template)
which is currently being scaffolded
- `values` - a json object which will resemble the `spec.schema` from the
[Template Entity](../../software-catalog/descriptor-format.md#kind-template)
which is defined here under spec.schema`. More info can be found here
[Register your own template](../adding-templates.md#adding-form-values-in-the-scaffolder-wizard)
- `directory` - a string containing the returned path from the `templater`. See
more information here in
[Create your own templater](./create-your-own-templater.md)
Now it's up to you to implement the `publish` function and return
`{ remoteUrl: string }` which can be used to identify the finished product.
Some good examples exist here:
- https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/src/scaffolder/stages/publish/github.ts
### Registering your own Publisher
Currently, we only support one `publisher` (PR's welcome), but you can register
any single `publisher` with the `createRouter`.
```ts
return await createRouter({
publisher: new MyGitlabPublisher(),
});
```
@@ -1,151 +0,0 @@
---
id: extending-templater
title: Creating your own Templater
description: Documentation on Creating your own Templater
---
Templaters are responsible for taking the directory path for the skeleton
returned by the preparers, and then executing the templating command on top of
the file and returning the completed template path. This may or may not be the
same directory as the input directory.
They also receive additional values from the frontend, which can be used to
interpolate into the skeleton files.
Currently we provide the following templaters:
- `cookiecutter`
This templater is added to the `TemplaterBuilder` and then passed into the
`createRouter` function of the `@backstage/plugin-scaffolder-backend`
An full example backend can be found
[here](https://github.com/backstage/backstage/blob/d91c10f654475a60829fa33a5c81018e517a319a/packages/backend/src/plugins/scaffolder.ts),
but it looks something like the following
```ts
import {
CookieCutter,
createRouter,
Templaters,
} from '@backstage/plugin-scaffolder-backend';
import type { PluginEnvironment } from '../types';
export default async function createPlugin({ logger }: PluginEnvironment) {
const templaters = new Templaters();
const cookiecutterTemplater = new CookieCutter();
templaters.register('cookiecutter', cookiecutterTemplater);
return await createRouter({
templaters,
});
}
```
As you can see in the above code a `TemplaterBuilder` is created and the default
`cookiecutter` `templater` is registered under the key `cookicutter`.
This `TemplaterKey` is used to select the correct templater from the
`spec.templater` in the
[Template Entity](../../software-catalog/descriptor-format.md#kind-template).
If you wish to add a new templater, you'll need to register it with the
`TemplaterBuilder`.
### Creating your own Templater to add to the `TemplaterBuilder`
All templaters need to implement the `TemplaterBase` type.
That type looks like the following:
```ts
export type TemplaterRunOptions = {
directory: string;
values: TemplaterValues;
logStream?: Writable;
dockerClient: Docker;
};
export type TemplaterBase = {
run(opts: TemplaterRunOptions): Promise<TemplaterRunResult>;
};
```
The `run` function will be given a `TemplaterRunOptions` object which is as
follows:
- `directory`- the skeleton directory returned from the `Preparer`, more info at
[Create your own preparer](./create-your-own-preparer.md).
- `values` - a json object which will resemble the `spec.schema` from the
[Template Entity](../../software-catalog/descriptor-format.md#kind-template)
which is defined here under spec.schema`. More info can be found here
[Register your own template](../adding-templates.md#adding-form-values-in-the-scaffolder-wizard)
- `logStream` - a stream that you can write to for displaying in the frontend.
- `dockerClient` - a [dockerode](https://github.com/apocas/dockerode) client to
be able to run docker containers.
_note_ Currently the templaters that we provide are basically Docker action
containers that are run on top of the skeleton folder. This keeps dependencies
to a minimum for running Backstage scaffolder, but you don't _have_ to use
Docker. You can `pip install cookiecutter` to run it locally in your backend.
You could create your own templater that spins up an EC2 instance and downloads
the folder and does everything using an AMI if you want. It's entirely up to
you!
Now it's up to you to implement the `run` function, and then return a
`TemplaterRunResult` which is `{ resultDir: string }`.
Some good examples exist here:
- https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/src/scaffolder/stages/templater/cookiecutter.ts
### Registering your own Templater
If you try to process a
[Template Entity](../../software-catalog/descriptor-format.md#kind-template)
with a new `spec.templater` value, you'll need to register that with the
`TemplaterBuilder`.
For example let's say you have the following
[Template Entity](../../software-catalog/descriptor-format.md#kind-template):
```yaml
apiVersion: backstage.io/v1alpha1
kind: Template
metadata:
name: react-ssr-template
title: React SSR Template
description:
Next.js application skeleton for creating isomorphic web applications.
tags:
- recommended
- react
spec:
owner: web@example.com
templater: handlebars
type: website
path: '.'
schema:
required:
- component_id
- description
properties:
component_id:
title: Name
type: string
description: Unique name of the component
description:
title: Description
type: string
description: Description of the component
```
You see that the `spec.templater` is set as `handlebars`, so you'll need to
register this with the `TemplaterBuilder` like so:
```ts
const templaters = new Templaters();
templaters.register('handlebars', new HandlebarsTemplater());
```
And then pass this into the `createRouter` function.
@@ -1,90 +0,0 @@
---
id: extending-index
title: Extending the Scaffolder
---
Welcome. Take a seat. You're at the Scaffolder Documentation.
So, you want to create stuff inside your company from some prebaked templates?
You're at the right place.
This guide is going to take you through how the Scaffolder in Backstage works.
We'll dive into some jargon and run through what's going on in the backend to be
able to create these templates. There's also more guides that you might find
useful at the bottom of this document. At its core, there are 3 simple stages.
1. Pick a skeleton
2. Template some variables into the skeleton
3. Send the templated skeleton somewhere
These three steps are translated to the following stages under the hood in the
scaffolder that you will need to know:
1. Prepare
2. Template
3. Publish
Each of these steps can be configured for your own use case, but we provide some
sensible defaults, too.
Let's dive a little deeper into these phases.
### Glossary and Jargon
**Preparer** - The preparer is responsible for fetching the skeleton code and
placing it into a directory and then will return that directory. It is
registered with the `Preparers` with a particular type, which is then used in
the router to pick the correct `Preparer` to run for the `Template` entity.
**Templater** - The templater is responsible for actually running the chosen
templater on top of the previously returned temporary directory from the
**Preparer**. We advise making these Docker containers as it can keep all
dependencies--for example Cookiecutter--self contained and not a dependency on
the host machine.
**Publisher** - The publisher is responsible for taking the finished directory,
and publishing it to a remote registry. This could be a Git repository or
something similar. Right now, the scaffolder only supports one publishing method
for the entire lifecycle, but it could be configured from the frontend and
passed through to the scaffolder backend.
### How it works
Most of the heavy lifting is done in the
[router.ts](https://github.com/backstage/backstage/blob/master/plugins/scaffolder-backend/src/service/router.ts#L93)
file in the `scaffolder-backend` plugin.
There are two routes defined in the router: `POST /v1/jobs` and
`GET /v1/job/:jobId`
To create a scaffolding job, a JSON object containing the
[Template Entity](../../software-catalog/descriptor-format.md#kind-template) +
additional templating values must be posted as the post body.
```js
{
"template": {
"apiVersion": "backstage/v1alpha1",
"kind": "Template",
// more stuff here
},
"values": {
"component_id": "test",
"description": "somethingelse"
}
}
```
The values should represent something that is valid with the `schema` part of
the [Template Entity](../../software-catalog/descriptor-format.md#kind-template)
Once that has been posted, a job will be setup with different stages, and the
job processor will complete each stage before moving onto the next stage, whilst
collecting logs and mutating the running job.
Here's some further reading that you might find useful:
- [Adding your own Template](../adding-templates.md)
- [Creating your own Templater](./create-your-own-templater.md)
- [Creating your own Publisher](./create-your-own-publisher.md)
- [Creating your own Preparer](./create-your-own-preparer.md)
@@ -99,6 +99,7 @@ export default async function createPlugin({
logger,
config,
database,
reader,
}: PluginEnvironment) {
const cookiecutterTemplater = new CookieCutter();
const craTemplater = new CreateReactAppTemplater();
@@ -124,6 +125,7 @@ export default async function createPlugin({
dockerClient,
database,
catalogClient,
reader,
});
}
```
@@ -136,10 +138,11 @@ import scaffolder from './plugins/scaffolder';
const scaffolderEnv = useHotMemoize(module, () => createEnv('scaffolder'));
const service = createServiceBuilder(module)
.loadConfig(configReader)
/** several different routers */
.addRouter('/scaffolder', await scaffolder(scaffolderEnv));
const apiRouter = Router();
/* several router .use calls */
/* add this line */
apiRouter.use('/scaffolder', await scaffolder(scaffolderEnv));
```
### Adding Templates
@@ -170,14 +173,10 @@ catalog:
### Runtime Dependencies / Configuration
For the scaffolder backend plugin to function, it needs a GitHub access token,
and access to a running Docker daemon. You can create a GitHub access token
[here](https://github.com/settings/tokens/new), select `repo` scope only. Full
docs on creating private GitHub access tokens is available
[here](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token).
Note that the need for private GitHub access tokens will be replaced with GitHub
Apps integration further down the line by using the existing `integrations`
config.
For the scaffolder backend plugin to function, you'll need to setup the
integrations config in your `app-config.yaml`.
You can find help for different providers below.
> Note: Some of this configuration may already be set up as part of your
> `app-config.yaml`. We're moving away from the duplicated config for
@@ -220,6 +219,32 @@ integrations:
$env: GITLAB_TOKEN
```
#### BitBucket
For Bitbucket there are two authentication methods supported. Either `token` or
a combination of `appPassword` and `username`. It looks like either of the
following:
```yaml
integrations:
bitbucket:
- host: bitbucket.org
token:
$env: BITBUCKET_TOKEN
```
or
```yaml
integrations:
bitbucket:
- host: bitbucket.org
appPassword:
$env: BITBUCKET_APP_PASSWORD
username:
$env: BITBUCKET_USERNAME
```
#### Azure DevOps
For Azure DevOps we support both the preparer and publisher stage with the
+118
View File
@@ -0,0 +1,118 @@
---
id: template-legacy
title: Writing Templates (Legacy)
description:
Old documentation describing the backstage.io/v1alpha1 format of the Template
Schema
---
## Kind: Template
Describes the following entity kind:
| Field | Value |
| ------------ | ----------------------- |
| `apiVersion` | `backstage.io/v1alpha1` |
| `kind` | `Template` |
A Template describes a skeleton for use with the Scaffolder. It is used for
describing what templating library is supported, and also for documenting the
variables that the template requires using
[JSON Forms Schema](https://jsonforms.io/).
Descriptor files for this kind may look as follows.
```yaml
apiVersion: backstage.io/v1alpha1
kind: Template
metadata:
name: react-ssr-template
title: React SSR Template
description:
Next.js application skeleton for creating isomorphic web applications.
tags:
- recommended
- react
spec:
owner: web@example.com
templater: cookiecutter
type: website
path: '.'
schema:
required:
- component_id
- description
properties:
component_id:
title: Name
type: string
description: Unique name of the component
description:
title: Description
type: string
description: Description of the component
```
In addition to the [common envelope metadata](#common-to-all-kinds-the-metadata)
shape, this kind has the following structure.
### `apiVersion` and `kind` [required]
Exactly equal to `backstage.io/v1alpha1` and `Template`, respectively.
### `metadata.title` [required]
The nice display name for the template as a string, e.g. `React SSR Template`.
This field is required as is used to reference the template to the user instead
of the `metadata.name` field.
### `metadata.tags` [optional]
A list of strings that can be associated with the template, e.g.
`['recommended', 'react']`.
This list will also be used in the frontend to display to the user so you can
potentially search and group templates by these tags.
### `spec.type` [optional]
The type of component as a string, e.g. `website`. This field is optional but
recommended.
The software catalog accepts any type value, but an organization should take
great care to establish a proper taxonomy for these. Tools including Backstage
itself may read this field and behave differently depending on its value. For
example, a website type component may present tooling in the Backstage interface
that is specific to just websites.
The current set of well-known and common values for this field is:
- `service` - a backend service, typically exposing an API
- `website` - a website
- `library` - a software library, such as an npm module or a Java library
### `spec.templater` [required]
The templating library that is supported by the template skeleton as a string,
e.g `cookiecutter`.
Different skeletons will use different templating syntax, so it's common that
the template will need to be run with a particular piece of software.
This key will be used to identify the correct templater which is registered into
the `TemplatersBuilder`.
The values which are available by default are:
- `cookiecutter` - [cookiecutter](https://github.com/cookiecutter/cookiecutter).
### `spec.path` [optional]
The string location where the templater should be run if it is not on the same
level as the `template.yaml` definition, e.g. `./cookiecutter/skeleton`.
This will set the `cwd` when running the templater to the folder path that you
specify relative to the `template.yaml` definition.
This is also particularly useful when you have multiple template definitions in
the same repository but only a single `template.yaml` registered in backstage.
@@ -0,0 +1,165 @@
---
id: writing-custom-actions
title: Writing Custom Actions
description: How to write your own actions
---
If you're wanting to extend the functionality of the Scaffolder, you can do so
by writing custom actions which can be used along side our
[built-in actions](./builtin-actions.md)
### Writing your Custom Action
Your custom action can live where you choose, but simplest is to include it
alongside your `backend` package in `packages/backend`.
Let's create a simple action that adds a new file and some contents that are
passed as `input` to the function.
In `packages/backend/src/actions/custom.ts` we can create a new action.
```ts
import { createTemplateAction } from '../../createTemplateAction';
import fs from 'fs-extra';
export const createNewFileAction = () => {
return createTemplateAction<{ contents: string; filename: string }>({
id: 'mycompany:create-file',
schema: {
input: {
required: ['contents', 'filename'],
type: 'object',
properties: {
contents: {
type: 'string',
title: 'Contents',
description: 'The contents of the file',
},
contents: {
type: 'string',
title: 'Filename',
description: 'The filename of the file that will be created',
},
},
},
},
async handler(ctx) {
await fs.outputFile(
`${ctx.workspacePath}/${ctx.input.filename}`,
ctx.input.content,
);
},
});
};
```
So let's break this down. The `createNewFileAction` is a function that returns a
`createTemplateAction`, and it's a good place to pass in dependencies which
close over the `TemplateAction`. Take a look at our
[built-in actions](https://github.com/backstage/backstage/blob/7f5716081f45a41dc8a4246134b50c893e15c5e1/../plugins/scaffolder-backend/src/scaffolder/actions/builtin/publish/github.ts)
for reference.
We set the type generic to `{ contents: string, filename: string}` which is
there to set the type on the handler `ctx` `inputs` property so we get good type
checking. This could be generated from the next part of this guide, the `input`
schema, but it's not supported right now. Feel free to contribute 🚀 👍.
The `createTemplateAction` takes an object which specifies the following:
- `id` - a unique ID for your custom action. We encourage you to namespace these
in someway so they wont collide with future built-in actions that we may ship
with the `scaffolder-backend` plugin.
- `schema.input` - A JSON schema for input values to your function
- `schema.output` - A JSON schema for values which are outputted from the
function using `ctx.output`
- `handler` the actual code which is run part of the action, with a context.
#### The context object
When the action `handler` is called, we provide you a `context` as the only
argument. It looks like the following:
- `ctx.baseUrl` - a string where the template is located
- `ctx.logger` - a winston logger for additional logging inside your action
- `ctx.logStream` - a stream version of the logger if needed
- `ctx.workspacePath` - a string of the working directory of the template run
- `ctx.input` - an object which should match the JSON schema provided in the
`schema.input` part of the action definition
- `ctx.output` - a function which you can call to set outputs that match the
JSON schema in `schema.output` for ex. `ctx.output('downloadUrl', something)`
- `createTemporaryDirectory` a function to call to give you a temporary
directory somewhere on the runner so you can store some files there rather
than polluting the `workspacePath`
### Registering Custom Actions
Once you have your Custom Action ready for usage with the scaffolder, you'll
need to pass this into the `scaffolder-backend` `createRouter` function. You
should have something similar to the below in
`packages/backend/src/plugins/scaffolder.ts`
```ts
return await createRouter({
preparers,
templaters,
publishers,
logger,
config,
dockerClient,
database,
catalogClient,
reader,
});
```
There's another property you can pass here, which is an array of `actions` which
will set the available actions that the scaffolder has access to.
```ts
const actions = [createNewFileAction()];
return await createRouter({
preparers,
templaters,
publishers,
logger,
config,
dockerClient,
database,
catalogClient,
reader,
actions,
});
```
**NOTE** - the actions array will replace the built-in actions too, so if you
want to have those as well as your new one, you'll need to do the following:
```ts
import { createBuiltinActions } from '@backstage/plugin-scaffolder-backend`;
const builtInActions = createBuiltinActions({
dockerClient,
integrations,
catalogClient,
templaters,
reader,
});
const actions = [...builtInActions, createNewFileAction()];
return await createRouter({
preparers,
templaters,
publishers,
logger,
config,
dockerClient,
database,
catalogClient,
reader,
actions,
});
```
Have fun! 🚀
@@ -0,0 +1,286 @@
---
id: writing-templates
title: Writing Templates
description: Details around creating your own custom Software Templates
---
Templates are stored in the **Service Catalog** under a kind `Template`. You can
create your own templates with a small `yaml` definition which describes the
template and it's metadata, along with some input variables that your template
will need, and then a list of actions which are then executed by the scaffolding
service.
Let's take a look at a simple example:
```yaml
# Notice the v1beta2 version
apiVersion: backstage.io/v1beta2
kind: Template
# some metadata about the template itself
metadata:
name: v1beta2-demo
title: Test Action template
description: scaffolder v1beta2 template demo
spec:
owner: backstage/techdocs-core
type: service
# these are the steps which are rendered in the frontend with the form input
parameters:
- title: Fill in some steps
required:
- name
properties:
name:
title: Name
type: string
description: Unique name of the component
ui:autofocus: true
ui:options:
rows: 5
- title: Choose a location
required:
- repoUrl
properties:
repoUrl:
title: Repository Location
type: string
ui:field: RepoUrlPicker
ui:options:
allowedHosts:
- github.com
# here's the steps that are executed in series in the scaffolder backend
steps:
- id: fetch-base
name: Fetch Base
action: fetch:cookiecutter
input:
url: ./template
values:
name: '{{ parameters.name }}'
- id: fetch-docs
name: Fetch Docs
action: fetch:plain
input:
targetPath: ./community
url: https://github.com/backstage/community/tree/main/backstage-community-sessions
- id: publish
name: Publish
action: publish:github
input:
allowedHosts: ['github.com']
description: 'This is {{ parameters.name }}'
repoUrl: '{{ parameters.repoUrl }}'
- id: register
name: Register
action: catalog:register
input:
repoContentsUrl: '{{ steps.publish.output.repoContentsUrl }}'
catalogInfoPath: '/catalog-info.yaml'
# some outputs which are saved along with the job for use in the frontend
output:
remoteUrl: '{{ steps.publish.output.remoteUrl }}'
entityRef: '{{ steps.register.output.entityRef }}'
```
Let's dive in an pick apart what each of these sections do and what they are.
### `spec.parameters` - `FormStep | FormStep[]`
These `parameters` are template variables which can be modified in the frontend
as a sequence. It can either be one `Step` if you just want one big list of
different fields in the frontend, or it can be broken up into multiple different
steps which would be rendered as different steps in the scaffolder plugin
frontend.
Each `Step` is `JSONSchema` with some extra goodies for styling what it might
look like in the frontend. For these steps we rely very heavily on this library:
https://github.com/rjsf-team/react-jsonschema-form. They have some great docs
too here: https://react-jsonschema-form.readthedocs.io/ and a playground where
you can play around with some examples here
https://rjsf-team.github.io/react-jsonschema-form.
There's another option for that library called `uiSchema` which we've taken
advantage of, and we've merged it with the existing `JSONSchema` that you
provide to the library. These are the little `ui:*` properties that you can see
in the step definitions.
For example if we take the **simple** example from the playground it looks like
this:
```json
// jsonSchema:
{
"title": "A registration form",
"description": "A simple form example.",
"type": "object",
"required": [
"firstName",
"lastName"
],
"properties": {
"firstName": {
"type": "string",
"title": "First name",
"default": "Chuck"
},
"lastName": {
"type": "string",
"title": "Last name"
},
"telephone": {
"type": "string",
"title": "Telephone",
"minLength": 10
}
}
}
// uiSchema:
{
"firstName": {
"ui:autofocus": true,
"ui:emptyValue": "",
"ui:autocomplete": "family-name"
},
"lastName": {
"ui:emptyValue": "",
"ui:autocomplete": "given-name"
},
"telephone": {
"ui:options": {
"inputType": "tel"
}
}
}
```
It would look something like the following in a template:
```yaml
apiVersion: backstage.io/v1beta2
kind: Template
metadata:
name: v1beta2-demo
title: Test Action template
description: scaffolder v1beta2 template demo
spec:
owner: backstage/techdocs-core
type: service
parameters:
- title: A registration form
description: A simple form example.
type: object
required:
- firstName
- lastName
properties:
firstName:
type: string
title: First name
default: Chuck
ui:autofocus: true
ui:emptyValue: ''
ui:autocomplete: family-name
lastName:
type: string
title: Last name
ui:emptyValue: ''
ui:autocomplete: given-name
telephone:
type: string
title: Telephone
minLength: 10
ui:options:
inputType: tel
```
#### The Repository Picker
So in order to make working with repository providers easier, we've built a
custom picker that can be used by overriding the `ui:field` option in the
`uiSchema` for a `string` field. Instead of displaying a text input block it
will render our custom component that we've built which makes it easy to select
a repository provider, and insert a project or owner, and repository name.
You can see it in the above full example which is a separate step and it looks a
little like this:
```yaml
- title: Choose a location
required:
- repoUrl
properties:
repoUrl:
title: Repository Location
type: string
ui:field: RepoUrlPicker
ui:options:
allowedHosts:
- github.com
```
The `allowedHosts` part should be set to where you wish to enable this template
to publish to. And it can be any host that is listed in your `integrations`
config in `app-config.yaml`.
The `RepoUrlPicker` is a custom field that we provide part of the
`plugin-scaffolder`. It's currently not possible to create your own fields yet,
but contributions are welcome! :)
### `spec.steps` - `Action[]`
The `steps` is an array of the things that you want to happen part of this
template. These follow the same standard format:
```yaml
- id: fetch-base # A unique id for the step
name: Fetch Base # A title displayed in the frontend
action: fetch:cookiecutter # an action to call
input: # input that is passed as arguments to the action handler
url: ./template
values:
name: '{{ parameters.name }}'
```
By default we ship some built in actions that you can take a look at
[here](./builtin-actions.md), or you can create your own custom actions by
looking at the docs [here](./writing-custom-actions.md)
### Outputs
Each individual step can output some variables that can be used in the
scaffolder frontend for after the job is finished. This is useful for things
like linking to the entity that has been created with the backend, and also
linking to the created repository.
The main two that are used are the following:
```yaml
output:
remoteUrl: '{{ steps.publish.output.remoteUrl }}' # link to the remote repository
entityRef: '{{ steps.register.output.entityRef }}' # link to the entitiy that has been ingested to the catalog
```
### The templating syntax
You might have noticed in the examples that there are `{{ }}`, and these are a
`handlebars` templates for linking and glueing all these different parts of
`yaml` together. All the form inputs from the `parameters` section, when passed
to the steps will be available by using the template syntax
`{{ parameters.something }}`. This is great for passing the values from the form
into different steps and reusing these input variables.
As you can see above in the `Outputs` section, `actions` and `steps` can also
output things. So you can grab that output by using
`steps.$stepId.output.$property`.
You can read more about all the `inputs` and `outputs` defined in the actions in
code part of the `JSONSchema` or you can read more about our built in ones
[here](./builtin-actions.md).