diff --git a/docs/features/techdocs/README.md b/docs/features/techdocs/README.md
index 16131a1ded..50cadc7b27 100644
--- a/docs/features/techdocs/README.md
+++ b/docs/features/techdocs/README.md
@@ -96,26 +96,38 @@ Build a widget (plugin) framework so that contributors can easily contribute
features to TechDocs - that others can use. And, also, so that we can easily
migrate Spotify's existing TechDocs features to open source.
-## Structure
+## Platforms Supported
-- [Getting Started]
-- [Concepts]
-- [Creating and Publishing Documentation]
-- [FAQ]
+See [TechDocs Architecture](architecture.md) to get an overview of where these
+providers are used.
+
+| Source Code Hosting Provider | Support Status |
+| ---------------------------- | -------------- |
+| GitHub | Yes ✅ |
+| GitHub Enterprise | Yes ✅ |
+| BitBucket | Yes ✅ |
+| Azure DevOps | Yes ✅ |
+| GitLab | Yes ✅ |
+| GitLab Enterprise | Yes ✅ |
+
+| File Storage Provider | Support Status | Track status |
+| --------------------------------- | -------------- | ----------------------------------------------------------- |
+| Local Filesystem of Backstage app | Yes ✅ | |
+| Google Cloud Storage (GCS) | Yes ✅ | |
+| Amazon Web Services (AWS) S3 | No ❌ | [#3714](https://github.com/backstage/backstage/issues/3714) |
+| Azure Storage | No ❌ | |
+
+[Reach out to us](#feedback) if you want to request more platforms.
## Tech Stack
-| Stack | Location |
-| ------------------------------------------- | -------------------------------------------------------- |
-| Frontend | [`@backstage/plugin-techdocs`][techdocs/frontend] |
-| Backend | [`@backstage/plugin-techdocs-backend`][techdocs/backend] |
-| Docker Container (for generating doc sites) | [`techdocs-container`][techdocs/container] |
-| CLI (for local development) | [`@techdocs/cli`][techdocs/cli] |
+| Stack | Location |
+| ----------------------------------------------- | -------------------------------------------------------- |
+| Frontend Plugin | [`@backstage/plugin-techdocs`][techdocs/frontend] |
+| Backend Plugin | [`@backstage/plugin-techdocs-backend`][techdocs/backend] |
+| CLI (for local development and generating docs) | [`@techdocs/cli`][techdocs/cli] |
+| Docker Container (for generating docs) | [`techdocs-container`][techdocs/container] |
-[getting started]: getting-started.md
-[concepts]: concepts.md
-[creating and publishing documentation]: creating-and-publishing.md
-[faq]: FAQ.md 'Frequently asked questions'
[techdocs/frontend]:
https://github.com/backstage/backstage/blob/master/plugins/techdocs
[techdocs/backend]:
@@ -131,3 +143,6 @@ https://docs.google.com/forms/d/e/1FAIpQLSdn5Vn3MQhCdyYRuW8cMzZkMQF0bFxXYN168gZR
This is to gather inputs from you (the Backstage community) which will help us
best serve TechDocs adopters and existing users. Your inputs will shape our
roadmap and we will share it in the open.
+
+For any other general queries, reach out to us in the `#docs-like-code` channel
+of our [Discord chatroom](https://github.com/backstage/backstage#community).
diff --git a/docs/features/techdocs/concepts.md b/docs/features/techdocs/concepts.md
index 1675c64288..26254a7e12 100644
--- a/docs/features/techdocs/concepts.md
+++ b/docs/features/techdocs/concepts.md
@@ -8,15 +8,45 @@ Spotify's docs-like-code solution in Backstage
This page describes concepts that are introduced with Spotify's docs-like-code
solution in Backstage.
-### TechDocs Core Plugin
+## Generating TechDocs Steps
-The TechDocs Core Plugin is an [MkDocs](https://www.mkdocs.org/) plugin created
-as a wrapper around multiple MkDocs plugins and Python Markdown extensions to
-standardize the configuration of MkDocs used for TechDocs.
+### TechDocs Preparer
-[TechDocs Core](https://github.com/backstage/mkdocs-techdocs-core)
+Preparing is the first step of generating documentation for an entity. It
+fetches the source markdown files from the source code hosting provider (GitHub,
+GitLab, etc.) and passes the files to the generator for next steps.
-### TechDocs container
+There are two kinds of preparers available -
+
+1. Common Git Preparer - Uses `git clone` on any repository url.
+2. Url Reader - Uses source code hosting provider's API to download files.
+ (Faster and recommended)
+
+### TechDocs Generator
+
+Generating is the second step after preparing the markdown source files. This
+step either runs the TechDocs container (defined below) or runs `mkdocs` CLI to
+generate static HTML files and its assets.
+
+### TechDocs Publisher
+
+Publishing is the third and final step after preparing and generating docs.
+TechDocs Publisher uploads the generated files to a storage.
+
+The `techdocs-backend` plugin currently comes with two publishers - Google Cloud
+Storage and Local Filesystem. You can configure them in your Backstage app.
+[See here](./configuration.md).
+
+A TechDocs publisher is responsible for two things (two-way communication
+between `techdocs-backend` and the storage)
+
+1. Publish generated static files to a storage (Configured by
+ `techdocs.builder`)
+2. Read files from the storage when users visit a TechDocs site
+
+[TechDocs Backend](https://github.com/backstage/backstage/tree/master/plugins/techdocs-backend)
+
+## TechDocs Container
The TechDocs container is a Docker container available at
[DockerHub](https://hub.docker.com/r/spotify/techdocs). It builds static HTML
@@ -25,16 +55,15 @@ MkDocs.
[TechDocs Container](https://github.com/backstage/techdocs-container)
-### TechDocs publisher
+## TechDocs Core Plugin
-The `techdocs-backend` plugin currently comes with one publisher -
-`LocalPublish`.
+The TechDocs Core Plugin is an [MkDocs](https://www.mkdocs.org/) plugin created
+as a wrapper around multiple MkDocs plugins and Python Markdown extensions to
+standardize the configuration of MkDocs used for TechDocs.
-[TechDocs Backend](https://github.com/backstage/backstage/tree/master/plugins/techdocs-backend)
+[TechDocs Core](https://github.com/backstage/mkdocs-techdocs-core)
-More standalone publishers will come in the near future...
-
-### TechDocs CLI
+## TechDocs CLI
The TechDocs CLI was created to make it easy to write, generate and preview
documentation for publishing. Currently it mostly acts as a wrapper around the
@@ -43,7 +72,7 @@ container.
[TechDocs CLI](https://github.com/backstage/techdocs-cli)
-### TechDocs Reader
+## TechDocs Reader
Documentation generated by TechDocs is generated as static HTML sites. The
TechDocs Reader was therefore created to be able to integrate pre-generated HTML
@@ -55,7 +84,7 @@ TechDocs widgets for a customized full-featured TechDocs experience.
[TechDocs Reader](https://github.com/backstage/backstage/blob/master/plugins/techdocs/src/reader/README.md)
-### Transformers
+## Transformers
Transformers are different pieces of functionality used inside the TechDocs
Reader. The reason why transformers were introduced was to provide a way to
diff --git a/docs/features/techdocs/getting-started.md b/docs/features/techdocs/getting-started.md
index e51c6a122e..c82bc8b95b 100644
--- a/docs/features/techdocs/getting-started.md
+++ b/docs/features/techdocs/getting-started.md
@@ -1,7 +1,7 @@
---
id: getting-started
title: Getting Started
-description: Getting Started Guidelines
+description: Getting Started Documentation
---
TechDocs functions as a plugin to Backstage, so you will need to use Backstage
@@ -10,50 +10,156 @@ to use TechDocs.
If you haven't setup Backstage already, start
[here](../../getting-started/index.md).
-## Installing TechDocs
+> If you used `npx @backstage/create-app`, TechDocs may already be present.
+>
+> You should skip to [`Setting the Configuration`](#setting-the-configuration)
+> below.
-TechDocs is provided with the Backstage application by default. If you want to
-set up TechDocs manually, keep following the instructions below.
-
-### Adding the package
+## Adding TechDocs frontend plugin
The first step is to add the TechDocs plugin to your Backstage application.
-Navigate to your new Backstage application folder:
-
-```bash
-cd hello-world/
-```
-
-Then navigate to your `packages/app` folder to install TechDocs:
+Navigate to your new Backstage application directory. And then to your
+`packages/app` directory, and install the `@backstage/plugin-techdocs` package.
```bash
+cd my-backstage-app/
cd packages/app
yarn add @backstage/plugin-techdocs
```
-After a short while, the TechDocs plugin should be successfully installed.
+Once the package has been installed, you need to import the plugin in your app.
-Next, you need to set up some basic configuration. Enter the following command:
-
-```bash
-yarn install
-```
-
-Add this to `packages/app/src/plugins.ts`:
+Add the following to `packages/app/src/plugins.ts`:
```typescript
export { plugin as TechDocs } from '@backstage/plugin-techdocs';
```
-### Setting the configuration
+Now let us embed the TechDocs router in our main Backstage frontend router. In
+`packages/app/src/App.tsx`, import the TechDocs router and add the following to
+`AppRoutes`:
-TechDocs allows for configuration of the docs storage URL through your
-`app-config.yaml` file. We provide two different values to be configured,
-`requestUrl` and `storageUrl`. The `requestUrl` is what the reader will request
-its data from, and `storageUrl` is where the backend can find the stored
-documentation.
+```tsx
+import { Router as DocsRouter } from '@backstage/plugin-techdocs';
-The default storage and request URLs:
+// ...
+
+const AppRoutes = () => {
+
+ // ... other plugin routes
+ } />
+ ;
+};
+```
+
+That's it! But now, we need the TechDocs Backend plugin for the frontend to
+work.
+
+## Adding TechDocs Backend plugin
+
+Navigate to `packages/backend` of your Backstage app, and install the
+`@backstage/plugin-techdocs-backend` package.
+
+```bash
+cd my-backstage-app/
+cd packages/backend
+yarn add @backstage/plugin-techdocs-backend
+```
+
+Create a file called `techdocs.ts` inside `packages/backend/src/plugins/` and
+add the following
+
+```typescript
+import {
+ createRouter,
+ DirectoryPreparer,
+ Preparers,
+ Generators,
+ TechdocsGenerator,
+ CommonGitPreparer,
+ UrlPreparer,
+ Publisher,
+} from '@backstage/plugin-techdocs-backend';
+import { PluginEnvironment } from '../types';
+import Docker from 'dockerode';
+
+export default async function createPlugin({
+ logger,
+ config,
+ discovery,
+ reader,
+}: PluginEnvironment) {
+ // Preparers are responsible for fetching source files for documentation.
+ const preparers = new Preparers();
+
+ const directoryPreparer = new DirectoryPreparer(logger);
+ preparers.register('dir', directoryPreparer);
+
+ const commonGitPreparer = new CommonGitPreparer(logger);
+ preparers.register('github', commonGitPreparer);
+ preparers.register('gitlab', commonGitPreparer);
+ preparers.register('azure/api', commonGitPreparer);
+
+ const urlPreparer = new UrlPreparer(reader, logger);
+ preparers.register('url', urlPreparer);
+
+ // Generators are used for generating documentation sites.
+ const generators = new Generators();
+ const techdocsGenerator = new TechdocsGenerator(logger, config);
+ generators.register('techdocs', techdocsGenerator);
+
+ // Publishers are used for
+ // 1. Publishing generated files to storage
+ // 2. Fetching files from storage and passing them to TechDocs frontend.
+ const publisher = Publisher.fromConfig(config, logger, discovery);
+
+ // Docker client used by the generators.
+ const dockerClient = new Docker();
+
+ return await createRouter({
+ preparers,
+ generators,
+ publisher,
+ dockerClient,
+ logger,
+ config,
+ discovery,
+ });
+}
+```
+
+You may need to install the `dockerode` package. But you may already have it in
+your backend since [Scaffolder plugin](../software-templates/index.md) also uses
+it.
+
+See [Concepts](concepts.md) and [TechDocs Architecture](architecture.md) to
+learn more about how preparers, generators and publishers work.
+
+Final step is to import the techdocs backend plugin in Backstage app backend.
+Add the following to your `packages/backend/src/index.ts`:
+
+```typescript
+import techdocs from './plugins/techdocs';
+
+// .... main should already be present.
+async function main() {
+ // ... other backend plugin envs
+ const techdocsEnv = useHotMemoize(module, () => createEnv('techdocs'));
+
+ // ... other backend plugin routes
+ apiRouter.use('/techdocs', await techdocs(techdocsEnv));
+}
+```
+
+That's it! TechDocs frontend and backend have now been added to your Backstage
+app. Now let us tweak some configurations to suit your needs.
+
+## Setting the configuration
+
+**See [TechDocs Configuration Options](configuration.md) for complete
+configuration reference.**
+
+### Setting TechDocs URLs
```yaml
techdocs:
@@ -61,22 +167,59 @@ techdocs:
requestUrl: http://localhost:7000/api/techdocs/
```
-If you want `techdocs-backend` to manage building and publishing, you want
-`requestUrl` to point to the default value (or wherever `techdocs-backend` is
-hosted). `storageUrl` should be where your publisher publishes your docs. Using
-the default `LocalPublish` that is the default value.
+`requestUrl` is used by TechDocs frontend plugin to discover techdocs-backend
+endpoints, and the `storageUrl` is another endpoint in `techdocs-backend` which
+acts as a middleware between TechDocs and the storage (where the static
+generated docs site are stored). These default values should mostly work for
+you. These options will soon be optional to set.
-If you have a setup where you are not using `techdocs-backend` for managing
-building and publishing of your documentation, you want to change the
-`requestUrl` to point to your storage. In this case `storageUrl` is not
-required.
+### Should TechDocs Backend generate docs?
-### Disable Docker in Docker situation (Optional)
+```yaml
+techdocs:
+ storageUrl: http://localhost:7000/api/techdocs/static/docs
+ requestUrl: http://localhost:7000/api/techdocs/
+ builder: 'local'
+```
-The TechDocs backend plugin runs a docker container with mkdocs installed to
+Set `techdocs.builder` to `'local'` if you want your TechDocs Backend to be
+responsible for generating documentation sites. If set to `'external'`,
+Backstage will assume that the sites are being generated on each entity's CI/CD
+pipeline, and are being stored in a storage somewhere.
+
+When `techdocs.builder` is set to `'external'`, TechDocs becomes more or less a
+read-only experience where it serves static files from a storage containing all
+the generated documentation. Read more in the "Basic" and "Recommended" setup of
+TechDocs [here](architecture.md)
+
+### Choosing storage (publisher)
+
+TechDocs needs to know where to store generated documentation sites and where to
+fetch the sites from. This is managed by a
+[Publisher](./concepts.md#techdocs-publisher). Examples: Google Cloud Storage,
+Amazon S3, or local filesystem of Backstage server.
+
+It is okay to use the local filesystem in a "Basic" setup when you are trying
+out Backstage for the first time. Using Cloud Storage is documented
+[here](./using-cloud-storage.md).
+
+```yaml
+techdocs:
+ storageUrl: http://localhost:7000/api/techdocs/static/docs
+ requestUrl: http://localhost:7000/api/techdocs/
+ builder: 'local'
+ publisher:
+ type: 'local'
+```
+
+### Disabling Docker in Docker situation (Optional)
+
+You can skip this if your `techdocs.builder` is set to `'external'`.
+
+The TechDocs Backend plugin runs a docker container with mkdocs installed to
generate the frontend of the docs from source files (Markdown). If you are
deploying Backstage using Docker, this will mean that your Backstage Docker
-container will try to run another Docker container for TechDocs backend.
+container will try to run another Docker container for TechDocs Backend.
To avoid this problem, we have a configuration available. You can set a value in
your `app-config.yaml` that tells the techdocs generator if it should run the
@@ -108,23 +251,13 @@ the dependencies. For example, we want `Markdown` version to be
You can also explicitly install `Markdown==3.2.2` after installing all other
Python packages.
-## Run Backstage locally
+## Running Backstage locally
-Change folder to `/packages/backend` and run the
-following command:
+Start the frontend and the backend app by
+[running backstage locally](../../getting-started/running-backstage-locally.md).
-```bash
-yarn start
-```
-
-Open a new command line window. Change directory to your Backstage application
-root and run the following command:
-
-```bash
-yarn start
-```
-
-Open your browser at [http://localhost:3000/docs/](http://localhost:3000/docs/).
+Open your browser at [http://localhost:3000/docs/](http://localhost:3000/docs/)
+to see all your documentation sites.
## Additional reading
diff --git a/docs/features/techdocs/using-cloud-storage.md b/docs/features/techdocs/using-cloud-storage.md
index 55352b6c00..c7b5ec9a9f 100644
--- a/docs/features/techdocs/using-cloud-storage.md
+++ b/docs/features/techdocs/using-cloud-storage.md
@@ -40,7 +40,7 @@ techdocs:
publisher:
type: 'googleGcs'
googleGcs:
- projectId: 'gcp-project-id
+ projectId: 'gcp-project-id'
```
**3. Service account API key**
@@ -76,7 +76,7 @@ Create a dedicated bucket for TechDocs sites. techdocs-backend will publish
documentation to this bucket. TechDocs will fetch files from here to serve
documentation in Backstage.
-Set the name of the bucket to `techdocs.publisher
+Set the name of the bucket to `techdocs.publisher.googleGcs.bucketName`.
```yaml
techdocs:
diff --git a/packages/backend/src/plugins/techdocs.ts b/packages/backend/src/plugins/techdocs.ts
index 4720b2568e..4822de9c5c 100644
--- a/packages/backend/src/plugins/techdocs.ts
+++ b/packages/backend/src/plugins/techdocs.ts
@@ -32,10 +32,7 @@ export default async function createPlugin({
discovery,
reader,
}: PluginEnvironment) {
- const generators = new Generators();
- const techdocsGenerator = new TechdocsGenerator(logger, config);
- generators.register('techdocs', techdocsGenerator);
-
+ // Preparers are responsible for fetching source files for documentation.
const preparers = new Preparers();
const directoryPreparer = new DirectoryPreparer(logger);
@@ -49,8 +46,17 @@ export default async function createPlugin({
const urlPreparer = new UrlPreparer(reader, logger);
preparers.register('url', urlPreparer);
+ // Generators are used for generating documentation sites.
+ const generators = new Generators();
+ const techdocsGenerator = new TechdocsGenerator(logger, config);
+ generators.register('techdocs', techdocsGenerator);
+
+ // Publishers are used for
+ // 1. Publishing generated files to storage
+ // 2. Fetching files from storage and passing them to TechDocs frontend.
const publisher = Publisher.fromConfig(config, logger, discovery);
+ // Docker client used by the generators.
const dockerClient = new Docker();
return await createRouter({