paulononato/backstage
1
0
Fork 0
Code Issues Pull Requests Actions 20 Packages Projects Releases Wiki Activity

Merge pull request #3718 from backstage/orkohunter/update-techdocs-project-status-docs

Browse Source
This commit is contained in:
Himanshu Mishra
2020-12-16 16:25:13 +01:00
committed by GitHub
parent d62be2f822 eb39a633e8
commit d96f61bd70
5 changed files with 273 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/features/techdocs/README.md
+30 -15
View File
@@ -96,26 +96,38 @@ Build a widget (plugin) framework so that contributors can easily contribute
features to TechDocs - that others can use. And, also, so that we can easily
migrate Spotify's existing TechDocs features to open source.
## Structure
## Platforms Supported
- [Getting Started]
- [Concepts]
- [Creating and Publishing Documentation]
- [FAQ]
See [TechDocs Architecture](architecture.md) to get an overview of where these
providers are used.
| Source Code Hosting Provider | Support Status |
| ---------------------------- | -------------- |
| GitHub | Yes ✅ |
| GitHub Enterprise | Yes ✅ |
| BitBucket | Yes ✅ |
| Azure DevOps | Yes ✅ |
| GitLab | Yes ✅ |
| GitLab Enterprise | Yes ✅ |
| File Storage Provider | Support Status | Track status |
| --------------------------------- | -------------- | ----------------------------------------------------------- |
| Local Filesystem of Backstage app | Yes ✅ | |
| Google Cloud Storage (GCS) | Yes ✅ | |
| Amazon Web Services (AWS) S3 | No ❌ | [#3714](https://github.com/backstage/backstage/issues/3714) |
| Azure Storage | No ❌ | |
[Reach out to us](#feedback) if you want to request more platforms.
## Tech Stack
| Stack | Location |
| ------------------------------------------- | -------------------------------------------------------- |
| Frontend | [`@backstage/plugin-techdocs`][techdocs/frontend] |
| Backend | [`@backstage/plugin-techdocs-backend`][techdocs/backend] |
| Docker Container (for generating doc sites) | [`techdocs-container`][techdocs/container] |
| CLI (for local development) | [`@techdocs/cli`][techdocs/cli] |
| Stack | Location |
| ----------------------------------------------- | -------------------------------------------------------- |
| Frontend Plugin | [`@backstage/plugin-techdocs`][techdocs/frontend] |
| Backend Plugin | [`@backstage/plugin-techdocs-backend`][techdocs/backend] |
| CLI (for local development and generating docs) | [`@techdocs/cli`][techdocs/cli] |
| Docker Container (for generating docs) | [`techdocs-container`][techdocs/container] |
[getting started]: getting-started.md
[concepts]: concepts.md
[creating and publishing documentation]: creating-and-publishing.md
[faq]: FAQ.md 'Frequently asked questions'
[techdocs/frontend]:
https://github.com/backstage/backstage/blob/master/plugins/techdocs
[techdocs/backend]:
@@ -131,3 +143,6 @@ https://docs.google.com/forms/d/e/1FAIpQLSdn5Vn3MQhCdyYRuW8cMzZkMQF0bFxXYN168gZR
This is to gather inputs from you (the Backstage community) which will help us
best serve TechDocs adopters and existing users. Your inputs will shape our
roadmap and we will share it in the open.
For any other general queries, reach out to us in the `#docs-like-code` channel
of our [Discord chatroom](https://github.com/backstage/backstage#community).
docs/features/techdocs/concepts.md
+44 -15
View File
@@ -8,15 +8,45 @@ Spotify's docs-like-code solution in Backstage
This page describes concepts that are introduced with Spotify's docs-like-code
solution in Backstage.
### TechDocs Core Plugin
## Generating TechDocs Steps
The TechDocs Core Plugin is an [MkDocs](https://www.mkdocs.org/) plugin created
as a wrapper around multiple MkDocs plugins and Python Markdown extensions to
standardize the configuration of MkDocs used for TechDocs.
### TechDocs Preparer
[TechDocs Core](https://github.com/backstage/mkdocs-techdocs-core)
Preparing is the first step of generating documentation for an entity. It
fetches the source markdown files from the source code hosting provider (GitHub,
GitLab, etc.) and passes the files to the generator for next steps.
### TechDocs container
There are two kinds of preparers available -
1. Common Git Preparer - Uses `git clone` on any repository url.
2. Url Reader - Uses source code hosting provider's API to download files.
(Faster and recommended)
### TechDocs Generator
Generating is the second step after preparing the markdown source files. This
step either runs the TechDocs container (defined below) or runs `mkdocs` CLI to
generate static HTML files and its assets.
### TechDocs Publisher
Publishing is the third and final step after preparing and generating docs.
TechDocs Publisher uploads the generated files to a storage.
The `techdocs-backend` plugin currently comes with two publishers - Google Cloud
Storage and Local Filesystem. You can configure them in your Backstage app.
[See here](./configuration.md).
A TechDocs publisher is responsible for two things (two-way communication
between `techdocs-backend` and the storage)
1. Publish generated static files to a storage (Configured by
`techdocs.builder`)
2. Read files from the storage when users visit a TechDocs site
[TechDocs Backend](https://github.com/backstage/backstage/tree/master/plugins/techdocs-backend)
## TechDocs Container
The TechDocs container is a Docker container available at
[DockerHub](https://hub.docker.com/r/spotify/techdocs). It builds static HTML
@@ -25,16 +55,15 @@ MkDocs.
[TechDocs Container](https://github.com/backstage/techdocs-container)
### TechDocs publisher
## TechDocs Core Plugin
The `techdocs-backend` plugin currently comes with one publisher -
`LocalPublish`.
The TechDocs Core Plugin is an [MkDocs](https://www.mkdocs.org/) plugin created
as a wrapper around multiple MkDocs plugins and Python Markdown extensions to
standardize the configuration of MkDocs used for TechDocs.
[TechDocs Backend](https://github.com/backstage/backstage/tree/master/plugins/techdocs-backend)
[TechDocs Core](https://github.com/backstage/mkdocs-techdocs-core)
More standalone publishers will come in the near future...
### TechDocs CLI
## TechDocs CLI
The TechDocs CLI was created to make it easy to write, generate and preview
documentation for publishing. Currently it mostly acts as a wrapper around the
@@ -43,7 +72,7 @@ container.
[TechDocs CLI](https://github.com/backstage/techdocs-cli)
### TechDocs Reader
## TechDocs Reader
Documentation generated by TechDocs is generated as static HTML sites. The
TechDocs Reader was therefore created to be able to integrate pre-generated HTML
@@ -55,7 +84,7 @@ TechDocs widgets for a customized full-featured TechDocs experience.
[TechDocs Reader](https://github.com/backstage/backstage/blob/master/plugins/techdocs/src/reader/README.md)
### Transformers
## Transformers
Transformers are different pieces of functionality used inside the TechDocs
Reader. The reason why transformers were introduced was to provide a way to
docs/features/techdocs/getting-started.md
+187 -54
View File
@@ -1,7 +1,7 @@
---
id: getting-started
title: Getting Started
description: Getting Started Guidelines
description: Getting Started Documentation
---
TechDocs functions as a plugin to Backstage, so you will need to use Backstage
@@ -10,50 +10,156 @@ to use TechDocs.
If you haven't setup Backstage already, start
[here](../../getting-started/index.md).
## Installing TechDocs
> If you used `npx @backstage/create-app`, TechDocs may already be present.
>
> You should skip to [`Setting the Configuration`](#setting-the-configuration)
> below.
TechDocs is provided with the Backstage application by default. If you want to
set up TechDocs manually, keep following the instructions below.
### Adding the package
## Adding TechDocs frontend plugin
The first step is to add the TechDocs plugin to your Backstage application.
Navigate to your new Backstage application folder:
```bash
cd hello-world/
```
Then navigate to your `packages/app` folder to install TechDocs:
Navigate to your new Backstage application directory. And then to your
`packages/app` directory, and install the `@backstage/plugin-techdocs` package.
```bash
cd my-backstage-app/
cd packages/app
yarn add @backstage/plugin-techdocs
```
After a short while, the TechDocs plugin should be successfully installed.
Once the package has been installed, you need to import the plugin in your app.
Next, you need to set up some basic configuration. Enter the following command:
```bash
yarn install
```
Add this to `packages/app/src/plugins.ts`:
Add the following to `packages/app/src/plugins.ts`:
```typescript
export { plugin as TechDocs } from '@backstage/plugin-techdocs';
```
### Setting the configuration
Now let us embed the TechDocs router in our main Backstage frontend router. In
`packages/app/src/App.tsx`, import the TechDocs router and add the following to
`AppRoutes`:
TechDocs allows for configuration of the docs storage URL through your
`app-config.yaml` file. We provide two different values to be configured,
`requestUrl` and `storageUrl`. The `requestUrl` is what the reader will request
its data from, and `storageUrl` is where the backend can find the stored
documentation.
```tsx
import { Router as DocsRouter } from '@backstage/plugin-techdocs';
The default storage and request URLs:
// ...
const AppRoutes = () => {
<Routes>
// ... other plugin routes
<Route path="/docs/*" element={<DocsRouter />} />
</Routes>;
};
```
That's it! But now, we need the TechDocs Backend plugin for the frontend to
work.
## Adding TechDocs Backend plugin
Navigate to `packages/backend` of your Backstage app, and install the
`@backstage/plugin-techdocs-backend` package.
```bash
cd my-backstage-app/
cd packages/backend
yarn add @backstage/plugin-techdocs-backend
```
Create a file called `techdocs.ts` inside `packages/backend/src/plugins/` and
add the following
```typescript
import {
createRouter,
DirectoryPreparer,
Preparers,
Generators,
TechdocsGenerator,
CommonGitPreparer,
UrlPreparer,
Publisher,
} from '@backstage/plugin-techdocs-backend';
import { PluginEnvironment } from '../types';
import Docker from 'dockerode';
export default async function createPlugin({
logger,
config,
discovery,
reader,
}: PluginEnvironment) {
// Preparers are responsible for fetching source files for documentation.
const preparers = new Preparers();
const directoryPreparer = new DirectoryPreparer(logger);
preparers.register('dir', directoryPreparer);
const commonGitPreparer = new CommonGitPreparer(logger);
preparers.register('github', commonGitPreparer);
preparers.register('gitlab', commonGitPreparer);
preparers.register('azure/api', commonGitPreparer);
const urlPreparer = new UrlPreparer(reader, logger);
preparers.register('url', urlPreparer);
// Generators are used for generating documentation sites.
const generators = new Generators();
const techdocsGenerator = new TechdocsGenerator(logger, config);
generators.register('techdocs', techdocsGenerator);
// Publishers are used for
// 1. Publishing generated files to storage
// 2. Fetching files from storage and passing them to TechDocs frontend.
const publisher = Publisher.fromConfig(config, logger, discovery);
// Docker client used by the generators.
const dockerClient = new Docker();
return await createRouter({
preparers,
generators,
publisher,
dockerClient,
logger,
config,
discovery,
});
}
```
You may need to install the `dockerode` package. But you may already have it in
your backend since [Scaffolder plugin](../software-templates/index.md) also uses
it.
See [Concepts](concepts.md) and [TechDocs Architecture](architecture.md) to
learn more about how preparers, generators and publishers work.
Final step is to import the techdocs backend plugin in Backstage app backend.
Add the following to your `packages/backend/src/index.ts`:
```typescript
import techdocs from './plugins/techdocs';
// .... main should already be present.
async function main() {
// ... other backend plugin envs
const techdocsEnv = useHotMemoize(module, () => createEnv('techdocs'));
// ... other backend plugin routes
apiRouter.use('/techdocs', await techdocs(techdocsEnv));
}
```
That's it! TechDocs frontend and backend have now been added to your Backstage
app. Now let us tweak some configurations to suit your needs.
## Setting the configuration
**See [TechDocs Configuration Options](configuration.md) for complete
configuration reference.**
### Setting TechDocs URLs
```yaml
techdocs:
@@ -61,22 +167,59 @@ techdocs:
requestUrl: http://localhost:7000/api/techdocs/
```
If you want `techdocs-backend` to manage building and publishing, you want
`requestUrl` to point to the default value (or wherever `techdocs-backend` is
hosted). `storageUrl` should be where your publisher publishes your docs. Using
the default `LocalPublish` that is the default value.
`requestUrl` is used by TechDocs frontend plugin to discover techdocs-backend
endpoints, and the `storageUrl` is another endpoint in `techdocs-backend` which
acts as a middleware between TechDocs and the storage (where the static
generated docs site are stored). These default values should mostly work for
you. These options will soon be optional to set.
If you have a setup where you are not using `techdocs-backend` for managing
building and publishing of your documentation, you want to change the
`requestUrl` to point to your storage. In this case `storageUrl` is not
required.
### Should TechDocs Backend generate docs?
### Disable Docker in Docker situation (Optional)
```yaml
techdocs:
storageUrl: http://localhost:7000/api/techdocs/static/docs
requestUrl: http://localhost:7000/api/techdocs/
builder: 'local'
```
The TechDocs backend plugin runs a docker container with mkdocs installed to
Set `techdocs.builder` to `'local'` if you want your TechDocs Backend to be
responsible for generating documentation sites. If set to `'external'`,
Backstage will assume that the sites are being generated on each entity's CI/CD
pipeline, and are being stored in a storage somewhere.
When `techdocs.builder` is set to `'external'`, TechDocs becomes more or less a
read-only experience where it serves static files from a storage containing all
the generated documentation. Read more in the "Basic" and "Recommended" setup of
TechDocs [here](architecture.md)
### Choosing storage (publisher)
TechDocs needs to know where to store generated documentation sites and where to
fetch the sites from. This is managed by a
[Publisher](./concepts.md#techdocs-publisher). Examples: Google Cloud Storage,
Amazon S3, or local filesystem of Backstage server.
It is okay to use the local filesystem in a "Basic" setup when you are trying
out Backstage for the first time. Using Cloud Storage is documented
[here](./using-cloud-storage.md).
```yaml
techdocs:
storageUrl: http://localhost:7000/api/techdocs/static/docs
requestUrl: http://localhost:7000/api/techdocs/
builder: 'local'
publisher:
type: 'local'
```
### Disabling Docker in Docker situation (Optional)
You can skip this if your `techdocs.builder` is set to `'external'`.
The TechDocs Backend plugin runs a docker container with mkdocs installed to
generate the frontend of the docs from source files (Markdown). If you are
deploying Backstage using Docker, this will mean that your Backstage Docker
container will try to run another Docker container for TechDocs backend.
container will try to run another Docker container for TechDocs Backend.
To avoid this problem, we have a configuration available. You can set a value in
your `app-config.yaml` that tells the techdocs generator if it should run the
@@ -108,23 +251,13 @@ the dependencies. For example, we want `Markdown` version to be
You can also explicitly install `Markdown==3.2.2` after installing all other
Python packages.
## Run Backstage locally
## Running Backstage locally
Change folder to `<backstage-project-root>/packages/backend` and run the
following command:
Start the frontend and the backend app by
[running backstage locally](../../getting-started/running-backstage-locally.md).
```bash
yarn start
```
Open a new command line window. Change directory to your Backstage application
root and run the following command:
```bash
yarn start
```
Open your browser at [http://localhost:3000/docs/](http://localhost:3000/docs/).
Open your browser at [http://localhost:3000/docs/](http://localhost:3000/docs/)
to see all your documentation sites.
## Additional reading
docs/features/techdocs/using-cloud-storage.md
+2 -2
View File
@@ -40,7 +40,7 @@ techdocs:
publisher:
type: 'googleGcs'
googleGcs:
projectId: 'gcp-project-id
projectId: 'gcp-project-id'
```
**3. Service account API key**
@@ -76,7 +76,7 @@ Create a dedicated bucket for TechDocs sites. techdocs-backend will publish
documentation to this bucket. TechDocs will fetch files from here to serve
documentation in Backstage.
Set the name of the bucket to `techdocs.publisher
Set the name of the bucket to `techdocs.publisher.googleGcs.bucketName`.
```yaml
techdocs: