Merge pull request #4335 from backstage/orkohunter/docs-techdocs-migrate-basic-to-recommended
This commit is contained in:
Himanshu Mishra
@@ -12,32 +12,31 @@ out-of-the box experience.
|
||||
<img data-zoomable src="../../assets/techdocs/architecture-basic.drawio.svg" alt="TechDocs Architecture diagram" />
|
||||
|
||||
> Note: See below for our recommended deployment architecture which takes care
|
||||
> of stability, scalability and speed.
|
||||
> of stability, scalability and speed. Also look at the
|
||||
> [HOW TO migrate guide](how-to-guides.md#how-to-migrate-from-techdocs-basic-to-recommended-deployment-approach).
|
||||
|
||||
When you open a TechDocs site in Backstage, the
|
||||
[TechDocs Reader](./concepts.md#techdocs-reader) makes a request to
|
||||
`techdocs-backend` with the entity ID and the path of the current page you are
|
||||
looking at. In response, it receives the static files (HTML, CSS, JSON, etc.) to
|
||||
render on the page in TechDocs/Backstage.
|
||||
`techdocs-backend` plugin with the entity ID and the path of the current page
|
||||
you are looking at. In response, it receives the static files (HTML, CSS, JSON,
|
||||
etc.) to render on the page in TechDocs/Backstage.
|
||||
|
||||
The static files consist of HTML, CSS and Images generated by MkDocs. We remove
|
||||
all the JavaScript before adding them to Backstage for security reasons. And
|
||||
there are some additional techdocs metadata JSON files that TechDocs needs to
|
||||
render a site.
|
||||
there is an additional `techdocs_metadata.json` file that TechDocs needs to
|
||||
render a site. It's important that you use either
|
||||
[techdocs-cli](https://github.com/backstage/techdocs-cli) or
|
||||
[techdocs-container](https://github.com/backstage/techdocs-container) to
|
||||
generate the docs for the expected output.
|
||||
|
||||
The TechDocs Reader then applies a list of "Transformers" (see
|
||||
[Concepts](./concepts.md)) which modify the generated static HTML files for a
|
||||
number of use cases e.g. Remove certain headers, filter out some HTML tags, etc.
|
||||
|
||||
Currently, we use the Backstage server's (or techdocs-backend's) local file
|
||||
system to store the generated files. Publishing to an external storage system
|
||||
(AWS S3, GCS, etc.) is also possible, but has not been implemented yet.
|
||||
|
||||
A word about `UrlReader` vs Git preparer - Right now, we have two ways to fetch
|
||||
files from its source repository for docs site generation. 1. By using Git
|
||||
and 2. By directly using Source control (GitHub, Azure, etc.) APIs. This work is
|
||||
heavily in progress. Please reach out to us on Discord in the #docs-like-code
|
||||
channel to talk about it.
|
||||
system to store the generated files. However, it's ideal to use an external
|
||||
storage system (e.g. AWS S3, GCS or Azure Blob Storage). Read more in
|
||||
[using cloud storage](using-cloud-storage.md).
|
||||
|
||||
## Recommended deployment
|
||||
|
||||
@@ -49,22 +48,16 @@ The key difference in the recommended deployment approach is where the docs are
|
||||
built.
|
||||
|
||||
We assume each entity lives in a repository somewhere (GitHub, GitLab, etc.). We
|
||||
recommend using a CI/CD pipeline with the repository that has a dedicated
|
||||
step/job to generate docs for TechDocs. The generated static files are then
|
||||
stored in a cloud storage solution of your choice.
|
||||
[Track progress here](https://github.com/backstage/backstage/issues/3096).
|
||||
recommend [using a CI/CD pipeline](configuring-ci-cd.md) with the repository
|
||||
that has a dedicated step/job to generate docs for TechDocs. The generated
|
||||
static files are then [stored in a cloud storage](using-cloud-storage.md)
|
||||
solution of your choice.
|
||||
|
||||
Similar to how it is done in the Basic setup, the TechDocs Reader requests
|
||||
`techdocs-backend` plugin for the docs site. `techdocs-backend` then requests
|
||||
your configured storage solution for the necessary files and returns them to
|
||||
TechDocs Reader.
|
||||
|
||||
We will provide instructions, scripts and/or templates (e.g. GitHub Actions) to
|
||||
generate docs in your CI/CD system.
|
||||
[Track progress here.](https://github.com/backstage/backstage/issues/3400) You
|
||||
will be able to use `techdocs-cli` to generate docs and publish the generated
|
||||
docs site files to your cloud storage system.
|
||||
|
||||
Note about caching: We have noticed internally that some storage providers can
|
||||
be quite slow, which is why we are recommending a cache that sits between the
|
||||
TechDocs Reader and the Storage.
|
||||
@@ -97,7 +90,7 @@ permissions to publish the generated docs site files.
|
||||
**Q: Why do you have separate "basic" and "recommended" deployment approaches?**
|
||||
|
||||
A: The basic or out-of-the-box setup is what you get when you create a new app
|
||||
or do a git clone of the Backstage repository. We want the first experience to
|
||||
or do a `git clone` of the Backstage repository. We want the first experience to
|
||||
_just work magically_ so that you can have your first experience with TechDocs
|
||||
which is smooth. However, if you decide to deploy Backstage/TechDocs for
|
||||
production use, the basic setup would work but there are going to be downsides
|
||||
@@ -109,19 +102,21 @@ and we welcome such "Alternative" ideas from the community.
|
||||
**Q: Why don't you recommend techdocs-backend local filesystem to serve static
|
||||
files?**
|
||||
|
||||
A: It would make scaling a Backstage instance harder. Think about the case where
|
||||
we have distributed Backstage deployments. Using a separate file storage system
|
||||
for TechDocs makes it easier to do some operations like delete a docs site and
|
||||
wipe its contents.
|
||||
A: It makes scaling a Backstage instance harder. Think about the case where we
|
||||
have distributed Backstage deployments (e.g. multiple Kubernetes pods of your
|
||||
Backstage app). Using a separate/central file storage system for TechDocs is
|
||||
necessary to make sure sites are persistent when the servers/pods are restarted
|
||||
and avoid duplicating sites per instance. By having an external storage, we are
|
||||
making it easier to do some operations like delete a docs site or wipe its
|
||||
contents.
|
||||
|
||||
**Q: Why aren't docs built on the fly i.e. when users visits a page, generate
|
||||
docs site in real-time?**
|
||||
|
||||
A: Generating the content from Markdown on the fly is not optimal (although that
|
||||
is how the basic out-of-the-box setup is implemented). Storage solutions act as
|
||||
a cache for the generated static content. TechDocs is also currently built on
|
||||
MkDocs which does not allow us to generate docs per-page, so we would have to
|
||||
build all docs for a entity on every request.
|
||||
A: Generating the content from Markdown on the fly is not optimal. Storage
|
||||
solutions act as a cache for the generated static content. TechDocs is also
|
||||
currently built on MkDocs which does not allow us to generate docs per-page, so
|
||||
we would have to build all docs for a entity on every request.
|
||||
|
||||
**Q. Can you use the techdocs plugin without the techdocs-backend plugin?**
|
||||
|
||||
@@ -146,21 +141,5 @@ We currently depend on MkDocs to parse doc sites written in Markdown. And we
|
||||
store the generated static assets and re-use it later to render in Backstage. A
|
||||
better (futuristic) approach will be to directly parse whatever type of source
|
||||
files you have in your docs repository and directly render in Backstage in
|
||||
real-time.
|
||||
|
||||
# Features status
|
||||
|
||||
Status of all the features mentioned above.
|
||||
|
||||
**In place ✅**
|
||||
|
||||
- Basic setup with techdocs-backend file server as storage.
|
||||
- Basic setup with cloud storage solution.
|
||||
- `techdocs-cli` is able to generate docs in CI/CD environment.
|
||||
- `techdocs-cli` is able to publish docs site to any storage.
|
||||
|
||||
**Work in progress 🚧**
|
||||
|
||||
**Not implemented yet ❌**
|
||||
|
||||
- `techdocs-backend` integration with Backstage access control management.
|
||||
real-time. You can read more in this
|
||||
[RFC - Simplifying TechDocs Frontend Architecture](https://github.com/backstage/backstage/issues/3998).
|
||||
|
||||
@@ -5,6 +5,36 @@ sidebar_label: "HOW TO" guides
|
||||
description: TechDocs "HOW TO" guides related to TechDocs
|
||||
---
|
||||
|
||||
## How to migrate from TechDocs Basic to Recommended deployment approach?
|
||||
|
||||
The main difference between TechDocs Basic and Recommended deployment approach
|
||||
is where the docs are generated and stored. In Basic or the out-of-the-box
|
||||
setup, docs are generated and stored at the server running your Backstage
|
||||
instance. But the recommended setup is to generate docs on CI/CD and store the
|
||||
generated sites to an external storage (e.g. AWS S3 or GCS). TechDocs in your
|
||||
Backstage instance should turn into read-only mode. Read more details and the
|
||||
benefits in the [TechDocs Architecture](architecture.md).
|
||||
|
||||
Here are the steps needed to switch from the Basic to Recommended setup -
|
||||
|
||||
### 1. Prepare a cloud storage
|
||||
|
||||
Choose a cloud storage provider like AWS, Google Cloud or Microsoft Azure.
|
||||
Follow the detailed instructions for
|
||||
[using cloud storage](using-cloud-storage.md) in TechDocs.
|
||||
|
||||
### 2. Publish to storage from CI/CD
|
||||
|
||||
Start publishing your TechDocs sites from the CI/CD workflow of each repository
|
||||
containing the source markdown files. Read the detailed instructions for
|
||||
[configuring CI/CD](configuring-ci-cd.md).
|
||||
|
||||
### 3. Switch TechDocs to read-only mode
|
||||
|
||||
In your Backstage instance's `app-config.yaml`, set `techdocs.builder` from
|
||||
`'local'` to `'external'`. By doing this, TechDocs will not try to generate
|
||||
docs. Look at [TechDocs configuration](configuration.md) for reference.
|
||||
|
||||
## How to use URL Reader in TechDocs Prepare step?
|
||||
|
||||
If TechDocs is configured to generate docs, it will first download the
|
||||
|
||||
Reference in New Issue
Block a user