paulononato/backstage
1
0
Fork 0
Code Issues Pull Requests Actions 42 Packages Projects Releases Wiki Activity

Merge pull request #6709 from backstage/techdocs-common/case-sensitive-migration

Browse Source 
[TechDocs] Beta!
This commit is contained in:
Eric Peterson
2021-08-25 13:06:31 +02:00
committed by GitHub
parent 8576195e38 5f8df0c69e
commit 30c0801e96
44 changed files with 1506 additions and 864 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.changeset/catalog-compose-my-techdocs.md
+10
View File
@@ -0,0 +1,10 @@
---
'@backstage/plugin-catalog': patch
---
**NOTE**: The entity `<AboutCard />` now uses an external route ref to link to
TechDocs sites. This external route must now be bound in order for the "View
TechDocs" link to continue working. See the [create-app changelog][cacl] for
details.
[cacl]: https://github.com/backstage/backstage/blob/master/packages/create-app/CHANGELOG.md
.changeset/create-incerteza-desafinado-tomara.md
+28
View File
@@ -0,0 +1,28 @@
---
'@backstage/create-app': patch
---
Wire up TechDocs, which now relies on the composability API for routing.
First, ensure you've mounted `<TechDocsReaderPage />`. If you already updated
to use the composable `<TechDocsIndexPage />` (see below), no action is
necessary. Otherwise, update your `App.tsx` so that `<TechDocsReaderPage />` is
mounted:
```diff
<Route path="/docs" element={<TechdocsPage />} />
+ <Route
+ path="/docs/:namespace/:kind/:name/*"
+ element={<TechDocsReaderPage />}
+ />
```
Next, ensure links from the Catalog Entity Page to its TechDocs site are bound:
```diff
bindRoutes({ bind }) {
bind(catalogPlugin.externalRoutes, {
createComponent: scaffolderPlugin.routes.root,
+ viewTechDoc: techdocsPlugin.routes.docRoot,
});
```
.changeset/techdocs-all-done.md
+20
View File
@@ -0,0 +1,20 @@
---
'@backstage/plugin-techdocs': minor
'@backstage/plugin-techdocs-backend': minor
'@backstage/techdocs-common': minor
---
TechDocs sites can now be accessed using paths containing entity triplets of
any case (e.g. `/docs/namespace/KIND/name` or `/docs/namespace/kind/name`).
If you do not use an external storage provider for serving TechDocs, this is a
transparent change and no action is required from you.
If you _do_ use an external storage provider for serving TechDocs (one of\* GCS,
AWS S3, or Azure Blob Storage), you must run a migration command against your
storage provider before updating.
[A migration guide is available here](https://backstage.io/docs/features/techdocs/how-to-guides#how-to-migrate-from-techdocs-alpha-to-beta).
- (\*) We're seeking help from the community to bring OpenStack Swift support
[to feature parity](https://github.com/backstage/backstage/issues/6763) with the above.
.changeset/techdocs-jobim-gilberto-moraes.md
+9
View File
@@ -0,0 +1,9 @@
---
'@backstage/plugin-techdocs': minor
---
The TechDocs plugin has completed the migration to the Composability API. In
order to update to this version, please ensure you've made all necessary
changes to your `App.tsx` file as outlined in the [create-app changelog][cacl].
[cacl]: https://github.com/backstage/backstage/blob/master/packages/create-app/CHANGELOG.md
docs/features/techdocs/README.md
+1 -12
View File
@@ -54,23 +54,12 @@ providers are used.
| Google Cloud Storage (GCS) | Yes ✅ |
| Amazon Web Services (AWS) S3 | Yes ✅ |
| Azure Blob Storage | Yes ✅ |
| OpenStack Swift | Yes ✅ |
| OpenStack Swift | Community ✅ |
[Reach out to us](#feedback) if you want to request more platforms.
## Project roadmap
### **Ongoing work 🚧**
**Beta release** -
[Milestone](https://github.com/backstage/backstage/milestone/29)
- It should be possible and easy to use TechDocs in most environments across
organizations.
- Minimal bugs, better error handling and scalable backend and frontend.
- Documentation Search
- TechDocs Homepage with basic features
### **Future work 🔮**
**General Availability (GA) release** -
docs/features/techdocs/configuration.md
+8
View File
@@ -113,6 +113,14 @@ techdocs:
# https://docs.microsoft.com/en-us/azure/storage/common/storage-auth?toc=/azure/storage/blobs/toc.json
accountKey: ${TECHDOCS_AZURE_BLOB_STORAGE_ACCOUNT_KEY}
# (Optional and not recommended) Prior to version [0.x.y] of TechDocs, docs
# sites could only be accessed over paths with case-sensitive entity triplets
# e.g. (namespace/Kind/name). If you are upgrading from an older version of
# TechDocs and are unable to perform the necessary migration of files in your
# external storage, you can set this value to `true` to temporarily revert to
# the old, case-sensitive entity triplet behavior.
legacyUseCaseSensitiveTripletPaths: false
# (Optional and Legacy) TechDocs makes API calls to techdocs-backend using this URL. e.g. get docs of an entity, get metadata, etc.
# You don't have to specify this anymore.
docs/features/techdocs/getting-started.md
+12 -2
View File
@@ -33,14 +33,24 @@ In `packages/app/src/App.tsx`, import `TechDocsPage` and add the following to
`FlatRoutes`:
```tsx
import { TechDocsPage } from '@backstage/plugin-techdocs';
import {
DefaultTechDocsHome,
TechDocsIndexPage,
TechDocsReaderPage,
} from '@backstage/plugin-techdocs';
// ...
const AppRoutes = () => {
<FlatRoutes>
// ... other plugin routes
<Route path="/docs" element={<TechdocsPage />} />
<Route path="/docs" element={<TechDocsIndexPage />}>
<DefaultTechDocsHome />
</Route>
<Route
path="/docs/:namespace/:kind/:name/*"
element={<TechDocsReaderPage />}
/>
</FlatRoutes>;
};
```
docs/features/techdocs/how-to-guides.md
+96 -69
View File
@@ -104,83 +104,110 @@ the repository. The archive does not have any git history attached to it. Also
it is a compressed file. Hence the file size is significantly smaller than how
much data git clone has to transfer.
## How to use a custom TechDocs home page?
## How to customize the TechDocs home page?
### 1st way: TechDocsCustomHome with a custom configuration
TechDocs uses a composability pattern similar to the Search and Catalog plugins
in Backstage. While a default table experience, similar to the one provided by
the Catalog plugin, is made available for ease-of-use, it's possible for you to
provide a completely custom experience, tailored to the needs of your
organization.
As an example, in your main App.tsx:
This is done in your `app` package. By default, you might see something like
this in your `App.tsx`:
```tsx
import {
TechDocsCustomHome,
PanelType,
TechDocsReaderPage,
} from '@backstage/plugin-techdocs';
import { Entity } from '@backstage/catalog-model';
const tabsConfig = [
{
label: 'Custom Tab',
panels: [
{
title: 'Custom Documents Cards 1',
description:
'Explore your internal technical ecosystem through documentation.',
panelType: 'DocsCardGrid' as PanelType,
// optional, is applied to a container of the panel (excludes header of panel)
panelCSS: { maxHeight: '400px', overflow:'auto' },
filterPredicate: (entity: Entity) => !!entity.metadata.annotations?.['customCardAnnotationOne'];
},
{
title: 'Custom Documents Cards 2',
description:
'Explore your internal technical ecosystem through documentation.',
panelType: 'DocsCardGrid' as PanelType,
panelCSS: { maxHeight: '400px', overflow:'auto' },
filterPredicate: (entity: Entity) => !!entity.metadata.annotations?.['customCardAnnotationTwo'];
},
],
},
{
label: 'Overview',
panels: [
{
title: 'Overview',
description:
'Explore your internal technical ecosystem through documentation.',
panelType: 'DocsTable' as PanelType,
filterPredicate: () => true,
},
],
},
];
const routes = (
const AppRoutes = () => {
<FlatRoutes>
<Route
path="/docs"
element={<TechDocsCustomHome tabsConfig={tabsConfig} />}
/>
<Route
path="/docs/:namespace/:kind/:name/*"
element={<TechDocsReaderPage />}
/>
</FlatRoutes>
<Route path="/docs" element={<TechDocsIndexPage />}>
<DefaultTechDocsHome />
</Route>
</FlatRoutes>;
};
```
An example of tabsConfig that corresponds to the default documentation home page
can be found at `plugins/techdocs/src/home/components/TechDocsHome.tsx`.
But you can replace `<DefaultTechDocsHome />` with any React component, which
will be rendered in its place. Most likely, you would want to create and
maintain such a component in a new directory at
`packages/app/src/components/techdocs`, and import and use it in `App.tsx`:
Currently `panelType` has DocsCardGrid and DocsTable available. We currently
recommend that DocsCardGrid can be optionally vertically stacked by setting a
maxHeight using `panelCSS`, and DocsTable to be in a tab by itself.
```tsx
import { CustomTechDocsHome } from './components/techdocs/CustomTechDocsHome';
// ...
const AppRoutes = () => {
<FlatRoutes>
<Route path="/docs" element={<TechDocsIndexPage />}>
<CustomTechDocsHome />
</Route>
</FlatRoutes>;
};
```
### 2nd way: Custom home page plugin
## How to migrate from TechDocs Alpha to Beta
A custom home page plugin can be built that uses the components extensions
DocsCardGrid and DocsTable, exported from @backstage/techdocs. They both take a
array of documentation entities ( i.e.have a 'backstage.io/techdocs-ref'
annotation ) as an 'entities' attribute.
> This guide only applies to the "recommended" TechDocs deployment method (where
> an external storage provider and external CI/CD is used). If you use the
> "basic" or "out-of-the-box" setup, you can stop here! No action needed.
For a reference to the React structure of the default home page, please refer to
`plugins/techdocs/src/home/components/TechDocsCustomHome.tsx`.
The beta version of TechDocs (v0.x.y) made a breaking change to the way TechDocs
content was accessed and stored, allowing pages to be accessed with
case-insensitive entity triplet paths (e.g. `/docs/namespace/kind/name` whereas
in prior versions, they could only be accessed at `/docs/namespace/Kind/name`).
In order to enable this change, documentation has to be stored in an external
storage provider using an object key whose entity triplet is lower-cased.
New installations of TechDocs since the beta version will work fine with no
action, but for those who were running TechDocs prior to this version, a
migration will need to be performed so that all existing content in your storage
bucket matches this lower-case entity triplet expectation.
1. **Ensure you have the right permissions on your storage provider**: In order
to migrate files in your storage provider, the `techdocs-cli` needs to be
able to read/copy/rename/move/delete files. The exact instructions vary by
storage provider, but check the [using cloud storage][using-cloud-storage]
page for details.
2. **Run a non-destructive migration of files**: Ensure you have the latest
version of `techdocs-cli` installed. Then run the following command, using
the details relevant for your provider / configuration. This will copy all
files from, e.g. `namespace/Kind/name/index.html` to
`namespace/kind/name/index.html`, without removing the original files.
```sh
techdocs-cli migrate --publisher-type <awsS3|googleGcs|azureBlobStorage> --storage-name <bucket/container name> --verbose
```
3. **Deploy the updated versions of the TechDocs plugins**: Once the migration
above has been run, you can deploy the beta versions of the TechDocs backend
and frontend plugins to your Backstage instance.
4. **Verify that your TechDocs sites are still loading/accessible**: Try
accessing a TechDocs site using different entity-triplet case variants, e.g.
`/docs/namespace/KIND/name` or `/docs/namespace/kind/name`. Your TechDocs
site should load regardless of the URL path casing you use.
5. **Clean up the old objects from storage**: Once you've verified that your
TechDocs site is accessible, you can clean up your storage bucket by
re-running the `migrate` command on the TechDocs CLI, but with an additional
`removeOriginal` flag passed:
```sh
techdocs-cli migrate --publisher-type <awsS3|googleGcs|azureBlobStorage> --storage-name <bucket/container name> --removeOriginal --verbose
```
6. **Update your CI/CD pipelines to use the beta version of the TechDocs CLI**:
Finally, you can update all of your CI/CD pipelines to use at least v0.x.y of
the TechDocs CLI, ensuring that all sites are published to the new,
lower-cased entity triplet paths going forward.
If you encounter problems running this migration, please [report the
issue][beta-migrate-bug]. You can temporarily revert to pre-beta storage
expectations with a configuration change:
```yaml
techdocs:
legacyUseCaseSensitiveTripletPaths: true
```
[beta-migrate-bug]:
https://github.com/backstage/backstage/issues/new?assignees=&labels=bug&template=bug_template.md&title=[TechDocs]%20Unable%20to%20run%20beta%20migration
[using-cloud-storage]: ./using-cloud-storage.md
packages/app/src/App.tsx
+2
View File
@@ -52,6 +52,7 @@ import { TechRadarPage } from '@backstage/plugin-tech-radar';
import {
DefaultTechDocsHome,
TechDocsIndexPage,
techdocsPlugin,
TechDocsReaderPage,
} from '@backstage/plugin-techdocs';
import { UserSettingsPage } from '@backstage/plugin-user-settings';
@@ -92,6 +93,7 @@ const app = createApp({
bindRoutes({ bind }) {
bind(catalogPlugin.externalRoutes, {
createComponent: scaffolderPlugin.routes.root,
viewTechDoc: techdocsPlugin.routes.docRoot,
});
bind(apiDocsPlugin.externalRoutes, {
createComponent: scaffolderPlugin.routes.root,
packages/create-app/templates/default-app/packages/app/src/App.tsx
+2
View File
@@ -16,6 +16,7 @@ import { TechRadarPage } from '@backstage/plugin-tech-radar';
import {
DefaultTechDocsHome,
TechDocsIndexPage,
techdocsPlugin,
TechDocsReaderPage,
} from '@backstage/plugin-techdocs';
import { UserSettingsPage } from '@backstage/plugin-user-settings';
@@ -31,6 +32,7 @@ const app = createApp({
bindRoutes({ bind }) {
bind(catalogPlugin.externalRoutes, {
createComponent: scaffolderPlugin.routes.root,
viewTechDoc: techdocsPlugin.routes.docRoot,
});
bind(apiDocsPlugin.externalRoutes, {
createComponent: scaffolderPlugin.routes.root,
packages/techdocs-common/__mocks__/@azure/storage-blob.ts
+13 -42
View File
@@ -13,34 +13,13 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import type {
import {
BlobUploadCommonResponse,
ContainerGetPropertiesResponse,
} from '@azure/storage-blob';
import { EventEmitter } from 'events';
import fs from 'fs-extra';
import os from 'os';
import path from 'path';
const rootDir = os.platform() === 'win32' ? 'C:\\rootDir' : '/rootDir';
/**
* @param sourceFile Relative path to entity root dir. Contains either / or \ as file separator
* depending upon the OS.
*/
const checkFileExists = async (sourceFile: string): Promise<boolean> => {
// sourceFile will always have / as file separator irrespective of OS since Azure expects /.
// Normalize sourceFile to OS specific path before checking if file exists.
const relativeFilePath = sourceFile.split(path.posix.sep).join(path.sep);
const filePath = path.join(rootDir, sourceFile);
try {
await fs.access(filePath, fs.constants.F_OK);
return true;
} catch (err) {
return false;
}
};
const storage = global.storageFilesMock;
export class BlockBlobClient {
private readonly blobName;
@@ -50,9 +29,7 @@ export class BlockBlobClient {
}
uploadFile(source: string): Promise<BlobUploadCommonResponse> {
if (!fs.existsSync(source)) {
return Promise.reject(new Error(`The file ${source} does not exist`));
}
storage.writeFile(this.blobName, source);
return Promise.resolve({
_response: {
request: {
@@ -65,20 +42,19 @@ export class BlockBlobClient {
}
exists() {
return checkFileExists(this.blobName);
return storage.fileExists(this.blobName);
}
download() {
const filePath = path.join(rootDir, this.blobName);
const emitter = new EventEmitter();
setTimeout(() => {
if (fs.existsSync(filePath)) {
emitter.emit('data', fs.readFileSync(filePath));
if (storage.fileExists(this.blobName)) {
emitter.emit('data', storage.readFile(this.blobName));
emitter.emit('end');
} else {
emitter.emit(
'error',
new Error(`The file ${filePath} does not exist !`),
new Error(`The file ${this.blobName} does not exist!`),
);
}
}, 0);
@@ -89,7 +65,7 @@ export class BlockBlobClient {
}
class BlockBlobClientFailUpload extends BlockBlobClient {
uploadFile(source: string): Promise<BlobUploadCommonResponse> {
uploadFile(): Promise<BlobUploadCommonResponse> {
return Promise.resolve({
_response: {
request: {
@@ -133,12 +109,6 @@ class ContainerClientIterator {
}
export class ContainerClient {
private readonly containerName;
constructor(containerName: string) {
this.containerName = containerName;
}
getProperties(): Promise<ContainerGetPropertiesResponse> {
return Promise.resolve({
_response: {
@@ -197,18 +167,19 @@ export class BlobServiceClient {
private readonly credential;
constructor(url: string, credential?: StorageSharedKeyCredential) {
storage.emptyFiles();
this.url = url;
this.credential = credential;
}
getContainerClient(containerName: string) {
if (containerName === 'bad_container') {
return new ContainerClientFailGetProperties(containerName);
return new ContainerClientFailGetProperties();
}
if (this.credential.accountName === 'failupload') {
return new ContainerClientFailUpload(containerName);
if (this.credential.accountName === 'bad_account_credentials') {
return new ContainerClientFailUpload();
}
return new ContainerClient(containerName);
return new ContainerClient();
}
}
packages/techdocs-common/__mocks__/@google-cloud/storage.ts
+15 -43
View File
@@ -14,42 +14,19 @@
* limitations under the License.
*/
import { Readable } from 'stream';
import fs from 'fs-extra';
import os from 'os';
import path from 'path';
type storageOptions = {
keyFilename?: string;
};
const rootDir = os.platform() === 'win32' ? 'C:\\rootDir' : '/rootDir';
/**
* @param sourceFile Absolute path. Contains either / or \ as file separator depending upon the OS.
*/
const checkFileExists = async (sourceFile: string): Promise<boolean> => {
// sourceFile will always have / as file separator irrespective of OS since GCS expects /.
// Normalize sourceFile to OS specific path before checking if file exists.
const filePath = sourceFile.split(path.posix.sep).join(path.sep);
try {
await fs.access(filePath, fs.constants.F_OK);
return true;
} catch (err) {
return false;
}
};
const storage = global.storageFilesMock;
class GCSFile {
private readonly localFilePath: string;
private readonly path: string;
constructor(private readonly destinationFilePath: string) {
this.destinationFilePath = destinationFilePath;
this.localFilePath = path.join(rootDir, this.destinationFilePath);
constructor(path: string) {
this.path = path;
}
exists() {
return new Promise(async (resolve, reject) => {
if (await checkFileExists(this.localFilePath)) {
if (storage.fileExists(this.path)) {
resolve([true]);
} else {
reject();
@@ -62,19 +39,20 @@ class GCSFile {
readable._read = () => {};
process.nextTick(() => {
if (fs.existsSync(this.localFilePath)) {
if (storage.fileExists(this.path)) {
if (readable.eventNames().includes('pipe')) {
readable.emit('pipe');
}
readable.emit('data', fs.readFileSync(this.localFilePath));
readable.emit('data', storage.readFile(this.path));
readable.emit('end');
} else {
readable.emit(
'error',
new Error(`The file ${this.localFilePath} does not exist !`),
new Error(`The file ${this.path} does not exist!`),
);
}
});
return readable;
}
@@ -91,20 +69,16 @@ class Bucket {
}
async getMetadata() {
if (this.bucketName === 'errorBucket') {
if (this.bucketName === 'bad_bucket_name') {
throw Error('Bucket does not exist');
}
return '';
}
upload(source: string, { destination }) {
return new Promise(async (resolve, reject) => {
if (await checkFileExists(source)) {
resolve({ source, destination });
} else {
reject(`Source file ${source} does not exist.`);
}
return new Promise(async resolve => {
storage.writeFile(destination, source);
resolve(null);
});
}
@@ -134,10 +108,8 @@ class Bucket {
}
export class Storage {
private readonly keyFilename;
constructor(options: storageOptions) {
this.keyFilename = options.keyFilename;
constructor() {
storage.emptyFiles();
}
bucket(bucketName) {
packages/techdocs-common/__mocks__/aws-sdk.ts
+19 -44
View File
@@ -13,45 +13,22 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import type { S3 as S3Types } from 'aws-sdk';
import { EventEmitter } from 'events';
import fs from 'fs-extra';
import os from 'os';
import path from 'path';
import { ReadStream } from 'fs';
export { Credentials } from 'aws-sdk';
const rootDir = os.platform() === 'win32' ? 'C:\\rootDir' : '/rootDir';
/**
* @param Key Relative path to entity root dir. Contains either / or \ as file separator
* depending upon the OS.
*/
const checkFileExists = async (Key: string): Promise<boolean> => {
// Key will always have / as file separator irrespective of OS since S3 expects /.
// Normalize Key to OS specific path before checking if file exists.
const relativeFilePath = Key.split(path.posix.sep).join(path.sep);
const filePath = path.join(rootDir, Key);
try {
await fs.access(filePath, fs.constants.F_OK);
return true;
} catch (err) {
return false;
}
};
const storage = global.storageFilesMock;
export class S3 {
private readonly options;
constructor(options: S3Types.ClientConfiguration) {
this.options = options;
constructor() {
storage.emptyFiles();
}
headObject({ Key }: { Key: string }) {
return {
promise: async () => {
if (!(await checkFileExists(Key))) {
if (!storage.fileExists(Key)) {
throw new Error('File does not exist');
}
},
@@ -59,20 +36,16 @@ export class S3 {
}
getObject({ Key }: { Key: string }) {
const filePath = path.join(rootDir, Key);
return {
promise: () => checkFileExists(filePath),
promise: async () => storage.fileExists(Key),
createReadStream: () => {
const emitter = new EventEmitter();
process.nextTick(() => {
if (fs.existsSync(filePath)) {
emitter.emit('data', Buffer.from(fs.readFileSync(filePath)));
if (storage.fileExists(Key)) {
emitter.emit('data', Buffer.from(storage.readFile(Key)));
emitter.emit('end');
} else {
emitter.emit(
'error',
new Error(`The file ${filePath} does not exist !`),
);
emitter.emit('error', new Error(`The file ${Key} does not exist!`));
}
});
return emitter;
@@ -86,21 +59,23 @@ export class S3 {
if (Bucket === 'errorBucket') {
throw new Error('Bucket does not exist');
}
return {};
},
};
}
upload({ Key }: { Key: string }) {
upload({ Key, Body }: { Key: string; Body: ReadStream }) {
return {
promise: () =>
new Promise(async (resolve, reject) => {
if (!(await checkFileExists(Key))) {
reject(`The file ${Key} does not exist`);
} else {
resolve('');
}
new Promise(async resolve => {
const chunks = [];
Body.on('data', chunk => {
chunks.push(chunk);
});
Body.once('end', () => {
storage.writeFile(Key, Buffer.concat(chunks));
resolve(null);
});
}),
};
}
packages/techdocs-common/__mocks__/globals.d.ts
Vendored
+22
View File
@@ -0,0 +1,22 @@
/*
* Copyright 2020 The Backstage Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
declare module NodeJS {
interface Global {
rootDir: string;
storageFilesMock: IStorageFilesMock;
}
}
packages/techdocs-common/__mocks__/types.d.ts
Vendored
+24
View File
@@ -0,0 +1,24 @@
/*
* Copyright 2020 The Backstage Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
interface IStorageFilesMock {
emptyFiles(): void;
fileExists(targetPath: string): boolean;
readFile(targetPath: string): Buffer;
writeFile(targetPath: string, sourcePath: string): void;
writeFile(targetPath: string, sourceBuffer: Buffer): void;
writeFile(targetPath: string, source: string | Buffer): void;
}
packages/techdocs-common/src/globals.d.ts
Vendored
+22
View File
@@ -0,0 +1,22 @@
/*
* Copyright 2020 The Backstage Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
declare module NodeJS {
interface Global {
rootDir: string;
storageFilesMock: IStorageFilesMock;
}
}
packages/techdocs-common/src/setupTests.ts
+60
View File
@@ -0,0 +1,60 @@
/*
* Copyright 2020 The Backstage Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import os from 'os';
import path from 'path';
import fs from 'fs-extra';
const rootDir: string = os.platform() === 'win32' ? 'C:\\rootDir' : '/rootDir';
const encoding = 'utf8';
class StorageFilesMock implements IStorageFilesMock {
private files: Record<string, string>;
constructor() {
this.files = {};
}
public emptyFiles(): void {
this.files = {};
}
public fileExists(targetPath: string): boolean {
const filePath = path.join(rootDir, targetPath);
const posixPath = filePath.split(path.posix.sep).join(path.sep);
return this.files[posixPath] !== undefined;
}
public readFile(targetPath: string): Buffer {
const filePath = path.join(rootDir, targetPath);
return Buffer.from(this.files[filePath] ?? '', encoding);
}
public writeFile(targetPath: string, sourcePath: string): void;
public writeFile(targetPath: string, sourceBuffer: Buffer): void;
public writeFile(targetPath: string, source: string | Buffer): void {
const filePath = path.join(rootDir, targetPath);
if (typeof source === 'string') {
this.files[filePath] = fs.readFileSync(source).toString(encoding);
} else {
this.files[filePath] = source.toString(encoding);
}
}
}
global.rootDir = rootDir;
global.storageFilesMock = new StorageFilesMock();
packages/techdocs-common/src/stages/publish/awsS3.test.ts
+173 -177
View File
@@ -15,43 +15,18 @@
*/
import { getVoidLogger } from '@backstage/backend-common';
import {
Entity,
EntityName,
ENTITY_DEFAULT_NAMESPACE,
} from '@backstage/catalog-model';
import { Entity, ENTITY_DEFAULT_NAMESPACE } from '@backstage/catalog-model';
import { ConfigReader } from '@backstage/config';
import express from 'express';
import request from 'supertest';
import mockFs from 'mock-fs';
import os from 'os';
import path from 'path';
import fs from 'fs-extra';
import { AwsS3Publish } from './awsS3';
import { PublisherBase, TechDocsMetadata } from './types';
// NOTE: /packages/techdocs-common/__mocks__ is being used to mock aws-sdk client library
const createMockEntity = (annotations = {}): Entity => {
return {
apiVersion: 'version',
kind: 'TestKind',
metadata: {
name: 'test-component-name',
namespace: 'test-namespace',
annotations: {
...annotations,
},
},
};
};
const createMockEntityName = (): EntityName => ({
kind: 'TestKind',
name: 'test-component-name',
namespace: 'test-namespace',
});
const rootDir = os.platform() === 'win32' ? 'C:\\rootDir' : '/rootDir';
const rootDir = global.rootDir;
const getEntityRootDir = (entity: Entity) => {
const {
@@ -66,7 +41,13 @@ const logger = getVoidLogger();
const loggerInfoSpy = jest.spyOn(logger, 'info');
const loggerErrorSpy = jest.spyOn(logger, 'error');
const createPublisherMock = (bucketName: string) => {
const createPublisherFromConfig = ({
bucketName = 'bucketName',
legacyUseCaseSensitiveTripletPaths = false,
}: {
bucketName?: string;
legacyUseCaseSensitiveTripletPaths?: boolean;
} = {}) => {
const mockConfig = new ConfigReader({
techdocs: {
requestUrl: 'http://localhost:7000',
@@ -80,71 +61,96 @@ const createPublisherMock = (bucketName: string) => {
bucketName,
},
},
legacyUseCaseSensitiveTripletPaths,
},
});
return AwsS3Publish.fromConfig(mockConfig, logger);
};
let publisher: PublisherBase;
beforeEach(() => {
loggerInfoSpy.mockClear();
loggerErrorSpy.mockClear();
mockFs.restore();
publisher = createPublisherMock('bucketName');
});
describe('AwsS3Publish', () => {
const entity = {
apiVersion: '1',
kind: 'Component',
metadata: {
name: 'backstage',
namespace: 'default',
annotations: {},
},
};
const entityName = {
kind: 'Component',
name: 'backstage',
namespace: 'default',
};
const techdocsMetadata = {
site_name: 'backstage',
site_description: 'site_content',
etag: 'etag',
};
const directory = getEntityRootDir(entity);
const files = {
'index.html': '',
'404.html': '',
'techdocs_metadata.json': JSON.stringify(techdocsMetadata),
assets: {
'main.css': '',
},
html: {
'unsafe.html': '<html></html>',
},
img: {
'with spaces.png': 'found it',
'unsafe.svg': '<svg></svg>',
},
'some folder': {
'also with spaces.js': 'found it too',
},
};
beforeEach(() => {
mockFs({
[directory]: files,
});
});
afterEach(() => {
mockFs.restore();
});
describe('getReadiness', () => {
it('should validate correct config', async () => {
const publisher = createPublisherFromConfig();
expect(await publisher.getReadiness()).toEqual({
isAvailable: true,
});
});
it('should reject incorrect config', async () => {
const errorPublisher = createPublisherMock('errorBucket');
expect(await errorPublisher.getReadiness()).toEqual({
const publisher = createPublisherFromConfig({
bucketName: 'errorBucket',
});
expect(await publisher.getReadiness()).toEqual({
isAvailable: false,
});
});
});
describe('publish', () => {
beforeEach(() => {
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
mockFs({
[entityRootDir]: {
'index.html': '',
'404.html': '',
assets: {
'main.css': '',
},
attachments: {
'image.png': Buffer.from([2, 3, 5, 3, 5, 3, 5, 9, 1]),
},
},
});
});
afterEach(() => {
mockFs.restore();
});
it('should publish a directory', async () => {
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const publisher = createPublisherFromConfig();
expect(await publisher.publish({ entity, directory })).toBeUndefined();
});
expect(
await publisher.publish({
entity,
directory: entityRootDir,
}),
).toBeUndefined();
it('should publish a directory as well when legacy casing is used', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
expect(await publisher.publish({ entity, directory })).toBeUndefined();
});
it('should fail to publish a directory', async () => {
@@ -156,36 +162,27 @@ describe('AwsS3Publish', () => {
'generatedDirectory',
);
const entity = createMockEntity();
await expect(
publisher.publish({
entity,
directory: wrongPathToGeneratedDirectory,
}),
).rejects.toThrowError();
const publisher = createPublisherFromConfig();
const fails = publisher.publish({
entity,
directory: wrongPathToGeneratedDirectory,
});
// Can not do exact error message match due to mockFs adding unexpected characters in the path when throwing the error
// Issue reported https://github.com/tschaub/mock-fs/issues/118
await expect(fails).rejects.toMatchObject({
message: expect.stringContaining(
'Unable to upload file(s) to AWS S3. Error: Failed to read template directory: ENOENT, no such file or directory',
),
});
await expect(fails).rejects.toMatchObject({
message: expect.stringContaining(wrongPathToGeneratedDirectory),
});
});
it('should delete stale files after upload', async () => {
const entity = createMockEntity();
const directory = getEntityRootDir(entity);
const bucketName = 'delete_stale_files_success';
publisher = createPublisherMock(bucketName);
const publisher = createPublisherFromConfig({ bucketName: bucketName });
await publisher.publish({ entity, directory });
expect(loggerInfoSpy).toHaveBeenLastCalledWith(
`Successfully deleted stale files for Entity ${entity.metadata.name}. Total number of files: 1`,
@@ -193,10 +190,8 @@ describe('AwsS3Publish', () => {
});
it('should log error when the stale files deletion fails', async () => {
const entity = createMockEntity();
const directory = getEntityRootDir(entity);
const bucketName = 'delete_stale_files_error';
publisher = createPublisherMock(bucketName);
const publisher = createPublisherFromConfig({ bucketName: bucketName });
await publisher.publish({ entity, directory });
expect(loggerErrorSpy).toHaveBeenLastCalledWith(
'Unable to delete file(s) from AWS S3. Error: Message',
@@ -206,142 +201,143 @@ describe('AwsS3Publish', () => {
describe('hasDocsBeenGenerated', () => {
it('should return true if docs has been generated', async () => {
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
mockFs({
[entityRootDir]: {
'index.html': 'file-content',
},
});
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
expect(await publisher.hasDocsBeenGenerated(entity)).toBe(true);
});
it('should return true if docs has been generated even if the legacy case is enabled', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
await publisher.publish({ entity, directory });
expect(await publisher.hasDocsBeenGenerated(entity)).toBe(true);
mockFs.restore();
});
it('should return false if docs has not been generated', async () => {
const entity = createMockEntity();
expect(await publisher.hasDocsBeenGenerated(entity)).toBe(false);
const publisher = createPublisherFromConfig();
expect(
await publisher.hasDocsBeenGenerated({
kind: 'entity',
metadata: {
namespace: 'invalid',
name: 'triplet',
},
} as Entity),
).toBe(false);
});
});
describe('fetchTechDocsMetadata', () => {
it('should return tech docs metadata', async () => {
const entityNameMock = createMockEntityName();
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
expect(await publisher.fetchTechDocsMetadata(entityName)).toStrictEqual(
techdocsMetadata,
);
});
mockFs({
[entityRootDir]: {
'techdocs_metadata.json':
'{"site_name": "backstage", "site_description": "site_content", "etag": "etag"}',
},
it('should return tech docs metadata even if the legacy case is enabled', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
const expectedMetadata: TechDocsMetadata = {
site_name: 'backstage',
site_description: 'site_content',
etag: 'etag',
};
expect(
await publisher.fetchTechDocsMetadata(entityNameMock),
).toStrictEqual(expectedMetadata);
mockFs.restore();
await publisher.publish({ entity, directory });
expect(await publisher.fetchTechDocsMetadata(entityName)).toStrictEqual(
techdocsMetadata,
);
});
it('should return tech docs metadata when json encoded with single quotes', async () => {
const entityNameMock = createMockEntityName();
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const techdocsMetadataPath = path.join(
directory,
'techdocs_metadata.json',
);
const techdocsMetadataContent = files['techdocs_metadata.json'];
mockFs({
[entityRootDir]: {
'techdocs_metadata.json': `{'site_name': 'backstage', 'site_description': 'site_content', 'etag': 'etag'}`,
},
});
fs.writeFileSync(
techdocsMetadataPath,
techdocsMetadataContent.replace(/"/g, "'"),
);
const expectedMetadata: TechDocsMetadata = {
site_name: 'backstage',
site_description: 'site_content',
etag: 'etag',
};
expect(
await publisher.fetchTechDocsMetadata(entityNameMock),
).toStrictEqual(expectedMetadata);
mockFs.restore();
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
expect(await publisher.fetchTechDocsMetadata(entityName)).toStrictEqual(
techdocsMetadata,
);
fs.writeFileSync(techdocsMetadataPath, techdocsMetadataContent);
});
it('should return an error if the techdocs_metadata.json file is not present', async () => {
const entityNameMock = createMockEntityName();
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const publisher = createPublisherFromConfig();
const fails = publisher.fetchTechDocsMetadata(entityNameMock);
const invalidEntityName = {
namespace: 'invalid',
kind: 'triplet',
name: 'path',
};
const techDocsMetadaFilePath = path.join(
...Object.values(invalidEntityName),
'techdocs_metadata.json',
);
const fails = publisher.fetchTechDocsMetadata(invalidEntityName);
const errorPath = path.join(entityRootDir, 'techdocs_metadata.json');
await expect(fails).rejects.toMatchObject({
message: `TechDocs metadata fetch failed, The file ${errorPath} does not exist !`,
message: `TechDocs metadata fetch failed, The file ${techDocsMetadaFilePath} does not exist!`,
});
});
});
describe('docsRouter', () => {
const entityTripletPath = `${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}`;
let app: express.Express;
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
beforeEach(() => {
beforeEach(async () => {
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
app = express().use(publisher.docsRouter());
mockFs.restore();
mockFs({
[entityRootDir]: {
html: {
'unsafe.html': '<html></html>',
},
img: {
'with spaces.png': 'found it',
'unsafe.svg': '<svg></svg>',
},
'some folder': {
'also with spaces.js': 'found it too',
},
},
});
});
afterEach(() => {
mockFs.restore();
});
it('should pass expected object path to bucket', async () => {
const {
kind,
metadata: { namespace, name },
} = entity;
// Ensures leading slash is trimmed and encoded path is decoded.
const pngResponse = await request(app).get(
`/${namespace}/${kind}/${name}/img/with%20spaces.png`,
`/${entityTripletPath}/img/with%20spaces.png`,
);
expect(Buffer.from(pngResponse.body).toString('utf8')).toEqual(
'found it',
);
const jsResponse = await request(app).get(
`/${namespace}/${kind}/${name}/some%20folder/also%20with%20spaces.js`,
`/${entityTripletPath}/some%20folder/also%20with%20spaces.js`,
);
expect(jsResponse.text).toEqual('found it too');
});
it('should pass expected object path to bucket even if the legacy case is enabled', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
await publisher.publish({ entity, directory });
app = express().use(publisher.docsRouter());
// Ensures leading slash is trimmed and encoded path is decoded.
const pngResponse = await request(app).get(
`/${entityTripletPath}/img/with%20spaces.png`,
);
expect(Buffer.from(pngResponse.body).toString('utf8')).toEqual(
'found it',
);
const jsResponse = await request(app).get(
`/${entityTripletPath}/some%20folder/also%20with%20spaces.js`,
);
expect(jsResponse.text).toEqual('found it too');
});
it('should pass text/plain content-type for html', async () => {
const {
kind,
metadata: { namespace, name },
} = entity;
const htmlResponse = await request(app).get(
`/${namespace}/${kind}/${name}/html/unsafe.html`,
`/${entityTripletPath}/html/unsafe.html`,
);
expect(htmlResponse.text).toEqual('<html></html>');
expect(htmlResponse.header).toMatchObject({
@@ -349,7 +345,7 @@ describe('AwsS3Publish', () => {
});
const svgResponse = await request(app).get(
`/${namespace}/${kind}/${name}/img/unsafe.svg`,
`/${entityTripletPath}/img/unsafe.svg`,
);
expect(svgResponse.text).toEqual('<svg></svg>');
expect(svgResponse.header).toMatchObject({
packages/techdocs-common/src/stages/publish/awsS3.ts
+41 -7
View File
@@ -31,6 +31,7 @@ import {
getFileTreeRecursively,
getHeadersForFileExtension,
getStaleFiles,
lowerCaseEntityTriplet,
lowerCaseEntityTripletInStoragePath,
} from './helpers';
import {
@@ -100,7 +101,17 @@ export class AwsS3Publish implements PublisherBase {
...(s3ForcePathStyle && { s3ForcePathStyle }),
});
return new AwsS3Publish(storageClient, bucketName, logger);
const legacyPathCasing =
config.getOptionalBoolean(
'techdocs.legacyUseCaseSensitiveTripletPaths',
) || false;
return new AwsS3Publish(
storageClient,
bucketName,
legacyPathCasing,
logger,
);
}
private static buildCredentials(
@@ -137,10 +148,12 @@ export class AwsS3Publish implements PublisherBase {
constructor(
private readonly storageClient: aws.S3,
private readonly bucketName: string,
private readonly legacyPathCasing: boolean,
private readonly logger: Logger,
) {
this.storageClient = storageClient;
this.bucketName = bucketName;
this.legacyPathCasing = legacyPathCasing;
this.logger = logger;
}
@@ -178,10 +191,15 @@ export class AwsS3Publish implements PublisherBase {
* Directory structure used in the bucket is - entityNamespace/entityKind/entityName/index.html
*/
async publish({ entity, directory }: PublishRequest): Promise<void> {
const useLegacyPathCasing = this.legacyPathCasing;
// First, try to retrieve a list of all individual files currently existing
let existingFiles: string[] = [];
try {
const remoteFolder = getCloudPathForLocalPath(entity);
const remoteFolder = getCloudPathForLocalPath(
entity,
undefined,
useLegacyPathCasing,
);
existingFiles = await this.getAllObjectsFromBucket({
prefix: remoteFolder,
});
@@ -206,7 +224,11 @@ export class AwsS3Publish implements PublisherBase {
const params = {
Bucket: this.bucketName,
Key: getCloudPathForLocalPath(entity, relativeFilePath),
Key: getCloudPathForLocalPath(
entity,
relativeFilePath,
useLegacyPathCasing,
),
Body: fileStream,
};
@@ -232,6 +254,7 @@ export class AwsS3Publish implements PublisherBase {
getCloudPathForLocalPath(
entity,
path.relative(directory, absoluteFilePath),
useLegacyPathCasing,
),
);
const staleFiles = getStaleFiles(relativeFilesToUpload, existingFiles);
@@ -263,7 +286,10 @@ export class AwsS3Publish implements PublisherBase {
): Promise<TechDocsMetadata> {
try {
return await new Promise<TechDocsMetadata>(async (resolve, reject) => {
const entityRootDir = `${entityName.namespace}/${entityName.kind}/${entityName.name}`;
const entityTriplet = `${entityName.namespace}/${entityName.kind}/${entityName.name}`;
const entityRootDir = this.legacyPathCasing
? entityTriplet
: lowerCaseEntityTriplet(entityTriplet);
const stream = this.storageClient
.getObject({
@@ -301,8 +327,12 @@ export class AwsS3Publish implements PublisherBase {
docsRouter(): express.Handler {
return async (req, res) => {
// Decode and trim the leading forward slash
// filePath example - /default/Component/documented-component/index.html
const filePath = decodeURI(req.path.replace(/^\//, ''));
const decodedUri = decodeURI(req.path.replace(/^\//, ''));
// filePath example - /default/component/documented-component/index.html
const filePath = this.legacyPathCasing
? decodedUri
: lowerCaseEntityTripletInStoragePath(decodedUri);
// Files with different extensions (CSS, HTML) need to be served with different headers
const fileExtension = path.extname(filePath);
@@ -333,7 +363,11 @@ export class AwsS3Publish implements PublisherBase {
*/
async hasDocsBeenGenerated(entity: Entity): Promise<boolean> {
try {
const entityRootDir = `${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}`;
const entityTriplet = `${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}`;
const entityRootDir = this.legacyPathCasing
? entityTriplet
: lowerCaseEntityTriplet(entityTriplet);
await this.storageClient
.headObject({
Bucket: this.bucketName,
packages/techdocs-common/src/stages/publish/azureBlobStorage.test.ts
+185 -215
View File
@@ -15,43 +15,18 @@
*/
import { getVoidLogger } from '@backstage/backend-common';
import {
Entity,
EntityName,
ENTITY_DEFAULT_NAMESPACE,
} from '@backstage/catalog-model';
import { Entity, ENTITY_DEFAULT_NAMESPACE } from '@backstage/catalog-model';
import { ConfigReader } from '@backstage/config';
import express from 'express';
import request from 'supertest';
import mockFs from 'mock-fs';
import os from 'os';
import path from 'path';
import fs from 'fs-extra';
import { AzureBlobStoragePublish } from './azureBlobStorage';
import { PublisherBase, TechDocsMetadata } from './types';
// NOTE: /packages/techdocs-common/__mocks__ is being used to mock Azure client library
const createMockEntity = (annotations = {}) => {
return {
apiVersion: 'version',
kind: 'TestKind',
metadata: {
name: 'test-component-name',
namespace: 'test-namespace',
annotations: {
...annotations,
},
},
};
};
const createMockEntityName = (): EntityName => ({
kind: 'TestKind',
name: 'test-component-name',
namespace: 'test-namespace',
});
const rootDir = os.platform() === 'win32' ? 'C:\\rootDir' : '/rootDir';
const rootDir = global.rootDir;
const getEntityRootDir = (entity: Entity) => {
const {
@@ -63,19 +38,18 @@ const getEntityRootDir = (entity: Entity) => {
};
const logger = getVoidLogger();
const loggerInfoSpy = jest.spyOn(logger, 'info');
const loggerErrorSpy = jest.spyOn(logger, 'error');
jest.spyOn(logger, 'error').mockReturnValue(logger);
const createPublisherMock = ({
const createPublisherFromConfig = ({
accountName = 'accountName',
containerName = 'containerName',
}:
| {
accountName?: string;
containerName?: string;
}
| undefined = {}) => {
const mockConfig = new ConfigReader({
legacyUseCaseSensitiveTripletPaths = false,
}: {
accountName?: string;
containerName?: string;
legacyUseCaseSensitiveTripletPaths?: boolean;
} = {}) => {
const config = new ConfigReader({
techdocs: {
requestUrl: 'http://localhost:7000',
publisher: {
@@ -88,30 +62,80 @@ const createPublisherMock = ({
containerName,
},
},
legacyUseCaseSensitiveTripletPaths,
},
});
return AzureBlobStoragePublish.fromConfig(mockConfig, logger);
return AzureBlobStoragePublish.fromConfig(config, logger);
};
let publisher: PublisherBase;
beforeEach(async () => {
loggerInfoSpy.mockClear();
loggerErrorSpy.mockClear();
mockFs.restore();
publisher = createPublisherMock();
});
describe('AzureBlobStoragePublish', () => {
const entity = {
apiVersion: '1',
kind: 'Component',
metadata: {
name: 'backstage',
namespace: 'default',
annotations: {},
},
};
const entityName = {
kind: 'Component',
name: 'backstage',
namespace: 'default',
};
const techdocsMetadata = {
site_name: 'backstage',
site_description: 'site_content',
etag: 'etag',
};
const directory = getEntityRootDir(entity);
beforeEach(() => {
(logger.error as jest.Mock).mockClear();
});
const files = {
'index.html': '',
'404.html': '',
'techdocs_metadata.json': JSON.stringify(techdocsMetadata),
assets: {
'main.css': '',
},
html: {
'unsafe.html': '<html></html>',
},
img: {
'with spaces.png': 'found it',
'unsafe.svg': '<svg></svg>',
},
'some folder': {
'also with spaces.js': 'found it too',
},
};
beforeEach(async () => {
mockFs({
[directory]: files,
});
});
afterEach(() => {
mockFs.restore();
});
describe('publishing with valid credentials', () => {
describe('getReadiness', () => {
it('should validate correct config', async () => {
const publisher = createPublisherFromConfig();
expect(await publisher.getReadiness()).toEqual({
isAvailable: true,
});
});
it('should reject incorrect config', async () => {
const errorPublisher = createPublisherMock({
const errorPublisher = createPublisherFromConfig({
containerName: 'bad_container',
});
@@ -119,7 +143,7 @@ describe('publishing with valid credentials', () => {
isAvailable: false,
});
expect(loggerErrorSpy).toHaveBeenCalledWith(
expect(logger.error).toHaveBeenCalledWith(
expect.stringContaining(
`Could not retrieve metadata about the Azure Blob Storage container bad_container.`,
),
@@ -128,35 +152,16 @@ describe('publishing with valid credentials', () => {
});
describe('publish', () => {
beforeEach(() => {
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
mockFs({
[entityRootDir]: {
'index.html': '',
'404.html': '',
assets: {
'main.css': '',
},
},
});
});
afterEach(() => {
mockFs.restore();
});
it('should publish a directory', async () => {
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const publisher = createPublisherFromConfig();
expect(await publisher.publish({ entity, directory })).toBeUndefined();
});
expect(
await publisher.publish({
entity,
directory: entityRootDir,
}),
).toBeUndefined();
it('should publish a directory as well when legacy casing is used', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
expect(await publisher.publish({ entity, directory })).toBeUndefined();
});
it('should fail to publish a directory', async () => {
@@ -168,7 +173,9 @@ describe('publishing with valid credentials', () => {
'generatedDirectory',
);
const entity = createMockEntity();
const publisher = createPublisherFromConfig({
containerName: 'bad_container',
});
const fails = publisher.publish({
entity,
@@ -187,177 +194,131 @@ describe('publishing with valid credentials', () => {
});
it('reports an error when bad account credentials', async () => {
publisher = createPublisherMock({
accountName: 'failupload',
});
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
mockFs({
[entityRootDir]: {
'index.html': '',
},
const publisher = createPublisherFromConfig({
accountName: 'bad_account_credentials',
});
let error;
try {
await publisher.publish({
entity,
directory: entityRootDir,
});
await publisher.publish({ entity, directory });
} catch (e) {
error = e;
}
expect(error.message).toContain(`Unable to upload file(s) to Azure`);
expect(loggerErrorSpy).toHaveBeenCalledWith(
expect(logger.error).toHaveBeenCalledWith(
expect.stringContaining(
`Unable to upload file(s) to Azure. Error: Upload failed for ${path.join(
entityRootDir,
'index.html',
directory,
'404.html',
)} with status code 500`,
),
);
});
it('should delete stale files after upload', async () => {
const entity = createMockEntity();
const directory = getEntityRootDir(entity);
const containerName = 'delete_stale_files_success';
publisher = createPublisherMock({ containerName });
await publisher.publish({ entity, directory });
expect(loggerInfoSpy).toHaveBeenLastCalledWith(
`Successfully deleted stale files for Entity ${entity.metadata.name}. Total number of files: 1`,
);
});
it('should log error when the stale files deletion fails', async () => {
const entity = createMockEntity();
const directory = getEntityRootDir(entity);
const containerName = 'delete_stale_files_error';
publisher = createPublisherMock({ containerName });
await publisher.publish({ entity, directory });
expect(loggerErrorSpy).toHaveBeenLastCalledWith(
'Unable to delete file(s) from Azure. Error: Message',
);
});
});
describe('hasDocsBeenGenerated', () => {
it('should return true if docs has been generated', async () => {
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
mockFs({
[entityRootDir]: {
'index.html': 'file-content',
},
});
it('should check expected file', async () => {
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
expect(await publisher.hasDocsBeenGenerated(entity)).toBe(true);
});
it('should check expected file when legacy case flag is passed', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
await publisher.publish({ entity, directory });
expect(await publisher.hasDocsBeenGenerated(entity)).toBe(true);
mockFs.restore();
});
it('should return false if docs has not been generated', async () => {
const entity = createMockEntity();
expect(await publisher.hasDocsBeenGenerated(entity)).toBe(false);
const publisher = createPublisherFromConfig();
expect(
await publisher.hasDocsBeenGenerated({
kind: 'triplet',
metadata: {
namespace: 'invalid',
name: 'path',
},
} as Entity),
).toBe(false);
});
});
describe('fetchTechDocsMetadata', () => {
it('should return tech docs metadata', async () => {
const entityNameMock = createMockEntityName();
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
expect(await publisher.fetchTechDocsMetadata(entityName)).toStrictEqual(
techdocsMetadata,
);
});
mockFs({
[entityRootDir]: {
'techdocs_metadata.json':
'{"site_name": "backstage", "site_description": "site_content", "etag": "etag"}',
},
it('should return tech docs metadata even if the legacy case is enabled', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
const expectedMetadata: TechDocsMetadata = {
site_name: 'backstage',
site_description: 'site_content',
etag: 'etag',
};
expect(
await publisher.fetchTechDocsMetadata(entityNameMock),
).toStrictEqual(expectedMetadata);
mockFs.restore();
await publisher.publish({ entity, directory });
expect(await publisher.fetchTechDocsMetadata(entityName)).toStrictEqual(
techdocsMetadata,
);
});
it('should return tech docs metadata when json encoded with single quotes', async () => {
const entityNameMock = createMockEntityName();
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const techdocsMetadataPath = path.join(
directory,
'techdocs_metadata.json',
);
const techdocsMetadataContent = files['techdocs_metadata.json'];
mockFs({
[entityRootDir]: {
'techdocs_metadata.json': `{'site_name': 'backstage', 'site_description': 'site_content', 'etag': 'etag'}`,
},
});
fs.writeFileSync(
techdocsMetadataPath,
techdocsMetadataContent.replace(/"/g, "'"),
);
const expectedMetadata: TechDocsMetadata = {
site_name: 'backstage',
site_description: 'site_content',
etag: 'etag',
};
expect(
await publisher.fetchTechDocsMetadata(entityNameMock),
).toStrictEqual(expectedMetadata);
mockFs.restore();
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
expect(await publisher.fetchTechDocsMetadata(entityName)).toStrictEqual(
techdocsMetadata,
);
fs.writeFileSync(techdocsMetadataPath, techdocsMetadataContent);
});
it('should return an error if the techdocs_metadata.json file is not present', async () => {
const entityNameMock = createMockEntityName();
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const publisher = createPublisherFromConfig();
let error;
try {
await publisher.fetchTechDocsMetadata(entityNameMock);
} catch (e) {
error = e;
}
const invalidEntityName = {
namespace: 'invalid',
kind: 'triplet',
name: 'path',
};
expect(error).toEqual(
new Error(
`TechDocs metadata fetch failed, The file ${path.join(
entityRootDir,
'techdocs_metadata.json',
)} does not exist !`,
),
const techDocsMetadaFilePath = path.join(
...Object.values(invalidEntityName),
'techdocs_metadata.json',
);
const fails = publisher.fetchTechDocsMetadata(invalidEntityName);
await expect(fails).rejects.toMatchObject({
message: `TechDocs metadata fetch failed, The file ${techDocsMetadaFilePath} does not exist!`,
});
});
});
describe('docsRouter', () => {
const entityTripletPath = `${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}`;
let app: express.Express;
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
beforeEach(() => {
beforeEach(async () => {
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
app = express().use(publisher.docsRouter());
mockFs.restore();
mockFs({
[entityRootDir]: {
html: {
'unsafe.html': '<html></html>',
},
img: {
'with spaces.png': 'found it',
'unsafe.svg': '<svg></svg>',
},
'some folder': {
'also with spaces.js': 'found it too',
},
},
});
});
afterEach(() => {
@@ -365,32 +326,41 @@ describe('publishing with valid credentials', () => {
});
it('should pass expected object path to bucket', async () => {
const {
kind,
metadata: { namespace, name },
} = entity;
// Ensures leading slash is trimmed and encoded path is decoded.
const pngResponse = await request(app).get(
`/${namespace}/${kind}/${name}/img/with%20spaces.png`,
`/${entityTripletPath}/img/with%20spaces.png`,
);
expect(Buffer.from(pngResponse.body).toString('utf8')).toEqual(
'found it',
);
const jsResponse = await request(app).get(
`/${namespace}/${kind}/${name}/some%20folder/also%20with%20spaces.js`,
`/${entityTripletPath}/some%20folder/also%20with%20spaces.js`,
);
expect(jsResponse.text).toEqual('found it too');
});
it('should pass expected object path to bucket even if the legacy case is enabled', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
await publisher.publish({ entity, directory });
app = express().use(publisher.docsRouter());
// Ensures leading slash is trimmed and encoded path is decoded.
const pngResponse = await request(app).get(
`/${entityTripletPath}/img/with%20spaces.png`,
);
expect(Buffer.from(pngResponse.body).toString('utf8')).toEqual(
'found it',
);
const jsResponse = await request(app).get(
`/${entityTripletPath}/some%20folder/also%20with%20spaces.js`,
);
expect(jsResponse.text).toEqual('found it too');
});
it('should pass text/plain content-type for html', async () => {
const {
kind,
metadata: { namespace, name },
} = entity;
const htmlResponse = await request(app).get(
`/${namespace}/${kind}/${name}/html/unsafe.html`,
`/${entityTripletPath}/html/unsafe.html`,
);
expect(htmlResponse.text).toEqual('<html></html>');
expect(htmlResponse.header).toMatchObject({
@@ -398,7 +368,7 @@ describe('publishing with valid credentials', () => {
});
const svgResponse = await request(app).get(
`/${namespace}/${kind}/${name}/img/unsafe.svg`,
`/${entityTripletPath}/img/unsafe.svg`,
);
expect(svgResponse.text).toEqual('<svg></svg>');
expect(svgResponse.header).toMatchObject({
packages/techdocs-common/src/stages/publish/azureBlobStorage.ts
+43 -6
View File
@@ -31,6 +31,7 @@ import {
getCloudPathForLocalPath,
getFileTreeRecursively,
getHeadersForFileExtension,
lowerCaseEntityTriplet,
getStaleFiles,
lowerCaseEntityTripletInStoragePath,
} from './helpers';
@@ -88,16 +89,28 @@ export class AzureBlobStoragePublish implements PublisherBase {
credential,
);
return new AzureBlobStoragePublish(storageClient, containerName, logger);
const legacyPathCasing =
config.getOptionalBoolean(
'techdocs.legacyUseCaseSensitiveTripletPaths',
) || false;
return new AzureBlobStoragePublish(
storageClient,
containerName,
legacyPathCasing,
logger,
);
}
constructor(
private readonly storageClient: BlobServiceClient,
private readonly containerName: string,
private readonly legacyPathCasing: boolean,
private readonly logger: Logger,
) {
this.storageClient = storageClient;
this.containerName = containerName;
this.legacyPathCasing = legacyPathCasing;
this.logger = logger;
}
@@ -137,8 +150,14 @@ export class AzureBlobStoragePublish implements PublisherBase {
* Directory structure used in the container is - entityNamespace/entityKind/entityName/index.html
*/
async publish({ entity, directory }: PublishRequest): Promise<void> {
const useLegacyPathCasing = this.legacyPathCasing;
// First, try to retrieve a list of all individual files currently existing
const remoteFolder = getCloudPathForLocalPath(entity);
const remoteFolder = getCloudPathForLocalPath(
entity,
undefined,
useLegacyPathCasing,
);
let existingFiles: string[] = [];
try {
existingFiles = await this.getAllBlobsFromContainer({
@@ -169,7 +188,11 @@ export class AzureBlobStoragePublish implements PublisherBase {
);
const response = await container
.getBlockBlobClient(
getCloudPathForLocalPath(entity, relativeFilePath),
getCloudPathForLocalPath(
entity,
relativeFilePath,
useLegacyPathCasing,
),
)
.uploadFile(absoluteFilePath);
@@ -212,6 +235,7 @@ export class AzureBlobStoragePublish implements PublisherBase {
getCloudPathForLocalPath(
entity,
path.relative(directory, absoluteFilePath),
useLegacyPathCasing,
),
);
@@ -263,7 +287,11 @@ export class AzureBlobStoragePublish implements PublisherBase {
async fetchTechDocsMetadata(
entityName: EntityName,
): Promise<TechDocsMetadata> {
const entityRootDir = `${entityName.namespace}/${entityName.kind}/${entityName.name}`;
const entityTriplet = `${entityName.namespace}/${entityName.kind}/${entityName.name}`;
const entityRootDir = this.legacyPathCasing
? entityTriplet
: lowerCaseEntityTriplet(entityTriplet);
try {
const techdocsMetadataJson = await this.download(
this.containerName,
@@ -289,8 +317,13 @@ export class AzureBlobStoragePublish implements PublisherBase {
docsRouter(): express.Handler {
return (req, res) => {
// Decode and trim the leading forward slash
const decodedUri = decodeURI(req.path.replace(/^\//, ''));
// filePath example - /default/Component/documented-component/index.html
const filePath = decodeURI(req.path.replace(/^\//, ''));
const filePath = this.legacyPathCasing
? decodedUri
: lowerCaseEntityTripletInStoragePath(decodedUri);
// Files with different extensions (CSS, HTML) need to be served with different headers
const fileExtension = platformPath.extname(filePath);
const responseHeaders = getHeadersForFileExtension(fileExtension);
@@ -317,7 +350,11 @@ export class AzureBlobStoragePublish implements PublisherBase {
* can be used to verify if there are any pre-generated docs available to serve.
*/
hasDocsBeenGenerated(entity: Entity): Promise<boolean> {
const entityRootDir = `${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}`;
const entityTriplet = `${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}`;
const entityRootDir = this.legacyPathCasing
? entityTriplet
: lowerCaseEntityTriplet(entityTriplet);
return this.storageClient
.getContainerClient(this.containerName)
.getBlockBlobClient(`${entityRootDir}/index.html`)
packages/techdocs-common/src/stages/publish/googleStorage.test.ts
+181 -186
View File
@@ -15,43 +15,18 @@
*/
import { getVoidLogger } from '@backstage/backend-common';
import {
Entity,
ENTITY_DEFAULT_NAMESPACE,
EntityName,
} from '@backstage/catalog-model';
import { Entity, ENTITY_DEFAULT_NAMESPACE } from '@backstage/catalog-model';
import { ConfigReader } from '@backstage/config';
import express from 'express';
import request from 'supertest';
import mockFs from 'mock-fs';
import os from 'os';
import path from 'path';
import fs from 'fs-extra';
import { GoogleGCSPublish } from './googleStorage';
import { PublisherBase, TechDocsMetadata } from './types';
// NOTE: /packages/techdocs-common/__mocks__ is being used to mock Google Cloud Storage client library
const createMockEntity = (annotations = {}): Entity => {
return {
apiVersion: 'version',
kind: 'TestKind',
metadata: {
name: 'test-component-name',
namespace: 'test-namespace',
annotations: {
...annotations,
},
},
};
};
const createMockEntityName = (): EntityName => ({
kind: 'TestKind',
name: 'test-component-name',
namespace: 'test-namespace',
});
const rootDir = os.platform() === 'win32' ? 'C:\\rootDir' : '/rootDir';
const rootDir = global.rootDir;
const getEntityRootDir = (entity: Entity) => {
const {
@@ -63,11 +38,17 @@ const getEntityRootDir = (entity: Entity) => {
};
const logger = getVoidLogger();
const loggerInfoSpy = jest.spyOn(logger, 'info').mockReturnValue(logger);
const loggerErrorSpy = jest.spyOn(logger, 'error').mockReturnValue(logger);
jest.spyOn(logger, 'info').mockReturnValue(logger);
jest.spyOn(logger, 'error').mockReturnValue(logger);
const createPublisherMock = (bucketName: string) => {
const mockConfig = new ConfigReader({
const createPublisherFromConfig = ({
bucketName = 'bucketName',
legacyUseCaseSensitiveTripletPaths = false,
}: {
bucketName?: string;
legacyUseCaseSensitiveTripletPaths?: boolean;
} = {}) => {
const config = new ConfigReader({
techdocs: {
requestUrl: 'http://localhost:7000',
publisher: {
@@ -77,67 +58,95 @@ const createPublisherMock = (bucketName: string) => {
bucketName,
},
},
legacyUseCaseSensitiveTripletPaths,
},
});
return GoogleGCSPublish.fromConfig(mockConfig, logger);
return GoogleGCSPublish.fromConfig(config, logger);
};
let publisher: PublisherBase;
beforeEach(async () => {
loggerInfoSpy.mockClear();
loggerErrorSpy.mockClear();
mockFs.restore();
publisher = createPublisherMock('bucketName');
});
describe('GoogleGCSPublish', () => {
const entity = {
apiVersion: 'version',
kind: 'Component',
metadata: {
name: 'backstage',
namespace: 'default',
annotations: {},
},
};
const entityName = {
kind: 'Component',
name: 'backstage',
namespace: 'default',
};
const techdocsMetadata = {
site_name: 'backstage',
site_description: 'site_content',
etag: 'etag',
};
const directory = getEntityRootDir(entity);
const files = {
'index.html': '',
'404.html': '',
assets: {
'main.css': '',
},
'techdocs_metadata.json': JSON.stringify(techdocsMetadata),
html: {
'unsafe.html': '<html></html>',
},
img: {
'with spaces.png': 'found it',
'unsafe.svg': '<svg></svg>',
},
'some folder': {
'also with spaces.js': 'found it too',
},
};
beforeAll(() => {
mockFs({
[directory]: files,
});
});
afterAll(() => {
mockFs.restore();
});
describe('getReadiness', () => {
it('should validate correct config', async () => {
const publisher = createPublisherFromConfig();
expect(await publisher.getReadiness()).toEqual({
isAvailable: true,
});
});
it('should reject incorrect config', async () => {
const errorPublisher = createPublisherMock('errorBucket');
expect(await errorPublisher.getReadiness()).toEqual({
const publisher = createPublisherFromConfig({
bucketName: 'bad_bucket_name',
});
expect(await publisher.getReadiness()).toEqual({
isAvailable: false,
});
});
});
describe('publish', () => {
beforeEach(() => {
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
mockFs({
[entityRootDir]: {
'index.html': '',
'404.html': '',
assets: {
'main.css': '',
},
},
});
});
afterEach(() => {
mockFs.restore();
});
it('should publish a directory', async () => {
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const publisher = createPublisherFromConfig();
expect(await publisher.publish({ entity, directory })).toBeUndefined();
});
expect(
await publisher.publish({
entity,
directory: entityRootDir,
}),
).toBeUndefined();
it('should publish a directory as well when legacy casing is used', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
expect(await publisher.publish({ entity, directory })).toBeUndefined();
});
it('should fail to publish a directory', async () => {
@@ -149,14 +158,7 @@ describe('GoogleGCSPublish', () => {
'generatedDirectory',
);
const entity = createMockEntity();
await expect(
publisher.publish({
entity,
directory: wrongPathToGeneratedDirectory,
}),
).rejects.toThrowError();
const publisher = createPublisherFromConfig();
const fails = publisher.publish({
entity,
@@ -170,29 +172,26 @@ describe('GoogleGCSPublish', () => {
`Unable to upload file(s) to Google Cloud Storage. Error: Failed to read template directory: ENOENT, no such file or directory`,
),
});
await expect(fails).rejects.toMatchObject({
message: expect.stringContaining(wrongPathToGeneratedDirectory),
});
});
it('should delete stale files after upload', async () => {
const entity = createMockEntity();
const directory = getEntityRootDir(entity);
const bucketName = 'delete_stale_files_success';
publisher = createPublisherMock(bucketName);
const publisher = createPublisherFromConfig({ bucketName });
await publisher.publish({ entity, directory });
expect(loggerInfoSpy).toHaveBeenLastCalledWith(
expect(logger.info).toHaveBeenLastCalledWith(
`Successfully deleted stale files for Entity ${entity.metadata.name}. Total number of files: 1`,
);
});
it('should log error when the stale files deletion fails', async () => {
const entity = createMockEntity();
const directory = getEntityRootDir(entity);
const bucketName = 'delete_stale_files_error';
publisher = createPublisherMock(bucketName);
const publisher = createPublisherFromConfig({ bucketName });
await publisher.publish({ entity, directory });
expect(loggerErrorSpy).toHaveBeenLastCalledWith(
expect(logger.error).toHaveBeenLastCalledWith(
'Unable to delete file(s) from Google Cloud Storage. Error: Message',
);
});
@@ -200,148 +199,144 @@ describe('GoogleGCSPublish', () => {
describe('hasDocsBeenGenerated', () => {
it('should return true if docs has been generated', async () => {
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
mockFs({
[entityRootDir]: {
'index.html': 'file-content',
},
});
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
expect(await publisher.hasDocsBeenGenerated(entity)).toBe(true);
});
it('should return true if docs has been generated even if the legacy case is enabled', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
await publisher.publish({ entity, directory });
expect(await publisher.hasDocsBeenGenerated(entity)).toBe(true);
mockFs.restore();
});
it('should return false if docs has not been generated', async () => {
const entity = createMockEntity();
expect(await publisher.hasDocsBeenGenerated(entity)).toBe(false);
const publisher = createPublisherFromConfig();
expect(
await publisher.hasDocsBeenGenerated({
kind: 'entity',
metadata: {
namespace: 'invalid',
name: 'triplet',
},
} as Entity),
).toBe(false);
});
});
describe('fetchTechDocsMetadata', () => {
beforeEach(() => {
mockFs.restore();
it('should return tech docs metadata', async () => {
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
expect(await publisher.fetchTechDocsMetadata(entityName)).toStrictEqual(
techdocsMetadata,
);
});
it('should return tech docs metadata', async () => {
const entityNameMock = createMockEntityName();
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
mockFs({
[entityRootDir]: {
'techdocs_metadata.json':
'{"site_name": "backstage", "site_description": "site_content", "etag": "etag"}',
},
it('should return tech docs metadata even if the legacy case is enabled', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
const expectedMetadata: TechDocsMetadata = {
site_name: 'backstage',
site_description: 'site_content',
etag: 'etag',
};
expect(
await publisher.fetchTechDocsMetadata(entityNameMock),
).toStrictEqual(expectedMetadata);
await publisher.publish({ entity, directory });
expect(await publisher.fetchTechDocsMetadata(entityName)).toStrictEqual(
techdocsMetadata,
);
});
it('should return tech docs metadata when json encoded with single quotes', async () => {
const entityNameMock = createMockEntityName();
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const techdocsMetadataPath = path.join(
directory,
'techdocs_metadata.json',
);
const techdocsMetadataContent = files['techdocs_metadata.json'];
mockFs({
[entityRootDir]: {
'techdocs_metadata.json':
"{'site_name': 'backstage', 'site_description': 'site_content', 'etag': 'etag'}",
},
});
fs.writeFileSync(
techdocsMetadataPath,
techdocsMetadataContent.replace(/"/g, "'"),
);
const expectedMetadata: TechDocsMetadata = {
site_name: 'backstage',
site_description: 'site_content',
etag: 'etag',
};
expect(
await publisher.fetchTechDocsMetadata(entityNameMock),
).toStrictEqual(expectedMetadata);
mockFs.restore();
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
expect(await publisher.fetchTechDocsMetadata(entityName)).toStrictEqual(
techdocsMetadata,
);
fs.writeFileSync(techdocsMetadataPath, techdocsMetadataContent);
});
it('should return an error if the techdocs_metadata.json file is not present', async () => {
const entityNameMock = createMockEntityName();
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const publisher = createPublisherFromConfig();
const fails = publisher.fetchTechDocsMetadata(entityNameMock);
const invalidEntityName = {
namespace: 'invalid',
kind: 'triplet',
name: 'path',
};
const techDocsMetadaFilePath = path.join(
...Object.values(invalidEntityName),
'techdocs_metadata.json',
);
const fails = publisher.fetchTechDocsMetadata(invalidEntityName);
await expect(fails).rejects.toMatchObject({
message: `The file ${path.join(
entityRootDir,
'techdocs_metadata.json',
)} does not exist !`,
message: `The file ${techDocsMetadaFilePath} does not exist!`,
});
});
});
describe('docsRouter', () => {
let app: express.Express;
const entity = createMockEntity();
const entityRootDir = getEntityRootDir(entity);
const entityTripletPath = `${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}`;
beforeEach(() => {
let app: Express.Application;
beforeEach(async () => {
const publisher = createPublisherFromConfig();
await publisher.publish({ entity, directory });
app = express().use(publisher.docsRouter());
mockFs.restore();
mockFs({
[entityRootDir]: {
html: {
'unsafe.html': '<html></html>',
},
img: {
'with spaces.png': 'found it',
'unsafe.svg': '<svg></svg>',
},
'some folder': {
'also with spaces.js': 'found it too',
},
},
});
});
afterEach(() => {
mockFs.restore();
});
it('should pass expected object path to bucket', async () => {
const {
kind,
metadata: { namespace, name },
} = entity;
// Ensures leading slash is trimmed and encoded path is decoded.
const pngResponse = await request(app).get(
`/${namespace}/${kind}/${name}/img/with%20spaces.png`,
`/${entityTripletPath}/img/with%20spaces.png`,
);
expect(Buffer.from(pngResponse.body).toString('utf8')).toEqual(
'found it',
);
const jsResponse = await request(app).get(
`/${namespace}/${kind}/${name}/some%20folder/also%20with%20spaces.js`,
`/${entityTripletPath}/some%20folder/also%20with%20spaces.js`,
);
expect(jsResponse.text).toEqual('found it too');
});
it('should pass expected object path to bucket even if the legacy case is enabled', async () => {
const publisher = createPublisherFromConfig({
legacyUseCaseSensitiveTripletPaths: true,
});
await publisher.publish({ entity, directory });
app = express().use(publisher.docsRouter());
// Ensures leading slash is trimmed and encoded path is decoded.
const pngResponse = await request(app).get(
`/${entityTripletPath}/img/with%20spaces.png`,
);
expect(Buffer.from(pngResponse.body).toString('utf8')).toEqual(
'found it',
);
const jsResponse = await request(app).get(
`/${entityTripletPath}/some%20folder/also%20with%20spaces.js`,
);
expect(jsResponse.text).toEqual('found it too');
});
it('should pass text/plain content-type for html', async () => {
const {
kind,
metadata: { namespace, name },
} = entity;
const htmlResponse = await request(app).get(
`/${namespace}/${kind}/${name}/html/unsafe.html`,
`/${entityTripletPath}/html/unsafe.html`,
);
expect(htmlResponse.text).toEqual('<html></html>');
expect(htmlResponse.header).toMatchObject({
@@ -349,7 +344,7 @@ describe('GoogleGCSPublish', () => {
});
const svgResponse = await request(app).get(
`/${namespace}/${kind}/${name}/img/unsafe.svg`,
`/${entityTripletPath}/img/unsafe.svg`,
);
expect(svgResponse.text).toEqual('<svg></svg>');
expect(svgResponse.header).toMatchObject({
packages/techdocs-common/src/stages/publish/googleStorage.ts
+44 -9
View File
@@ -22,10 +22,12 @@ import path from 'path';
import { Readable } from 'stream';
import { Logger } from 'winston';
import {
bulkStorageOperation,
getCloudPathForLocalPath,
getFileTreeRecursively,
getHeadersForFileExtension,
lowerCaseEntityTriplet,
lowerCaseEntityTripletInStoragePath,
bulkStorageOperation,
getCloudPathForLocalPath,
getStaleFiles,
} from './helpers';
import { MigrateWriteStream } from './migrations';
@@ -70,16 +72,28 @@ export class GoogleGCSPublish implements PublisherBase {
}),
});
return new GoogleGCSPublish(storageClient, bucketName, logger);
const legacyPathCasing =
config.getOptionalBoolean(
'techdocs.legacyUseCaseSensitiveTripletPaths',
) || false;
return new GoogleGCSPublish(
storageClient,
bucketName,
legacyPathCasing,
logger,
);
}
constructor(
private readonly storageClient: Storage,
private readonly bucketName: string,
private readonly legacyPathCasing: boolean,
private readonly logger: Logger,
) {
this.storageClient = storageClient;
this.bucketName = bucketName;
this.legacyPathCasing = legacyPathCasing;
this.logger = logger;
}
@@ -115,12 +129,17 @@ export class GoogleGCSPublish implements PublisherBase {
* Directory structure used in the bucket is - entityNamespace/entityKind/entityName/index.html
*/
async publish({ entity, directory }: PublishRequest): Promise<void> {
const useLegacyPathCasing = this.legacyPathCasing;
const bucket = this.storageClient.bucket(this.bucketName);
// First, try to retrieve a list of all individual files currently existing
let existingFiles: string[] = [];
try {
const remoteFolder = getCloudPathForLocalPath(entity);
const remoteFolder = getCloudPathForLocalPath(
entity,
undefined,
useLegacyPathCasing,
);
existingFiles = await this.getFilesForFolder(remoteFolder);
} catch (e) {
this.logger.error(
@@ -140,7 +159,11 @@ export class GoogleGCSPublish implements PublisherBase {
async absoluteFilePath => {
const relativeFilePath = path.relative(directory, absoluteFilePath);
return await bucket.upload(absoluteFilePath, {
destination: getCloudPathForLocalPath(entity, relativeFilePath),
destination: getCloudPathForLocalPath(
entity,
relativeFilePath,
useLegacyPathCasing,
),
});
},
absoluteFilesToUpload,
@@ -163,6 +186,7 @@ export class GoogleGCSPublish implements PublisherBase {
getCloudPathForLocalPath(
entity,
path.relative(directory, absoluteFilePath),
useLegacyPathCasing,
),
);
const staleFiles = getStaleFiles(relativeFilesToUpload, existingFiles);
@@ -186,7 +210,10 @@ export class GoogleGCSPublish implements PublisherBase {
fetchTechDocsMetadata(entityName: EntityName): Promise<TechDocsMetadata> {
return new Promise((resolve, reject) => {
const entityRootDir = `${entityName.namespace}/${entityName.kind}/${entityName.name}`;
const entityTriplet = `${entityName.namespace}/${entityName.kind}/${entityName.name}`;
const entityRootDir = this.legacyPathCasing
? entityTriplet
: lowerCaseEntityTriplet(entityTriplet);
const fileStreamChunks: Array<any> = [];
this.storageClient
@@ -214,8 +241,12 @@ export class GoogleGCSPublish implements PublisherBase {
docsRouter(): express.Handler {
return (req, res) => {
// Decode and trim the leading forward slash
// filePath example - /default/Component/documented-component/index.html
const filePath = decodeURI(req.path.replace(/^\//, ''));
const decodedUri = decodeURI(req.path.replace(/^\//, ''));
// filePath example - /default/component/documented-component/index.html
const filePath = this.legacyPathCasing
? decodedUri
: lowerCaseEntityTripletInStoragePath(decodedUri);
// Files with different extensions (CSS, HTML) need to be served with different headers
const fileExtension = path.extname(filePath);
@@ -248,7 +279,11 @@ export class GoogleGCSPublish implements PublisherBase {
*/
async hasDocsBeenGenerated(entity: Entity): Promise<boolean> {
return new Promise(resolve => {
const entityRootDir = `${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}`;
const entityTriplet = `${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}`;
const entityRootDir = this.legacyPathCasing
? entityTriplet
: lowerCaseEntityTriplet(entityTriplet);
this.storageClient
.bucket(this.bucketName)
.file(`${entityRootDir}/index.html`)
packages/techdocs-common/src/stages/publish/helpers.test.ts
+9 -8
View File
@@ -131,23 +131,19 @@ describe('getStaleFiles', () => {
describe('getCloudPathForLocalPath', () => {
const entity: Entity = {
apiVersion: 'version',
metadata: { namespace: 'default', name: 'backstage' },
metadata: { namespace: 'custom', name: 'backstage' },
kind: 'Component',
};
it('should compose a remote bucket path including entity information', () => {
const remoteBucket = getCloudPathForLocalPath(entity);
expect(remoteBucket).toBe(
`${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}/`,
);
expect(remoteBucket).toBe('custom/component/backstage/');
});
it('should compose a remote filename including entity information', () => {
const localPath = 'index.html';
const remoteBucket = getCloudPathForLocalPath(entity, localPath);
expect(remoteBucket).toBe(
`${entity.metadata.namespace}/${entity.kind}/${entity.metadata.name}/${localPath}`,
);
expect(remoteBucket).toBe(`custom/component/backstage/${localPath}`);
});
it('should use the default namespace when it is undefined', () => {
@@ -161,10 +157,15 @@ describe('getCloudPathForLocalPath', () => {
localPath,
);
expect(remoteBucket).toBe(
`${ENTITY_DEFAULT_NAMESPACE}/${entity.kind}/${entity.metadata.name}/${localPath}`,
`${ENTITY_DEFAULT_NAMESPACE}/component/backstage/${localPath}`,
);
});
it('should preserve case when legacy flag is passed', () => {
const remoteBucket = getCloudPathForLocalPath(entity, undefined, true);
expect(remoteBucket).toBe('custom/Component/backstage/');
});
it('should throw error when entity is invalid', () => {
expect(() => getCloudPathForLocalPath({} as Entity)).toThrow();
});
packages/techdocs-common/src/stages/publish/helpers.ts
+30 -8
View File
@@ -86,6 +86,23 @@ export const getFileTreeRecursively = async (
return fileList;
};
/**
* Returns a lower-cased version of entity's triplet in the form of a path.
*
* Path must not include a starting slash.
*
* @example
* lowerCaseEntityTriplet('default/Component/backstage')
* // return default/component/backstage
*/
export const lowerCaseEntityTriplet = (originalPath: string): string => {
const [namespace, kind, name, ...parts] = originalPath.split('/');
const lowerNamespace = namespace.toLowerCase();
const lowerKind = kind.toLowerCase();
const lowerName = name.toLowerCase();
return [lowerNamespace, lowerKind, lowerName, ...parts].join('/');
};
/**
* Returns the version of an object's storage path where the first three parts
* of the path (the entity triplet of namespace, kind, and name) are
@@ -93,9 +110,11 @@ export const getFileTreeRecursively = async (
*
* Path must not include a starting slash.
*
* Throws an error if the path does not appear to be an entity triplet.
*
* @example
* lowerCaseEntityTripletInStoragePath('default/Component/backstage')
* // return default/component/backstage
* lowerCaseEntityTripletInStoragePath('default/Component/backstage/file.txt')
* // return default/component/backstage/file.txt
*/
export const lowerCaseEntityTripletInStoragePath = (
originalPath: string,
@@ -108,11 +127,7 @@ export const lowerCaseEntityTripletInStoragePath = (
`Encountered file unmanaged by TechDocs ${originalPath}. Skipping.`,
);
}
const [namespace, kind, name, ...parts] = originalPath.split('/');
const lowerNamespace = namespace.toLowerCase();
const lowerKind = kind.toLowerCase();
const lowerName = name.toLowerCase();
return [lowerNamespace, lowerKind, lowerName, ...parts].join('/');
return lowerCaseEntityTriplet(originalPath);
};
// Only returns the files that existed previously and are not present anymore.
@@ -131,6 +146,7 @@ export const getStaleFiles = (
export const getCloudPathForLocalPath = (
entity: Entity,
localPath = '',
useLegacyPathCasing = false,
): string => {
// Convert destination file path to a POSIX path for uploading.
// GCS expects / as path separator and relativeFilePath will contain \\ on Windows.
@@ -141,7 +157,13 @@ export const getCloudPathForLocalPath = (
const entityRootDir = `${
entity.metadata?.namespace ?? ENTITY_DEFAULT_NAMESPACE
}/${entity.kind}/${entity.metadata.name}`;
return `${entityRootDir}/${relativeFilePathPosix}`; // GCS Bucket file relative path
const relativeFilePathTriplet = `${entityRootDir}/${relativeFilePathPosix}`;
const destination = useLegacyPathCasing
? relativeFilePathTriplet
: lowerCaseEntityTripletInStoragePath(relativeFilePathTriplet);
return destination; // Remote storage file relative path
};
// Perform rate limited generic operations by passing a function and a list of arguments
packages/techdocs-common/src/types.d.ts
Vendored
+24
View File
@@ -0,0 +1,24 @@
/*
* Copyright 2020 The Backstage Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
interface IStorageFilesMock {
emptyFiles(): void;
fileExists(targetPath: string): boolean;
readFile(targetPath: string): Buffer;
writeFile(targetPath: string, sourcePath: string): void;
writeFile(targetPath: string, sourceBuffer: Buffer): void;
writeFile(targetPath: string, source: string | Buffer): void;
}
plugins/catalog/api-report.md
+9 -1
View File
@@ -128,6 +128,14 @@ const catalogPlugin: BackstagePlugin<
},
{
createComponent: ExternalRouteRef<undefined, true>;
viewTechDoc: ExternalRouteRef<
{
name: string;
kind: string;
namespace: string;
},
true
>;
}
>;
export { catalogPlugin };
@@ -399,7 +407,7 @@ export const Router: ({
// src/components/EntityLayout/EntityLayout.d.ts:43:5 - (ae-forgotten-export) The symbol "EntityLayoutProps" needs to be exported by the entry point index.d.ts
// src/components/EntityLayout/EntityLayout.d.ts:44:5 - (ae-forgotten-export) The symbol "SubRoute" needs to be exported by the entry point index.d.ts
// src/components/EntityPageLayout/EntityPageLayout.d.ts:17:5 - (ae-forgotten-export) The symbol "EntityPageLayoutProps" needs to be exported by the entry point index.d.ts
// src/plugin.d.ts:17:5 - (ae-forgotten-export) The symbol "ColumnBreakpoints" needs to be exported by the entry point index.d.ts
// src/plugin.d.ts:22:5 - (ae-forgotten-export) The symbol "ColumnBreakpoints" needs to be exported by the entry point index.d.ts
// (No @packageDocumentation comment for this package)
```
plugins/catalog/src/components/AboutCard/AboutCard.test.tsx
+135
View File
@@ -29,6 +29,7 @@ import {
ApiRegistry,
ConfigReader,
} from '@backstage/core-app-api';
import { viewTechDocRouteRef } from '../../routes';
describe('<AboutCard />', () => {
it('renders info', async () => {
@@ -203,4 +204,138 @@ describe('<AboutCard />', () => {
);
expect(getByText('View Source').closest('a')).not.toHaveAttribute('href');
});
it('renders techdocs link', async () => {
const entity = {
apiVersion: 'v1',
kind: 'Component',
metadata: {
name: 'software',
annotations: {
'backstage.io/techdocs-ref': './',
},
},
spec: {
owner: 'guest',
type: 'service',
lifecycle: 'production',
},
};
const apis = ApiRegistry.with(
scmIntegrationsApiRef,
ScmIntegrationsApi.fromConfig(
new ConfigReader({
integrations: {
github: [
{
host: 'github.com',
token: '...',
},
],
},
}),
),
);
const { getByText } = await renderInTestApp(
<ApiProvider apis={apis}>
<EntityProvider entity={entity}>
<AboutCard />
</EntityProvider>
</ApiProvider>,
{
mountedRoutes: {
'/docs/:namespace/:kind/:name': viewTechDocRouteRef,
},
},
);
expect(getByText('View TechDocs').closest('a')).toHaveAttribute(
'href',
'/docs/default/Component/software',
);
});
it('renders disabled techdocs link when no docs exist', async () => {
const entity = {
apiVersion: 'v1',
kind: 'Component',
metadata: {
name: 'software',
},
spec: {
owner: 'guest',
type: 'service',
lifecycle: 'production',
},
};
const apis = ApiRegistry.with(
scmIntegrationsApiRef,
ScmIntegrationsApi.fromConfig(
new ConfigReader({
integrations: {
github: [
{
host: 'github.com',
token: '...',
},
],
},
}),
),
);
const { getByText } = await renderInTestApp(
<ApiProvider apis={apis}>
<EntityProvider entity={entity}>
<AboutCard />
</EntityProvider>
</ApiProvider>,
);
expect(getByText('View TechDocs').closest('a')).not.toHaveAttribute('href');
});
it('renders disbaled techdocs link when route is not bound', async () => {
const entity = {
apiVersion: 'v1',
kind: 'Component',
metadata: {
name: 'software',
annotations: {
'backstage.io/techdocs-ref': './',
},
},
spec: {
owner: 'guest',
type: 'service',
lifecycle: 'production',
},
};
const apis = ApiRegistry.with(
scmIntegrationsApiRef,
ScmIntegrationsApi.fromConfig(
new ConfigReader({
integrations: {
github: [
{
host: 'github.com',
token: '...',
},
],
},
}),
),
);
const { getByText } = await renderInTestApp(
<ApiProvider apis={apis}>
<EntityProvider entity={entity}>
<AboutCard />
</EntityProvider>
</ApiProvider>,
);
expect(getByText('View TechDocs').closest('a')).not.toHaveAttribute('href');
});
});
plugins/catalog/src/components/AboutCard/AboutCard.tsx
+14 -5
View File
@@ -49,7 +49,8 @@ import {
IconLinkVerticalProps,
InfoCardVariants,
} from '@backstage/core-components';
import { useApi } from '@backstage/core-plugin-api';
import { useApi, useRouteRef } from '@backstage/core-plugin-api';
import { viewTechDocRouteRef } from '../../routes';
const useStyles = makeStyles({
gridItemCard: {
@@ -81,6 +82,8 @@ export function AboutCard({ variant }: AboutCardProps) {
const classes = useStyles();
const { entity } = useEntity();
const scmIntegrationsApi = useApi(scmIntegrationsApiRef);
const viewTechdocLink = useRouteRef(viewTechDocRouteRef);
const entitySourceLocation = getEntitySourceLocation(
entity,
scmIntegrationsApi,
@@ -105,11 +108,17 @@ export function AboutCard({ variant }: AboutCardProps) {
};
const viewInTechDocs: IconLinkVerticalProps = {
label: 'View TechDocs',
disabled: !entity.metadata.annotations?.['backstage.io/techdocs-ref'],
disabled:
!entity.metadata.annotations?.['backstage.io/techdocs-ref'] ||
!viewTechdocLink,
icon: <DocsIcon />,
href: `/docs/${entity.metadata.namespace || ENTITY_DEFAULT_NAMESPACE}/${
entity.kind
}/${entity.metadata.name}`,
href:
viewTechdocLink &&
viewTechdocLink({
namespace: entity.metadata.namespace || ENTITY_DEFAULT_NAMESPACE,
kind: entity.kind,
name: entity.metadata.name,
}),
};
const viewApi: IconLinkVerticalProps = {
title: hasApis ? '' : 'No APIs available',
plugins/catalog/src/plugin.ts
+2 -1
View File
@@ -21,7 +21,7 @@ import {
entityRouteRef,
} from '@backstage/plugin-catalog-react';
import { CatalogClientWrapper } from './CatalogClientWrapper';
import { createComponentRouteRef } from './routes';
import { createComponentRouteRef, viewTechDocRouteRef } from './routes';
import {
createApiFactory,
createComponentExtension,
@@ -50,6 +50,7 @@ export const catalogPlugin = createPlugin({
},
externalRoutes: {
createComponent: createComponentRouteRef,
viewTechDoc: viewTechDocRouteRef,
},
});
plugins/catalog/src/routes.ts
+6
View File
@@ -20,3 +20,9 @@ export const createComponentRouteRef = createExternalRouteRef({
id: 'create-component',
optional: true,
});
export const viewTechDocRouteRef = createExternalRouteRef({
id: 'view-techdoc',
optional: true,
params: ['namespace', 'kind', 'name'],
});
plugins/techdocs-backend/config.d.ts
Vendored
+10
View File
@@ -246,5 +246,15 @@ export interface Config {
* @deprecated
*/
storageUrl?: string;
/**
* (Optional and not recommended) Prior to version [0.x.y] of TechDocs, docs
* sites could only be accessed over paths with case-sensitive entity triplets
* e.g. (namespace/Kind/name). If you are upgrading from an older version of
* TechDocs and are unable to perform the necessary migration of files in your
* external storage, you can set this value to `true` to temporarily revert to
* the old, case-sensitive entity triplet behavior.
*/
legacyUseCaseSensitiveTripletPaths?: boolean;
};
}
plugins/techdocs/api-report.md
+6 -1
View File
@@ -271,6 +271,11 @@ export const TechDocsPicker: () => null;
const techdocsPlugin: BackstagePlugin<
{
root: RouteRef<undefined>;
docRoot: RouteRef<{
name: string;
kind: string;
namespace: string;
}>;
entityContent: RouteRef<undefined>;
},
{}
@@ -374,7 +379,7 @@ export class TechDocsStorageClient implements TechDocsStorageApi {
//
// src/home/components/EntityListDocsTable.d.ts:11:5 - (ae-forgotten-export) The symbol "columnFactories" needs to be exported by the entry point index.d.ts
// src/home/components/EntityListDocsTable.d.ts:12:5 - (ae-forgotten-export) The symbol "actionFactories" needs to be exported by the entry point index.d.ts
// src/plugin.d.ts:24:5 - (ae-forgotten-export) The symbol "TabsConfig" needs to be exported by the entry point index.d.ts
// src/plugin.d.ts:29:5 - (ae-forgotten-export) The symbol "TabsConfig" needs to be exported by the entry point index.d.ts
// (No @packageDocumentation comment for this package)
```
plugins/techdocs/config.d.ts
Vendored
+7
View File
@@ -26,6 +26,13 @@ export interface Config {
*/
builder: 'local' | 'external';
/**
* Allows fallback to case-sensitive triplets in case of migration issues.
* @visibility frontend
* @see https://backstage.io/docs/features/techdocs/how-to-guides#how-to-migrate-from-techdocs-alpha-to-beta
*/
legacyUseCaseSensitiveTripletPaths?: boolean;
/**
* @example http://localhost:7000/api/techdocs
* @visibility frontend
plugins/techdocs/src/Router.tsx
+3 -11
View File
@@ -18,11 +18,6 @@ import React from 'react';
import { Entity } from '@backstage/catalog-model';
import { useEntity } from '@backstage/plugin-catalog-react';
import { Route, Routes } from 'react-router-dom';
import {
rootRouteRef,
rootDocsRouteRef,
rootCatalogDocsRouteRef,
} from './routes';
import { TechDocsIndexPage } from './home/components/TechDocsIndexPage';
import { TechDocsPage as TechDocsReaderPage } from './reader/components/TechDocsPage';
import { EntityPageDocs } from './EntityPageDocs';
@@ -33,9 +28,9 @@ const TECHDOCS_ANNOTATION = 'backstage.io/techdocs-ref';
export const Router = () => {
return (
<Routes>
<Route path={`/${rootRouteRef.path}`} element={<TechDocsIndexPage />} />
<Route path="/" element={<TechDocsIndexPage />} />
<Route
path={`/${rootDocsRouteRef.path}`}
path="/:namespace/:kind/:name/*"
element={<TechDocsReaderPage />}
/>
</Routes>
@@ -58,10 +53,7 @@ export const EmbeddedDocsRouter = (_props: Props) => {
return (
<Routes>
<Route
path={`/${rootCatalogDocsRouteRef.path}`}
element={<EntityPageDocs entity={entity} />}
/>
<Route path="/*" element={<EntityPageDocs entity={entity} />} />
</Routes>
);
};
plugins/techdocs/src/home/components/DefaultTechDocsHome.test.tsx
+6
View File
@@ -30,6 +30,7 @@ import {
configApiRef,
storageApiRef,
} from '@backstage/core-plugin-api';
import { rootDocsRouteRef } from '../../routes';
jest.mock('@backstage/plugin-catalog-react', () => {
const actual = jest.requireActual('@backstage/plugin-catalog-react');
@@ -73,6 +74,11 @@ describe('TechDocs Home', () => {
<ApiProvider apis={apiRegistry}>
<DefaultTechDocsHome />
</ApiProvider>,
{
mountedRoutes: {
'/docs/:namespace/:kind/:name/*': rootDocsRouteRef,
},
},
);
// Header
plugins/techdocs/src/home/components/DocsCardGrid.test.tsx
+75 -1
View File
@@ -17,11 +17,36 @@
import { wrapInTestApp } from '@backstage/test-utils';
import { render } from '@testing-library/react';
import React from 'react';
import { configApiRef } from '@backstage/core-plugin-api';
import { DocsCardGrid } from './DocsCardGrid';
import { rootDocsRouteRef } from '../../routes';
// Hacky way to mock a specific boolean config value.
const getOptionalBooleanMock = jest.fn().mockReturnValue(false);
jest.mock('@backstage/core-plugin-api', () => ({
...jest.requireActual('@backstage/core-plugin-api'),
useApi: (apiRef: any) => {
const actualUseApi = jest.requireActual(
'@backstage/core-plugin-api',
).useApi;
const actualApi = actualUseApi(apiRef);
if (apiRef === configApiRef) {
const configReader = actualApi;
configReader.getOptionalBoolean = getOptionalBooleanMock;
return configReader;
}
return actualApi;
},
}));
describe('Entity Docs Card Grid', () => {
beforeEach(() => {
jest.resetAllMocks();
});
it('should render all entities passed ot it', async () => {
const { findByText } = render(
const { findByText, findAllByRole } = render(
wrapInTestApp(
<DocsCardGrid
entities={[
@@ -47,9 +72,58 @@ describe('Entity Docs Card Grid', () => {
},
]}
/>,
{
mountedRoutes: {
'/docs/:namespace/:kind/:name/*': rootDocsRouteRef,
},
},
),
);
expect(await findByText('testName')).toBeInTheDocument();
expect(await findByText('testName2')).toBeInTheDocument();
const [button1, button2] = await findAllByRole('button');
expect(button1.getAttribute('href')).toContain(
'/docs/default/testkind/testname',
);
expect(button2.getAttribute('href')).toContain(
'/docs/default/testkind2/testname2',
);
});
it('should fall back to case-sensitive links when configured', async () => {
getOptionalBooleanMock.mockReturnValue(true);
const { findByRole } = render(
wrapInTestApp(
<DocsCardGrid
entities={[
{
apiVersion: 'version',
kind: 'TestKind',
metadata: {
name: 'testName',
namespace: 'SomeNamespace',
},
spec: {
owner: 'techdocs@example.com',
},
},
]}
/>,
{
mountedRoutes: {
'/techdocs/:namespace/:kind/:name/*': rootDocsRouteRef,
},
},
),
);
const button = await findByRole('button');
expect(getOptionalBooleanMock).toHaveBeenCalledWith(
'techdocs.legacyUseCaseSensitiveTripletPaths',
);
expect(button.getAttribute('href')).toContain(
'/techdocs/SomeNamespace/TestKind/testName',
);
});
});
plugins/techdocs/src/home/components/DocsCardGrid.tsx
+16 -5
View File
@@ -15,9 +15,9 @@
*/
import React from 'react';
import { generatePath } from 'react-router-dom';
import { Entity } from '@backstage/catalog-model';
import { configApiRef, useApi, useRouteRef } from '@backstage/core-plugin-api';
import { Card, CardActions, CardContent, CardMedia } from '@material-ui/core';
import { rootDocsRouteRef } from '../../routes';
@@ -32,6 +32,15 @@ export const DocsCardGrid = ({
}: {
entities: Entity[] | undefined;
}) => {
const getRouteToReaderPageFor = useRouteRef(rootDocsRouteRef);
// Lower-case entity triplets by default, but allow override.
const toLowerMaybe = useApi(configApiRef).getOptionalBoolean(
'techdocs.legacyUseCaseSensitiveTripletPaths',
)
? (str: string) => str
: (str: string) => str.toLocaleLowerCase();
if (!entities) return null;
return (
<ItemCardGrid data-testid="docs-explore">
@@ -45,10 +54,12 @@ export const DocsCardGrid = ({
<CardContent>{entity.metadata.description}</CardContent>
<CardActions>
<Button
to={generatePath(rootDocsRouteRef.path, {
namespace: entity.metadata.namespace ?? 'default',
kind: entity.kind,
name: entity.metadata.name,
to={getRouteToReaderPageFor({
namespace: toLowerMaybe(
entity.metadata.namespace ?? 'default',
),
kind: toLowerMaybe(entity.kind),
name: toLowerMaybe(entity.metadata.name),
})}
color="primary"
>
plugins/techdocs/src/home/components/DocsTable.test.tsx
+94 -3
View File
@@ -16,9 +16,34 @@
import React from 'react';
import { render } from '@testing-library/react';
import { wrapInTestApp } from '@backstage/test-utils';
import { configApiRef } from '@backstage/core-plugin-api';
import { DocsTable } from './DocsTable';
import { rootDocsRouteRef } from '../../routes';
// Hacky way to mock a specific boolean config value.
const getOptionalBooleanMock = jest.fn().mockReturnValue(false);
jest.mock('@backstage/core-plugin-api', () => ({
...jest.requireActual('@backstage/core-plugin-api'),
useApi: (apiRef: any) => {
const actualUseApi = jest.requireActual(
'@backstage/core-plugin-api',
).useApi;
const actualApi = actualUseApi(apiRef);
if (apiRef === configApiRef) {
const configReader = actualApi;
configReader.getOptionalBoolean = getOptionalBooleanMock;
return configReader;
}
return actualApi;
},
}));
describe('DocsTable test', () => {
beforeEach(() => {
jest.resetAllMocks();
});
it('should render documents passed', async () => {
const { findByText } = render(
wrapInTestApp(
@@ -66,15 +91,81 @@ describe('DocsTable test', () => {
},
]}
/>,
{
mountedRoutes: {
'/docs/:namespace/:kind/:name/*': rootDocsRouteRef,
},
},
),
);
expect(await findByText('testName')).toBeInTheDocument();
expect(await findByText('testName2')).toBeInTheDocument();
const link1 = await findByText('testName');
const link2 = await findByText('testName2');
expect(link1).toBeInTheDocument();
expect(link1.getAttribute('href')).toContain(
'/docs/default/testkind/testname',
);
expect(link2).toBeInTheDocument();
expect(link2.getAttribute('href')).toContain(
'/docs/default/testkind2/testname2',
);
});
it('should fall back to case-sensitive links when configured', async () => {
getOptionalBooleanMock.mockReturnValue(true);
const { findByText } = render(
wrapInTestApp(
<DocsTable
entities={[
{
apiVersion: 'version',
kind: 'TestKind',
metadata: {
name: 'testName',
namespace: 'SomeNamespace',
},
spec: {
owner: 'user:owned',
},
relations: [
{
target: {
kind: 'user',
namespace: 'default',
name: 'owned',
},
type: 'ownedBy',
},
],
},
]}
/>,
{
mountedRoutes: {
'/techdocs/:namespace/:kind/:name/*': rootDocsRouteRef,
},
},
),
);
const button = await findByText('testName');
expect(getOptionalBooleanMock).toHaveBeenCalledWith(
'techdocs.legacyUseCaseSensitiveTripletPaths',
);
expect(button.getAttribute('href')).toContain(
'/techdocs/SomeNamespace/TestKind/testName',
);
});
it('should render empty state if no owned documents exist', async () => {
const { findByText } = render(wrapInTestApp(<DocsTable entities={[]} />));
const { findByText } = render(
wrapInTestApp(<DocsTable entities={[]} />, {
mountedRoutes: {
'/docs/:namespace/:kind/:name/*': rootDocsRouteRef,
},
}),
);
expect(await findByText('No documents to show')).toBeInTheDocument();
});
plugins/techdocs/src/home/components/DocsTable.tsx
+13 -5
View File
@@ -16,8 +16,8 @@
import React from 'react';
import { useCopyToClipboard } from 'react-use';
import { generatePath } from 'react-router-dom';
import { configApiRef, useApi, useRouteRef } from '@backstage/core-plugin-api';
import { Entity, RELATION_OWNED_BY } from '@backstage/catalog-model';
import {
formatEntityRefTitle,
@@ -49,6 +49,14 @@ export const DocsTable = ({
actions?: TableProps<DocsTableRow>['actions'];
}) => {
const [, copyToClipboard] = useCopyToClipboard();
const getRouteToReaderPageFor = useRouteRef(rootDocsRouteRef);
// Lower-case entity triplets by default, but allow override.
const toLowerMaybe = useApi(configApiRef).getOptionalBoolean(
'techdocs.legacyUseCaseSensitiveTripletPaths',
)
? (str: string) => str
: (str: string) => str.toLocaleLowerCase();
if (!entities) return null;
@@ -58,10 +66,10 @@ export const DocsTable = ({
return {
entity,
resolved: {
docsUrl: generatePath(rootDocsRouteRef.path, {
namespace: entity.metadata.namespace ?? 'default',
kind: entity.kind,
name: entity.metadata.name,
docsUrl: getRouteToReaderPageFor({
namespace: toLowerMaybe(entity.metadata.namespace ?? 'default'),
kind: toLowerMaybe(entity.kind),
name: toLowerMaybe(entity.metadata.name),
}),
ownedByRelations,
ownedByRelationsTitle: ownedByRelations
plugins/techdocs/src/home/components/LegacyTechDocsHome.test.tsx
+6
View File
@@ -26,6 +26,7 @@ import {
ConfigReader,
} from '@backstage/core-app-api';
import { ConfigApi, configApiRef } from '@backstage/core-plugin-api';
import { rootDocsRouteRef } from '../../routes';
jest.mock('@backstage/plugin-catalog-react', () => {
const actual = jest.requireActual('@backstage/plugin-catalog-react');
@@ -68,6 +69,11 @@ describe('Legacy TechDocs Home', () => {
<ApiProvider apis={apiRegistry}>
<LegacyTechDocsHome />
</ApiProvider>,
{
mountedRoutes: {
'/docs/:namespace/:kind/:name/*': rootDocsRouteRef,
},
},
);
// Header
plugins/techdocs/src/home/components/TechDocsCustomHome.test.tsx
+6
View File
@@ -20,6 +20,7 @@ import { screen } from '@testing-library/react';
import React from 'react';
import { TechDocsCustomHome, PanelType } from './TechDocsCustomHome';
import { ApiProvider, ApiRegistry } from '@backstage/core-app-api';
import { rootDocsRouteRef } from '../../routes';
jest.mock('@backstage/plugin-catalog-react', () => {
const actual = jest.requireActual('@backstage/plugin-catalog-react');
@@ -78,6 +79,11 @@ describe('TechDocsCustomHome', () => {
<ApiProvider apis={apiRegistry}>
<TechDocsCustomHome tabsConfig={tabsConfig} />
</ApiProvider>,
{
mountedRoutes: {
'/docs/:namespace/:kind/:name/*': rootDocsRouteRef,
},
},
);
// Header
plugins/techdocs/src/plugin.ts
+1
View File
@@ -65,6 +65,7 @@ export const techdocsPlugin = createPlugin({
],
routes: {
root: rootRouteRef,
docRoot: rootDocsRouteRef,
entityContent: rootCatalogDocsRouteRef,
},
});
plugins/techdocs/src/routes.ts
+4 -3
View File
@@ -17,16 +17,17 @@
import { createRouteRef } from '@backstage/core-plugin-api';
export const rootRouteRef = createRouteRef({
path: '',
id: 'techdocs-index-page',
title: 'TechDocs Landing Page',
});
export const rootDocsRouteRef = createRouteRef({
path: ':namespace/:kind/:name/*',
id: 'techdocs-reader-page',
title: 'Docs',
params: ['namespace', 'kind', 'name'],
});
export const rootCatalogDocsRouteRef = createRouteRef({
path: '*',
id: 'catalog-techdocs-reader-view',
title: 'Docs',
});