From ff176942c83b57bcac879214646175737794785a Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Wed, 28 Feb 2024 10:57:31 +0100 Subject: [PATCH 01/90] docs/frontend-system: starting point for app config docs Signed-off-by: Patrik Oldsberg --- .../02-configuring-extensions.md | 90 ++++++++++++++++++- 1 file changed, 89 insertions(+), 1 deletion(-) diff --git a/docs/frontend-system/building-apps/02-configuring-extensions.md b/docs/frontend-system/building-apps/02-configuring-extensions.md index 2fd4ce69fc..57a2110da6 100644 --- a/docs/frontend-system/building-apps/02-configuring-extensions.md +++ b/docs/frontend-system/building-apps/02-configuring-extensions.md @@ -6,4 +6,92 @@ sidebar_label: Configuring Extensions description: Documentation for how to configure extensions in a Backstage app --- -TODO +All extensions in a Backstage app can be configured through static configuration. This configuration is all done under a the `app.extensions` configuration key. For more general information on how to write configuration for Backstage, see the section on [writing configuration](../../conf/writing.md). + +## Extension Configuration Schema + +This section focuses on the format of the `app.extensions` configuration and the various shorthands that are available. + +The most complete and verbose format for configuring an individual extensions is as follows: + +```yaml +app: + extensions: + - : + attachTo: + id: + input: + disabled: + config: +``` + +All of the top-level fields are optional: `attachTo`, `disabled`, and `config`. Every extension implementation must provide defaults for all of these fields that will be used if they are not provided in the configuration. + +Note that `app.extensions` is always an array rather than an object. For example, the following is invalid: + +```yaml title="INVALID" +app: + extensions: + : # Invalid, this should be an array item, `app.extensions` is now an object + config: ... +``` + +In addition to this schema, there are a number of shorthands available: + +Rather than a full object, you can specify just the ID of the extension as a string. This is equivalent to setting `disabled` to `false`: + +```yaml +app: + extensions: + - ‘’ +``` + +You can enable/disable individual extension by ID, in this case the value is a boolean: + +```yaml +extensions: + - : +``` + +You can override the implementation of an extension by ID, in this case the value is a string: + +```yaml +extensions: + - : ‘’ +``` + +You can **create a new extension instance with a generated ID** by including an input name in the key: + +```yaml +extensions: + - /: + extension: + config: +``` + +This syntax is only for use in the app configuration itself, every extension provided by default from a plugin must have an explicit ID. For example, the following two configurations are equivalent, except that the former does not have an explicit instance ID: + +```yaml +extensions: + # Generated ID + - core.router/routes: + extension: '@backstage/plugin-tech-radar#TechRadarPage' + # Explicit ID + - tech-radar.page: + at: core.router/routes + extension: '@backstage/plugin-tech-radar#TechRadarPage' +``` + +Lastly, if you do not need to provide additional configuration, you can combine the key input format with the implementation value format as a shorthand for creating a new extension instance with a generated ID and no configuration: + +```yaml +extensions: + - /: ‘’ +``` + +For example: + +```yaml +extensions: + - core.router/routes: '@backstage/plugin-tech-radar#TechRadarPage' +``` From faf3fdf3ada5f922976baa0bcc82be7d2f69b00e Mon Sep 17 00:00:00 2001 From: Camila Belo Date: Wed, 28 Feb 2024 15:54:41 +0100 Subject: [PATCH 02/90] docs: add more examples to the frontend app migration guide Signed-off-by: Camila Belo --- .../building-apps/08-migrating.md | 116 +++++++++++++++++- 1 file changed, 113 insertions(+), 3 deletions(-) diff --git a/docs/frontend-system/building-apps/08-migrating.md b/docs/frontend-system/building-apps/08-migrating.md index 98d8b3bab8..8bcc1f973f 100644 --- a/docs/frontend-system/building-apps/08-migrating.md +++ b/docs/frontend-system/building-apps/08-migrating.md @@ -117,7 +117,7 @@ You can then also add any additional extensions that you may need to create as p [Utility API](../utility-apis/01-index.md) factories are now installed as extensions instead. Pass the existing factory to `createApiExtension` and install it in the app. For more information, see the section on [configuring Utility APIs](../utility-apis/04-configuring.md). -For example, the following API configuration: +For example, the following apis configuration: ```ts const app = createApp({ @@ -151,15 +151,75 @@ Icons are currently installed through the usual options to `createApp`, but will Plugins are now passed through the `features` options instead. +For example, the following plugins configuration: + +```tsx +import { homePlugin } from '@backstage/plugin-home'; + +createApp({ + // ... + plugins: [homePlugin], + // ... +}); +``` + +Can be converted to the following features configuration: + +```tsx +// plugins are now default exported via alpha subpath +import homePlugin from '@backstage/plugin-home/alpha'; + +createApp({ + // ... + features: [homePlugin], + // ... +}); +``` + +Plugins don't even have to be imported manually after installing their package if [features discovery](../architecture/02-app.md#feature-discovery) is enabled. + +```yaml title="in app-config.yaml" +app: + # Enabling plugin and override features discovery + experimental: 'all' +``` + ### `featureFlags` Declaring features flags in the app is no longer supported, move these declarations to the appropriate plugins instead. +For example, the following app feature flags configuration: + +```tsx +createApp({ + // ... + featureFlags: [ + { + pluginId: '', + name: 'tech-radar', + description: 'Enables the tech radar plugin', + }, + ], + // ... +}); +``` + +Can be converted to the following plugin configuration: + +```tsx +createPlugin({ + id: 'tech-radar', + // ... + featureFlags: [{ name: 'tech-radar' }], + // ... +}); +``` + ### `components` Many app components are now installed as extensions instead using `createComponentExtension`. See the section on [configuring app components](./01-index.md#configure-your-app) for more information. -The `Router` component is now a built-in extension that you can override using `createRouterExtension`. +The `Router` component is now a built-in extension that you can [override](../architecture/05-extension-overrides.md) using `createRouterExtension`. The Sign-in page is now installed as an extension using the `createSignInPageExtension` instead. @@ -277,6 +337,35 @@ const app = createApp({ Translations are now installed as extensions, using `createTranslationExtension`. +For example, the following translations configuration: + +```tsx +import { catalogTranslationRef } from '@backstage/plugin-catalog/alpha'; +createApp({ + // ... + __experimentalTranslations: { + resources: [ + createTranslationMessages({ + ref: catalogTranslationRef, + catalog_page_create_button_title: 'Create Software', + }), + ], + }, + // ... +}); +``` + +Can be converted to the following extension: + +```tsx +createTranslationExtension({ + resource: createTranslationMessages({ + ref: catalogTranslationRef, + catalog_page_create_button_title: 'Create Software', + }), +}); +``` + ## Gradual Migration After updating all `createApp` options as well as using `convertLegacyApp` to use your existing app structure, you should be able to start up the app and see that it still works. If that is not the case, make sure you read any error messages that you may see in the app as they can provide hints on what you need to fix. If you are still stuck, you can check if anyone else ran into the same issue in our [GitHub issues](https://github.com/backstage/backstage/issues), or ask for help in our [community Discord](https://discord.gg/backstage-687207715902193673). @@ -368,7 +457,7 @@ The entity pages are typically defined in `packages/app/src/components/catalog` New apps feature a built-in sidebar extension (`app/nav`) that will render all nav item extensions provided by plugins. This is a placeholder implementation and not intended as a long-term solution. In the future we will aim to provide a more flexible sidebar extension that allows for more customization out of the box. -Because the built-in sidebar is quite limited you may want to override the sidebar with your own custom implementation. To do so, use `createExtension` directly and refer to the [original sidebar implementation](https://github.com/backstage/backstage/blob/master/packages/frontend-app-api/src/extensions/AppNav.tsx). The following is an example of how to take your existing sidebar from the `Root` component that you typically find in `packages/app/src/components/Root.tsx`, and use it in an extension override: +Because the built-in sidebar is quite limited you may want to override the sidebar with your own custom implementation. To do so, use `createExtension` directly and refer to the [original sidebar implementation](https://github.com/backstage/backstage/blob/master/packages/frontend-app-api/src/extensions/AppNav.tsx). The following is an example of how to take your existing sidebar from the `Root` component that you typically find in `packages/app/src/components/Root.tsx`, and use it in an [extension override](../architecture/05-extension-overrides.md): ```tsx const nav = createExtension({ @@ -435,3 +524,24 @@ export default app.createRoot( ``` Any app root wrapper needs to be migrated to be an extension, using `createAppRootWrapperExtension`. Note that if you have multiple wrappers they must be completely independent of each other, i.e. the order in which they the appear in the React tree should not matter. If that is not the case then you should group them into a single wrapper. + +Here is an example converting the `CustomAppBarrier` into extension: + +```tsx +createApp({ + // ... + features: [ + createExtensionOverrides({ + extensions: [ + createAppRootWrapperExtension({ + name: 'CustomAppBarrier', + // Whenever your component uses legacy core packages, wrap it with "compatWrapper" + // e.g. props => compatWrapper() + Component: CustomAppBarrier, + }), + ], + }), + ], + // ... +}); +``` From e2e39faa710b09ed0fa6cfebd32987d1e1b39129 Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Fri, 1 Mar 2024 11:37:05 +0100 Subject: [PATCH 03/90] docs/frontend-system: removed outdated content from app config section Signed-off-by: Patrik Oldsberg --- .../02-configuring-extensions.md | 48 ++----------------- 1 file changed, 3 insertions(+), 45 deletions(-) diff --git a/docs/frontend-system/building-apps/02-configuring-extensions.md b/docs/frontend-system/building-apps/02-configuring-extensions.md index 57a2110da6..5f090e23cd 100644 --- a/docs/frontend-system/building-apps/02-configuring-extensions.md +++ b/docs/frontend-system/building-apps/02-configuring-extensions.md @@ -49,49 +49,7 @@ app: You can enable/disable individual extension by ID, in this case the value is a boolean: ```yaml -extensions: - - : -``` - -You can override the implementation of an extension by ID, in this case the value is a string: - -```yaml -extensions: - - : ‘’ -``` - -You can **create a new extension instance with a generated ID** by including an input name in the key: - -```yaml -extensions: - - /: - extension: - config: -``` - -This syntax is only for use in the app configuration itself, every extension provided by default from a plugin must have an explicit ID. For example, the following two configurations are equivalent, except that the former does not have an explicit instance ID: - -```yaml -extensions: - # Generated ID - - core.router/routes: - extension: '@backstage/plugin-tech-radar#TechRadarPage' - # Explicit ID - - tech-radar.page: - at: core.router/routes - extension: '@backstage/plugin-tech-radar#TechRadarPage' -``` - -Lastly, if you do not need to provide additional configuration, you can combine the key input format with the implementation value format as a shorthand for creating a new extension instance with a generated ID and no configuration: - -```yaml -extensions: - - /: ‘’ -``` - -For example: - -```yaml -extensions: - - core.router/routes: '@backstage/plugin-tech-radar#TechRadarPage' +app: + extensions: + - : ``` From 665d118422bf3d61e453bf566f0a3052187cf134 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 14:05:23 -0500 Subject: [PATCH 04/90] feat(openapi-tooling): add breaking changes checks to the verify command Signed-off-by: aramissennyeydd --- packages/repo-tools/cli-report.md | 3 +- packages/repo-tools/src/commands/index.ts | 6 ++- .../commands/repo/schema/openapi/verify.ts | 37 +++++++++++++++++-- 3 files changed, 41 insertions(+), 5 deletions(-) diff --git a/packages/repo-tools/cli-report.md b/packages/repo-tools/cli-report.md index 9d37ce7aff..7c7dd445bc 100644 --- a/packages/repo-tools/cli-report.md +++ b/packages/repo-tools/cli-report.md @@ -168,7 +168,7 @@ Options: -h, --help Commands: - verify [paths...] + verify [options] [paths...] lint [options] [paths...] test [options] [paths...] fuzz [options] @@ -211,6 +211,7 @@ Options: Usage: backstage-repo-tools repo schema openapi verify [options] [paths...] Options: + --from -h, --help ``` diff --git a/packages/repo-tools/src/commands/index.ts b/packages/repo-tools/src/commands/index.ts index d2fe507941..9f0461e7ea 100644 --- a/packages/repo-tools/src/commands/index.ts +++ b/packages/repo-tools/src/commands/index.ts @@ -96,7 +96,11 @@ function registerRepoCommand(program: Command) { openApiCommand .command('verify [paths...]') .description( - 'Verify that all OpenAPI schemas are valid and have a matching `schemas/openapi.generated.ts` file.', + 'Verify that all OpenAPI schemas are valid and set up correctly. This also verifies that your API has not changed in a breaking way.', + ) + .option( + '--from ', + 'The base ref to compare against. Defaults to the fork point of the current branch.', ) .action( lazy(() => diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts b/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts index 442494610c..78b5a966c3 100644 --- a/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts +++ b/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts @@ -29,8 +29,10 @@ import { YAML_SCHEMA_PATH, } from '../../../../lib/openapi/constants'; import { getPathToOpenApiSpec } from '../../../../lib/openapi/helpers'; +import { exec } from '../../../../lib/exec'; +import { OptionValues } from 'commander'; -async function verify(directoryPath: string) { +async function verify(directoryPath: string, options: OptionValues) { let openapiPath = ''; try { openapiPath = await getPathToOpenApiSpec(directoryPath); @@ -58,10 +60,39 @@ async function verify(directoryPath: string) { `\`${YAML_SCHEMA_PATH}\` and \`${TS_SCHEMA_PATH}\` do not match. Please run \`yarn backstage-repo-tools package schema openapi generate\` from '${path}' to regenerate \`${TS_SCHEMA_PATH}\`.`, ); } + + let baseRef = options.from ?? process.env.GITHUB_BASE_REF; + if (!baseRef) { + const { stdout: branch } = await exec('git merge-base --fork-point HEAD'); + baseRef = branch.toString().trim(); + } + + try { + const { stdout } = await exec('optic diff', [ + openapiPath, + '--check', + '--base', + baseRef, + ]); + // Log out the results as this still shows API changes that aren't breakages. + console.log( + stdout + .toString() + .split('\n') + .filter(e => !e.startsWith('Rerun') && e.trim()) + .join('\n'), + ); + } catch (err) { + err.message = err.stdout; + throw err; + } } -export async function bulkCommand(paths: string[] = []): Promise { - const resultsList = await runner(paths, dir => verify(dir)); +export async function bulkCommand( + paths: string[] = [], + options: OptionValues, +): Promise { + const resultsList = await runner(paths, dir => verify(dir, options)); let failed = false; for (const { relativeDir, resultText } of resultsList) { From 0b3fac608a9cb8052ad6644380add93c8116b72f Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 14:05:30 -0500 Subject: [PATCH 05/90] example breakage Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- plugins/catalog-backend/src/schema/openapi.generated.ts | 2 +- plugins/catalog-backend/src/schema/openapi.yaml | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/plugins/catalog-backend/src/schema/openapi.generated.ts b/plugins/catalog-backend/src/schema/openapi.generated.ts index 550db58f2e..c8c9c73a84 100644 --- a/plugins/catalog-backend/src/schema/openapi.generated.ts +++ b/plugins/catalog-backend/src/schema/openapi.generated.ts @@ -1072,7 +1072,7 @@ export const spec = { 'application/json': { schema: { type: 'object', - required: ['entityRefs'], + required: ['entityRefs', 'fields'], properties: { entityRefs: { type: 'array', diff --git a/plugins/catalog-backend/src/schema/openapi.yaml b/plugins/catalog-backend/src/schema/openapi.yaml index 75dc1d9bbc..d3a65490bf 100644 --- a/plugins/catalog-backend/src/schema/openapi.yaml +++ b/plugins/catalog-backend/src/schema/openapi.yaml @@ -841,6 +841,7 @@ paths: type: object required: - entityRefs + - fields properties: entityRefs: type: array From 7cd15860dc065afd44d1d56e4eb498f7093b7693 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 16:13:03 -0500 Subject: [PATCH 06/90] testing using the uffizzi workflow Signed-off-by: aramissennyeydd --- .../api-breaking-changes-comment.yml | 111 ++++++++ .github/workflows/api-breaking-changes.yml | 62 +++++ NOTICE | 1 + packages/repo-tools/package.json | 1 + packages/repo-tools/src/commands/index.ts | 28 +- .../commands/package/schema/openapi/check.ts | 93 +++++++ .../src/commands/repo/schema/openapi/check.ts | 80 ++++++ .../commands/repo/schema/openapi/verify.ts | 56 ++-- .../src/lib/openapi/optic/helpers.ts | 244 ++++++++++++++++++ plugins/catalog-backend/package.json | 1 + yarn.lock | 36 +++ 11 files changed, 670 insertions(+), 43 deletions(-) create mode 100644 .github/workflows/api-breaking-changes-comment.yml create mode 100644 .github/workflows/api-breaking-changes.yml create mode 100644 packages/repo-tools/src/commands/package/schema/openapi/check.ts create mode 100644 packages/repo-tools/src/commands/repo/schema/openapi/check.ts create mode 100644 packages/repo-tools/src/lib/openapi/optic/helpers.ts diff --git a/.github/workflows/api-breaking-changes-comment.yml b/.github/workflows/api-breaking-changes-comment.yml new file mode 100644 index 0000000000..91741bba5b --- /dev/null +++ b/.github/workflows/api-breaking-changes-comment.yml @@ -0,0 +1,111 @@ +name: API Breaking Changes (comment) + +on: + workflow_run: + workflows: + - 'API Breaking Changes (Trigger)' + types: + - completed + +jobs: + setup: + name: Add values from previous step + runs-on: ubuntu-latest + if: ${{ github.event.workflow_run.conclusion == 'success' }} + permissions: + # "If you specify the access for any of these scopes, all of those that are not specified are set to none." + # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions + actions: read # Access cache + outputs: + git-ref: ${{ steps.event.outputs.GIT_REF }} + pr-number: ${{ steps.event.outputs.PR_NUMBER }} + action: ${{ steps.event.outputs.ACTION }} + steps: + - name: Harden Runner + uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0 + with: + disable-sudo: true + egress-policy: block + allowed-endpoints: > + api.github.com:443 + + - name: 'Download artifacts' + # Fetch output (zip archive) from the workflow run that triggered this workflow. + uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 + with: + script: | + let allArtifacts = await github.rest.actions.listWorkflowRunArtifacts({ + owner: context.repo.owner, + repo: context.repo.repo, + run_id: context.payload.workflow_run.id, + }); + let matchArtifact = allArtifacts.data.artifacts.filter((artifact) => { + return artifact.name == "preview-spec" + })[0]; + if (matchArtifact === undefined) { + throw TypeError('Build Artifact not found!'); + } + let download = await github.rest.actions.downloadArtifact({ + owner: context.repo.owner, + repo: context.repo.repo, + artifact_id: matchArtifact.id, + archive_format: 'zip', + }); + let fs = require('fs'); + fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/preview-spec.zip`, Buffer.from(download.data)); + + - name: 'Accept event from first stage' + run: unzip preview-spec.zip event.json + + - name: Read Event into ENV + id: event + run: | + echo PR_NUMBER=$(jq '.number | tonumber' < event.json) >> $GITHUB_OUTPUT + echo ACTION=$(jq --raw-output '.action | tostring | [scan("\\w+")][0]' < event.json) >> $GITHUB_OUTPUT + echo GIT_REF=$(jq --raw-output '.pull_request.head.sha | tostring | [scan("\\w+")][0]' < event.json) >> $GITHUB_OUTPUT + + - name: DEBUG - Print Job Outputs + if: ${{ runner.debug }} + run: | + echo "PR number: ${{ steps.event.outputs.PR_NUMBER }}" + echo "Git Ref: ${{ steps.event.outputs.GIT_REF }}" + echo "Action: ${{ steps.event.outputs.ACTION }}" + cat event.json + + - name: Get Comment + id: get-comment + run: | + unzip preview-spec.zip comment.md + ls + echo "MANIFESTS_FILE_HASH=$(md5sum manifests.rendered.yml | awk '{ print $1 }')" >> $GITHUB_OUTPUT + + add-comment: + name: Write comment about issues + needs: + - setup + if: ${{ github.event.workflow_run.conclusion == 'success' }} + permissions: + contents: read + pull-requests: write + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4 + + # Identify comment to be updated + - name: Find comment for Ephemeral Environment + uses: peter-evans/find-comment@d5fe37641ad8451bdd80312415672ba26c86575e # v3 + id: find-comment + with: + issue-number: ${{ needs.cache-manifests-file.outputs.pr-number }} + comment-author: 'github-actions[bot]' + body-includes: pr-changes-${{ needs.cache-manifests-file.outputs.pr-number }} + direction: last + + - name: Create or Update Comment with Deployment URL + uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4 + with: + comment-id: ${{ steps.notification.outputs.comment-id }} + issue-number: ${{ github.event.pull_request.number }} + body-path: comment.md + edit-mode: replace diff --git a/.github/workflows/api-breaking-changes.yml b/.github/workflows/api-breaking-changes.yml new file mode 100644 index 0000000000..ed2dcd9a0e --- /dev/null +++ b/.github/workflows/api-breaking-changes.yml @@ -0,0 +1,62 @@ +name: API Breaking Changes (Trigger) +on: + pull_request: + types: [opened, synchronize, reopened, closed] + paths-ignore: + - '.changeset/**' + - 'contrib/**' + - 'docs/**' + - 'microsite/**' + - 'beps/**' + - 'scripts/**' + - 'storybook/**' + - '**/*.test.*' + - '**/package.json' + - '*.md' + +jobs: + get-backstage-changes: + env: + NODE_OPTIONS: --max-old-space-size=4096 + name: Build PR image + runs-on: ubuntu-latest + if: ${{ github.event_name != 'pull_request' || github.event.action != 'closed' }} + outputs: + tags: ${{ steps.meta.outputs.tags }} + steps: + - name: Harden Runner + uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0 + with: + egress-policy: audit + + - name: checkout + uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 + + - name: setup-node + uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 + with: + node-version: 18.x + registry-url: https://registry.npmjs.org/ + + - name: yarn install + uses: backstage/actions/yarn-install@a674369920067381b450d398b27df7039b7ef635 # v0.6.5 + with: + cache-prefix: linux-v18 + + - name: breaking changes check + run: | + yarn backstage-repo-tools repo schema openapi check > comment.md + + - name: Upload Rendered Comment as Artifact + uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3 + with: + name: preview-spec + path: comment.md + retention-days: 2 + + - name: Upload PR Event as Artifact + uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3 + with: + name: preview-spec + path: ${{ github.event_path }} + retention-days: 2 diff --git a/NOTICE b/NOTICE index fb23d28ebc..1f1dfbccbb 100644 --- a/NOTICE +++ b/NOTICE @@ -5,3 +5,4 @@ Portions of this software were developed by third-party software vendors: - Tech Radar Plugin (https://opensource.zalando.com/tech-radar/), Copyright (c) 2017 Zalando SE - [OpenAPI Generator Templates](./packages/repo-tools/templates), Copyright 2018 OpenAPI-Generator Contributors (https://openapi-generator.tech) Copyright 2018 SmartBear Software +- Optic CLI (https://github.com/opticdev/optic), Copyright 2022, Optic Labs Corporation diff --git a/packages/repo-tools/package.json b/packages/repo-tools/package.json index abf134b5be..126a392265 100644 --- a/packages/repo-tools/package.json +++ b/packages/repo-tools/package.json @@ -49,6 +49,7 @@ "@stoplight/spectral-rulesets": "^1.18.0", "@stoplight/spectral-runtime": "^1.1.2", "@stoplight/types": "^14.0.0", + "@useoptic/openapi-utilities": "^0.54.8", "chalk": "^4.0.0", "codeowners-utils": "^1.0.2", "command-exists": "^1.2.9", diff --git a/packages/repo-tools/src/commands/index.ts b/packages/repo-tools/src/commands/index.ts index 9f0461e7ea..6a557d909e 100644 --- a/packages/repo-tools/src/commands/index.ts +++ b/packages/repo-tools/src/commands/index.ts @@ -78,6 +78,14 @@ function registerPackageCommand(program: Command) { .action( lazy(() => import('./package/schema/openapi/fuzz').then(m => m.command)), ); + + openApiCommand + .command('check') + .option('--ignore', 'Ignore linting failures and only log the results.') + .option('--json', 'Output the results as JSON') + .action( + lazy(() => import('./package/schema/openapi/check').then(m => m.command)), + ); } function registerRepoCommand(program: Command) { @@ -96,11 +104,7 @@ function registerRepoCommand(program: Command) { openApiCommand .command('verify [paths...]') .description( - 'Verify that all OpenAPI schemas are valid and set up correctly. This also verifies that your API has not changed in a breaking way.', - ) - .option( - '--from ', - 'The base ref to compare against. Defaults to the fork point of the current branch.', + 'Verify that all OpenAPI schemas are valid and set up correctly.', ) .action( lazy(() => @@ -137,6 +141,20 @@ function registerRepoCommand(program: Command) { .action( lazy(() => import('./repo/schema/openapi/fuzz').then(m => m.command)), ); + + openApiCommand + .command('check') + .description( + 'Check the repository against a specific ref, will run all package `check:api` scripts.', + ) + .option( + '--since ', + 'Check the API against a specific ref', + 'origin/master', + ) + .action( + lazy(() => import('./repo/schema/openapi/check').then(m => m.command)), + ); } export function registerCommands(program: Command) { diff --git a/packages/repo-tools/src/commands/package/schema/openapi/check.ts b/packages/repo-tools/src/commands/package/schema/openapi/check.ts new file mode 100644 index 0000000000..22d2d0a0d6 --- /dev/null +++ b/packages/repo-tools/src/commands/package/schema/openapi/check.ts @@ -0,0 +1,93 @@ +/* + * Copyright 2023 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +import chalk from 'chalk'; +import { exec } from '../../../../lib/exec'; +import { getPathToCurrentOpenApiSpec } from '../../../../lib/openapi/helpers'; +import { paths as cliPaths } from '../../../../lib/paths'; +import { OptionValues } from 'commander'; +import { env } from 'process'; +import { readFile, rm } from 'fs/promises'; +import { resolve } from 'path'; + +const reduceOpticOutput = (output: string) => { + return output + .split('\n') + .filter(e => !e.startsWith('Rerun') && e.trim()) + .join('\n'); +}; + +async function check(opts: OptionValues) { + const resolvedOpenapiPath = await getPathToCurrentOpenApiSpec(); + + let baseRef = opts.since ?? process.env.GITHUB_BASE_REF; + if (!baseRef) { + const { stdout: branch } = await exec( + 'git merge-base --fork-point origin/master', + ); + baseRef = branch.toString().trim(); + } + + let failed = false; + let output = ''; + try { + const { stdout } = await exec( + 'yarn optic diff', + [ + resolvedOpenapiPath, + '--check', + opts.json ? '--json' : '', + '--base', + baseRef, + ], + { + cwd: cliPaths.targetRoot, + env: { CI: opts.json ? '1' : undefined, ...env }, + }, + ); + output = stdout.toString(); + } catch (err) { + output = err.stdout; + failed = true; + } + + if (opts.json) { + const file = ( + await readFile(resolve(cliPaths.targetRoot, 'ci-run-details.json')) + ).toString(); + const results = JSON.parse(file); + console.log(file); + if (!opts.ignore && results.failed) { + throw new Error('Some checks failed'); + } + + await rm(resolve(cliPaths.targetRoot, 'ci-run-details.json')); + } else { + console.log(reduceOpticOutput(output)); + if (!opts.ignore && failed) { + throw new Error('Some checks failed'); + } + } +} + +export async function command(opts: OptionValues) { + try { + await check(opts); + if (!opts.json) console.log(chalk.green(`All checks passed.`)); + } catch (err) { + if (!opts.json) console.log(chalk.red(err.message)); + process.exit(1); + } +} diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts new file mode 100644 index 0000000000..f77586a840 --- /dev/null +++ b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts @@ -0,0 +1,80 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +import { PackageGraph } from '@backstage/cli-node'; +import { OptionValues } from 'commander'; +import { exec } from '../../../../lib/exec'; +import { + CiRunDetails, + generateCompareSummaryMarkdown, +} from '../../../../lib/openapi/optic/helpers'; + +export async function command(opts: OptionValues) { + let packages = await PackageGraph.listTargetPackages(); + if (opts.since) { + const graph = PackageGraph.fromPackages(packages); + const changedPackages = await graph.listChangedPackages({ + ref: opts.since, + analyzeLockfile: true, + }); + const withDevDependents = graph.collectPackageNames( + changedPackages.map(pkg => pkg.name), + pkg => pkg.localDevDependents.keys(), + ); + packages = Array.from(withDevDependents).map(name => graph.get(name)!); + } + + const checkablePackages = packages.filter( + e => e.packageJson.scripts?.['check:api'], + ); + try { + const outputs = { + completed: [], + failed: [], + noop: [], + severity: 0, + } as CiRunDetails; + for (const pkg of checkablePackages) { + const { stdout } = await exec( + 'yarn', + ['check:api', '--ignore', '--json'], + { + cwd: pkg.dir, + }, + ); + const result = JSON.parse(stdout.toString()); + outputs.completed.push(...(result.completed ?? [])); + outputs.failed.push(...(result.failed ?? [])); + outputs.noop.push(...(result.noop ?? [])); + } + + const { stdout: currentSha } = await exec('git', ['rev-parse', 'HEAD']); + console.log( + generateCompareSummaryMarkdown( + { sha: currentSha.toString().trim() }, + outputs, + { verbose: true }, + ), + ); + + const failed = outputs.failed.length > 0; + if (failed) { + throw new Error('Some checks failed'); + } + } catch (err) { + console.error(err); + process.exit(1); + } +} diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts b/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts index 78b5a966c3..2902d99ea4 100644 --- a/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts +++ b/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts @@ -29,17 +29,14 @@ import { YAML_SCHEMA_PATH, } from '../../../../lib/openapi/constants'; import { getPathToOpenApiSpec } from '../../../../lib/openapi/helpers'; -import { exec } from '../../../../lib/exec'; -import { OptionValues } from 'commander'; -async function verify(directoryPath: string, options: OptionValues) { - let openapiPath = ''; - try { - openapiPath = await getPathToOpenApiSpec(directoryPath); - } catch { - // Unable to find spec at path. - return; - } +const verifySpecAndGeneratedSpecMatch = async ( + openapiPath: string, + directoryPath: string, +) => { + const openapiTempDirectory = resolvePath(cliPaths.targetDir, '.openapi'); + await fs.mkdirp(openapiTempDirectory); + console.log(openapiTempDirectory); const yaml = YAML.load(await fs.readFile(openapiPath, 'utf8')); await Parser.validate(cloneDeep(yaml) as any); @@ -60,39 +57,22 @@ async function verify(directoryPath: string, options: OptionValues) { `\`${YAML_SCHEMA_PATH}\` and \`${TS_SCHEMA_PATH}\` do not match. Please run \`yarn backstage-repo-tools package schema openapi generate\` from '${path}' to regenerate \`${TS_SCHEMA_PATH}\`.`, ); } +}; - let baseRef = options.from ?? process.env.GITHUB_BASE_REF; - if (!baseRef) { - const { stdout: branch } = await exec('git merge-base --fork-point HEAD'); - baseRef = branch.toString().trim(); - } - +async function verify(directoryPath: string) { + let openapiPath = ''; try { - const { stdout } = await exec('optic diff', [ - openapiPath, - '--check', - '--base', - baseRef, - ]); - // Log out the results as this still shows API changes that aren't breakages. - console.log( - stdout - .toString() - .split('\n') - .filter(e => !e.startsWith('Rerun') && e.trim()) - .join('\n'), - ); - } catch (err) { - err.message = err.stdout; - throw err; + openapiPath = await getPathToOpenApiSpec(directoryPath); + } catch { + // Unable to find spec at path. + return; } + + await verifySpecAndGeneratedSpecMatch(openapiPath, directoryPath); } -export async function bulkCommand( - paths: string[] = [], - options: OptionValues, -): Promise { - const resultsList = await runner(paths, dir => verify(dir, options)); +export async function bulkCommand(paths: string[] = []): Promise { + const resultsList = await runner(paths, dir => verify(dir)); let failed = false; for (const { relativeDir, resultText } of resultsList) { diff --git a/packages/repo-tools/src/lib/openapi/optic/helpers.ts b/packages/repo-tools/src/lib/openapi/optic/helpers.ts new file mode 100644 index 0000000000..4c50f9bbd9 --- /dev/null +++ b/packages/repo-tools/src/lib/openapi/optic/helpers.ts @@ -0,0 +1,244 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +/* eslint-disable no-nested-ternary */ + +import { + compareSpecs, + groupDiffsByEndpoint, + Severity, + getOperationsChangedLabel, + getOperationsChanged, +} from '@useoptic/openapi-utilities'; +import { GroupedDiffs } from '@useoptic/openapi-utilities/build/openapi3/group-diff'; +import { relative } from 'path'; +import { paths as cliPaths } from '../../paths'; + +type Comparison = { + groupedDiffs: ReturnType; + results: Awaited>['results']; +}; + +export type CiRunDetails = { + completed: { + warnings: string[]; + apiName: string; + opticWebUrl?: string | null; + comparison: Comparison; + specUrl?: string | null; + capture?: any; + }[]; + failed: { apiName: string; error: string }[]; + noop: { apiName: string }[]; + severity: Severity; +}; + +const getChecksLabel = ( + results: CiRunDetails['completed'][number]['comparison']['results'], + severity: Severity, +) => { + const totalChecks = results.length; + let failingChecks = 0; + let exemptedFailingChecks = 0; + + for (const result of results) { + if (result.passed) continue; + if (result.severity < severity) continue; + if (result.exempted) exemptedFailingChecks += 1; + else failingChecks += 1; + } + + const exemptedChunk = + exemptedFailingChecks > 0 ? `, ${exemptedFailingChecks} exempted` : ''; + + return failingChecks > 0 + ? `⚠️ **${failingChecks}**/**${totalChecks}** failed${exemptedChunk}` + : totalChecks > 0 + ? `✅ **${totalChecks}** passed${exemptedChunk}` + : `ℹ️ No automated checks have run`; +}; + +function getOperationsText( + groupedDiffs: GroupedDiffs, + options: { webUrl?: string | null; verbose: boolean; labelJoiner?: string }, +) { + const ops = getOperationsChanged(groupedDiffs); + + const operationsText = options.verbose + ? [ + ...[...ops.added].map(o => `\`${o}\` (added)`), + ...[...ops.changed].map(o => `\`${o}\` (changed)`), + ...[...ops.removed].map(o => `\`${o}\` (removed)`), + ].join('\n') + : ''; + return `${getOperationsChangedLabel(groupedDiffs, { + joiner: options.labelJoiner, + })} + + ${operationsText} + `; +} + +const getCaptureIssuesLabel = ({ + unmatchedInteractions, + mismatchedEndpoints, +}: { + unmatchedInteractions: number; + mismatchedEndpoints: number; +}) => { + return [ + ...(unmatchedInteractions + ? [ + `🆕 ${unmatchedInteractions} undocumented path${ + unmatchedInteractions > 1 ? 's' : '' + }`, + ] + : []), + ...(mismatchedEndpoints + ? [ + `⚠️ ${mismatchedEndpoints} mismatch${ + mismatchedEndpoints > 1 ? 'es' : '' + }`, + ] + : []), + ].join('\n'); +}; + +export const generateCompareSummaryMarkdown = ( + commit: { sha: string }, + results: CiRunDetails, + options: { verbose: boolean }, +) => { + const anyCompletedHasWarning = results.completed.some( + s => s.warnings.length > 0, + ); + const anyCompletedHasCapture = results.completed.some(s => s.capture); + return ` + ${ + results.completed.length > 0 + ? `### APIs Changed + + + + + + + + ${anyCompletedHasWarning ? '' : ''} + ${anyCompletedHasCapture ? '' : ''} + + + + + ${results.completed + .map( + s => + ` + + + + + ${anyCompletedHasWarning ? `` : ''} + + ${ + anyCompletedHasCapture + ? ` + + + + ` + : '' + } + `, + ) + .join('\n')} + +
APIChangesRulesWarningsTests
+ + ${relative(cliPaths.targetDir, s.apiName)} + + + + ${getOperationsText(s.comparison.groupedDiffs, { + webUrl: s.opticWebUrl, + verbose: options.verbose, + labelJoiner: ',\n', + })} + + + + ${getChecksLabel(s.comparison.results, results.severity)} + + ${s.warnings.join('\n')} + + ${ + s.capture + ? s.capture.success + ? s.capture.mismatchedEndpoints || s.capture.unmatchedInteractions + ? getCaptureIssuesLabel({ + unmatchedInteractions: s.capture.unmatchedInteractions, + mismatchedEndpoints: s.capture.mismatchedEndpoints, + }) + : `✅ ${s.capture.percentCovered}% coverage` + : '❌ Failed to run' + : '' + } + +
+ ` + : '' + } + ${ + results.failed.length > 0 + ? `### Errors running optic + + + + + + + + + + ${results.failed + .map( + s => ` + + + `, + ) + .join('\n')} + +
APIError
${s.apiName} + + ${'```'} + ${s.error} + ${'```'} + +
+ ` + : '' + } + + Summary of API changes for commit (${commit.sha}) + + ${ + results.noop.length > 0 + ? `${ + results.noop.length === 1 ? '1 API' : `${results.noop.length} APIs` + } had no changes.` + : '' + }`; +}; diff --git a/plugins/catalog-backend/package.json b/plugins/catalog-backend/package.json index 0a20041ee4..058b9318d7 100644 --- a/plugins/catalog-backend/package.json +++ b/plugins/catalog-backend/package.json @@ -42,6 +42,7 @@ ], "scripts": { "build": "backstage-cli package build", + "check:api": "backstage-repo-tools package schema openapi check", "clean": "backstage-cli package clean", "fuzz": "backstage-repo-tools package schema openapi fuzz --exclude-checks response_schema_conformance", "generate": "backstage-repo-tools package schema openapi generate --server --client-package packages/catalog-client", diff --git a/yarn.lock b/yarn.lock index b7a8b0ce2c..b6a138814b 100644 --- a/yarn.lock +++ b/yarn.lock @@ -10108,6 +10108,7 @@ __metadata: "@types/is-glob": ^4.0.2 "@types/node": ^18.17.8 "@types/prettier": ^2.0.0 + "@useoptic/openapi-utilities": ^0.54.8 chalk: ^4.0.0 codeowners-utils: ^1.0.2 command-exists: ^1.2.9 @@ -20292,6 +20293,16 @@ __metadata: languageName: node linkType: hard +"@useoptic/json-pointer-helpers@npm:0.54.8": + version: 0.54.8 + resolution: "@useoptic/json-pointer-helpers@npm:0.54.8" + dependencies: + jsonpointer: ^5.0.1 + minimatch: 9.0.3 + checksum: 4eddabb6dce3ca8160dcd4904299b6964945c3fe47d39bfeca6c68b9a50b058b901a6fb10ab168295475d651df3349149faa5f27f77293e15b6eee8d4417432e + languageName: node + linkType: hard + "@useoptic/openapi-io@npm:0.50.10": version: 0.50.10 resolution: "@useoptic/openapi-io@npm:0.50.10" @@ -20346,6 +20357,31 @@ __metadata: languageName: node linkType: hard +"@useoptic/openapi-utilities@npm:^0.54.8": + version: 0.54.8 + resolution: "@useoptic/openapi-utilities@npm:0.54.8" + dependencies: + "@useoptic/json-pointer-helpers": 0.54.8 + ajv: ^8.6.0 + ajv-errors: ~3.0.0 + ajv-formats: ~2.1.0 + chalk: ^4.1.2 + fast-deep-equal: ^3.1.3 + is-url: ^1.2.4 + js-yaml: ^4.1.0 + json-stable-stringify: ^1.0.1 + lodash.groupby: ^4.6.0 + lodash.isequal: ^4.5.0 + lodash.omit: ^4.5.0 + node-machine-id: ^1.1.12 + openapi-types: ^12.0.2 + ts-invariant: ^0.9.3 + url-join: ^4.0.1 + yaml-ast-parser: ^0.0.43 + checksum: fa9e9f430c77687591aaf8b43b7b31a7c2f80fe9c140aaa978f1948f84d3e974181c91c3d8ec3e06efca9735c7826290baf4be72063bf733887aa632b40c3c4a + languageName: node + linkType: hard + "@useoptic/optic@npm:^0.50.10": version: 0.50.10 resolution: "@useoptic/optic@npm:0.50.10" From 7b06c6c78fbd398b1f70b5986448beb810e7ad3c Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 16:14:17 -0500 Subject: [PATCH 07/90] add attribution comment as well Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- packages/repo-tools/src/lib/openapi/optic/helpers.ts | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/packages/repo-tools/src/lib/openapi/optic/helpers.ts b/packages/repo-tools/src/lib/openapi/optic/helpers.ts index 4c50f9bbd9..cd2a1637d4 100644 --- a/packages/repo-tools/src/lib/openapi/optic/helpers.ts +++ b/packages/repo-tools/src/lib/openapi/optic/helpers.ts @@ -26,6 +26,11 @@ import { GroupedDiffs } from '@useoptic/openapi-utilities/build/openapi3/group-d import { relative } from 'path'; import { paths as cliPaths } from '../../paths'; +/** + * The below code is copied from https://github.com/opticdev/optic/blob/main/projects/optic/src/commands/ci/comment/common.ts#L82 for use + * with a security flow for forked repositories. + */ + type Comparison = { groupedDiffs: ReturnType; results: Awaited>['results']; From 53c1ec25e221b8d0bd4fb0ded8412f7788938315 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 16:14:52 -0500 Subject: [PATCH 08/90] update paths requirement Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- .github/workflows/api-breaking-changes.yml | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/.github/workflows/api-breaking-changes.yml b/.github/workflows/api-breaking-changes.yml index ed2dcd9a0e..c6227792c1 100644 --- a/.github/workflows/api-breaking-changes.yml +++ b/.github/workflows/api-breaking-changes.yml @@ -2,17 +2,8 @@ name: API Breaking Changes (Trigger) on: pull_request: types: [opened, synchronize, reopened, closed] - paths-ignore: - - '.changeset/**' - - 'contrib/**' - - 'docs/**' - - 'microsite/**' - - 'beps/**' - - 'scripts/**' - - 'storybook/**' - - '**/*.test.*' - - '**/package.json' - - '*.md' + paths: + - '**/openapi.yaml' jobs: get-backstage-changes: From fe4c26532bee4514f4cb171780cb4e9333b4fea9 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 16:22:00 -0500 Subject: [PATCH 09/90] update git command for GA env Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- packages/repo-tools/src/commands/index.ts | 1 + .../repo-tools/src/commands/package/schema/openapi/check.ts | 2 +- packages/repo-tools/src/commands/repo/schema/openapi/check.ts | 3 ++- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/packages/repo-tools/src/commands/index.ts b/packages/repo-tools/src/commands/index.ts index 6a557d909e..3fa16aa512 100644 --- a/packages/repo-tools/src/commands/index.ts +++ b/packages/repo-tools/src/commands/index.ts @@ -83,6 +83,7 @@ function registerPackageCommand(program: Command) { .command('check') .option('--ignore', 'Ignore linting failures and only log the results.') .option('--json', 'Output the results as JSON') + .option('--since ', 'Check the API against a specific ref') .action( lazy(() => import('./package/schema/openapi/check').then(m => m.command)), ); diff --git a/packages/repo-tools/src/commands/package/schema/openapi/check.ts b/packages/repo-tools/src/commands/package/schema/openapi/check.ts index 22d2d0a0d6..3cb0baf313 100644 --- a/packages/repo-tools/src/commands/package/schema/openapi/check.ts +++ b/packages/repo-tools/src/commands/package/schema/openapi/check.ts @@ -32,7 +32,7 @@ const reduceOpticOutput = (output: string) => { async function check(opts: OptionValues) { const resolvedOpenapiPath = await getPathToCurrentOpenApiSpec(); - let baseRef = opts.since ?? process.env.GITHUB_BASE_REF; + let baseRef = opts.since; if (!baseRef) { const { stdout: branch } = await exec( 'git merge-base --fork-point origin/master', diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts index f77586a840..41d0bec0e5 100644 --- a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts +++ b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts @@ -47,9 +47,10 @@ export async function command(opts: OptionValues) { severity: 0, } as CiRunDetails; for (const pkg of checkablePackages) { + const baseRef = opts.since ?? process.env.GITHUB_BASE_REF; const { stdout } = await exec( 'yarn', - ['check:api', '--ignore', '--json'], + ['check:api', '--ignore', '--json', '--since', baseRef], { cwd: pkg.dir, }, From b0b1371075e58308dcc7aaef45c6aa8809d12163 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 16:28:04 -0500 Subject: [PATCH 10/90] use base ref instead Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- .github/workflows/api-breaking-changes.yml | 2 +- packages/repo-tools/src/commands/repo/schema/openapi/check.ts | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/api-breaking-changes.yml b/.github/workflows/api-breaking-changes.yml index c6227792c1..fbe4764a9f 100644 --- a/.github/workflows/api-breaking-changes.yml +++ b/.github/workflows/api-breaking-changes.yml @@ -36,7 +36,7 @@ jobs: - name: breaking changes check run: | - yarn backstage-repo-tools repo schema openapi check > comment.md + yarn backstage-repo-tools repo schema openapi check --since ${{ github.base_ref }} > comment.md - name: Upload Rendered Comment as Artifact uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3 diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts index 41d0bec0e5..492dd3b5be 100644 --- a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts +++ b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts @@ -47,7 +47,7 @@ export async function command(opts: OptionValues) { severity: 0, } as CiRunDetails; for (const pkg of checkablePackages) { - const baseRef = opts.since ?? process.env.GITHUB_BASE_REF; + const baseRef = opts.since; const { stdout } = await exec( 'yarn', ['check:api', '--ignore', '--json', '--since', baseRef], From d1b44c9a4af18c380aa0d26884209250c25d0952 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 16:34:22 -0500 Subject: [PATCH 11/90] get the actual sha and run with that Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- .../repo-tools/src/commands/repo/schema/openapi/check.ts | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts index 492dd3b5be..5599937322 100644 --- a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts +++ b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts @@ -23,7 +23,11 @@ import { export async function command(opts: OptionValues) { let packages = await PackageGraph.listTargetPackages(); + + let since = ''; if (opts.since) { + const { stdout: sinceRaw } = await exec('git', ['rev-parse', opts.since]); + since = sinceRaw.toString().trim(); const graph = PackageGraph.fromPackages(packages); const changedPackages = await graph.listChangedPackages({ ref: opts.since, @@ -47,10 +51,10 @@ export async function command(opts: OptionValues) { severity: 0, } as CiRunDetails; for (const pkg of checkablePackages) { - const baseRef = opts.since; + const sinceCommands = since ? ['--since', since] : []; const { stdout } = await exec( 'yarn', - ['check:api', '--ignore', '--json', '--since', baseRef], + ['check:api', '--ignore', '--json', ...sinceCommands], { cwd: pkg.dir, }, From dce3d7870dd468a4e7e7daede7e915e1429a4e8f Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 16:38:23 -0500 Subject: [PATCH 12/90] actually check out the needed branches Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- .github/workflows/api-breaking-changes.yml | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/.github/workflows/api-breaking-changes.yml b/.github/workflows/api-breaking-changes.yml index fbe4764a9f..406205d578 100644 --- a/.github/workflows/api-breaking-changes.yml +++ b/.github/workflows/api-breaking-changes.yml @@ -20,8 +20,13 @@ jobs: with: egress-policy: audit - - name: checkout - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 + - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 + with: + # Fetch the commit that's merged into the base rather than the target ref + # This will let us diff only the contents of the PR, without fetching more history + ref: 'refs/pull/${{ github.event.pull_request.number }}/merge' + - name: fetch base + run: git fetch --depth 1 origin ${{ github.base_ref }} - name: setup-node uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 From e2c9b91fe899feee8308756cafc25af2c18327dd Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 16:41:06 -0500 Subject: [PATCH 13/90] add origin to base ref Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- .github/workflows/api-breaking-changes.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/api-breaking-changes.yml b/.github/workflows/api-breaking-changes.yml index 406205d578..5c75910546 100644 --- a/.github/workflows/api-breaking-changes.yml +++ b/.github/workflows/api-breaking-changes.yml @@ -41,7 +41,7 @@ jobs: - name: breaking changes check run: | - yarn backstage-repo-tools repo schema openapi check --since ${{ github.base_ref }} > comment.md + yarn backstage-repo-tools repo schema openapi check --since origin/${{ github.base_ref }} > comment.md - name: Upload Rendered Comment as Artifact uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3 From 8f681584e8f4b6cbc91732885f322823c819c6b6 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 16:48:23 -0500 Subject: [PATCH 14/90] update comment to show correctly Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- .../src/lib/openapi/optic/helpers.ts | 219 +++++++++--------- 1 file changed, 104 insertions(+), 115 deletions(-) diff --git a/packages/repo-tools/src/lib/openapi/optic/helpers.ts b/packages/repo-tools/src/lib/openapi/optic/helpers.ts index cd2a1637d4..9bcd8c45a3 100644 --- a/packages/repo-tools/src/lib/openapi/optic/helpers.ts +++ b/packages/repo-tools/src/lib/openapi/optic/helpers.ts @@ -131,119 +131,108 @@ export const generateCompareSummaryMarkdown = ( ); const anyCompletedHasCapture = results.completed.some(s => s.capture); return ` - ${ - results.completed.length > 0 - ? `### APIs Changed - - - - - - - - ${anyCompletedHasWarning ? '' : ''} - ${anyCompletedHasCapture ? '' : ''} - - - - - ${results.completed - .map( - s => - ` - - - - - ${anyCompletedHasWarning ? `` : ''} - - ${ - anyCompletedHasCapture - ? ` - - - - ` - : '' - } - `, - ) - .join('\n')} - -
APIChangesRulesWarningsTests
- - ${relative(cliPaths.targetDir, s.apiName)} - - - - ${getOperationsText(s.comparison.groupedDiffs, { - webUrl: s.opticWebUrl, - verbose: options.verbose, - labelJoiner: ',\n', - })} - - - - ${getChecksLabel(s.comparison.results, results.severity)} - - ${s.warnings.join('\n')} - - ${ - s.capture - ? s.capture.success - ? s.capture.mismatchedEndpoints || s.capture.unmatchedInteractions - ? getCaptureIssuesLabel({ - unmatchedInteractions: s.capture.unmatchedInteractions, - mismatchedEndpoints: s.capture.mismatchedEndpoints, - }) - : `✅ ${s.capture.percentCovered}% coverage` - : '❌ Failed to run' - : '' - } - -
- ` - : '' - } - ${ - results.failed.length > 0 - ? `### Errors running optic - - - - - - - - - - ${results.failed - .map( - s => ` - - - `, - ) - .join('\n')} - -
APIError
${s.apiName} - - ${'```'} - ${s.error} - ${'```'} - -
- ` - : '' - } - - Summary of API changes for commit (${commit.sha}) - - ${ - results.noop.length > 0 - ? `${ - results.noop.length === 1 ? '1 API' : `${results.noop.length} APIs` - } had no changes.` - : '' - }`; +${ + results.completed.length > 0 + ? `### APIs Changed + + + + + + + +${anyCompletedHasWarning ? '' : ''} +${anyCompletedHasCapture ? '' : ''} + + + +${results.completed + .map( + s => + ` + + + + +${anyCompletedHasWarning ? `` : ''} + +${ + anyCompletedHasCapture + ? ` + +` + : '' +} +`, + ) + .join('\n')} + +
APIChangesRulesWarningsTests
+${relative(cliPaths.targetDir, s.apiName)} + +${getOperationsText(s.comparison.groupedDiffs, { + webUrl: s.opticWebUrl, + verbose: options.verbose, + labelJoiner: ',\n', +})} + +${getChecksLabel(s.comparison.results, results.severity)} +${s.warnings.join('\n')} +${ + s.capture + ? s.capture.success + ? s.capture.mismatchedEndpoints || s.capture.unmatchedInteractions + ? getCaptureIssuesLabel({ + unmatchedInteractions: s.capture.unmatchedInteractions, + mismatchedEndpoints: s.capture.mismatchedEndpoints, + }) + : `✅ ${s.capture.percentCovered}% coverage` + : '❌ Failed to run' + : '' +} +
+` + : '' +} +${ + results.failed.length > 0 + ? `### Errors running optic + + + + + + + + + +${results.failed + .map( + s => ` + + +`, + ) + .join('\n')} + +
APIError
${s.apiName} + +${'```'} +${s.error} +${'```'} + +
+` + : '' +} + +Summary of API changes for commit (${commit.sha}) + +${ + results.noop.length > 0 + ? `${ + results.noop.length === 1 ? '1 API' : `${results.noop.length} APIs` + } had no changes.` + : '' +}`; }; From 351ae33e34b8f160d020004548291e1549a72a66 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 18:30:11 -0500 Subject: [PATCH 15/90] update workflow with improved comment structure Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- .../api-breaking-changes-comment.yml | 12 +- .github/workflows/api-breaking-changes.yml | 2 - .../src/commands/repo/schema/openapi/check.ts | 62 ++++++++-- .../src/lib/openapi/optic/helpers.ts | 112 +++++++++++++++--- .../src/schema/openapi.generated.ts | 2 +- 5 files changed, 157 insertions(+), 33 deletions(-) diff --git a/.github/workflows/api-breaking-changes-comment.yml b/.github/workflows/api-breaking-changes-comment.yml index 91741bba5b..4c7fa452e7 100644 --- a/.github/workflows/api-breaking-changes-comment.yml +++ b/.github/workflows/api-breaking-changes-comment.yml @@ -77,7 +77,7 @@ jobs: run: | unzip preview-spec.zip comment.md ls - echo "MANIFESTS_FILE_HASH=$(md5sum manifests.rendered.yml | awk '{ print $1 }')" >> $GITHUB_OUTPUT + grep add-comment: name: Write comment about issues @@ -93,19 +93,19 @@ jobs: uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4 # Identify comment to be updated - - name: Find comment for Ephemeral Environment + - name: Find comment for API Changes uses: peter-evans/find-comment@d5fe37641ad8451bdd80312415672ba26c86575e # v3 id: find-comment with: - issue-number: ${{ needs.cache-manifests-file.outputs.pr-number }} + issue-number: ${{ needs.setup.outputs.pr-number }} comment-author: 'github-actions[bot]' - body-includes: pr-changes-${{ needs.cache-manifests-file.outputs.pr-number }} + body-includes: API changes direction: last - - name: Create or Update Comment with Deployment URL + - name: Create or Update Comment with API Changes uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4 with: - comment-id: ${{ steps.notification.outputs.comment-id }} + comment-id: ${{ steps.find-comment.outputs.comment-id }} issue-number: ${{ github.event.pull_request.number }} body-path: comment.md edit-mode: replace diff --git a/.github/workflows/api-breaking-changes.yml b/.github/workflows/api-breaking-changes.yml index 5c75910546..07206718bb 100644 --- a/.github/workflows/api-breaking-changes.yml +++ b/.github/workflows/api-breaking-changes.yml @@ -12,8 +12,6 @@ jobs: name: Build PR image runs-on: ubuntu-latest if: ${{ github.event_name != 'pull_request' || github.event.action != 'closed' }} - outputs: - tags: ${{ steps.meta.outputs.tags }} steps: - name: Harden Runner uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0 diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts index 5599937322..d434e0cd4d 100644 --- a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts +++ b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts @@ -20,6 +20,8 @@ import { CiRunDetails, generateCompareSummaryMarkdown, } from '../../../../lib/openapi/optic/helpers'; +import { paths as cliPaths } from '../../../../lib/paths'; +import { YAML_SCHEMA_PATH } from '../../../../lib/openapi/constants'; export async function command(opts: OptionValues) { let packages = await PackageGraph.listTargetPackages(); @@ -28,26 +30,34 @@ export async function command(opts: OptionValues) { if (opts.since) { const { stdout: sinceRaw } = await exec('git', ['rev-parse', opts.since]); since = sinceRaw.toString().trim(); - const graph = PackageGraph.fromPackages(packages); - const changedPackages = await graph.listChangedPackages({ - ref: opts.since, - analyzeLockfile: true, - }); - const withDevDependents = graph.collectPackageNames( - changedPackages.map(pkg => pkg.name), - pkg => pkg.localDevDependents.keys(), + const { stdout: changedFilesRaw } = await exec('git', [ + 'diff', + '--name-only', + since, + ]); + const changedFiles = changedFilesRaw.toString().trim(); + + const changedOpenApiSpecs = changedFiles + .split('\n') + .filter(e => e.endsWith(YAML_SCHEMA_PATH)) + .map(e => cliPaths.resolveTarget(e)); + + // filter packages by changedFiles + packages = packages.filter(pkg => + changedOpenApiSpecs.some(e => e.startsWith(`${pkg.dir}/`)), ); - packages = Array.from(withDevDependents).map(name => graph.get(name)!); } const checkablePackages = packages.filter( e => e.packageJson.scripts?.['check:api'], ); + try { const outputs = { completed: [], failed: [], noop: [], + warning: [], severity: 0, } as CiRunDetails; for (const pkg of checkablePackages) { @@ -65,6 +75,40 @@ export async function command(opts: OptionValues) { outputs.noop.push(...(result.noop ?? [])); } + for (const pkg of packages.filter( + e => !e.packageJson.scripts?.['check:api'], + )) { + outputs.warning?.push({ + apiName: `${pkg.dir}/`, + warning: 'No check:api script found in package.json', + }); + } + + outputs.completed.forEach( + e => + (e.apiName = e.apiName + .replace(cliPaths.targetDir, '') + .replace(YAML_SCHEMA_PATH, '')), + ); + outputs.failed.forEach( + e => + (e.apiName = e.apiName + .replace(cliPaths.targetDir, '') + .replace(YAML_SCHEMA_PATH, '')), + ); + outputs.noop.forEach( + e => + (e.apiName = e.apiName + .replace(cliPaths.targetDir, '') + .replace(YAML_SCHEMA_PATH, '')), + ); + outputs.warning?.forEach( + e => + (e.apiName = e.apiName + .replace(cliPaths.targetDir, '') + .replace(YAML_SCHEMA_PATH, '')), + ); + const { stdout: currentSha } = await exec('git', ['rev-parse', 'HEAD']); console.log( generateCompareSummaryMarkdown( diff --git a/packages/repo-tools/src/lib/openapi/optic/helpers.ts b/packages/repo-tools/src/lib/openapi/optic/helpers.ts index 9bcd8c45a3..2393e71ce8 100644 --- a/packages/repo-tools/src/lib/openapi/optic/helpers.ts +++ b/packages/repo-tools/src/lib/openapi/optic/helpers.ts @@ -23,8 +23,6 @@ import { getOperationsChanged, } from '@useoptic/openapi-utilities'; import { GroupedDiffs } from '@useoptic/openapi-utilities/build/openapi3/group-diff'; -import { relative } from 'path'; -import { paths as cliPaths } from '../../paths'; /** * The below code is copied from https://github.com/opticdev/optic/blob/main/projects/optic/src/commands/ci/comment/common.ts#L82 for use @@ -45,6 +43,7 @@ export type CiRunDetails = { specUrl?: string | null; capture?: any; }[]; + warning?: { apiName: string; warning: string }[]; failed: { apiName: string; error: string }[]; noop: { apiName: string }[]; severity: Severity; @@ -121,6 +120,18 @@ const getCaptureIssuesLabel = ({ ].join('\n'); }; +const getBreakagesRow = (breakage: CiRunDetails['completed'][number]) => { + return ` + - ${breakage.apiName} + ${breakage.comparison.results.map( + s => ` + - ${s.where} + ${'```'} + ${s.error} + ${'```'}`, + )}`; +}; + export const generateCompareSummaryMarkdown = ( commit: { sha: string }, results: CiRunDetails, @@ -130,10 +141,61 @@ export const generateCompareSummaryMarkdown = ( s => s.warnings.length > 0, ); const anyCompletedHasCapture = results.completed.some(s => s.capture); - return ` + if ( + results.completed.length === 0 && + results.failed.length === 0 && + results.failed.length === 0 + ) { + return `No API changes detected for commit (${commit.sha})`; + } + const breakages = results.completed + .filter(s => s.comparison.results.some(e => !e.passed)) + .map(e => ({ + ...e, + comparison: { + ...e.comparison, + results: e.comparison.results.filter(f => !f.passed), + }, + })); + const successfullyCompletedCount = + results.completed.length - breakages.length; + return `### Summary for commit (${commit.sha}) + +${ + results.noop.length > 0 + ? `${ + results.noop.length === 1 ? '1 API' : `${results.noop.length} APIs` + } had no changes.` + : '' +} +${ + breakages.length > 0 + ? `${ + breakages.length === 1 ? '1 API' : `${breakages.length} APIs` + } had breaking changes.` + : '' +} +${ + successfullyCompletedCount > 0 + ? `${ + successfullyCompletedCount === 1 + ? '1 API' + : `${successfullyCompletedCount} APIs` + } had non-breaking changes.` + : '' +} +${ + results.warning && results.warning.length > 0 + ? `${ + results.warning.length === 1 + ? '1 API' + : `${results.warning.length} APIs` + } had warnings.` + : '' +} ${ results.completed.length > 0 - ? `### APIs Changed + ? `### APIs with Changes @@ -151,7 +213,7 @@ ${results.completed s => `
-${relative(cliPaths.targetDir, s.apiName)} +${s.apiName} ${getOperationsText(s.comparison.groupedDiffs, { @@ -190,13 +252,13 @@ ${ ) .join('\n')} -
-` +` : '' } + ${ results.failed.length > 0 - ? `### Errors running optic + ? `### APIs with Errors @@ -226,13 +288,33 @@ ${'```'} : '' } -Summary of API changes for commit (${commit.sha}) - ${ - results.noop.length > 0 - ? `${ - results.noop.length === 1 ? '1 API' : `${results.noop.length} APIs` - } had no changes.` + results.warning && results.warning.length + ? ` +### APIs with Warnings +
+ + + + + + + + ${results.warning + .map(e => ``) + .join('\n')} + +
APIWarning
${e.apiName}${e.warning}
` : '' -}`; +} +${ + breakages.length > 0 + ? ` +### Routes with Breakages + +${breakages.map(getBreakagesRow).join('\n')} +` + : '' +} +`; }; diff --git a/plugins/catalog-backend/src/schema/openapi.generated.ts b/plugins/catalog-backend/src/schema/openapi.generated.ts index c8c9c73a84..550db58f2e 100644 --- a/plugins/catalog-backend/src/schema/openapi.generated.ts +++ b/plugins/catalog-backend/src/schema/openapi.generated.ts @@ -1072,7 +1072,7 @@ export const spec = { 'application/json': { schema: { type: 'object', - required: ['entityRefs', 'fields'], + required: ['entityRefs'], properties: { entityRefs: { type: 'array', From 683870a29ada01c4971a5a6efa4ac3c3fe330bc6 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 18:35:09 -0500 Subject: [PATCH 16/90] add changeset Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- .changeset/flat-countries-clap.md | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 .changeset/flat-countries-clap.md diff --git a/.changeset/flat-countries-clap.md b/.changeset/flat-countries-clap.md new file mode 100644 index 0000000000..ccd779a2b5 --- /dev/null +++ b/.changeset/flat-countries-clap.md @@ -0,0 +1,5 @@ +--- +'@backstage/repo-tools': minor +--- + +Adds 2 new commands `repo schema openapi check` and `package schema openapi check`. `repo schema openapi check` is intended to power a new breaking changes check on pull requests and the package level command allows plugin developers to quickly see new API breaking changes.They're intended to be used in complement with the existing `repo schema openapi verify` command to validate your OpenAPI spec against a variety of things. From d3d227d44fba792da9cdce1b184e86b137fb45dd Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 18:36:14 -0500 Subject: [PATCH 17/90] revert catalog changes Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- plugins/catalog-backend/src/schema/openapi.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/plugins/catalog-backend/src/schema/openapi.yaml b/plugins/catalog-backend/src/schema/openapi.yaml index d3a65490bf..75dc1d9bbc 100644 --- a/plugins/catalog-backend/src/schema/openapi.yaml +++ b/plugins/catalog-backend/src/schema/openapi.yaml @@ -841,7 +841,6 @@ paths: type: object required: - entityRefs - - fields properties: entityRefs: type: array From 9341347dc129e53a343ebcef742a583568dec45c Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 18:36:33 -0500 Subject: [PATCH 18/90] revert prettier change Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- plugins/catalog-backend/src/schema/openapi.generated.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/catalog-backend/src/schema/openapi.generated.ts b/plugins/catalog-backend/src/schema/openapi.generated.ts index 550db58f2e..823ac40a14 100644 --- a/plugins/catalog-backend/src/schema/openapi.generated.ts +++ b/plugins/catalog-backend/src/schema/openapi.generated.ts @@ -1,5 +1,5 @@ /* - * Copyright 2024 The Backstage Authors + * Copyright 2023 The Backstage Authors * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. From f4aebb8e88012d1f9f80f63962b5a7b2ad866971 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 2 Mar 2024 18:44:06 -0500 Subject: [PATCH 19/90] small refactoring Signed-off-by: aramissennyeydd Signed-off-by: web-next-automation --- .../src/commands/repo/schema/openapi/check.ts | 34 ++++--------- .../src/lib/openapi/optic/helpers.ts | 48 +++++++------------ 2 files changed, 26 insertions(+), 56 deletions(-) diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts index d434e0cd4d..95bd4dffed 100644 --- a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts +++ b/packages/repo-tools/src/commands/repo/schema/openapi/check.ts @@ -23,6 +23,12 @@ import { import { paths as cliPaths } from '../../../../lib/paths'; import { YAML_SCHEMA_PATH } from '../../../../lib/openapi/constants'; +function cleanUpApiName(e: { apiName: string }) { + e.apiName = e.apiName + .replace(cliPaths.targetDir, '') + .replace(YAML_SCHEMA_PATH, ''); +} + export async function command(opts: OptionValues) { let packages = await PackageGraph.listTargetPackages(); @@ -84,30 +90,10 @@ export async function command(opts: OptionValues) { }); } - outputs.completed.forEach( - e => - (e.apiName = e.apiName - .replace(cliPaths.targetDir, '') - .replace(YAML_SCHEMA_PATH, '')), - ); - outputs.failed.forEach( - e => - (e.apiName = e.apiName - .replace(cliPaths.targetDir, '') - .replace(YAML_SCHEMA_PATH, '')), - ); - outputs.noop.forEach( - e => - (e.apiName = e.apiName - .replace(cliPaths.targetDir, '') - .replace(YAML_SCHEMA_PATH, '')), - ); - outputs.warning?.forEach( - e => - (e.apiName = e.apiName - .replace(cliPaths.targetDir, '') - .replace(YAML_SCHEMA_PATH, '')), - ); + outputs.completed.forEach(cleanUpApiName); + outputs.failed.forEach(cleanUpApiName); + outputs.noop.forEach(cleanUpApiName); + outputs.warning?.forEach(cleanUpApiName); const { stdout: currentSha } = await exec('git', ['rev-parse', 'HEAD']); console.log( diff --git a/packages/repo-tools/src/lib/openapi/optic/helpers.ts b/packages/repo-tools/src/lib/openapi/optic/helpers.ts index 2393e71ce8..df8ecfc723 100644 --- a/packages/repo-tools/src/lib/openapi/optic/helpers.ts +++ b/packages/repo-tools/src/lib/openapi/optic/helpers.ts @@ -132,6 +132,14 @@ const getBreakagesRow = (breakage: CiRunDetails['completed'][number]) => { )}`; }; +const addSummaryLine = (items: any[] | number | undefined, label: string) => { + const length = Array.isArray(items) ? items.length : items; + if (!length) return ''; + let text = length === 1 ? `1 API` : `${length} APIs`; + text += ` had ${label}`; + return text; +}; + export const generateCompareSummaryMarkdown = ( commit: { sha: string }, results: CiRunDetails, @@ -161,38 +169,14 @@ export const generateCompareSummaryMarkdown = ( results.completed.length - breakages.length; return `### Summary for commit (${commit.sha}) -${ - results.noop.length > 0 - ? `${ - results.noop.length === 1 ? '1 API' : `${results.noop.length} APIs` - } had no changes.` - : '' -} -${ - breakages.length > 0 - ? `${ - breakages.length === 1 ? '1 API' : `${breakages.length} APIs` - } had breaking changes.` - : '' -} -${ - successfullyCompletedCount > 0 - ? `${ - successfullyCompletedCount === 1 - ? '1 API' - : `${successfullyCompletedCount} APIs` - } had non-breaking changes.` - : '' -} -${ - results.warning && results.warning.length > 0 - ? `${ - results.warning.length === 1 - ? '1 API' - : `${results.warning.length} APIs` - } had warnings.` - : '' -} +${addSummaryLine(results.noop, 'no changes')} + +${addSummaryLine(breakages.length, 'breaking changes')} + +${addSummaryLine(successfullyCompletedCount, 'non-breaking changes')} + +${addSummaryLine(results.warning, 'warnings')} + ${ results.completed.length > 0 ? `### APIs with Changes From 61a9e8801f20671e08b9cf64dad2286611a42cbc Mon Sep 17 00:00:00 2001 From: Aramis Sennyey <159921952+aramissennyeydd@users.noreply.github.com> Date: Mon, 1 Apr 2024 11:13:19 -0400 Subject: [PATCH 20/90] add changeset Signed-off-by: Aramis Sennyey <159921952+aramissennyeydd@users.noreply.github.com> --- packages/repo-tools/cli-report.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/packages/repo-tools/cli-report.md b/packages/repo-tools/cli-report.md index 7c7dd445bc..65c0f50163 100644 --- a/packages/repo-tools/cli-report.md +++ b/packages/repo-tools/cli-report.md @@ -168,7 +168,7 @@ Options: -h, --help Commands: - verify [options] [paths...] + verify [paths...] lint [options] [paths...] test [options] [paths...] fuzz [options] @@ -179,6 +179,14 @@ Commands: ``` Usage: backstage-repo-tools repo schema openapi fuzz [options] + check [options] + help [command] +``` + +### `backstage-repo-tools repo schema openapi check` + +``` +Usage: backstage-repo-tools repo schema openapi check [options] Options: --since @@ -211,7 +219,6 @@ Options: Usage: backstage-repo-tools repo schema openapi verify [options] [paths...] Options: - --from -h, --help ``` From 5d091a6405dc6bca517f49c2e889e46f256751fd Mon Sep 17 00:00:00 2001 From: Aramis Sennyey <159921952+aramissennyeydd@users.noreply.github.com> Date: Mon, 1 Apr 2024 19:46:05 -0400 Subject: [PATCH 21/90] revert verify changes Signed-off-by: Aramis Sennyey <159921952+aramissennyeydd@users.noreply.github.com> Signed-off-by: web-next-automation --- .../commands/repo/schema/openapi/verify.ts | 28 ++++++------------- 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts b/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts index 2902d99ea4..23e07d1230 100644 --- a/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts +++ b/packages/repo-tools/src/commands/repo/schema/openapi/verify.ts @@ -30,14 +30,14 @@ import { } from '../../../../lib/openapi/constants'; import { getPathToOpenApiSpec } from '../../../../lib/openapi/helpers'; -const verifySpecAndGeneratedSpecMatch = async ( - openapiPath: string, - directoryPath: string, -) => { - const openapiTempDirectory = resolvePath(cliPaths.targetDir, '.openapi'); - await fs.mkdirp(openapiTempDirectory); - console.log(openapiTempDirectory); - +async function verify(directoryPath: string) { + let openapiPath = ''; + try { + openapiPath = await getPathToOpenApiSpec(directoryPath); + } catch { + // Unable to find spec at path. + return; + } const yaml = YAML.load(await fs.readFile(openapiPath, 'utf8')); await Parser.validate(cloneDeep(yaml) as any); @@ -57,18 +57,6 @@ const verifySpecAndGeneratedSpecMatch = async ( `\`${YAML_SCHEMA_PATH}\` and \`${TS_SCHEMA_PATH}\` do not match. Please run \`yarn backstage-repo-tools package schema openapi generate\` from '${path}' to regenerate \`${TS_SCHEMA_PATH}\`.`, ); } -}; - -async function verify(directoryPath: string) { - let openapiPath = ''; - try { - openapiPath = await getPathToOpenApiSpec(directoryPath); - } catch { - // Unable to find spec at path. - return; - } - - await verifySpecAndGeneratedSpecMatch(openapiPath, directoryPath); } export async function bulkCommand(paths: string[] = []): Promise { From 490d96829b582c0a707a20d81b8167d94fda5e0d Mon Sep 17 00:00:00 2001 From: web-next-automation Date: Mon, 8 Apr 2024 21:01:31 -0400 Subject: [PATCH 22/90] fix merge issue Signed-off-by: web-next-automation --- packages/repo-tools/src/lib/openapi/optic/helpers.ts | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/packages/repo-tools/src/lib/openapi/optic/helpers.ts b/packages/repo-tools/src/lib/openapi/optic/helpers.ts index df8ecfc723..9f5deb72e0 100644 --- a/packages/repo-tools/src/lib/openapi/optic/helpers.ts +++ b/packages/repo-tools/src/lib/openapi/optic/helpers.ts @@ -87,9 +87,7 @@ function getOperationsText( ...[...ops.removed].map(o => `\`${o}\` (removed)`), ].join('\n') : ''; - return `${getOperationsChangedLabel(groupedDiffs, { - joiner: options.labelJoiner, - })} + return `${getOperationsChangedLabel(groupedDiffs)} ${operationsText} `; From 5826d70b544472d394b72bc2162a5fbbe8821473 Mon Sep 17 00:00:00 2001 From: web-next-automation Date: Mon, 8 Apr 2024 21:08:15 -0400 Subject: [PATCH 23/90] add cli report Signed-off-by: web-next-automation --- packages/repo-tools/cli-report.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/packages/repo-tools/cli-report.md b/packages/repo-tools/cli-report.md index 65c0f50163..68d679fb06 100644 --- a/packages/repo-tools/cli-report.md +++ b/packages/repo-tools/cli-report.md @@ -172,13 +172,6 @@ Commands: lint [options] [paths...] test [options] [paths...] fuzz [options] - help [command] -``` - -### `backstage-repo-tools repo schema openapi fuzz` - -``` -Usage: backstage-repo-tools repo schema openapi fuzz [options] check [options] help [command] ``` @@ -193,6 +186,16 @@ Options: -h, --help ``` +### `backstage-repo-tools repo schema openapi fuzz` + +``` +Usage: backstage-repo-tools repo schema openapi fuzz [options] + +Options: + --since + -h, --help +``` + ### `backstage-repo-tools repo schema openapi lint` ``` From fd84ca431702a0faa267b475a9a81b8c7d4b4021 Mon Sep 17 00:00:00 2001 From: web-next-automation Date: Mon, 8 Apr 2024 21:09:13 -0400 Subject: [PATCH 24/90] revert Signed-off-by: web-next-automation --- plugins/catalog-backend/src/schema/openapi.generated.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/catalog-backend/src/schema/openapi.generated.ts b/plugins/catalog-backend/src/schema/openapi.generated.ts index 823ac40a14..550db58f2e 100644 --- a/plugins/catalog-backend/src/schema/openapi.generated.ts +++ b/plugins/catalog-backend/src/schema/openapi.generated.ts @@ -1,5 +1,5 @@ /* - * Copyright 2023 The Backstage Authors + * Copyright 2024 The Backstage Authors * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. From 0501243a575569e894c8e29949f91b6779fd8259 Mon Sep 17 00:00:00 2001 From: JeevaRamanathan Date: Tue, 9 Apr 2024 21:49:45 +0530 Subject: [PATCH 25/90] Enhance Accessibility: Add ARIA Attributes to SearchModal Component Signed-off-by: JeevaRamanathan --- .changeset/thirty-mangos-travel.md | 5 +++++ plugins/search/src/components/SearchModal/SearchModal.tsx | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) create mode 100644 .changeset/thirty-mangos-travel.md diff --git a/.changeset/thirty-mangos-travel.md b/.changeset/thirty-mangos-travel.md new file mode 100644 index 0000000000..2281cc06b6 --- /dev/null +++ b/.changeset/thirty-mangos-travel.md @@ -0,0 +1,5 @@ +--- +'@backstage/plugin-search': patch +--- + +Added `aria-label` attribute to DialogTitle element and set `aria-modal` attribute to `true` for improved accessibility in the search modal. diff --git a/plugins/search/src/components/SearchModal/SearchModal.tsx b/plugins/search/src/components/SearchModal/SearchModal.tsx index 814779fb45..07eec4438e 100644 --- a/plugins/search/src/components/SearchModal/SearchModal.tsx +++ b/plugins/search/src/components/SearchModal/SearchModal.tsx @@ -190,7 +190,8 @@ export const SearchModal = (props: SearchModalProps) => { paperFullWidth: classes.paperFullWidth, }} onClose={toggleModal} - aria-labelledby="search-modal-title" + aria-label="Search Modal" + aria-modal="true" fullWidth maxWidth="lg" open={open} From fade5e020fe1a3e114262e6bf719753a9a58c31d Mon Sep 17 00:00:00 2001 From: CiscoRob <133238823+CiscoRob@users.noreply.github.com> Date: Sat, 6 Apr 2024 14:38:20 -0500 Subject: [PATCH 26/90] Update CatalogTable to default total count to 0 instead of undefined Depending on latency in making the request the context can load without items and then refresh moments later with the correct count. Until the entities are loaded the total will show as "All (undefined)", which is not confidence inspiring. Signed-off-by: CiscoRob <133238823+CiscoRob@users.noreply.github.com> Signed-off-by: Coderrob --- .../src/components/CatalogTable/CatalogTable.tsx | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx b/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx index 674271a044..ba52b7d3a0 100644 --- a/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx +++ b/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx @@ -88,10 +88,15 @@ export const CatalogTable = (props: CatalogTableProps) => { } = props; const { isStarredEntity, toggleStarredEntity } = useStarredEntities(); const entityListContext = useEntityList(); - const { loading, error, entities, filters, pageInfo, totalItems } = - entityListContext; + const { + loading, + error, + entities, + filters, + pageInfo, + totalItems = 0, + } = entityListContext; const enablePagination = !!pageInfo; - const tableColumns = useMemo( () => typeof columns === 'function' ? columns(entityListContext) : columns, From 411853058ffd516f113f84daf9a467152dea4c4f Mon Sep 17 00:00:00 2001 From: Coderrob Date: Tue, 9 Apr 2024 15:46:11 -0500 Subject: [PATCH 27/90] Add changeset per contributor guide Change display to avoid displaying counts when not loaded yet Signed-off-by: Coderrob --- .changeset/swift-humans-hunt.md | 5 ++++ .../CatalogTable/CatalogTable.test.tsx | 4 ++-- .../components/CatalogTable/CatalogTable.tsx | 23 ++++++++++--------- 3 files changed, 19 insertions(+), 13 deletions(-) create mode 100644 .changeset/swift-humans-hunt.md diff --git a/.changeset/swift-humans-hunt.md b/.changeset/swift-humans-hunt.md new file mode 100644 index 0000000000..b96b3e41bc --- /dev/null +++ b/.changeset/swift-humans-hunt.md @@ -0,0 +1,5 @@ +--- +'@backstage/plugin-catalog': patch +--- + +Avoiding pre-loading display total count undefined for table counts diff --git a/plugins/catalog/src/components/CatalogTable/CatalogTable.test.tsx b/plugins/catalog/src/components/CatalogTable/CatalogTable.test.tsx index 052d8f4787..35639069df 100644 --- a/plugins/catalog/src/components/CatalogTable/CatalogTable.test.tsx +++ b/plugins/catalog/src/components/CatalogTable/CatalogTable.test.tsx @@ -343,7 +343,7 @@ describe('CatalogTable component', () => { expect(screen.getByText('Should be rendered')).toBeInTheDocument(); }); - it('should render the label column with customised title and value as specified', async () => { + it('should render the label column with customized title and value as specified', async () => { const columns = [ CatalogTable.columns.createNameColumn({ defaultKind: 'API' }), CatalogTable.columns.createLabelColumn('category', { title: 'Category' }), @@ -381,7 +381,7 @@ describe('CatalogTable component', () => { expect(labelCellValue).toBeInTheDocument(); }); - it('should render the label column with customised title and value as specified using function', async () => { + it('should render the label column with customized title and value as specified using function', async () => { const columns: CatalogTableColumnsFunc = ({ filters, entities: entities1, diff --git a/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx b/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx index ba52b7d3a0..08da250051 100644 --- a/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx +++ b/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx @@ -88,14 +88,8 @@ export const CatalogTable = (props: CatalogTableProps) => { } = props; const { isStarredEntity, toggleStarredEntity } = useStarredEntities(); const entityListContext = useEntityList(); - const { - loading, - error, - entities, - filters, - pageInfo, - totalItems = 0, - } = entityListContext; + const { loading, error, entities, filters, pageInfo, totalItems } = + entityListContext; const enablePagination = !!pageInfo; const tableColumns = useMemo( () => @@ -175,13 +169,20 @@ export const CatalogTable = (props: CatalogTableProps) => { const currentKind = filters.kind?.value || ''; const currentType = filters.type?.value || ''; + const currentCount = Number.isSafeInteger(totalItems) + ? `(${totalItems})` + : ''; // TODO(timbonicus): remove the title from the CatalogTable once using EntitySearchBar const titlePreamble = capitalize(filters.user?.value ?? 'all'); - const titleDisplay = [titlePreamble, currentType, pluralize(currentKind)] + const title = [ + titlePreamble, + currentType, + pluralize(currentKind), + currentCount, + ] .filter(s => s) .join(' '); - const title = `${titleDisplay} (${totalItems})`; const actions = props.actions || defaultActions; const options = { actionsColumnIndex: -1, @@ -197,7 +198,7 @@ export const CatalogTable = (props: CatalogTableProps) => { columns={tableColumns} emptyContent={emptyContent} isLoading={loading} - title={titleDisplay} + title={title} actions={actions} subtitle={subtitle} options={options} From 4039d328ae08f51b550c0d6b191776b004335446 Mon Sep 17 00:00:00 2001 From: Coderrob Date: Wed, 17 Apr 2024 11:14:53 -0500 Subject: [PATCH 28/90] Adjust to make lighthouse happier Signed-off-by: Coderrob --- plugins/catalog/src/components/CatalogTable/CatalogTable.tsx | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx b/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx index 08da250051..b1590bdff3 100644 --- a/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx +++ b/plugins/catalog/src/components/CatalogTable/CatalogTable.tsx @@ -169,9 +169,7 @@ export const CatalogTable = (props: CatalogTableProps) => { const currentKind = filters.kind?.value || ''; const currentType = filters.type?.value || ''; - const currentCount = Number.isSafeInteger(totalItems) - ? `(${totalItems})` - : ''; + const currentCount = typeof totalItems === 'number' ? `(${totalItems})` : ''; // TODO(timbonicus): remove the title from the CatalogTable once using EntitySearchBar const titlePreamble = capitalize(filters.user?.value ?? 'all'); const title = [ From 821f902bfec625628ce82e38003a37c0bc9cd78f Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Sat, 20 Apr 2024 22:15:57 -0400 Subject: [PATCH 29/90] update to diff instead of check Signed-off-by: aramissennyeydd --- .changeset/flat-countries-clap.md | 2 +- packages/repo-tools/src/commands/index.ts | 14 +++++++------- .../package/schema/openapi/{check.ts => diff.ts} | 0 .../repo/schema/openapi/{check.ts => diff.ts} | 12 ++++-------- plugins/catalog-backend/package.json | 2 +- 5 files changed, 13 insertions(+), 17 deletions(-) rename packages/repo-tools/src/commands/package/schema/openapi/{check.ts => diff.ts} (100%) rename packages/repo-tools/src/commands/repo/schema/openapi/{check.ts => diff.ts} (90%) diff --git a/.changeset/flat-countries-clap.md b/.changeset/flat-countries-clap.md index ccd779a2b5..9c962b04a3 100644 --- a/.changeset/flat-countries-clap.md +++ b/.changeset/flat-countries-clap.md @@ -2,4 +2,4 @@ '@backstage/repo-tools': minor --- -Adds 2 new commands `repo schema openapi check` and `package schema openapi check`. `repo schema openapi check` is intended to power a new breaking changes check on pull requests and the package level command allows plugin developers to quickly see new API breaking changes.They're intended to be used in complement with the existing `repo schema openapi verify` command to validate your OpenAPI spec against a variety of things. +Adds 2 new commands `repo schema openapi diff` and `package schema openapi diff`. `repo schema openapi diff` is intended to power a new breaking changes check on pull requests and the package level command allows plugin developers to quickly see new API breaking changes. They're intended to be used in complement with the existing `repo schema openapi verify` command to validate your OpenAPI spec against a variety of things. diff --git a/packages/repo-tools/src/commands/index.ts b/packages/repo-tools/src/commands/index.ts index 3fa16aa512..83266ea9f9 100644 --- a/packages/repo-tools/src/commands/index.ts +++ b/packages/repo-tools/src/commands/index.ts @@ -80,12 +80,12 @@ function registerPackageCommand(program: Command) { ); openApiCommand - .command('check') + .command('diff') .option('--ignore', 'Ignore linting failures and only log the results.') .option('--json', 'Output the results as JSON') - .option('--since ', 'Check the API against a specific ref') + .option('--since ', 'Diff the API against a specific ref') .action( - lazy(() => import('./package/schema/openapi/check').then(m => m.command)), + lazy(() => import('./package/schema/openapi/diff').then(m => m.command)), ); } @@ -144,17 +144,17 @@ function registerRepoCommand(program: Command) { ); openApiCommand - .command('check') + .command('diff') .description( - 'Check the repository against a specific ref, will run all package `check:api` scripts.', + 'Diff the repository against a specific ref, will run all package `diff` scripts.', ) .option( '--since ', - 'Check the API against a specific ref', + 'Diff the API against a specific ref', 'origin/master', ) .action( - lazy(() => import('./repo/schema/openapi/check').then(m => m.command)), + lazy(() => import('./repo/schema/openapi/diff').then(m => m.command)), ); } diff --git a/packages/repo-tools/src/commands/package/schema/openapi/check.ts b/packages/repo-tools/src/commands/package/schema/openapi/diff.ts similarity index 100% rename from packages/repo-tools/src/commands/package/schema/openapi/check.ts rename to packages/repo-tools/src/commands/package/schema/openapi/diff.ts diff --git a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts b/packages/repo-tools/src/commands/repo/schema/openapi/diff.ts similarity index 90% rename from packages/repo-tools/src/commands/repo/schema/openapi/check.ts rename to packages/repo-tools/src/commands/repo/schema/openapi/diff.ts index 95bd4dffed..02f3db0c43 100644 --- a/packages/repo-tools/src/commands/repo/schema/openapi/check.ts +++ b/packages/repo-tools/src/commands/repo/schema/openapi/diff.ts @@ -54,9 +54,7 @@ export async function command(opts: OptionValues) { ); } - const checkablePackages = packages.filter( - e => e.packageJson.scripts?.['check:api'], - ); + const checkablePackages = packages.filter(e => e.packageJson.scripts?.diff); try { const outputs = { @@ -70,7 +68,7 @@ export async function command(opts: OptionValues) { const sinceCommands = since ? ['--since', since] : []; const { stdout } = await exec( 'yarn', - ['check:api', '--ignore', '--json', ...sinceCommands], + ['diff', '--ignore', '--json', ...sinceCommands], { cwd: pkg.dir, }, @@ -81,12 +79,10 @@ export async function command(opts: OptionValues) { outputs.noop.push(...(result.noop ?? [])); } - for (const pkg of packages.filter( - e => !e.packageJson.scripts?.['check:api'], - )) { + for (const pkg of packages.filter(e => !e.packageJson.scripts?.diff)) { outputs.warning?.push({ apiName: `${pkg.dir}/`, - warning: 'No check:api script found in package.json', + warning: 'No diff script found in package.json', }); } diff --git a/plugins/catalog-backend/package.json b/plugins/catalog-backend/package.json index 058b9318d7..04a7778926 100644 --- a/plugins/catalog-backend/package.json +++ b/plugins/catalog-backend/package.json @@ -42,8 +42,8 @@ ], "scripts": { "build": "backstage-cli package build", - "check:api": "backstage-repo-tools package schema openapi check", "clean": "backstage-cli package clean", + "diff": "backstage-repo-tools package schema openapi diff", "fuzz": "backstage-repo-tools package schema openapi fuzz --exclude-checks response_schema_conformance", "generate": "backstage-repo-tools package schema openapi generate --server --client-package packages/catalog-client", "lint": "backstage-cli package lint", From f4856e9f9b1d950565a32e431a4c835a82354e91 Mon Sep 17 00:00:00 2001 From: aramissennyeydd Date: Mon, 22 Apr 2024 09:26:20 -0400 Subject: [PATCH 30/90] add api report Signed-off-by: aramissennyeydd --- packages/repo-tools/cli-report.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/packages/repo-tools/cli-report.md b/packages/repo-tools/cli-report.md index 68d679fb06..a06f426baf 100644 --- a/packages/repo-tools/cli-report.md +++ b/packages/repo-tools/cli-report.md @@ -172,14 +172,14 @@ Commands: lint [options] [paths...] test [options] [paths...] fuzz [options] - check [options] + diff [options] help [command] ``` -### `backstage-repo-tools repo schema openapi check` +### `backstage-repo-tools repo schema openapi diff` ``` -Usage: backstage-repo-tools repo schema openapi check [options] +Usage: backstage-repo-tools repo schema openapi diff [options] Options: --since From a2ee4df20a6884d00328556948c81448d1487dd3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gustaf=20R=C3=A4ntil=C3=A4?= Date: Fri, 22 Mar 2024 08:46:49 +0100 Subject: [PATCH 31/90] feat: Allow GaugeCard to handle multi-line titles MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit By adding a fullHeightFixedContent variant. Also, add support for a small version. Signed-off-by: Gustaf Räntilä --- .changeset/selfish-walls-visit.md | 5 ++ packages/core-components/api-report.md | 7 +- .../src/components/ProgressBars/Gauge.tsx | 14 +++- .../ProgressBars/GaugeCard.stories.tsx | 76 +++++++++++++++++++ .../src/components/ProgressBars/GaugeCard.tsx | 17 ++++- .../src/layout/InfoCard/InfoCard.tsx | 25 +++++- 6 files changed, 137 insertions(+), 7 deletions(-) create mode 100644 .changeset/selfish-walls-visit.md diff --git a/.changeset/selfish-walls-visit.md b/.changeset/selfish-walls-visit.md new file mode 100644 index 0000000000..8748b3717e --- /dev/null +++ b/.changeset/selfish-walls-visit.md @@ -0,0 +1,5 @@ +--- +'@backstage/core-components': patch +--- + +Add a fullHeightFixedContent variant of the GaugeCard, and a small size version. Fixed content will vertically align the gauge in the cards, even when the card titles span across multiple lines. diff --git a/packages/core-components/api-report.md b/packages/core-components/api-report.md index 7fd706db43..1e5c1b892f 100644 --- a/packages/core-components/api-report.md +++ b/packages/core-components/api-report.md @@ -448,6 +448,7 @@ export type GaugeProps = { inverse?: boolean; unit?: string; max?: number; + size?: 'normal' | 'small'; description?: ReactNode; getColor?: GaugePropsGetColor; }; @@ -609,7 +610,11 @@ export type InfoCardClassKey = | 'headerContent'; // @public (undocumented) -export type InfoCardVariants = 'flex' | 'fullHeight' | 'gridItem'; +export type InfoCardVariants = + | 'flex' + | 'fullHeight' + | 'fullHeightFixedContent' + | 'gridItem'; // Warning: (ae-forgotten-export) The symbol "ItemCardProps" needs to be exported by the entry point index.d.ts // Warning: (ae-missing-release-tag) "ItemCard" is part of the package's API, but it is missing a release tag (@alpha, @beta, @public, or @internal) diff --git a/packages/core-components/src/components/ProgressBars/Gauge.tsx b/packages/core-components/src/components/ProgressBars/Gauge.tsx index 088e369e62..87d2bf4f58 100644 --- a/packages/core-components/src/components/ProgressBars/Gauge.tsx +++ b/packages/core-components/src/components/ProgressBars/Gauge.tsx @@ -19,6 +19,7 @@ import { makeStyles, useTheme } from '@material-ui/core/styles'; import { Circle } from 'rc-progress'; import React, { ReactNode, useEffect, useState } from 'react'; import Box from '@material-ui/core/Box'; +import classNames from 'classnames'; /** @public */ export type GaugeClassKey = @@ -43,6 +44,9 @@ const useStyles = makeStyles( fontWeight: theme.typography.fontWeightBold, color: theme.palette.textContrast, }, + overlaySmall: { + fontSize: theme.typography.pxToRem(25), + }, description: { fontSize: '100%', top: '50%', @@ -68,6 +72,7 @@ export type GaugeProps = { inverse?: boolean; unit?: string; max?: number; + size?: 'normal' | 'small'; description?: ReactNode; getColor?: GaugePropsGetColor; }; @@ -121,7 +126,7 @@ export const getProgressColor: GaugePropsGetColor = ({ export function Gauge(props: GaugeProps) { const [hoverRef, setHoverRef] = useState(null); - const { getColor = getProgressColor } = props; + const { getColor = getProgressColor, size = 'normal' } = props; const classes = useStyles(props); const { palette } = useTheme(); const { value, fractional, inverse, unit, max, description } = { @@ -165,7 +170,12 @@ export function Gauge(props: GaugeProps) { {description && isHovering ? ( {description} ) : ( - + {isNaN(value) ? 'N/A' : `${asActual}${unit}`} )} diff --git a/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx b/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx index a0b6faea60..21d023999e 100644 --- a/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx +++ b/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx @@ -175,6 +175,82 @@ export const InfoMessage = () => ( ); +export const AlignedBottom = () => ( + + + + + + + + + + + + + + +); + +export const Small = () => ( + + + + + + + + + + + + + + +); + export const HoverMessage = () => ( diff --git a/packages/core-components/src/components/ProgressBars/GaugeCard.tsx b/packages/core-components/src/components/ProgressBars/GaugeCard.tsx index 3442395380..bbb870141b 100644 --- a/packages/core-components/src/components/ProgressBars/GaugeCard.tsx +++ b/packages/core-components/src/components/ProgressBars/GaugeCard.tsx @@ -27,6 +27,7 @@ type Props = { variant?: InfoCardVariants; /** Progress in % specified as decimal, e.g. "0.23" */ progress: number; + size?: 'normal' | 'small'; description?: ReactNode; icon?: ReactNode; inverse?: boolean; @@ -43,6 +44,10 @@ const useStyles = makeStyles( height: '100%', width: 250, }, + rootSmall: { + height: '100%', + width: 160, + }, }, { name: 'BackstageGaugeCard' }, ); @@ -64,6 +69,7 @@ export function GaugeCard(props: Props) { description, icon, variant, + size = 'normal', getColor, } = props; @@ -75,15 +81,22 @@ export function GaugeCard(props: Props) { }; return ( - + - + ); diff --git a/packages/core-components/src/layout/InfoCard/InfoCard.tsx b/packages/core-components/src/layout/InfoCard/InfoCard.tsx index c80e71b609..9fd72e50b6 100644 --- a/packages/core-components/src/layout/InfoCard/InfoCard.tsx +++ b/packages/core-components/src/layout/InfoCard/InfoCard.tsx @@ -46,6 +46,10 @@ const useStyles = makeStyles( header: { padding: theme.spacing(2, 2, 2, 2.5), }, + headerFixedContent: { + flexGrow: 1, + alignItems: 'flex-start', + }, headerTitle: { fontWeight: theme.typography.fontWeightBold, }, @@ -87,6 +91,11 @@ const VARIANT_STYLES = { flexDirection: 'column', height: '100%', }, + fullHeightFixedContent: { + display: 'flex', + flexDirection: 'column', + height: '100%', + }, gridItem: { display: 'flex', flexDirection: 'column', @@ -102,6 +111,9 @@ const VARIANT_STYLES = { fullHeight: { flex: 1, }, + fullHeightFixedContent: { + flex: '0 1 0%', + }, gridItem: { flex: 1, }, @@ -109,7 +121,11 @@ const VARIANT_STYLES = { }; /** @public */ -export type InfoCardVariants = 'flex' | 'fullHeight' | 'gridItem'; +export type InfoCardVariants = + | 'flex' + | 'fullHeight' + | 'fullHeightFixedContent' + | 'gridItem'; /** * InfoCard is used to display a paper-styled block on the screen, similar to a panel. @@ -228,7 +244,12 @@ export function InfoCard(props: Props): JSX.Element { {title && ( Date: Fri, 22 Mar 2024 16:09:05 +0100 Subject: [PATCH 32/90] Changed to not having a fixed/full sized header, but allow the content to grow, with an option to align the content to the bottom MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Gustaf Räntilä --- .changeset/selfish-walls-visit.md | 3 +- packages/core-components/api-report.md | 6 +--- .../ProgressBars/GaugeCard.stories.tsx | 24 +++++++++----- .../src/components/ProgressBars/GaugeCard.tsx | 3 ++ .../src/layout/InfoCard/InfoCard.tsx | 32 ++++++------------- 5 files changed, 31 insertions(+), 37 deletions(-) diff --git a/.changeset/selfish-walls-visit.md b/.changeset/selfish-walls-visit.md index 8748b3717e..64c87a64f5 100644 --- a/.changeset/selfish-walls-visit.md +++ b/.changeset/selfish-walls-visit.md @@ -2,4 +2,5 @@ '@backstage/core-components': patch --- -Add a fullHeightFixedContent variant of the GaugeCard, and a small size version. Fixed content will vertically align the gauge in the cards, even when the card titles span across multiple lines. +Add `alignGauge` prop to the `GaugeCard`, and a small size version. When `alignGauge` is `'bottom'` the gauge will vertically align the gauge in the cards, even when the card titles span across multiple lines. +Add `alignContent` prop to the `InfoCard`, defaulting to `'normal'` with the option of `'bottom'` which vertically aligns the content to the bottom of the card. diff --git a/packages/core-components/api-report.md b/packages/core-components/api-report.md index 1e5c1b892f..293695db0c 100644 --- a/packages/core-components/api-report.md +++ b/packages/core-components/api-report.md @@ -610,11 +610,7 @@ export type InfoCardClassKey = | 'headerContent'; // @public (undocumented) -export type InfoCardVariants = - | 'flex' - | 'fullHeight' - | 'fullHeightFixedContent' - | 'gridItem'; +export type InfoCardVariants = 'flex' | 'fullHeight' | 'gridItem'; // Warning: (ae-forgotten-export) The symbol "ItemCardProps" needs to be exported by the entry point index.d.ts // Warning: (ae-missing-release-tag) "ItemCard" is part of the package's API, but it is missing a release tag (@alpha, @beta, @public, or @internal) diff --git a/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx b/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx index 21d023999e..12cdef52cb 100644 --- a/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx +++ b/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx @@ -179,7 +179,8 @@ export const AlignedBottom = () => ( ( ( ( ( ( ( ( From 0b9d63a02519628a5a3546376451b6dc3ff33966 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gustaf=20R=C3=A4ntil=C3=A4?= Date: Sat, 6 Apr 2024 08:55:14 +0200 Subject: [PATCH 33/90] fix: Minor refactoring of styling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Gustaf Räntilä --- .../core-components/src/components/ProgressBars/Gauge.tsx | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/packages/core-components/src/components/ProgressBars/Gauge.tsx b/packages/core-components/src/components/ProgressBars/Gauge.tsx index 87d2bf4f58..e72172fd59 100644 --- a/packages/core-components/src/components/ProgressBars/Gauge.tsx +++ b/packages/core-components/src/components/ProgressBars/Gauge.tsx @@ -171,10 +171,9 @@ export function Gauge(props: GaugeProps) { {description} ) : ( {isNaN(value) ? 'N/A' : `${asActual}${unit}`} From a7648561962ac8d80c1145aa3b4e488e3063d427 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gustaf=20R=C3=A4ntil=C3=A4?= Date: Sat, 6 Apr 2024 09:28:13 +0200 Subject: [PATCH 34/90] feat: Added subheaderTypographyProps prop to InfoCard, allowing it to be used from GaugeCard. Also made GaugeCard 'small' variant to have smaller text MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Gustaf Räntilä --- .../components/ProgressBars/GaugeCard.stories.tsx | 1 + .../src/components/ProgressBars/GaugeCard.tsx | 13 ++++++------- .../src/layout/InfoCard/InfoCard.tsx | 3 +++ 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx b/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx index 12cdef52cb..f7e2eb69c6 100644 --- a/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx +++ b/packages/core-components/src/components/ProgressBars/GaugeCard.stories.tsx @@ -234,6 +234,7 @@ export const Small = () => ( alignGauge="bottom" size="small" title="Progress" + subheader="With a subheader" progress={0.57} /> diff --git a/packages/core-components/src/components/ProgressBars/GaugeCard.tsx b/packages/core-components/src/components/ProgressBars/GaugeCard.tsx index 4e73487983..9f6de8eafe 100644 --- a/packages/core-components/src/components/ProgressBars/GaugeCard.tsx +++ b/packages/core-components/src/components/ProgressBars/GaugeCard.tsx @@ -91,13 +91,12 @@ export function GaugeCard(props: Props) { variant={variant} alignContent={alignGauge} icon={icon} - titleTypographyProps={ - size === 'small' - ? { - variant: 'h6', - } - : undefined - } + titleTypographyProps={{ + ...(size === 'small' ? { variant: 'subtitle2' } : undefined), + }} + subheaderTypographyProps={{ + ...(size === 'small' ? { variant: 'body2' } : undefined), + }} > diff --git a/packages/core-components/src/layout/InfoCard/InfoCard.tsx b/packages/core-components/src/layout/InfoCard/InfoCard.tsx index 1f41a315b8..e80c0d3384 100644 --- a/packages/core-components/src/layout/InfoCard/InfoCard.tsx +++ b/packages/core-components/src/layout/InfoCard/InfoCard.tsx @@ -155,6 +155,7 @@ export type Props = { className?: string; noPadding?: boolean; titleTypographyProps?: object; + subheaderTypographyProps?: object; }; /** @@ -185,6 +186,7 @@ export function InfoCard(props: Props): JSX.Element { className, noPadding, titleTypographyProps, + subheaderTypographyProps, } = props; const classes = useStyles(); /** @@ -246,6 +248,7 @@ export function InfoCard(props: Props): JSX.Element { action={action} style={{ ...headerStyle }} titleTypographyProps={titleTypographyProps} + subheaderTypographyProps={subheaderTypographyProps} {...headerProps} /> )} From f72f3a076dd911a2f836b5a8b534e6e0fbe0c885 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gustaf=20R=C3=A4ntil=C3=A4?= Date: Sat, 6 Apr 2024 09:32:23 +0200 Subject: [PATCH 35/90] fix: Made the subhead in InfoCard not duplicate top padding props twice MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Gustaf Räntilä --- packages/core-components/src/layout/InfoCard/InfoCard.tsx | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/packages/core-components/src/layout/InfoCard/InfoCard.tsx b/packages/core-components/src/layout/InfoCard/InfoCard.tsx index e80c0d3384..a08a816c2e 100644 --- a/packages/core-components/src/layout/InfoCard/InfoCard.tsx +++ b/packages/core-components/src/layout/InfoCard/InfoCard.tsx @@ -217,10 +217,7 @@ export function InfoCard(props: Props): JSX.Element { } return ( -
+
{subheader &&
{subheader}
} {icon}
From 73a963492e5453a118ede38cb790ccf3a8be438e Mon Sep 17 00:00:00 2001 From: Himesh Ladva Date: Thu, 25 Apr 2024 09:52:04 +0100 Subject: [PATCH 36/90] chore(docs): update openapi generate-client command Signed-off-by: Himesh Ladva --- docs/openapi/generate-client.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/openapi/generate-client.md b/docs/openapi/generate-client.md index 640036960e..b8acc331e5 100644 --- a/docs/openapi/generate-client.md +++ b/docs/openapi/generate-client.md @@ -20,7 +20,7 @@ info: ### Generating your client -1. Run `yarn backstage-repo-tools schema openapi generate client --output-package `. This will create a new folder in `/src/generated` to house the generated content. +1. Run `yarn backstage-repo-tools package schema openapi generate client --client-package `. This will create a new folder in `/src/generated` to house the generated content. 2. You should use the generated files as follows, - `apis/DefaultApi.client.ts` - this is the client that you should use. It has types for all of the various operations on your API. From 3ef8cbc9cc4cf99782aedbfaabc6fe3344ddf4c7 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Fri, 26 Apr 2024 00:20:29 +0000 Subject: [PATCH 37/90] chore(deps): update github/codeql-action action to v3.25.3 Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> --- .github/workflows/scorecard.yml | 2 +- .github/workflows/sync_snyk-monitor.yml | 2 +- .github/workflows/verify_codeql.yml | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 91f1b02fee..67fdff4020 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -66,6 +66,6 @@ jobs: # Upload the results to GitHub's code scanning dashboard. - name: 'Upload to code-scanning' - uses: github/codeql-action/upload-sarif@c7f9125735019aa87cfc361530512d50ea439c71 # v3.25.1 + uses: github/codeql-action/upload-sarif@d39d31e687223d841ef683f52467bd88e9b21c14 # v3.25.3 with: sarif_file: results.sarif diff --git a/.github/workflows/sync_snyk-monitor.yml b/.github/workflows/sync_snyk-monitor.yml index 70fb8a44d9..e2507996b1 100644 --- a/.github/workflows/sync_snyk-monitor.yml +++ b/.github/workflows/sync_snyk-monitor.yml @@ -58,6 +58,6 @@ jobs: SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }} NODE_OPTIONS: --max-old-space-size=7168 - name: Upload Snyk report - uses: github/codeql-action/upload-sarif@c7f9125735019aa87cfc361530512d50ea439c71 # v3.25.1 + uses: github/codeql-action/upload-sarif@d39d31e687223d841ef683f52467bd88e9b21c14 # v3.25.3 with: sarif_file: snyk.sarif diff --git a/.github/workflows/verify_codeql.yml b/.github/workflows/verify_codeql.yml index e82577763e..920c9bab17 100644 --- a/.github/workflows/verify_codeql.yml +++ b/.github/workflows/verify_codeql.yml @@ -55,7 +55,7 @@ jobs: # Initializes the CodeQL tools for scanning. - name: Initialize CodeQL - uses: github/codeql-action/init@c7f9125735019aa87cfc361530512d50ea439c71 # v3.25.1 + uses: github/codeql-action/init@d39d31e687223d841ef683f52467bd88e9b21c14 # v3.25.3 with: languages: ${{ matrix.language }} # If you wish to specify custom queries, you can do so here or in a config file. @@ -66,7 +66,7 @@ jobs: # Autobuild attempts to build any compiled languages (C/C++, C#, or Java). # If this step fails, then you should remove it and run the build manually (see below) - name: Autobuild - uses: github/codeql-action/autobuild@c7f9125735019aa87cfc361530512d50ea439c71 # v3.25.1 + uses: github/codeql-action/autobuild@d39d31e687223d841ef683f52467bd88e9b21c14 # v3.25.3 # ℹ️ Command-line programs to run using the OS shell. # 📚 https://git.io/JvXDl @@ -80,4 +80,4 @@ jobs: # make release - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@c7f9125735019aa87cfc361530512d50ea439c71 # v3.25.1 + uses: github/codeql-action/analyze@d39d31e687223d841ef683f52467bd88e9b21c14 # v3.25.3 From 18f736ffc5efbd2e2dcdf08c12fa8c14ca883791 Mon Sep 17 00:00:00 2001 From: JeevaRamanathan Date: Sat, 27 Apr 2024 22:22:06 +0530 Subject: [PATCH 38/90] Add examples for scaffolder action & improve related tests Signed-off-by: JeevaRamanathan --- .changeset/tame-jars-double.md | 5 + ...tlabProjectVariableAction.examples.test.ts | 236 ++++++++++++++++++ ...ateGitlabProjectVariableAction.examples.ts | 160 ++++++++++++ .../createGitlabProjectVariableAction.ts | 3 +- 4 files changed, 403 insertions(+), 1 deletion(-) create mode 100644 .changeset/tame-jars-double.md create mode 100644 plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.examples.test.ts create mode 100644 plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.examples.ts diff --git a/.changeset/tame-jars-double.md b/.changeset/tame-jars-double.md new file mode 100644 index 0000000000..8af92a2d72 --- /dev/null +++ b/.changeset/tame-jars-double.md @@ -0,0 +1,5 @@ +--- +'@backstage/plugin-scaffolder-backend-module-gitlab': minor +--- + +Add examples for `gitlab:projectVariable:create` scaffolder action & improve related tests diff --git a/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.examples.test.ts b/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.examples.test.ts new file mode 100644 index 0000000000..d1c37ccd9d --- /dev/null +++ b/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.examples.test.ts @@ -0,0 +1,236 @@ +/* + * Copyright 2021 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { createGitlabProjectVariableAction } from './createGitlabProjectVariableAction'; +import { createMockActionContext } from '@backstage/plugin-scaffolder-node-test-utils'; +import { ScmIntegrations } from '@backstage/integration'; +import { ConfigReader } from '@backstage/config'; +import yaml from 'yaml'; +import { examples } from './createGitlabProjectVariableAction.examples'; + +const mockGitlabClient = { + ProjectVariables: { + create: jest.fn(), + }, +}; +jest.mock('@gitbeaker/node', () => ({ + Gitlab: class { + constructor() { + return mockGitlabClient; + } + }, +})); + +describe('gitlab:projectVariableAction: create examples', () => { + const config = new ConfigReader({ + integrations: { + gitlab: [ + { + host: 'gitlab.com', + token: 'tokenlols', + apiBaseUrl: 'https://api.gitlab.com', + }, + { + host: 'hosted.gitlab.com', + apiBaseUrl: 'https://api.hosted.gitlab.com', + }, + ], + }, + }); + + const integrations = ScmIntegrations.fromConfig(config); + const action = createGitlabProjectVariableAction({ integrations }); + const mockContext = createMockActionContext({ + input: { + repoUrl: 'gitlab.com?repo=repo&owner=owner', + projectId: '123', + key: 'MY_VARIABLE', + value: 'my_value', + variableType: 'env_var', + }, + }); + + beforeEach(() => { + jest.resetAllMocks(); + }); + + it(`Should ${examples[0].description}`, async () => { + mockGitlabClient.ProjectVariables.create.mockResolvedValue({ + token: 'TOKEN', + }); + + await action.handler({ + ...mockContext, + input: yaml.parse(examples[0].example).steps[0].input, + }); + + expect(mockGitlabClient.ProjectVariables.create).toHaveBeenCalledWith( + '123', + { + key: 'MY_VARIABLE', + value: 'my_value', + variable_type: 'env_var', + environment_scope: '*', + masked: false, + protected: false, + raw: false, + }, + ); + }); + it(`Should ${examples[1].description}`, async () => { + mockGitlabClient.ProjectVariables.create.mockResolvedValue({ + token: 'TOKEN', + }); + + await action.handler({ + ...mockContext, + input: yaml.parse(examples[1].example).steps[0].input, + }); + + expect(mockGitlabClient.ProjectVariables.create).toHaveBeenCalledWith( + '123', + { + key: 'MY_VARIABLE', + value: 'my-file-content', + protected: false, + masked: false, + raw: false, + environment_scope: '*', + variable_type: 'file', + }, + ); + }); + + it(`Should ${examples[2].description}`, async () => { + mockGitlabClient.ProjectVariables.create.mockResolvedValue({ + token: 'TOKEN', + }); + + await action.handler({ + ...mockContext, + input: yaml.parse(examples[2].example).steps[0].input, + }); + + expect(mockGitlabClient.ProjectVariables.create).toHaveBeenCalledWith( + '456', + { + key: 'MY_VARIABLE', + value: 'my_value', + masked: false, + raw: false, + environment_scope: '*', + variable_type: 'env_var', + protected: true, + }, + ); + }); + + it(`Should ${examples[3].description}`, async () => { + mockGitlabClient.ProjectVariables.create.mockResolvedValue({ + token: 'TOKEN', + }); + + await action.handler({ + ...mockContext, + input: yaml.parse(examples[3].example).steps[0].input, + }); + + expect(mockGitlabClient.ProjectVariables.create).toHaveBeenCalledWith( + '789', + { + key: 'DB_PASSWORD', + value: 'password123', + protected: false, + raw: false, + environment_scope: '*', + variable_type: 'env_var', + masked: true, + }, + ); + }); + + it(`Should ${examples[4].description}`, async () => { + mockGitlabClient.ProjectVariables.create.mockResolvedValue({ + token: 'TOKEN', + }); + + await action.handler({ + ...mockContext, + input: yaml.parse(examples[4].example).steps[0].input, + }); + + expect(mockGitlabClient.ProjectVariables.create).toHaveBeenCalledWith( + '123', + { + key: 'MY_VARIABLE', + value: 'my_value', + protected: false, + environment_scope: '*', + variable_type: 'env_var', + masked: false, + raw: true, + }, + ); + }); + + it(`Should ${examples[5].description}`, async () => { + mockGitlabClient.ProjectVariables.create.mockResolvedValue({ + token: 'TOKEN', + }); + + await action.handler({ + ...mockContext, + input: yaml.parse(examples[5].example).steps[0].input, + }); + + expect(mockGitlabClient.ProjectVariables.create).toHaveBeenCalledWith( + '123', + { + key: 'MY_VARIABLE', + value: 'my_value', + protected: false, + variable_type: 'env_var', + masked: false, + raw: false, + environment_scope: 'production', + }, + ); + }); + + it(`Should ${examples[6].description}`, async () => { + mockGitlabClient.ProjectVariables.create.mockResolvedValue({ + token: 'TOKEN', + }); + + await action.handler({ + ...mockContext, + input: yaml.parse(examples[6].example).steps[0].input, + }); + + expect(mockGitlabClient.ProjectVariables.create).toHaveBeenCalledWith( + '123', + { + key: 'MY_VARIABLE', + value: 'my_value', + protected: false, + variable_type: 'env_var', + masked: false, + raw: false, + environment_scope: '*', + }, + ); + }); +}); diff --git a/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.examples.ts b/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.examples.ts new file mode 100644 index 0000000000..81d526f7c8 --- /dev/null +++ b/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.examples.ts @@ -0,0 +1,160 @@ +/* + * Copyright 2023 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +import { TemplateExample } from '@backstage/plugin-scaffolder-node'; +import yaml from 'yaml'; + +export const examples: TemplateExample[] = [ + { + description: 'Creating a GitLab project variable of type env_var', + example: yaml.stringify({ + steps: [ + { + id: 'createVariable', + action: 'gitlab:createGitlabProjectVariableAction', + name: 'Create GitLab Project Variable', + input: { + repoUrl: 'gitlab.com?repo=repo&owner=owner', + projectId: '123', + key: 'MY_VARIABLE', + value: 'my_value', + variableType: 'env_var', + }, + }, + ], + }), + }, + { + description: 'Creating a GitLab project variable of type file', + example: yaml.stringify({ + steps: [ + { + id: 'createVariable', + action: 'gitlab:createGitlabProjectVariableAction', + name: 'Create GitLab Project Variable', + input: { + repoUrl: 'gitlab.com?repo=repo&owner=owner', + projectId: '123', + key: 'MY_VARIABLE', + value: 'my-file-content', + variableType: 'file', + }, + }, + ], + }), + }, + { + description: 'Create a GitLab project variable that is protected.', + example: yaml.stringify({ + steps: [ + { + id: 'createVariable', + action: 'gitlab:createGitlabProjectVariableAction', + name: 'Create GitLab Project Variable', + input: { + repoUrl: 'gitlab.com?repo=repo&owner=owner', + projectId: '456', + key: 'MY_VARIABLE', + value: 'my_value', + variableType: 'env_var', + variableProtected: true, + }, + }, + ], + }), + }, + { + description: 'Create a GitLab project variable with masked flag as true', + example: yaml.stringify({ + steps: [ + { + id: 'createVariable', + action: 'gitlab:createGitlabProjectVariableAction', + name: 'Create GitLab Project Variable', + input: { + repoUrl: 'gitlab.com?repo=repo&owner=owner', + projectId: '789', + key: 'DB_PASSWORD', + value: 'password123', + variableType: 'env_var', + masked: true, + }, + }, + ], + }), + }, + { + description: 'Create a GitLab project variable that is expandable.', + example: yaml.stringify({ + steps: [ + { + id: 'createVariable', + action: 'gitlab:projectVariable:create', + name: 'Create GitLab Project Variable', + input: { + repoUrl: 'gitlab.com?repo=repo&owner=owner', + projectId: '123', + key: 'MY_VARIABLE', + value: 'my_value', + variableType: 'env_var', + raw: true, + }, + }, + ], + }), + }, + { + description: + 'Create a GitLab project variable with a specific environment scope.', + example: yaml.stringify({ + steps: [ + { + id: 'createVariable', + action: 'gitlab:projectVariable:create', + name: 'Create GitLab Project Variable', + input: { + repoUrl: 'gitlab.com?repo=repo&owner=owner', + projectId: '123', + key: 'MY_VARIABLE', + value: 'my_value', + variableType: 'env_var', + environmentScope: 'production', + }, + }, + ], + }), + }, + { + description: + 'Create a GitLab project variable with a wildcard environment scope.', + example: yaml.stringify({ + steps: [ + { + id: 'createVariable', + action: 'gitlab:projectVariable:create', + name: 'Create GitLab Project Variable', + input: { + repoUrl: 'gitlab.com?repo=repo&owner=owner', + projectId: '123', + key: 'MY_VARIABLE', + value: 'my_value', + variableType: 'env_var', + environmentScope: '*', + }, + }, + ], + }), + }, +]; diff --git a/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.ts b/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.ts index a154c16662..e09a074701 100644 --- a/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.ts +++ b/plugins/scaffolder-backend-module-gitlab/src/actions/createGitlabProjectVariableAction.ts @@ -20,6 +20,7 @@ import { Gitlab } from '@gitbeaker/node'; import { getToken } from '../util'; import commonGitlabConfig from '../commonGitlabConfig'; import { z } from 'zod'; +import { examples } from './createGitlabProjectVariableAction.examples'; /** * Creates a `gitlab:projectVariable:create` Scaffolder action. @@ -33,6 +34,7 @@ export const createGitlabProjectVariableAction = (options: { const { integrations } = options; return createTemplateAction({ id: 'gitlab:projectVariable:create', + examples, schema: { input: commonGitlabConfig.merge( z.object({ @@ -85,7 +87,6 @@ export const createGitlabProjectVariableAction = (options: { host: integrationConfig.config.baseUrl, token: token, }); - await api.ProjectVariables.create(projectId, { key: key, value: value, From 2cc750d36766a43c277760af681a28e373867243 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Fri, 19 Apr 2024 00:55:19 -0600 Subject: [PATCH 39/90] feat: integration support for harness Signed-off-by: Calvin Lee --- .changeset/empty-beers-relax.md | 5 + .changeset/tasty-rats-explain.md | 5 + .../config/vocabularies/Backstage/accept.txt | 2 + docs/integrations/harness/locations.md | 33 +++ microsite/sidebars.json | 5 + packages/backend-common/api-report.md | 26 +++ .../src/reading/HarnessCodeUrlReader.test.ts | 204 ++++++++++++++++++ .../src/reading/HarnessUrlReader.ts | 124 +++++++++++ .../backend-common/src/reading/UrlReaders.ts | 3 +- packages/backend-common/src/reading/index.ts | 1 + packages/integration/api-report.md | 55 ++++- packages/integration/config.d.ts | 24 +++ .../integration/src/ScmIntegrations.test.ts | 9 + packages/integration/src/ScmIntegrations.ts | 7 + .../src/harness/HarnessIntegration.test.ts | 128 +++++++++++ .../src/harness/HarnessIntegration.ts | 58 +++++ .../integration/src/harness/config.test.ts | 108 ++++++++++ packages/integration/src/harness/config.ts | 78 +++++++ packages/integration/src/harness/core.test.ts | 95 ++++++++ packages/integration/src/harness/core.ts | 135 ++++++++++++ packages/integration/src/harness/index.ts | 19 ++ packages/integration/src/index.ts | 1 + packages/integration/src/registry.ts | 2 + 23 files changed, 1125 insertions(+), 2 deletions(-) create mode 100644 .changeset/empty-beers-relax.md create mode 100644 .changeset/tasty-rats-explain.md create mode 100644 docs/integrations/harness/locations.md create mode 100644 packages/backend-common/src/reading/HarnessCodeUrlReader.test.ts create mode 100644 packages/backend-common/src/reading/HarnessUrlReader.ts create mode 100644 packages/integration/src/harness/HarnessIntegration.test.ts create mode 100644 packages/integration/src/harness/HarnessIntegration.ts create mode 100644 packages/integration/src/harness/config.test.ts create mode 100644 packages/integration/src/harness/config.ts create mode 100644 packages/integration/src/harness/core.test.ts create mode 100644 packages/integration/src/harness/core.ts create mode 100644 packages/integration/src/harness/index.ts diff --git a/.changeset/empty-beers-relax.md b/.changeset/empty-beers-relax.md new file mode 100644 index 0000000000..526f88c1c7 --- /dev/null +++ b/.changeset/empty-beers-relax.md @@ -0,0 +1,5 @@ +--- +'@backstage/backend-common': patch +--- + +This patch adds HarnessURLReader to the available classes. It currently only reads single files via Harness codes public repo api. diff --git a/.changeset/tasty-rats-explain.md b/.changeset/tasty-rats-explain.md new file mode 100644 index 0000000000..aee8057915 --- /dev/null +++ b/.changeset/tasty-rats-explain.md @@ -0,0 +1,5 @@ +--- +'@backstage/integration': minor +--- + +This patch brings Harness Code as a valid integration via the ScmIntgration interface. It adds harness code to the relevant static properties ( get integration by name, get integration by type) for plugs to be able to reference the same harness code server diff --git a/.github/vale/config/vocabularies/Backstage/accept.txt b/.github/vale/config/vocabularies/Backstage/accept.txt index 026e5a9b18..2e42ff7998 100644 --- a/.github/vale/config/vocabularies/Backstage/accept.txt +++ b/.github/vale/config/vocabularies/Backstage/accept.txt @@ -152,6 +152,8 @@ graphviz Hackathons haproxy hardcoded +Harness +harness Helidon Henneke Heroku diff --git a/docs/integrations/harness/locations.md b/docs/integrations/harness/locations.md new file mode 100644 index 0000000000..8911d7918c --- /dev/null +++ b/docs/integrations/harness/locations.md @@ -0,0 +1,33 @@ +--- +id: locations +title: Harness Locations +sidebar_label: Locations +description: Integrating source code stored in Harness Code into the Backstage catalog +--- + +The Harness Code integration supports loading catalog entities from a hosted repository. Entities can be added to +[static catalog configuration](../../features/software-catalog/configuration.md), +registered with the +[catalog-import](https://github.com/backstage/backstage/tree/master/plugins/catalog-import) +plugin. + +## Configuration + +To use this integration, add configuration to your root `app-config.yaml`: + +```yaml +integrations: + harness: + - host: app.harness.io + token: ${HARNESS_CODE_BEARER_TOKEN} +``` + +Directly under the `harnessCode` key is a list of provider configurations, where you +can list the Gitea instances you want to be able to fetch +data from. Each entry is a structure with up to four elements: + +- `host`: The host of the Harness Code instance that you want to match on. +- `baseUrl` (optional): Needed if the Harness Code instance is not reachable at + the base of the `host` option (e.g. `https://app.harness.io`). This is the address that you would open in a browser. +- `username` (optional): The gitea username to use in API requests. +- `token` (optional): The password or api token to authenticate with. diff --git a/microsite/sidebars.json b/microsite/sidebars.json index 32863c61db..8291549153 100644 --- a/microsite/sidebars.json +++ b/microsite/sidebars.json @@ -238,6 +238,11 @@ "label": "Gitea", "items": ["integrations/gitea/locations"] }, + { + "type": "subcategory", + "label": "Harness", + "ids": ["integrations/harness/locations"] + }, { "type": "category", "label": "Google GCS", diff --git a/packages/backend-common/api-report.md b/packages/backend-common/api-report.md index f9e0d76aab..37bfda9861 100644 --- a/packages/backend-common/api-report.md +++ b/packages/backend-common/api-report.md @@ -30,6 +30,7 @@ import { GiteaIntegration } from '@backstage/integration'; import { GithubCredentialsProvider } from '@backstage/integration'; import { GithubIntegration } from '@backstage/integration'; import { GitLabIntegration } from '@backstage/integration'; +import { HarnessIntegration } from '@backstage/integration'; import { HostDiscovery as HostDiscovery_2 } from '@backstage/backend-app-api'; import { HttpAuthService } from '@backstage/backend-plugin-api'; import { IdentityService } from '@backstage/backend-plugin-api'; @@ -530,6 +531,31 @@ export class GitlabUrlReader implements UrlReader { toString(): string; } +// @public +export class HarnessUrlReader implements UrlReader { + constructor(integration: HarnessIntegration); + // (undocumented) + static factory: ReaderFactory; + // (undocumented) + read(url: string): Promise; + // (undocumented) + readTree(): Promise; + // (undocumented) + readUrl(url: string, options?: ReadUrlOptions): Promise; + // (undocumented) + search(): Promise; + // (undocumented) + toString(): string; +} + +// @public +export type HarnessIntegrationConfig = { + host: string; + baseUrl?: string; + username?: string; + token?: string; +}; + // @public export const HostDiscovery: typeof HostDiscovery_2; diff --git a/packages/backend-common/src/reading/HarnessCodeUrlReader.test.ts b/packages/backend-common/src/reading/HarnessCodeUrlReader.test.ts new file mode 100644 index 0000000000..3e8b70a6aa --- /dev/null +++ b/packages/backend-common/src/reading/HarnessCodeUrlReader.test.ts @@ -0,0 +1,204 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { setupRequestMockHandlers } from '@backstage/backend-test-utils'; +import { ConfigReader } from '@backstage/config'; +import { HarnessIntegration, readHarnessConfig } from '@backstage/integration'; +import { JsonObject } from '@backstage/types'; +import { rest } from 'msw'; +import { setupServer } from 'msw/node'; +import { getVoidLogger } from '../logging'; +import { UrlReaderPredicateTuple } from './types'; +import { DefaultReadTreeResponseFactory } from './tree'; +import getRawBody from 'raw-body'; +import { HarnessUrlReader } from './HarnessUrlReader'; +import { NotFoundError } from '@backstage/errors'; + +const treeResponseFactory = DefaultReadTreeResponseFactory.create({ + config: new ConfigReader({}), +}); + +jest.mock('../scm', () => ({ + Git: { + fromAuth: () => ({ + clone: jest.fn(() => Promise.resolve({})), + }), + }, +})); + +const harnessProcessor = new HarnessUrlReader( + new HarnessIntegration( + readHarnessConfig( + new ConfigReader({ + host: 'app.harness.io', + }), + ), + ), +); + +const createReader = (config: JsonObject): UrlReaderPredicateTuple[] => { + return HarnessUrlReader.factory({ + config: new ConfigReader(config), + logger: getVoidLogger(), + treeResponseFactory, + }); +}; + +describe('HarnessUrlReader', () => { + const worker = setupServer(); + setupRequestMockHandlers(worker); + + afterAll(() => { + jest.clearAllMocks(); + }); + + describe('reader factory', () => { + it('creates a reader.', () => { + const readers = createReader({ + integrations: { + harness: [{ host: 'app.harness.io' }], + }, + }); + expect(readers).toHaveLength(1); + }); + + it('should not create a default entry.', () => { + const readers = createReader({ + integrations: {}, + }); + expect(readers).toHaveLength(0); + }); + }); + + describe('predicates', () => { + it('returns true for the configured host', () => { + const readers = createReader({ + integrations: { + harness: [{ host: 'app.harness.io' }], + }, + }); + const predicate = readers[0].predicate; + + expect(predicate(new URL('https://app.harness.io/path'))).toBe(true); + }); + + it('returns false for a different host.', () => { + const readers = createReader({ + integrations: { + harness: [{ host: 'app.harness.io' }], + }, + }); + const predicate = readers[0].predicate; + + expect(predicate(new URL('https://github.com/path'))).toBe(false); + }); + }); + + describe('readUrl', () => { + const responseBuffer = Buffer.from('Apache License'); + const harnessApiResponse = (content: any) => { + return JSON.stringify({ + encoding: 'base64', + content: Buffer.from(content).toString('base64'), + }); + }; + + it.skip('should be able to read file contents as buffer', async () => { + worker.use( + rest.get( + 'https://app.harness.io/api/v1/repos/owner/project/contents/LICENSE', + (req, res, ctx) => { + // Test utils prefers matching URL directly but it is part of Gitea's API + if (req.url.searchParams.get('ref') === 'branch2') { + return res( + ctx.status(200), + ctx.body(harnessApiResponse(responseBuffer.toString())), + ); + } + + return res(ctx.status(500)); + }, + ), + ); + + const result = await harnessProcessor.readUrl( + 'https://app.harness.io/owner/project/src/branch/branch2/LICENSE', + ); + const buffer = await result.buffer(); + expect(buffer.toString()).toBe(responseBuffer.toString()); + }); + + it.skip('should be able to read file contents as stream', async () => { + worker.use( + rest.get( + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/LICENSE.txt', + (req, res, ctx) => { + if (req.url.searchParams.get('ref') === 'refMain') { + return res( + ctx.status(200), + ctx.body(harnessApiResponse(responseBuffer.toString())), + ); + } + + return res(ctx.status(500)); + }, + ), + ); + + const result = await harnessProcessor.readUrl( + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/LICENSE.TXT', + ); + const fromStream = await getRawBody(result.stream!()); + expect(fromStream.toString()).toBe(responseBuffer.toString()); + }); + + it.skip('should raise NotFoundError on 404.', async () => { + worker.use( + rest.get( + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', + (_, res, ctx) => { + return res(ctx.status(404, 'File not found.')); + }, + ), + ); + + await expect( + harnessProcessor.readUrl( + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', + ), + ).rejects.toThrow(NotFoundError); + }); + + it.skip('should throw an error on non 404 errors.', async () => { + worker.use( + rest.get( + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', + (_, res, ctx) => { + return res(ctx.status(500, 'Error!!!')); + }, + ), + ); + + await expect( + harnessProcessor.readUrl( + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', + ), + ).rejects.toThrow( + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/content/all-apis.yaml?routingId=accountId&include_commit=false&ref=refMain, 500 Error!!!', + ); + }); + }); +}); diff --git a/packages/backend-common/src/reading/HarnessUrlReader.ts b/packages/backend-common/src/reading/HarnessUrlReader.ts new file mode 100644 index 0000000000..fd01e4f8a2 --- /dev/null +++ b/packages/backend-common/src/reading/HarnessUrlReader.ts @@ -0,0 +1,124 @@ +/* + * Copyright 2025 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +import { + getHarnessRequestOptions, + getHarnessFileContentsUrl, + HarnessIntegration, + ScmIntegrations, +} from '@backstage/integration'; +import { ReadUrlOptions, ReadUrlResponse } from './types'; +import { + ReaderFactory, + ReadTreeResponse, + SearchResponse, + UrlReader, +} from './types'; +import fetch, { Response } from 'node-fetch'; +import { ReadUrlResponseFactory } from './ReadUrlResponseFactory'; +import { + AuthenticationError, + NotFoundError, + NotModifiedError, +} from '@backstage/errors'; +import { Readable } from 'stream'; + +/** + * Implements a {@link UrlReader} for the Harness code v1 api. + * + * @public + */ +export class HarnessUrlReader implements UrlReader { + static factory: ReaderFactory = ({ config }) => { + return ScmIntegrations.fromConfig(config) + .harness.list() + .map(integration => { + const reader = new HarnessUrlReader(integration); + const predicate = (url: URL) => { + return url.host === integration.config.host; + }; + return { reader, predicate }; + }); + }; + + constructor(private readonly integration: HarnessIntegration) {} + + async read(url: string): Promise { + const response = await this.readUrl(url); + return response.buffer(); + } + + async readUrl( + url: string, + options?: ReadUrlOptions, + ): Promise { + let response: Response; + const blobUrl = getHarnessFileContentsUrl(this.integration.config, url); + + try { + response = await fetch(blobUrl, { + method: 'GET', + ...getHarnessRequestOptions(this.integration.config), + signal: options?.signal as any, + }); + } catch (e) { + throw new Error(`Unable to read ${blobUrl}, ${e}`); + } + + if (response.ok) { + // Harness Code returns an object with the file contents encoded, not the file itself + const jsonResponse = await response.json(); + if (jsonResponse?.content?.encoding === 'base64') { + return ReadUrlResponseFactory.fromReadable( + Readable.from(Buffer.from(jsonResponse?.content?.data, 'base64')), + { + etag: response.headers.get('ETag') ?? undefined, + }, + ); + } + + throw new Error(`Unknown encoding: ${jsonResponse?.content?.encoding}`); + } + + const message = `${url} x ${blobUrl}, ${response.status} ${response.statusText}`; + if (response.status === 404) { + throw new NotFoundError(message); + } + + if (response.status === 304) { + throw new NotModifiedError(); + } + + if (response.status === 403) { + throw new AuthenticationError(); + } + + throw new Error(message); + } + + readTree(): Promise { + throw new Error('HarnessUrlReader readTree not implemented.'); + } + search(): Promise { + throw new Error('HarnessUrlReader search not implemented.'); + } + + toString() { + const { host } = this.integration.config; + return `harness{host=${host},authed=${Boolean( + this.integration.config.token, + )}}`; + } +} diff --git a/packages/backend-common/src/reading/UrlReaders.ts b/packages/backend-common/src/reading/UrlReaders.ts index e987eb9186..7aaaf120b4 100644 --- a/packages/backend-common/src/reading/UrlReaders.ts +++ b/packages/backend-common/src/reading/UrlReaders.ts @@ -31,6 +31,7 @@ import { GoogleGcsUrlReader } from './GoogleGcsUrlReader'; import { AwsS3UrlReader } from './AwsS3UrlReader'; import { GiteaUrlReader } from './GiteaUrlReader'; import { AwsCodeCommitUrlReader } from './AwsCodeCommitUrlReader'; +import { HarnessUrlReader } from './HarnessUrlReader'; /** * Creation options for {@link @backstage/backend-plugin-api#UrlReaderService}. @@ -61,7 +62,6 @@ export class UrlReaders { const treeResponseFactory = DefaultReadTreeResponseFactory.create({ config, }); - for (const factory of factories ?? []) { const tuples = factory({ config, logger: logger, treeResponseFactory }); @@ -94,6 +94,7 @@ export class UrlReaders { GiteaUrlReader.factory, GitlabUrlReader.factory, GoogleGcsUrlReader.factory, + HarnessUrlReader.factory, AwsS3UrlReader.factory, AwsCodeCommitUrlReader.factory, FetchUrlReader.factory, diff --git a/packages/backend-common/src/reading/index.ts b/packages/backend-common/src/reading/index.ts index 21e82da9b5..a99bb6dda1 100644 --- a/packages/backend-common/src/reading/index.ts +++ b/packages/backend-common/src/reading/index.ts @@ -22,6 +22,7 @@ export { GerritUrlReader } from './GerritUrlReader'; export { GithubUrlReader } from './GithubUrlReader'; export { GitlabUrlReader } from './GitlabUrlReader'; export { GiteaUrlReader } from './GiteaUrlReader'; +export { HarnessUrlReader } from './HarnessUrlReader'; export { AwsS3UrlReader } from './AwsS3UrlReader'; export { FetchUrlReader } from './FetchUrlReader'; export { ReadUrlResponseFactory } from './ReadUrlResponseFactory'; diff --git a/packages/integration/api-report.md b/packages/integration/api-report.md index 6b691dff80..a693c1314c 100644 --- a/packages/integration/api-report.md +++ b/packages/integration/api-report.md @@ -513,6 +513,19 @@ export function getGitLabRequestOptions(config: GitLabIntegrationConfig): { headers: Record; }; +// @public +export function getHarnessFileContentsUrl( + config: HarnessIntegrationConfig, + url: string, +): string; + +// @public +export function getHarnessRequestOptions( + config: HarnessIntegrationConfig, +): { + headers?: Record; +}; + // @public export class GiteaIntegration implements ScmIntegration { constructor(config: GiteaIntegrationConfig); @@ -539,7 +552,7 @@ export type GiteaIntegrationConfig = { host: string; baseUrl?: string; username?: string; - password?: string; + token?: string; }; // @public @@ -674,6 +687,35 @@ export type GoogleGcsIntegrationConfig = { privateKey?: string; }; +// @public +export class HarnessIntegration implements ScmIntegration { + constructor(config: HarnessIntegrationConfig); + // (undocumented) + readonly config: HarnessIntegrationConfig; + // (undocumented) + static factory: ScmIntegrationsFactory; + // (undocumented) + resolveEditUrl(url: string): string; + // (undocumented) + resolveUrl(options: { + url: string; + base: string; + lineNumber?: number | undefined; + }): string; + // (undocumented) + get title(): string; + // (undocumented) + get type(): string; +} + +// @public +export type HarnessIntegrationConfig = { + host: string; + baseUrl?: string; + username?: string; + token?: string; +}; + // @public export interface IntegrationsByType { // (undocumented) @@ -696,6 +738,8 @@ export interface IntegrationsByType { github: ScmIntegrationsGroup; // (undocumented) gitlab: ScmIntegrationsGroup; + // (undocumented) + harness: ScmIntegrationsGroup; } // @public @@ -839,6 +883,11 @@ export function readGoogleGcsIntegrationConfig( config: Config, ): GoogleGcsIntegrationConfig; +// @public +export function readHarnessConfig( + config: Config, +): HarnessIntegrationConfig; + // @public @deprecated (undocumented) export const replaceGitHubUrlType: typeof replaceGithubUrlType; @@ -889,6 +938,8 @@ export interface ScmIntegrationRegistry github: ScmIntegrationsGroup; // (undocumented) gitlab: ScmIntegrationsGroup; + // (undocumented) + harness: ScmIntegrationsGroup; resolveEditUrl(url: string): string; resolveUrl(options: { url: string; @@ -927,6 +978,8 @@ export class ScmIntegrations implements ScmIntegrationRegistry { // (undocumented) get gitlab(): ScmIntegrationsGroup; // (undocumented) + get harness(): ScmIntegrationsGroup; + // (undocumented) list(): ScmIntegration[]; // (undocumented) resolveEditUrl(url: string): string; diff --git a/packages/integration/config.d.ts b/packages/integration/config.d.ts index 44303bf25c..01f1f44d73 100644 --- a/packages/integration/config.d.ts +++ b/packages/integration/config.d.ts @@ -345,5 +345,29 @@ export interface Config { */ password?: string; }>; + /** Integration configuration for Harness Code */ + harness?: Array<{ + /** + * The hostname of the given Harness Code instance + * @visibility frontend + */ + host: string; + /** + * The base url for the Gitea instance. + * @visibility frontend + */ + baseUrl?: string; + + /** + * The username to use for authenticated requests. + * @visibility secret + */ + username?: string; + /** + * Harness Code token used to authenticate requests. This can be either a generated access token. + * @visibility secret + */ + token?: string; + }>; }; } diff --git a/packages/integration/src/ScmIntegrations.test.ts b/packages/integration/src/ScmIntegrations.test.ts index acf602e38d..c758816c03 100644 --- a/packages/integration/src/ScmIntegrations.test.ts +++ b/packages/integration/src/ScmIntegrations.test.ts @@ -38,6 +38,7 @@ import { ScmIntegrations } from './ScmIntegrations'; import { GiteaIntegration, GiteaIntegrationConfig } from './gitea'; import { AwsCodeCommitIntegration } from './awsCodeCommit/AwsCodeCommitIntegration'; import { AwsCodeCommitIntegrationConfig } from './awsCodeCommit'; +import { HarnessIntegration, HarnessIntegrationConfig } from './harness'; describe('ScmIntegrations', () => { const awsS3 = new AwsS3Integration({ @@ -80,6 +81,10 @@ describe('ScmIntegrations', () => { host: 'gitea.local', } as GiteaIntegrationConfig); + const harness = new HarnessIntegration({ + host: 'harness.local', + } as HarnessIntegrationConfig); + const i = new ScmIntegrations({ awsS3: basicIntegrations([awsS3], item => item.config.host), awsCodeCommit: basicIntegrations([awsCodeCommit], item => item.config.host), @@ -94,6 +99,7 @@ describe('ScmIntegrations', () => { github: basicIntegrations([github], item => item.config.host), gitlab: basicIntegrations([gitlab], item => item.config.host), gitea: basicIntegrations([gitea], item => item.config.host), + harness: basicIntegrations([harness], item => item.config.host), }); it('can get the specifics', () => { @@ -113,6 +119,7 @@ describe('ScmIntegrations', () => { expect(i.github.byUrl('https://github.local')).toBe(github); expect(i.gitlab.byUrl('https://gitlab.local')).toBe(gitlab); expect(i.gitea.byUrl('https://gitea.local')).toBe(gitea); + expect(i.harness.byUrl('https://harness.local')).toBe(harness); }); it('can list', () => { @@ -128,6 +135,7 @@ describe('ScmIntegrations', () => { github, gitlab, gitea, + harness, ]), ); }); @@ -143,6 +151,7 @@ describe('ScmIntegrations', () => { expect(i.byUrl('https://github.local')).toBe(github); expect(i.byUrl('https://gitlab.local')).toBe(gitlab); expect(i.byUrl('https://gitea.local')).toBe(gitea); + expect(i.byUrl('https://harness.local')).toBe(harness); expect(i.byHost('awss3.local')).toBe(awsS3); expect(i.byHost('awscodecommit.local')).toBe(awsCodeCommit); diff --git a/packages/integration/src/ScmIntegrations.ts b/packages/integration/src/ScmIntegrations.ts index 6a1faaa75b..e0d430d834 100644 --- a/packages/integration/src/ScmIntegrations.ts +++ b/packages/integration/src/ScmIntegrations.ts @@ -28,6 +28,7 @@ import { defaultScmResolveUrl } from './helpers'; import { ScmIntegration, ScmIntegrationsGroup } from './types'; import { ScmIntegrationRegistry } from './registry'; import { GiteaIntegration } from './gitea'; +import { HarnessIntegration } from './harness/HarnessIntegration'; /** * The set of supported integrations. @@ -48,6 +49,7 @@ export interface IntegrationsByType { github: ScmIntegrationsGroup; gitlab: ScmIntegrationsGroup; gitea: ScmIntegrationsGroup; + harness: ScmIntegrationsGroup; } /** @@ -70,6 +72,7 @@ export class ScmIntegrations implements ScmIntegrationRegistry { github: GithubIntegration.factory({ config }), gitlab: GitLabIntegration.factory({ config }), gitea: GiteaIntegration.factory({ config }), + harness: HarnessIntegration.factory({ config }), }); } @@ -120,6 +123,10 @@ export class ScmIntegrations implements ScmIntegrationRegistry { return this.byType.gitea; } + get harness(): ScmIntegrationsGroup { + return this.byType.harness; + } + list(): ScmIntegration[] { return Object.values(this.byType).flatMap( i => i.list() as ScmIntegration[], diff --git a/packages/integration/src/harness/HarnessIntegration.test.ts b/packages/integration/src/harness/HarnessIntegration.test.ts new file mode 100644 index 0000000000..9014c0bb79 --- /dev/null +++ b/packages/integration/src/harness/HarnessIntegration.test.ts @@ -0,0 +1,128 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { ConfigReader } from '@backstage/config'; +import { HarnessIntegration } from './HarnessIntegration'; + +describe('HarnessIntegration', () => { + it('has a working factory', () => { + const integrations = HarnessIntegration.factory({ + config: new ConfigReader({ + integrations: { + harness: [ + { + host: 'app.harness.io', + username: 'git', + baseUrl: 'https://app.harness.io/route', + token: '1234', + }, + ], + }, + }), + }); + expect(integrations.list().length).toBe(1); + expect(integrations.list()[0].config.host).toBe('app.harness.io'); + expect(integrations.list()[0].config.baseUrl).toBe( + 'https://app.harness.io/route', + ); + }); + + it('returns the basics', () => { + const integration = new HarnessIntegration({ + host: 'app.harness.io', + }); + expect(integration.type).toBe('harness'); + expect(integration.title).toBe('app.harness.io'); + }); + + describe('resolveUrl', () => { + it('works for valid urls, ignoring line number', () => { + const integration = new HarnessIntegration({ + host: 'app.harness.io', + }); + + expect( + integration.resolveUrl({ + url: 'https://app.harness.io/catalog-info.yaml', + base: 'https://app.harness.io/catalog-info.yaml', + lineNumber: 9, + }), + ).toBe('https://app.harness.io/catalog-info.yaml'); + }); + + it('handles line numbers', () => { + const integration = new HarnessIntegration({ + host: 'app.harness.io', + }); + + expect( + integration.resolveUrl({ + url: '', + base: 'https://app.harness.io/catalog-info.yaml#4', + lineNumber: 9, + }), + ).toBe('https://app.harness.io/catalog-info.yaml#L9'); + }); + }); + + describe('resolves with a relative url', () => { + it('works for valid urls', () => { + const integration = new HarnessIntegration({ + host: 'app.harness.io', + }); + + expect( + integration.resolveUrl({ + url: './skeleton', + base: 'https://app.harness.io/git/plugins/repo/+/refs/heads/master/template.yaml', + }), + ).toBe( + 'https://app.harness.io/git/plugins/repo/+/refs/heads/master/skeleton', + ); + }); + }); + + describe('resolves with an absolute url', () => { + it('works for valid urls', () => { + const integration = new HarnessIntegration({ + host: 'app.harness.io', + }); + + expect( + integration.resolveUrl({ + url: '/catalog-info.yaml', + base: 'https://app.harness.io/git/repo/+/refs/heads/master/', + }), + ).toBe( + 'https://app.harness.io/git/repo/+/refs/heads/master/catalog-info.yaml', + ); + }); + }); + + it('resolve edit URL', () => { + const integration = new HarnessIntegration({ + host: 'app.harness.io', + }); + + expect( + integration.resolveEditUrl( + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/edit/refMain/~/all-apis.yaml', + ), + ).toBe( + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/edit/all-apis.yaml', + ); + }); +}); diff --git a/packages/integration/src/harness/HarnessIntegration.ts b/packages/integration/src/harness/HarnessIntegration.ts new file mode 100644 index 0000000000..5d1b274149 --- /dev/null +++ b/packages/integration/src/harness/HarnessIntegration.ts @@ -0,0 +1,58 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +import { basicIntegrations, defaultScmResolveUrl } from '../helpers'; +import { ScmIntegration, ScmIntegrationsFactory } from '../types'; +import { HarnessIntegrationConfig, readHarnessConfig } from './config'; +import { getHarnessEditContentsUrl } from './core'; + +/** + * A Harness Code based integration. + * + * @public + */ +export class HarnessIntegration implements ScmIntegration { + static factory: ScmIntegrationsFactory = ({ config }) => { + const configs = config.getOptionalConfigArray('integrations.harness') ?? []; + const harnessConfigs = configs.map(c => readHarnessConfig(c)); + + return basicIntegrations( + harnessConfigs.map(c => new HarnessIntegration(c)), + (harness: HarnessIntegration) => harness.config.host, + ); + }; + + constructor(readonly config: HarnessIntegrationConfig) {} + + get type(): string { + return 'harness'; + } + + get title(): string { + return this.config.host; + } + + resolveUrl(options: { + url: string; + base: string; + lineNumber?: number | undefined; + }): string { + return defaultScmResolveUrl(options); + } + + resolveEditUrl(url: string): string { + return getHarnessEditContentsUrl(this.config, url); + } +} diff --git a/packages/integration/src/harness/config.test.ts b/packages/integration/src/harness/config.test.ts new file mode 100644 index 0000000000..6482e89de8 --- /dev/null +++ b/packages/integration/src/harness/config.test.ts @@ -0,0 +1,108 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { Config, ConfigReader } from '@backstage/config'; +import { loadConfigSchema } from '@backstage/config-loader'; +import { HarnessIntegrationConfig, readHarnessConfig } from './config'; + +describe('readHarnessConfig', () => { + function buildConfig(data: Partial): Config { + return new ConfigReader(data); + } + + async function buildFrontendConfig( + data: Partial, + ): Promise { + const fullSchema = await loadConfigSchema({ + dependencies: ['@backstage/integration'], + }); + const serializedSchema = fullSchema.serialize() as { + schemas: { value: { properties?: { integrations?: object } } }[]; + }; + const schema = await loadConfigSchema({ + serialized: { + ...serializedSchema, // only include schemas that apply to integrations + schemas: serializedSchema.schemas.filter( + s => s.value?.properties?.integrations, + ), + }, + }); + const processed = schema.process( + [{ data: { integrations: { harness: [data] } }, context: 'app' }], + { visibility: ['frontend'] }, + ); + return new ConfigReader((processed[0].data as any).integrations.harness[0]); + } + + it('reads all values', () => { + const output = readHarnessConfig( + buildConfig({ + host: 'a.com', + baseUrl: 'https://a.com/route/api', + username: 'u', + token: 'p', + }), + ); + expect(output).toEqual({ + host: 'a.com', + baseUrl: 'https://a.com/route/api', + username: 'u', + token: 'p', + }); + }); + + it('can create a default value if the API base URL is missing', () => { + const output = readHarnessConfig( + buildConfig({ + host: 'a.com', + }), + ); + expect(output).toEqual({ + host: 'a.com', + baseUrl: 'https://a.com', + username: undefined, + token: undefined, + }); + }); + + it('rejects funky configs', () => { + const valid: any = { + host: 'a.com', + }; + expect(() => readHarnessConfig(buildConfig({ ...valid, host: 2 }))).toThrow( + /host/, + ); + expect(() => + readHarnessConfig(buildConfig({ ...valid, baseUrl: 2 })), + ).toThrow(/baseUrl/); + }); + + it('works on the frontend', async () => { + expect( + readHarnessConfig( + await buildFrontendConfig({ + host: 'a.com', + baseUrl: 'https://a.com/route', + username: 'u', + token: 'p', + }), + ), + ).toEqual({ + host: 'a.com', + baseUrl: 'https://a.com/route', + }); + }); +}); diff --git a/packages/integration/src/harness/config.ts b/packages/integration/src/harness/config.ts new file mode 100644 index 0000000000..75482a9b39 --- /dev/null +++ b/packages/integration/src/harness/config.ts @@ -0,0 +1,78 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { Config } from '@backstage/config'; +import { trimEnd } from 'lodash'; +import { isValidHost } from '../helpers'; + +/** + * The configuration for a single Gitea integration. + * + * @public + */ +export type HarnessIntegrationConfig = { + /** + * The host of the target that this matches on, e.g. "app.harness.io" + */ + host: string; + /** + * The optional base URL of the Harness code instance. It is assumed that https + * is used and that the base path is "/" on the host. If that is not the + * case set the complete base url to the Harness code instance, e.g. + * "https://harnesscode.website.com/". This is the url that you would open + * in a browser. + */ + baseUrl?: string; + /** + * The username to use for requests to harness code. + */ + username?: string; + + /** + * The password or http token to use for authentication. + */ + token?: string; +}; + +/** + * Parses a location config block for use in HarnessIntegration + * + * @public + */ +export function readHarnessConfig(config: Config): HarnessIntegrationConfig { + const host = config.getString('host'); + let baseUrl = config.getOptionalString('baseUrl'); + const username = config.getOptionalString('username'); + const token = config.getOptionalString('token'); + if (!isValidHost(host)) { + throw new Error( + `Invalid Harness Code integration config, '${host}' is not a valid host`, + ); + } + + if (baseUrl) { + baseUrl = trimEnd(baseUrl, '/'); + } else { + baseUrl = `https://${host}`; + } + + return { + host, + baseUrl, + username, + token, + }; +} diff --git a/packages/integration/src/harness/core.test.ts b/packages/integration/src/harness/core.test.ts new file mode 100644 index 0000000000..6502fd3e48 --- /dev/null +++ b/packages/integration/src/harness/core.test.ts @@ -0,0 +1,95 @@ +/* + * Copyright 2020 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { setupServer } from 'msw/node'; +import { setupRequestMockHandlers } from '@backstage/test-utils'; +import { HarnessIntegrationConfig } from './config'; +import { + getHarnessEditContentsUrl, + getHarnessFileContentsUrl, + getHarnessRequestOptions, +} from './core'; + +describe('Harness code core', () => { + const worker = setupServer(); + setupRequestMockHandlers(worker); + + describe('getHarnessFileContentsUrl', () => { + it('can create an url from arguments', () => { + const config: HarnessIntegrationConfig = { + host: 'app.harness.io', + }; + expect( + getHarnessFileContentsUrl( + config, + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', + ), + ).toEqual( + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/content/all-apis.yaml?routingId=accountId&include_commit=false&ref=refMain', + ); + }); + }); + + describe('getHarnessEditContentsUrl', () => { + it('can create an url from arguments', () => { + const config: HarnessIntegrationConfig = { + host: 'app.harness.io', + }; + expect( + getHarnessEditContentsUrl( + config, + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/edit/refMain/~/all-apis.yaml', + ), + ).toEqual( + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/edit/all-apis.yaml', + ); + }); + }); + + describe('getGerritRequestOptions', () => { + it('adds token header when only a token is specified', () => { + const authRequest: HarnessIntegrationConfig = { + host: 'gerrit.com', + token: 'P', + }; + const anonymousRequest: HarnessIntegrationConfig = { + host: 'gerrit.com', + }; + expect( + (getHarnessRequestOptions(authRequest).headers as any).Authorization, + ).toEqual('Bearer P'); + expect( + getHarnessRequestOptions(anonymousRequest).headers, + ).toBeUndefined(); + }); + + it('adds basic auth when username and token are specified', () => { + const authRequest: HarnessIntegrationConfig = { + host: 'gerrit.com', + username: 'username', + token: 'P', + }; + + const basicAuthentication = `basic ${Buffer.from( + `${authRequest.username}:${authRequest.token}`, + ).toString('base64')}`; + + expect( + (getHarnessRequestOptions(authRequest).headers as any).Authorization, + ).toEqual(basicAuthentication); + }); + }); +}); diff --git a/packages/integration/src/harness/core.ts b/packages/integration/src/harness/core.ts new file mode 100644 index 0000000000..0c4fb01970 --- /dev/null +++ b/packages/integration/src/harness/core.ts @@ -0,0 +1,135 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +import { HarnessIntegrationConfig } from './config'; + +/** + * Given a URL pointing to a file, returns a URL + * for editing the contents of the data. + * + * @remarks + * + * Converts + * from: https://app.harness.io/a/b/src/branchname/path/to/c.yaml + * or: https://app.harness.io/a/b/_edit/branchname/path/to/c.yaml + * + * @param url - A URL pointing to a file + * @param config - The relevant provider config + * @public + */ +export function getHarnessEditContentsUrl( + config: HarnessIntegrationConfig, + url: string, +) { + try { + const baseUrl = config.baseUrl ?? `https://${config.host}`; + const [ + _blank, + _ng, + _account, + accountId, + _module, + _moduleName, + _org, + orgName, + _projects, + projectName, + _repos, + repoName, + _files, + _ref, + _branch, + ...path + ] = url.replace(baseUrl, '').split('/'); + const pathWithoutSlash = path.join('/').replace(/^\//, ''); + return `${baseUrl}/gateway/code/api/v1/repos/${accountId}/${orgName}/${projectName}/${repoName}/+/edit/${pathWithoutSlash}`; + } catch (e) { + throw new Error(`Incorrect URL: ${url}, ${e}`); + } +} + +/** + * Given a URL pointing to a file, returns an api URL + * for fetching the contents of the data. + * + * @remarks + * + * Converts + * from: https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml + * to: https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/content/all-apis.yaml?routingId=accountId&include_commit=false&ref=refMain + * + * @param url - A URL pointing to a file + * @param config - The relevant provider config + * @public + */ +export function getHarnessFileContentsUrl( + config: HarnessIntegrationConfig, + url: string, +) { + try { + const baseUrl = config.baseUrl ?? `https://${config.host}`; + const [ + _blank, + _ng, + _account, + accountId, + _module, + _moduleName, + _org, + orgName, + _projects, + projectName, + _repos, + repoName, + _files, + ref, + _branch, + ...path + ] = url.replace(baseUrl, '').split('/'); + const pathWithoutSlash = path.join('/').replace(/^\//, ''); + return `${baseUrl}/gateway/code/api/v1/repos/${accountId}/${orgName}/${projectName}/${repoName}/+/content/${pathWithoutSlash}?routingId=${accountId}&include_commit=false&ref=${ref}`; + } catch (e) { + throw new Error(`Incorrect URL: ${url}, ${e}`); + } +} + +/** + * Return request headers for a Harness Code provider. + * + * @param config - A Harness Code provider config + * @public + */ +export function getHarnessRequestOptions(config: HarnessIntegrationConfig): { + headers?: Record; +} { + const headers: Record = {}; + const { username, token } = config; + + if (!token) { + return headers; + } + + if (username) { + headers.Authorization = `basic ${Buffer.from( + `${username}:${token}`, + ).toString('base64')}`; + } else { + headers.Authorization = `Bearer ${token}`; + } + + return { + headers, + }; +} diff --git a/packages/integration/src/harness/index.ts b/packages/integration/src/harness/index.ts new file mode 100644 index 0000000000..264df657f0 --- /dev/null +++ b/packages/integration/src/harness/index.ts @@ -0,0 +1,19 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +export { HarnessIntegration } from './HarnessIntegration'; +export { getHarnessRequestOptions, getHarnessFileContentsUrl } from './core'; +export { readHarnessConfig } from './config'; +export type { HarnessIntegrationConfig } from './config'; diff --git a/packages/integration/src/index.ts b/packages/integration/src/index.ts index b20477ee6a..32c573abef 100644 --- a/packages/integration/src/index.ts +++ b/packages/integration/src/index.ts @@ -31,6 +31,7 @@ export * from './gitea'; export * from './github'; export * from './gitlab'; export * from './googleGcs'; +export * from './harness'; export { defaultScmResolveUrl } from './helpers'; export { ScmIntegrations } from './ScmIntegrations'; export type { IntegrationsByType } from './ScmIntegrations'; diff --git a/packages/integration/src/registry.ts b/packages/integration/src/registry.ts index 00706acc1c..7e4b34cc34 100644 --- a/packages/integration/src/registry.ts +++ b/packages/integration/src/registry.ts @@ -25,6 +25,7 @@ import { GerritIntegration } from './gerrit/GerritIntegration'; import { GithubIntegration } from './github/GithubIntegration'; import { GitLabIntegration } from './gitlab/GitLabIntegration'; import { GiteaIntegration } from './gitea/GiteaIntegration'; +import { HarnessIntegration } from './harness/HarnessIntegration'; /** * Holds all registered SCM integrations, of all types. @@ -46,6 +47,7 @@ export interface ScmIntegrationRegistry github: ScmIntegrationsGroup; gitlab: ScmIntegrationsGroup; gitea: ScmIntegrationsGroup; + harness: ScmIntegrationsGroup; /** * Resolves an absolute or relative URL in relation to a base URL. * From 4750bf66223a0befc33ab8226c7036a5f0d2e898 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Sun, 21 Apr 2024 19:39:59 -0600 Subject: [PATCH 40/90] Update .changeset/empty-beers-relax.md Co-authored-by: Himanshu Mishra Signed-off-by: Calvin Lee --- .changeset/empty-beers-relax.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.changeset/empty-beers-relax.md b/.changeset/empty-beers-relax.md index 526f88c1c7..a285b9e7b4 100644 --- a/.changeset/empty-beers-relax.md +++ b/.changeset/empty-beers-relax.md @@ -2,4 +2,4 @@ '@backstage/backend-common': patch --- -This patch adds HarnessURLReader to the available classes. It currently only reads single files via Harness codes public repo api. +This patch adds HarnessURLReader. It only supports readUrl for now. readTree and search will be implemented next. From d422716946e6bd1ca2072f21b510add272eb0b59 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Sun, 21 Apr 2024 20:31:45 -0600 Subject: [PATCH 41/90] integration support for harness p2-comments Signed-off-by: Calvin Lee --- docs/integrations/harness/locations.md | 12 +++---- .../src/reading/HarnessUrlReader.ts | 2 +- packages/integration/api-report.md | 5 ++- .../src/harness/HarnessIntegration.test.ts | 5 --- .../integration/src/harness/config.test.ts | 11 ++----- packages/integration/src/harness/config.ts | 32 ++++++------------- packages/integration/src/harness/core.test.ts | 12 +++---- packages/integration/src/harness/core.ts | 17 ++++------ 8 files changed, 31 insertions(+), 65 deletions(-) diff --git a/docs/integrations/harness/locations.md b/docs/integrations/harness/locations.md index 8911d7918c..a3d0e2cfd4 100644 --- a/docs/integrations/harness/locations.md +++ b/docs/integrations/harness/locations.md @@ -20,14 +20,14 @@ integrations: harness: - host: app.harness.io token: ${HARNESS_CODE_BEARER_TOKEN} + apiKey: ${HARNESS_CODE_APIKEY} ``` -Directly under the `harnessCode` key is a list of provider configurations, where you -can list the Gitea instances you want to be able to fetch -data from. Each entry is a structure with up to four elements: +Directly under the `harness` key is a list of provider configurations, where you +can list the Harness instances you want to be able to fetch + +check out https://developer.harness.io/docs/platform/automation/api/add-and-manage-api-keys/ for more information - `host`: The host of the Harness Code instance that you want to match on. -- `baseUrl` (optional): Needed if the Harness Code instance is not reachable at - the base of the `host` option (e.g. `https://app.harness.io`). This is the address that you would open in a browser. -- `username` (optional): The gitea username to use in API requests. - `token` (optional): The password or api token to authenticate with. +- `apiKey` (optional): The apiKey to authenticate with. diff --git a/packages/backend-common/src/reading/HarnessUrlReader.ts b/packages/backend-common/src/reading/HarnessUrlReader.ts index fd01e4f8a2..95114ae9d6 100644 --- a/packages/backend-common/src/reading/HarnessUrlReader.ts +++ b/packages/backend-common/src/reading/HarnessUrlReader.ts @@ -118,7 +118,7 @@ export class HarnessUrlReader implements UrlReader { toString() { const { host } = this.integration.config; return `harness{host=${host},authed=${Boolean( - this.integration.config.token, + this.integration.config.token || this.integration.config.apiKey, )}}`; } } diff --git a/packages/integration/api-report.md b/packages/integration/api-report.md index a693c1314c..b40b66b0b6 100644 --- a/packages/integration/api-report.md +++ b/packages/integration/api-report.md @@ -552,7 +552,7 @@ export type GiteaIntegrationConfig = { host: string; baseUrl?: string; username?: string; - token?: string; + password?: string; }; // @public @@ -711,8 +711,7 @@ export class HarnessIntegration implements ScmIntegration { // @public export type HarnessIntegrationConfig = { host: string; - baseUrl?: string; - username?: string; + apiKey?: string; token?: string; }; diff --git a/packages/integration/src/harness/HarnessIntegration.test.ts b/packages/integration/src/harness/HarnessIntegration.test.ts index 9014c0bb79..2d85204f16 100644 --- a/packages/integration/src/harness/HarnessIntegration.test.ts +++ b/packages/integration/src/harness/HarnessIntegration.test.ts @@ -25,8 +25,6 @@ describe('HarnessIntegration', () => { harness: [ { host: 'app.harness.io', - username: 'git', - baseUrl: 'https://app.harness.io/route', token: '1234', }, ], @@ -35,9 +33,6 @@ describe('HarnessIntegration', () => { }); expect(integrations.list().length).toBe(1); expect(integrations.list()[0].config.host).toBe('app.harness.io'); - expect(integrations.list()[0].config.baseUrl).toBe( - 'https://app.harness.io/route', - ); }); it('returns the basics', () => { diff --git a/packages/integration/src/harness/config.test.ts b/packages/integration/src/harness/config.test.ts index 6482e89de8..4f6d8496e3 100644 --- a/packages/integration/src/harness/config.test.ts +++ b/packages/integration/src/harness/config.test.ts @@ -51,16 +51,14 @@ describe('readHarnessConfig', () => { const output = readHarnessConfig( buildConfig({ host: 'a.com', - baseUrl: 'https://a.com/route/api', - username: 'u', token: 'p', + apiKey: 'a', }), ); expect(output).toEqual({ host: 'a.com', - baseUrl: 'https://a.com/route/api', - username: 'u', token: 'p', + apiKey: 'a', }); }); @@ -72,8 +70,6 @@ describe('readHarnessConfig', () => { ); expect(output).toEqual({ host: 'a.com', - baseUrl: 'https://a.com', - username: undefined, token: undefined, }); }); @@ -95,14 +91,11 @@ describe('readHarnessConfig', () => { readHarnessConfig( await buildFrontendConfig({ host: 'a.com', - baseUrl: 'https://a.com/route', - username: 'u', token: 'p', }), ), ).toEqual({ host: 'a.com', - baseUrl: 'https://a.com/route', }); }); }); diff --git a/packages/integration/src/harness/config.ts b/packages/integration/src/harness/config.ts index 75482a9b39..748cd141c5 100644 --- a/packages/integration/src/harness/config.ts +++ b/packages/integration/src/harness/config.ts @@ -15,11 +15,10 @@ */ import { Config } from '@backstage/config'; -import { trimEnd } from 'lodash'; import { isValidHost } from '../helpers'; /** - * The configuration for a single Gitea integration. + * The configuration for a single Harness integration. * * @public */ @@ -28,23 +27,14 @@ export type HarnessIntegrationConfig = { * The host of the target that this matches on, e.g. "app.harness.io" */ host: string; - /** - * The optional base URL of the Harness code instance. It is assumed that https - * is used and that the base path is "/" on the host. If that is not the - * case set the complete base url to the Harness code instance, e.g. - * "https://harnesscode.website.com/". This is the url that you would open - * in a browser. - */ - baseUrl?: string; - /** - * The username to use for requests to harness code. - */ - username?: string; - /** * The password or http token to use for authentication. */ token?: string; + /** + * The API key to use for authentication. + */ + apiKey?: string; }; /** @@ -55,24 +45,20 @@ export type HarnessIntegrationConfig = { export function readHarnessConfig(config: Config): HarnessIntegrationConfig { const host = config.getString('host'); let baseUrl = config.getOptionalString('baseUrl'); - const username = config.getOptionalString('username'); const token = config.getOptionalString('token'); + const apiKey = config.getOptionalString('apiKey'); + if (!isValidHost(host)) { throw new Error( `Invalid Harness Code integration config, '${host}' is not a valid host`, ); } - if (baseUrl) { - baseUrl = trimEnd(baseUrl, '/'); - } else { - baseUrl = `https://${host}`; - } + baseUrl = `https://${host}`; return { host, - baseUrl, - username, + apiKey, token, }; } diff --git a/packages/integration/src/harness/core.test.ts b/packages/integration/src/harness/core.test.ts index 6502fd3e48..2a214cc06f 100644 --- a/packages/integration/src/harness/core.test.ts +++ b/packages/integration/src/harness/core.test.ts @@ -76,20 +76,16 @@ describe('Harness code core', () => { ).toBeUndefined(); }); - it('adds basic auth when username and token are specified', () => { + it('adds basic auth when apikey and token are specified', () => { const authRequest: HarnessIntegrationConfig = { host: 'gerrit.com', - username: 'username', token: 'P', + apiKey: 'a', }; - const basicAuthentication = `basic ${Buffer.from( - `${authRequest.username}:${authRequest.token}`, - ).toString('base64')}`; - expect( - (getHarnessRequestOptions(authRequest).headers as any).Authorization, - ).toEqual(basicAuthentication); + (getHarnessRequestOptions(authRequest).headers as any)['x-api-key'], + ).toEqual('a'); }); }); }); diff --git a/packages/integration/src/harness/core.ts b/packages/integration/src/harness/core.ts index 0c4fb01970..324a57071c 100644 --- a/packages/integration/src/harness/core.ts +++ b/packages/integration/src/harness/core.ts @@ -34,7 +34,7 @@ export function getHarnessEditContentsUrl( url: string, ) { try { - const baseUrl = config.baseUrl ?? `https://${config.host}`; + const baseUrl = `https://${config.host}`; const [ _blank, _ng, @@ -61,9 +61,8 @@ export function getHarnessEditContentsUrl( } /** - * Given a URL pointing to a file, returns an api URL - * for fetching the contents of the data. - * + * Given a file path URL, + * it returns an API URL which returns the contents of the file. * @remarks * * Converts @@ -79,7 +78,7 @@ export function getHarnessFileContentsUrl( url: string, ) { try { - const baseUrl = config.baseUrl ?? `https://${config.host}`; + const baseUrl = `https://${config.host}`; const [ _blank, _ng, @@ -115,16 +114,14 @@ export function getHarnessRequestOptions(config: HarnessIntegrationConfig): { headers?: Record; } { const headers: Record = {}; - const { username, token } = config; + const { token, apiKey } = config; if (!token) { return headers; } - if (username) { - headers.Authorization = `basic ${Buffer.from( - `${username}:${token}`, - ).toString('base64')}`; + if (apiKey) { + headers['x-api-key'] = apiKey; } else { headers.Authorization = `Bearer ${token}`; } From 6a0e918dc69d6e46d350485c4fea14c54cea3847 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 00:23:33 -0600 Subject: [PATCH 42/90] integration support for harness p3-comments Signed-off-by: Calvin Lee --- .../{HarnessCodeUrlReader.test.ts => HarnessUrlReader.test.ts} | 2 +- packages/integration/src/harness/core.ts | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) rename packages/backend-common/src/reading/{HarnessCodeUrlReader.test.ts => HarnessUrlReader.test.ts} (98%) diff --git a/packages/backend-common/src/reading/HarnessCodeUrlReader.test.ts b/packages/backend-common/src/reading/HarnessUrlReader.test.ts similarity index 98% rename from packages/backend-common/src/reading/HarnessCodeUrlReader.test.ts rename to packages/backend-common/src/reading/HarnessUrlReader.test.ts index 3e8b70a6aa..18c523b959 100644 --- a/packages/backend-common/src/reading/HarnessCodeUrlReader.test.ts +++ b/packages/backend-common/src/reading/HarnessUrlReader.test.ts @@ -60,7 +60,7 @@ const createReader = (config: JsonObject): UrlReaderPredicateTuple[] => { describe('HarnessUrlReader', () => { const worker = setupServer(); setupRequestMockHandlers(worker); - + beforeAll(() => worker.listen({ onUnhandledRequest: 'bypass' })); afterAll(() => { jest.clearAllMocks(); }); diff --git a/packages/integration/src/harness/core.ts b/packages/integration/src/harness/core.ts index 324a57071c..6de8039dd5 100644 --- a/packages/integration/src/harness/core.ts +++ b/packages/integration/src/harness/core.ts @@ -122,7 +122,7 @@ export function getHarnessRequestOptions(config: HarnessIntegrationConfig): { if (apiKey) { headers['x-api-key'] = apiKey; - } else { + } else if (token) { headers.Authorization = `Bearer ${token}`; } From ed8b5324dad91129efa850a42f2d27b77b340806 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 16:41:40 -0600 Subject: [PATCH 43/90] integration support for harness p4-fixed test Signed-off-by: Calvin Lee --- .../src/reading/HarnessUrlReader.test.ts | 136 +++++++++--------- packages/integration/src/harness/core.ts | 8 +- 2 files changed, 71 insertions(+), 73 deletions(-) diff --git a/packages/backend-common/src/reading/HarnessUrlReader.test.ts b/packages/backend-common/src/reading/HarnessUrlReader.test.ts index 18c523b959..4d43cebaec 100644 --- a/packages/backend-common/src/reading/HarnessUrlReader.test.ts +++ b/packages/backend-common/src/reading/HarnessUrlReader.test.ts @@ -25,7 +25,6 @@ import { UrlReaderPredicateTuple } from './types'; import { DefaultReadTreeResponseFactory } from './tree'; import getRawBody from 'raw-body'; import { HarnessUrlReader } from './HarnessUrlReader'; -import { NotFoundError } from '@backstage/errors'; const treeResponseFactory = DefaultReadTreeResponseFactory.create({ config: new ConfigReader({}), @@ -44,6 +43,7 @@ const harnessProcessor = new HarnessUrlReader( readHarnessConfig( new ConfigReader({ host: 'app.harness.io', + token: 'p', }), ), ), @@ -56,9 +56,60 @@ const createReader = (config: JsonObject): UrlReaderPredicateTuple[] => { treeResponseFactory, }); }; +const responseBuffer = Buffer.from('Apache License'); +const harnessApiResponse = (content: any) => { + return JSON.stringify({ + content: { + data: Buffer.from(content).toString('base64'), + encoding: 'base64', + }, + }); +}; + +const handlers = [ + rest.get( + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/all-apis.yaml', + (req, res, ctx) => { + return res(ctx.status(500), ctx.json({ message: 'Error!!!' })); + }, + ), + rest.get( + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/404error.yaml', + (req, res, ctx) => { + return res(ctx.status(404), ctx.json({ message: 'File not found.' })); + }, + ), + rest.get( + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/stream.TXT', + (req, res, ctx) => { + return res( + ctx.status(200), + ctx.body(harnessApiResponse(responseBuffer.toString())), + ); + }, + ), + + rest.get( + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/buffer.TXT', + (req, res, ctx) => { + return res( + ctx.status(200), + ctx.body(harnessApiResponse(responseBuffer.toString())), + ); + }, + ), + rest.post('/api/login', (req, res, ctx) => { + const { username } = req.body; + + if (username === 'admin') { + return res(ctx.status(200), ctx.json({ token: 'fake-token' })); + } + return res(ctx.status(403), ctx.json({ message: 'Access Denied' })); + }), +]; describe('HarnessUrlReader', () => { - const worker = setupServer(); + const worker = setupServer(...handlers); setupRequestMockHandlers(worker); beforeAll(() => worker.listen({ onUnhandledRequest: 'bypass' })); afterAll(() => { @@ -107,97 +158,40 @@ describe('HarnessUrlReader', () => { }); }); - describe('readUrl', () => { - const responseBuffer = Buffer.from('Apache License'); - const harnessApiResponse = (content: any) => { - return JSON.stringify({ - encoding: 'base64', - content: Buffer.from(content).toString('base64'), - }); - }; - - it.skip('should be able to read file contents as buffer', async () => { - worker.use( - rest.get( - 'https://app.harness.io/api/v1/repos/owner/project/contents/LICENSE', - (req, res, ctx) => { - // Test utils prefers matching URL directly but it is part of Gitea's API - if (req.url.searchParams.get('ref') === 'branch2') { - return res( - ctx.status(200), - ctx.body(harnessApiResponse(responseBuffer.toString())), - ); - } - - return res(ctx.status(500)); - }, - ), - ); - + describe('readUrl part 1', () => { + it('should be able to read file contents as buffer', async () => { const result = await harnessProcessor.readUrl( - 'https://app.harness.io/owner/project/src/branch/branch2/LICENSE', + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/buffer.TXT', ); const buffer = await result.buffer(); expect(buffer.toString()).toBe(responseBuffer.toString()); }); - it.skip('should be able to read file contents as stream', async () => { - worker.use( - rest.get( - 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/LICENSE.txt', - (req, res, ctx) => { - if (req.url.searchParams.get('ref') === 'refMain') { - return res( - ctx.status(200), - ctx.body(harnessApiResponse(responseBuffer.toString())), - ); - } - - return res(ctx.status(500)); - }, - ), - ); - + it('should be able to read file contents as stream', async () => { const result = await harnessProcessor.readUrl( - 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/LICENSE.TXT', + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/stream.TXT', ); const fromStream = await getRawBody(result.stream!()); expect(fromStream.toString()).toBe(responseBuffer.toString()); }); - it.skip('should raise NotFoundError on 404.', async () => { - worker.use( - rest.get( - 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', - (_, res, ctx) => { - return res(ctx.status(404, 'File not found.')); - }, - ), - ); - + it('should raise NotFoundError on 404.', async () => { await expect( harnessProcessor.readUrl( - 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/404error.yaml', ), - ).rejects.toThrow(NotFoundError); + ).rejects.toThrow( + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/404error.yaml x https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/content/404error.yaml?routingId=accountId&include_commit=false&ref=refMain, 404 Not Found', + ); }); - it.skip('should throw an error on non 404 errors.', async () => { - worker.use( - rest.get( - 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', - (_, res, ctx) => { - return res(ctx.status(500, 'Error!!!')); - }, - ), - ); - + it('should throw an error on non 404 errors.', async () => { await expect( harnessProcessor.readUrl( 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', ), ).rejects.toThrow( - 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/content/all-apis.yaml?routingId=accountId&include_commit=false&ref=refMain, 500 Error!!!', + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml x https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/content/all-apis.yaml?routingId=accountId&include_commit=false&ref=refMain, 500 Internal Server Error', ); }); }); diff --git a/packages/integration/src/harness/core.ts b/packages/integration/src/harness/core.ts index 6de8039dd5..f762cebb56 100644 --- a/packages/integration/src/harness/core.ts +++ b/packages/integration/src/harness/core.ts @@ -93,12 +93,16 @@ export function getHarnessFileContentsUrl( _repos, repoName, _files, - ref, + _ref, _branch, ...path ] = url.replace(baseUrl, '').split('/'); + const urlParts = url.replace(baseUrl, '').split('/'); + const refAndPath = urlParts.slice(13); + const refIndex = refAndPath.findIndex(item => item === '~'); + const refString = refAndPath.slice(0, refIndex); const pathWithoutSlash = path.join('/').replace(/^\//, ''); - return `${baseUrl}/gateway/code/api/v1/repos/${accountId}/${orgName}/${projectName}/${repoName}/+/content/${pathWithoutSlash}?routingId=${accountId}&include_commit=false&ref=${ref}`; + return `${baseUrl}/gateway/code/api/v1/repos/${accountId}/${orgName}/${projectName}/${repoName}/+/content/${pathWithoutSlash}?routingId=${accountId}&include_commit=false&ref=${refString}`; } catch (e) { throw new Error(`Incorrect URL: ${url}, ${e}`); } From 0cf356671a830937d09872a933edcbc80e676841 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 16:43:47 -0600 Subject: [PATCH 44/90] integration support for harness p4-fixed test Signed-off-by: Calvin Lee --- .../backend-common/src/reading/HarnessUrlReader.test.ts | 8 -------- 1 file changed, 8 deletions(-) diff --git a/packages/backend-common/src/reading/HarnessUrlReader.test.ts b/packages/backend-common/src/reading/HarnessUrlReader.test.ts index 4d43cebaec..5e1d2d9b2e 100644 --- a/packages/backend-common/src/reading/HarnessUrlReader.test.ts +++ b/packages/backend-common/src/reading/HarnessUrlReader.test.ts @@ -98,14 +98,6 @@ const handlers = [ ); }, ), - rest.post('/api/login', (req, res, ctx) => { - const { username } = req.body; - - if (username === 'admin') { - return res(ctx.status(200), ctx.json({ token: 'fake-token' })); - } - return res(ctx.status(403), ctx.json({ message: 'Access Denied' })); - }), ]; describe('HarnessUrlReader', () => { From 1cfa4aa35ea08f16dc7635e87a608c6e8c41851c Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 21:13:04 -0600 Subject: [PATCH 45/90] integration support for harness p5-fixed lint Signed-off-by: Calvin Lee --- packages/backend-common/api-report.md | 3 +-- .../src/reading/HarnessUrlReader.test.ts | 8 ++++---- packages/integration/config.d.ts | 10 ++-------- packages/integration/src/harness/config.test.ts | 3 --- packages/integration/src/harness/config.ts | 3 --- 5 files changed, 7 insertions(+), 20 deletions(-) diff --git a/packages/backend-common/api-report.md b/packages/backend-common/api-report.md index 37bfda9861..33145fc47f 100644 --- a/packages/backend-common/api-report.md +++ b/packages/backend-common/api-report.md @@ -551,8 +551,7 @@ export class HarnessUrlReader implements UrlReader { // @public export type HarnessIntegrationConfig = { host: string; - baseUrl?: string; - username?: string; + apiKey?: string; token?: string; }; diff --git a/packages/backend-common/src/reading/HarnessUrlReader.test.ts b/packages/backend-common/src/reading/HarnessUrlReader.test.ts index 5e1d2d9b2e..d6cf9ab018 100644 --- a/packages/backend-common/src/reading/HarnessUrlReader.test.ts +++ b/packages/backend-common/src/reading/HarnessUrlReader.test.ts @@ -69,19 +69,19 @@ const harnessApiResponse = (content: any) => { const handlers = [ rest.get( 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/all-apis.yaml', - (req, res, ctx) => { + (_req, res, ctx) => { return res(ctx.status(500), ctx.json({ message: 'Error!!!' })); }, ), rest.get( 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/404error.yaml', - (req, res, ctx) => { + (_req, res, ctx) => { return res(ctx.status(404), ctx.json({ message: 'File not found.' })); }, ), rest.get( 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/stream.TXT', - (req, res, ctx) => { + (_req, res, ctx) => { return res( ctx.status(200), ctx.body(harnessApiResponse(responseBuffer.toString())), @@ -91,7 +91,7 @@ const handlers = [ rest.get( 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/buffer.TXT', - (req, res, ctx) => { + (_req, res, ctx) => { return res( ctx.status(200), ctx.body(harnessApiResponse(responseBuffer.toString())), diff --git a/packages/integration/config.d.ts b/packages/integration/config.d.ts index 01f1f44d73..58dd9153c4 100644 --- a/packages/integration/config.d.ts +++ b/packages/integration/config.d.ts @@ -353,16 +353,10 @@ export interface Config { */ host: string; /** - * The base url for the Gitea instance. - * @visibility frontend - */ - baseUrl?: string; - - /** - * The username to use for authenticated requests. + * The apikey to use for authenticated requests. * @visibility secret */ - username?: string; + apiKey?: string; /** * Harness Code token used to authenticate requests. This can be either a generated access token. * @visibility secret diff --git a/packages/integration/src/harness/config.test.ts b/packages/integration/src/harness/config.test.ts index 4f6d8496e3..93be1fcc32 100644 --- a/packages/integration/src/harness/config.test.ts +++ b/packages/integration/src/harness/config.test.ts @@ -81,9 +81,6 @@ describe('readHarnessConfig', () => { expect(() => readHarnessConfig(buildConfig({ ...valid, host: 2 }))).toThrow( /host/, ); - expect(() => - readHarnessConfig(buildConfig({ ...valid, baseUrl: 2 })), - ).toThrow(/baseUrl/); }); it('works on the frontend', async () => { diff --git a/packages/integration/src/harness/config.ts b/packages/integration/src/harness/config.ts index 748cd141c5..2f75915567 100644 --- a/packages/integration/src/harness/config.ts +++ b/packages/integration/src/harness/config.ts @@ -44,7 +44,6 @@ export type HarnessIntegrationConfig = { */ export function readHarnessConfig(config: Config): HarnessIntegrationConfig { const host = config.getString('host'); - let baseUrl = config.getOptionalString('baseUrl'); const token = config.getOptionalString('token'); const apiKey = config.getOptionalString('apiKey'); @@ -54,8 +53,6 @@ export function readHarnessConfig(config: Config): HarnessIntegrationConfig { ); } - baseUrl = `https://${host}`; - return { host, apiKey, From 2bf97f0aa15b1940fb2a485584ae14c773cf7591 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 21:31:44 -0600 Subject: [PATCH 46/90] integration support for harness p5-fixed check Signed-off-by: Calvin Lee --- packages/integration/package.json | 1 + yarn.lock | 1 + 2 files changed, 2 insertions(+) diff --git a/packages/integration/package.json b/packages/integration/package.json index bbd0eecbe7..b57b94f982 100644 --- a/packages/integration/package.json +++ b/packages/integration/package.json @@ -50,6 +50,7 @@ "devDependencies": { "@backstage/cli": "workspace:^", "@backstage/config-loader": "workspace:^", + "@backstage/test-utils": "workspace:^", "@types/luxon": "^3.0.0", "msw": "^1.0.0" }, diff --git a/yarn.lock b/yarn.lock index 3cf5a79175..60b03d4399 100644 --- a/yarn.lock +++ b/yarn.lock @@ -4368,6 +4368,7 @@ __metadata: "@backstage/config": "workspace:^" "@backstage/config-loader": "workspace:^" "@backstage/errors": "workspace:^" + "@backstage/test-utils": "workspace:^" "@octokit/auth-app": ^4.0.0 "@octokit/rest": ^19.0.3 "@types/luxon": ^3.0.0 From 049a69f223e9c986312750e55c4fab028398fc25 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 21:46:58 -0600 Subject: [PATCH 47/90] integration support for harness p5-fixed api report Signed-off-by: Calvin Lee --- packages/integration/api-report.md | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/packages/integration/api-report.md b/packages/integration/api-report.md index b40b66b0b6..2029dac600 100644 --- a/packages/integration/api-report.md +++ b/packages/integration/api-report.md @@ -520,9 +520,7 @@ export function getHarnessFileContentsUrl( ): string; // @public -export function getHarnessRequestOptions( - config: HarnessIntegrationConfig, -): { +export function getHarnessRequestOptions(config: HarnessIntegrationConfig): { headers?: Record; }; @@ -711,8 +709,8 @@ export class HarnessIntegration implements ScmIntegration { // @public export type HarnessIntegrationConfig = { host: string; - apiKey?: string; token?: string; + apiKey?: string; }; // @public @@ -883,9 +881,7 @@ export function readGoogleGcsIntegrationConfig( ): GoogleGcsIntegrationConfig; // @public -export function readHarnessConfig( - config: Config, -): HarnessIntegrationConfig; +export function readHarnessConfig(config: Config): HarnessIntegrationConfig; // @public @deprecated (undocumented) export const replaceGitHubUrlType: typeof replaceGithubUrlType; From 7362e25a24086d47c28c98029cb4ff982af102f5 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 21:59:08 -0600 Subject: [PATCH 48/90] integration support for harness p5-fixed api report Signed-off-by: Calvin Lee --- packages/backend-common/api-report.md | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/packages/backend-common/api-report.md b/packages/backend-common/api-report.md index 33145fc47f..1ae7a9dceb 100644 --- a/packages/backend-common/api-report.md +++ b/packages/backend-common/api-report.md @@ -531,6 +531,8 @@ export class GitlabUrlReader implements UrlReader { toString(): string; } +// Warning: (ae-unresolved-link) The @link reference could not be resolved: This type of declaration is not supported yet by the resolver +// // @public export class HarnessUrlReader implements UrlReader { constructor(integration: HarnessIntegration); @@ -548,13 +550,6 @@ export class HarnessUrlReader implements UrlReader { toString(): string; } -// @public -export type HarnessIntegrationConfig = { - host: string; - apiKey?: string; - token?: string; -}; - // @public export const HostDiscovery: typeof HostDiscovery_2; From a102a02c27a380513f04f8807c7eea2f5718998b Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 22:42:20 -0600 Subject: [PATCH 49/90] integration support for harness p5-fixed api report Signed-off-by: Calvin Lee --- packages/backend-common/api-report.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/packages/backend-common/api-report.md b/packages/backend-common/api-report.md index 1ae7a9dceb..e815973fb1 100644 --- a/packages/backend-common/api-report.md +++ b/packages/backend-common/api-report.md @@ -531,8 +531,6 @@ export class GitlabUrlReader implements UrlReader { toString(): string; } -// Warning: (ae-unresolved-link) The @link reference could not be resolved: This type of declaration is not supported yet by the resolver -// // @public export class HarnessUrlReader implements UrlReader { constructor(integration: HarnessIntegration); From a25b566c6453d2ad5f04e3b4a31b896a3864c797 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 22:57:03 -0600 Subject: [PATCH 50/90] integration support for harness p5-fixed api report Signed-off-by: Calvin Lee --- packages/backend-common/src/reading/HarnessUrlReader.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/backend-common/src/reading/HarnessUrlReader.ts b/packages/backend-common/src/reading/HarnessUrlReader.ts index 95114ae9d6..55854d102f 100644 --- a/packages/backend-common/src/reading/HarnessUrlReader.ts +++ b/packages/backend-common/src/reading/HarnessUrlReader.ts @@ -36,7 +36,7 @@ import { import { Readable } from 'stream'; /** - * Implements a {@link UrlReader} for the Harness code v1 api. + * Implements a {@link @backstage/backend-plugin-api#UrlReaderService} for the Harness code v1 api. * * @public */ From 645580361b68f525c227c9c2f5fcedc3cb89c3f2 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 23 Apr 2024 23:06:29 -0600 Subject: [PATCH 51/90] integration support for harness p5-fixed api microsite Signed-off-by: Calvin Lee --- microsite/sidebars.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/microsite/sidebars.json b/microsite/sidebars.json index 8291549153..eadcbbf806 100644 --- a/microsite/sidebars.json +++ b/microsite/sidebars.json @@ -239,9 +239,9 @@ "items": ["integrations/gitea/locations"] }, { - "type": "subcategory", + "type": "category", "label": "Harness", - "ids": ["integrations/harness/locations"] + "items": ["integrations/harness/locations"] }, { "type": "category", From d01f3ab8901274470ff160e3bca38d3b22372f96 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Thu, 25 Apr 2024 11:41:13 -0600 Subject: [PATCH 52/90] integration support for harness p5-fixed api microsite-fake Signed-off-by: Calvin Lee --- packages/backend-common/src/reading/HarnessUrlReader.ts | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/backend-common/src/reading/HarnessUrlReader.ts b/packages/backend-common/src/reading/HarnessUrlReader.ts index 55854d102f..7d65e674b8 100644 --- a/packages/backend-common/src/reading/HarnessUrlReader.ts +++ b/packages/backend-common/src/reading/HarnessUrlReader.ts @@ -38,6 +38,7 @@ import { Readable } from 'stream'; /** * Implements a {@link @backstage/backend-plugin-api#UrlReaderService} for the Harness code v1 api. * + * * @public */ export class HarnessUrlReader implements UrlReader { From 9093f35e8aa1a99c1d7620a2ca3da6d19deeeefb Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Mon, 29 Apr 2024 12:41:19 -0600 Subject: [PATCH 53/90] integration support for harness p5-fixed api microsite-fix content api Signed-off-by: Calvin Lee --- .../src/reading/HarnessUrlReader.test.ts | 19 +++++++------------ .../src/reading/HarnessUrlReader.ts | 10 +++++----- packages/integration/src/harness/core.test.ts | 2 +- packages/integration/src/harness/core.ts | 2 +- 4 files changed, 14 insertions(+), 19 deletions(-) diff --git a/packages/backend-common/src/reading/HarnessUrlReader.test.ts b/packages/backend-common/src/reading/HarnessUrlReader.test.ts index d6cf9ab018..bb09baa140 100644 --- a/packages/backend-common/src/reading/HarnessUrlReader.test.ts +++ b/packages/backend-common/src/reading/HarnessUrlReader.test.ts @@ -58,29 +58,24 @@ const createReader = (config: JsonObject): UrlReaderPredicateTuple[] => { }; const responseBuffer = Buffer.from('Apache License'); const harnessApiResponse = (content: any) => { - return JSON.stringify({ - content: { - data: Buffer.from(content).toString('base64'), - encoding: 'base64', - }, - }); + return content; }; const handlers = [ rest.get( - 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/all-apis.yaml', + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/raw/all-apis.yaml', (_req, res, ctx) => { return res(ctx.status(500), ctx.json({ message: 'Error!!!' })); }, ), rest.get( - 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/404error.yaml', + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/raw/404error.yaml', (_req, res, ctx) => { return res(ctx.status(404), ctx.json({ message: 'File not found.' })); }, ), rest.get( - 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/stream.TXT', + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/raw/stream.TXT', (_req, res, ctx) => { return res( ctx.status(200), @@ -90,7 +85,7 @@ const handlers = [ ), rest.get( - 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/content/buffer.TXT', + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/:path+/raw/buffer.TXT', (_req, res, ctx) => { return res( ctx.status(200), @@ -173,7 +168,7 @@ describe('HarnessUrlReader', () => { 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/404error.yaml', ), ).rejects.toThrow( - 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/404error.yaml x https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/content/404error.yaml?routingId=accountId&include_commit=false&ref=refMain, 404 Not Found', + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/404error.yaml x https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/raw/404error.yaml?routingId=accountId&git_ref=refMain, 404 Not Found', ); }); @@ -183,7 +178,7 @@ describe('HarnessUrlReader', () => { 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', ), ).rejects.toThrow( - 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml x https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/content/all-apis.yaml?routingId=accountId&include_commit=false&ref=refMain, 500 Internal Server Error', + 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml x https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/raw/all-apis.yaml?routingId=accountId&git_ref=refMain, 500 Internal Server Error', ); }); }); diff --git a/packages/backend-common/src/reading/HarnessUrlReader.ts b/packages/backend-common/src/reading/HarnessUrlReader.ts index 7d65e674b8..09291c5176 100644 --- a/packages/backend-common/src/reading/HarnessUrlReader.ts +++ b/packages/backend-common/src/reading/HarnessUrlReader.ts @@ -79,18 +79,18 @@ export class HarnessUrlReader implements UrlReader { } if (response.ok) { - // Harness Code returns an object with the file contents encoded, not the file itself - const jsonResponse = await response.json(); - if (jsonResponse?.content?.encoding === 'base64') { + // Harness Code returns the raw content object + const jsonResponse = { data: response.body }; + if (jsonResponse) { return ReadUrlResponseFactory.fromReadable( - Readable.from(Buffer.from(jsonResponse?.content?.data, 'base64')), + Readable.from(jsonResponse.data), { etag: response.headers.get('ETag') ?? undefined, }, ); } - throw new Error(`Unknown encoding: ${jsonResponse?.content?.encoding}`); + throw new Error(`Unknown json: ${jsonResponse}`); } const message = `${url} x ${blobUrl}, ${response.status} ${response.statusText}`; diff --git a/packages/integration/src/harness/core.test.ts b/packages/integration/src/harness/core.test.ts index 2a214cc06f..abfbb5d4c2 100644 --- a/packages/integration/src/harness/core.test.ts +++ b/packages/integration/src/harness/core.test.ts @@ -38,7 +38,7 @@ describe('Harness code core', () => { 'https://app.harness.io/ng/account/accountId/module/code/orgs/orgName/projects/projName/repos/repoName/files/refMain/~/all-apis.yaml', ), ).toEqual( - 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/content/all-apis.yaml?routingId=accountId&include_commit=false&ref=refMain', + 'https://app.harness.io/gateway/code/api/v1/repos/accountId/orgName/projName/repoName/+/raw/all-apis.yaml?routingId=accountId&git_ref=refMain', ); }); }); diff --git a/packages/integration/src/harness/core.ts b/packages/integration/src/harness/core.ts index f762cebb56..8eba850ec2 100644 --- a/packages/integration/src/harness/core.ts +++ b/packages/integration/src/harness/core.ts @@ -102,7 +102,7 @@ export function getHarnessFileContentsUrl( const refIndex = refAndPath.findIndex(item => item === '~'); const refString = refAndPath.slice(0, refIndex); const pathWithoutSlash = path.join('/').replace(/^\//, ''); - return `${baseUrl}/gateway/code/api/v1/repos/${accountId}/${orgName}/${projectName}/${repoName}/+/content/${pathWithoutSlash}?routingId=${accountId}&include_commit=false&ref=${refString}`; + return `${baseUrl}/gateway/code/api/v1/repos/${accountId}/${orgName}/${projectName}/${repoName}/+/raw/${pathWithoutSlash}?routingId=${accountId}&git_ref=${refString}`; } catch (e) { throw new Error(`Incorrect URL: ${url}, ${e}`); } From 84bb2ed37df5312b0a0ed5b9195e2cd329487ec5 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 30 Apr 2024 02:57:29 -0600 Subject: [PATCH 54/90] integration support for harness - fixed comments Signed-off-by: Calvin Lee --- .changeset/empty-beers-relax.md | 2 +- .changeset/tasty-rats-explain.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.changeset/empty-beers-relax.md b/.changeset/empty-beers-relax.md index a285b9e7b4..bbfcef0933 100644 --- a/.changeset/empty-beers-relax.md +++ b/.changeset/empty-beers-relax.md @@ -2,4 +2,4 @@ '@backstage/backend-common': patch --- -This patch adds HarnessURLReader. It only supports readUrl for now. readTree and search will be implemented next. +Added `HarnessURLReader` with `readUrl` support. diff --git a/.changeset/tasty-rats-explain.md b/.changeset/tasty-rats-explain.md index aee8057915..b97ee28fef 100644 --- a/.changeset/tasty-rats-explain.md +++ b/.changeset/tasty-rats-explain.md @@ -2,4 +2,4 @@ '@backstage/integration': minor --- -This patch brings Harness Code as a valid integration via the ScmIntgration interface. It adds harness code to the relevant static properties ( get integration by name, get integration by type) for plugs to be able to reference the same harness code server +Added `HarnessIntegration` via the `ScmIntegrations` interface. From 0a63f600a999b6aab2a3425a325c22e2578d6e3f Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Tue, 30 Apr 2024 11:52:23 +0200 Subject: [PATCH 55/90] remove GOVERNANCE.md Signed-off-by: Patrik Oldsberg --- GOVERNANCE.md | 233 ------------------ OWNERS.md | 2 +- docs/faq/technical.md | 2 +- ...021-06-22-spotify-backstage-is-growing.mdx | 4 +- microsite/blog/2023-04-26-kubecon-eu-2023.mdx | 2 +- .../blog/2024-04-19-community-plugins.mdx | 2 +- 6 files changed, 6 insertions(+), 239 deletions(-) delete mode 100644 GOVERNANCE.md diff --git a/GOVERNANCE.md b/GOVERNANCE.md deleted file mode 100644 index 2367dab99c..0000000000 --- a/GOVERNANCE.md +++ /dev/null @@ -1,233 +0,0 @@ -# Project Areas - -The Backstage project is divided into several project areas, each covering particular parts of the project. The main driver for each area is ownership of code in the main Backstage repository, as well as other repositories in the Backstage GitHub organization. Each area has a set of maintainers and repository content that they own. There may be no overlap in ownership between areas, which means that any given line in the GitHub code owners file should have only one owner specified. Each area is represented by a team in the Backstage GitHub organization. Apart from certain project-wide concerns, such as the release process, each area is self-governing and chooses their own ways of working. Project areas may also have special interest groups (SIGs) related to their area, but this is not required. - -The project areas as well as their maintainers are listed in the [OWNERS.md](./OWNERS.md) file. - -Each project area must have at least one maintainer. Project area maintainers may have shared ownership with the core maintainers, which in that case is considered an incubating area. The project area maintainers help drive work forward in the area, but they might not yet feel ready to take on ownership. The goal should generally be that these project area maintainers eventually become sole maintainers of the project area. This is to allow for a more smooth onboarding and transition of ownership, where members of the community might for example be new to open source maintainership. - -## Adding new project areas - -Project areas are added by nominating new maintainers for that area. See the sections for becoming a [Project Area Maintainer](#project-area-maintainer). - -Project areas may also by added by splitting existing areas. Every area that is created through this process must have at least one maintainer. - -## Removing project areas - -Project areas are removed by removing all maintainers for that area and removing the corresponding team from the Backstage GitHub organization. The project area can be re-added later if there is a need for it. Reasons for removal may include lack of activity, lack of maintainers, or lack of relevance to the project. - -# Project Roles - -## Contributor - -A Contributor contributes directly to the project and adds value to it. Contributions need not be code. People at the Contributor level may be new contributors, or they may only contribute occasionally. - -### Responsibilities - -- Follow the [CNCF CoC](https://github.com/cncf/foundation/blob/main/code-of-conduct.md) -- Follow the project [contributing guide](CONTRIBUTING.md) - -### How to get involved - -- Participate in community discussions -- Help other users -- Submit bug reports -- Comment on issues -- Try out new releases -- Attend community events - -### How to contribute - -- Report and sometimes resolve issues -- Occasionally submit PRs -- Contribute to the documentation -- Show up at meetings, take notes -- Answer questions from other community members -- Submit feedback on issues and PRs -- Test releases and patches and submit reviews -- Run or help run events -- Promote the project in public - -## Organization Member - -An org member is a frequent contributor that has become a member of the Backstage GitHub organization. In addition to the responsibilities of contributors, an org member is also expected to be reasonably active in the community through continuous contributions of any type. - -An Organization Member must meet the responsibilities and has the requirements of a Contributor. - -### Responsibilities - -- Continues to contribute regularly, as demonstrated by having at least 10 GitHub contributions per year in [Devstats](https://backstage.devstats.cncf.io/d/48/users-statistics-by-repository-group?orgId=1&var-period=y&var-metric=contributions&var-repogroup_name=All&from=now-1y&to=now&var-users=All), or contributions of a similar effort that might not be captured in Devstats. - -### Requirements - -- Must have at least 10 contributions to the projects in the form of: - - Accepted PRs - - Helpful PR reviews - - Resolving GitHub issues - - Or some equivalent contributions to the project -- Must have been contributing for at least 3 months -- Or is the member of a team that owns a project area, in which case the above requirements do not apply and the member is instead vetted by the project area maintainers - -### Becoming an Organization Member - -Open an issue towards [the community repository](https://github.com/backstage/community) using the [org membership request template](https://github.com/backstage/community/issues/new?template=org_member.yaml&title=Org+Member%3A+%3Cyour-github-login%3E). - -### Privileges - -- Membership in the Backstage GitHub organization - -## Plugin Maintainer - -A Plugin Maintainer is responsible for maintaining an individual Backstage plugin or module. This includes reviewing contributions and responding to issues towards an individual plugin, as well as keeping the plugin up to date. - -Plugin Maintainer is a lightweight form of ownership that is primarily reflected though code owners of the plugin packages in the [CODEOWNERS](./.github/CODEOWNERS) file. Each plugin can have one or more maintainers. If a plugin becomes a significant part of the Backstage ecosystem, it may be promoted to be a distinct project area instead. - -A Plugin Maintainer has all the rights and responsibilities of an Organization Member. - -### Responsibilities - -- Review the majority of PRs towards the plugin -- Respond to GitHub issues related to the plugin -- Keep the plugin up-to-date with Backstage libraries and other dependencies -- Follow the [reviewing guide](REVIEWING.md) - -### Requirements - -- Is an Organization Member -- Display knowledge of Backstage's review process and best practices for plugin design -- Is supportive of new and occasional contributors and helps get useful PRs in shape to merge - -### Privileges - -- GitHub code owner of the plugin directory, with rights to approve and merge PRs towards the plugin - -### Becoming a Plugin Maintainer - -To become a Plugin Maintainer, you first need to be an Organization Member. You can then file a pull request towards [CODEOWNERS.md](./.github/CODEOWNERS) requesting to be added as a code owner of the plugin directory. Existing code owners of that plugin alongside the core maintainers will then review the request. - -## Project Area Maintainer - -Project Area Maintainers are owners of a particular project area. They are expected to review and merge pull requests towards their area, and also drive development and manage tech health. A Project Area Maintainer also need to commit a certain number of hours per month towards the project, and exercise judgment for the good of the project, independent of their employer. Project Area Maintainers should also mentor new maintainers and participate in and lead community meetings related to their area. New Project Area Maintainers need to be approved by the existing project area maintainers, or the core maintainers if it is a new area. - -The maintainers of a project area may be represented by a team in external organization. In this case, the state as a project area maintainer is tied to the membership in that team. New members of the team may automatically be added as maintainers, as well as removed when they leave. This process is governed autonomously by the team of project area maintainers. - -A Project Area Maintainer has all the rights and responsibilities of an Organization Member. - -### Responsibilities - -- Review PRs towards their project area. Project area maintainers are expected to review at least 20 PRs per year, or the majority of all PRs towards the area, if it is less than 20 -- Follow the [reviewing guide](REVIEWING.md) -- Triage and respond to issues related to their project area -- Mentor new project area maintainers -- Write refactoring PRs -- Determine strategy and policy for the project area -- Participate in or leading community meetings related to their project area - -### Requirements - -- Is an Organization Member -- Have made at least 5 meaningful contributions towards the project area -- Demonstrates knowledge of their project area, and how it fits into the larger Backstage project -- Is able to exercise judgment for the good of the project, independent of their employer, friends, or team -- Mentors other contributors and project area maintainers -- Can commit to spending at least 16 hours per month working on the project, preferably distributed evenly across the month - -### Privileges - -- Approve and merge PRs towards their project area -- Drive the direction and roadmap of their project area - -### Becoming a Project Area Maintainer - -If you are interested in becoming a project area maintainer, reach out to the existing maintainers for that area. If you wish to become a maintainer for a new area, reach out to the core maintainers. - -Any current project area maintainer or core maintainer may nominate a new project area maintainer by opening a PR towards the [OWNERS.md](OWNERS.md) file. A majority of the project area maintainers for that area must approve the PR. If there are no existing maintainers for that area, the PR must be approved by a majority of the core maintainers. - -## Core Maintainer - -Core Maintainers are responsible for the Backstage project as a whole. They help review and merge project-level pull requests as well as coordinate work affecting multiple project areas. A core maintainer needs to commit the majority of their working time towards the project, and exercise judgment for the good of the project, independent of their employer. Core maintainers should also mentor and seek out new maintainers, lead community meetings, and communicate with the CNCF on behalf of the project. To become a core maintainer one needs to have been the maintainer of a number of different project areas, demonstrate a deep knowledge of large parts of the Backstage project, and be backed by the existing core maintainers. - -A Core Maintainer have all the rights and responsibilities of a Project Area Maintainer. - -### Responsibilities - -- Take part in the incoming issue and PR triage and review process. PRs are shared equally among all maintainers -- Mentor new Project Area Maintainers and Plugin Maintainers -- Drive refactoring and manage tech health across the entire project -- Participate in CNCF maintainer activities -- Respond to security incidents in accordance to our [security policy](./SECURITY.md) -- Determine strategy and policy for the project -- Participate in or leading community meetings - -### Requirements - -- Experience as a Project Area Maintainer for at least 6 months -- Demonstrates a broad knowledge of the project across multiple areas -- Is able to exercise judgment for the good of the project, independent of their employer, friends, or team -- Mentors other contributors -- Can commit to spending at least 10 days per month working on the project - -### Privileges - -- Approve PRs that fall outside any specific project area -- Merge PRs to any area of the project -- Represent the project in public as a Maintainer -- Communicate with the CNCF on behalf of the project -- Have a vote in Maintainer decision-making meetings - -### Becoming a Core Maintainer - -Any core maintainer or end user sponsor may nominate a new core maintainer by opening a PR towards the [OWNERS.md](OWNERS.md) file. Core maintainers must be approved by a majority of the existing core maintainers and end user sponsors. - -## End User Sponsors - -### Role of a Backstage End User Sponsor - -- Provide support for Backstage by removing blockers, securing funding, providing advocacy, feedback, and ensuring project continuity and long term success -- Assist Backstage maintainers in prioritizing upcoming roadmap items and planned work -- Provide neutral mediation for any disputes that arise as part of the project - -### Backstage End User Sponsor Membership - -The End User Sponsors group comprises at most 5 people. To be eligible for membership in the group, you or the company where you work you must: - -- Be responsible for and end user of a production Backstage deployment of non-trivial size -- Be active contributors to the open source project -- Be willing and able to attend regularly-scheduled End User Sponsor meetings -- Abide by [CNCF CoC](https://github.com/cncf/foundation/blob/main/code-of-conduct.md) - -Candidates for membership will be nominated by current Sponsor members or by Backstage maintainers. If there are more nominations than Sponsor seats remaining, existing sponsors shall vote on the candidates, and the candidates with the most votes will become Sponsors. Any ties will be broken by current Backstage sponsors. - -# Conflict resolution and voting - -In general, we prefer that technical issues and membership are amicably worked out between the persons involved. If a dispute cannot be decided independently, the sponsors and core maintainers can be called in to decide an issue. If the sponsors and maintainers themselves cannot decide an issue, the issue will be resolved by voting. - -In all cases in this document where voting is mentioned, the voting process is a simple majority in which each sponsor receives two votes and each core maintainer receives one vote. If such a majority is reached, the vote is said to have _passed_. - -## Inactivity - -It is important for contributors to be and stay active to set an example and show commitment to the project. Inactivity is harmful to the project as it may lead to unexpected delays, contributor attrition, and a loss of trust in the project. - -Inactivity is measured by periods of no contributions without explanation, for longer than: - -- Core Maintainer: 2 months -- Project Area Maintainer: 4 months -- Plugin Maintainer: 6 months -- Organization Member: 12 months - -Consequences of being inactive include: - -- Involuntary removal or demotion -- Being asked to move to Emeritus status - -## Involuntary Removal or Demotion - -Involuntary removal/demotion of a contributor happens when responsibilities and requirements aren't being met. This may include repeated patterns of inactivity, extended period of inactivity, a period of failing to meet the requirements of your role, and/or a violation of the Code of Conduct. This process is important because it protects the community and its deliverables while also opens up opportunities for new contributors to step in. - -Involuntary removal or demotion is handled through a vote by a majority of the current Core Maintainers. Some aspects of this process may be automated, such as removal after periods of inactivity. - -## Stepping Down/Emeritus Process - -If and when contributors' commitment levels change, contributors can consider stepping down (moving down the contributor ladder) vs moving to emeritus status (completely stepping away from the project). - -Contact the Maintainers about changing to Emeritus status, or reducing your contributor level. diff --git a/OWNERS.md b/OWNERS.md index ab35b5b1e5..d8967ca873 100644 --- a/OWNERS.md +++ b/OWNERS.md @@ -1,5 +1,5 @@ - See [CONTRIBUTING.md](CONTRIBUTING.md) for general contribution guidelines. -- See [GOVERNANCE.md](GOVERNANCE.md) for governance guidelines and responsibilities. +- See [GOVERNANCE.md](https://github.com/backstage/community/blob/main/GOVERNANCE.md) for governance guidelines and responsibilities. ## Core Maintainers diff --git a/docs/faq/technical.md b/docs/faq/technical.md index ffd99dadcd..d64d81d499 100644 --- a/docs/faq/technical.md +++ b/docs/faq/technical.md @@ -154,7 +154,7 @@ maintains Backstage in your own environment. For more information, see our [Owners](https://github.com/backstage/backstage/blob/master/OWNERS.md) and -[Governance](https://github.com/backstage/backstage/blob/master/GOVERNANCE.md). +[Governance](https://github.com/backstage/community/blob/main/GOVERNANCE.md). ### Does Spotify provide a managed version of Backstage? diff --git a/microsite/blog/2021-06-22-spotify-backstage-is-growing.mdx b/microsite/blog/2021-06-22-spotify-backstage-is-growing.mdx index 6cae5c907e..308effaeab 100644 --- a/microsite/blog/2021-06-22-spotify-backstage-is-growing.mdx +++ b/microsite/blog/2021-06-22-spotify-backstage-is-growing.mdx @@ -66,9 +66,9 @@ Speaking of reviewers and maintainers… ## Adding reviewers and maintainers -[![GitHub logo](assets/21-06-22/gh-reviewers.png)](https://github.com/backstage/backstage/blob/master/GOVERNANCE.md#reviewers) +[![GitHub logo](assets/21-06-22/gh-reviewers.png)](https://github.com/backstage/community/blob/main/GOVERNANCE.md#reviewers) -We have introduced [reviewers](https://github.com/backstage/backstage/blob/master/GOVERNANCE.md#reviewers) to the project! By adding this new role, we’ve expanded the number of people who are permitted to approve and merge pull requests. This will offload some of the review work from the maintainers, simplifying and speeding up the review process for contributors. +We have introduced [reviewers](https://github.com/backstage/community/blob/main/GOVERNANCE.md#reviewers) to the project! By adding this new role, we’ve expanded the number of people who are permitted to approve and merge pull requests. This will offload some of the review work from the maintainers, simplifying and speeding up the review process for contributors. Of course, with these new efforts, we expect even more companies to adopt Backstage, which means the platform will continue to grow, and the number of PRs will continue to grow with it. As that happens, we hope to add to both the maintainer and reviewer teams in the future. diff --git a/microsite/blog/2023-04-26-kubecon-eu-2023.mdx b/microsite/blog/2023-04-26-kubecon-eu-2023.mdx index 7ee549c088..8f369c2d14 100644 --- a/microsite/blog/2023-04-26-kubecon-eu-2023.mdx +++ b/microsite/blog/2023-04-26-kubecon-eu-2023.mdx @@ -21,7 +21,7 @@ On Tuesday the Backstage maintainers hosted a jam-packed project meeting. The co ![Patrik and Ben onstage for the State of Backstage talk](assets/2023-04-26/IMG_0120.png) -Core maintainers [Ben Lambert](https://github.com/benjdlambert) and [Patrik Oldsberg](https://github.com/Rugvip) took center stage on Wednesday for the Backstage Maintainer Track: State of Backstage in 2023 talk. Backstage has officially hit over 1,000 adopters and 1,000 contributors – so it’s apt timing to modernize the governance model for the project. Taking pointers from the [CNCF Contributor Ladder Governance Template](https://contribute.cncf.io/maintainers/templates/), a new [Backstage Governance Model](https://github.com/backstage/backstage/blob/master/GOVERNANCE.md) is now in effect! Patrik walked us through the new ladder model which introduces a number of changes, one being the addition of [project area maintainers](https://github.com/backstage/backstage/blob/master/GOVERNANCE.md#project-area-maintainer). This role lets members of the community take increased ownership over a specific area of interest, like Catalog, Discoverability, TechDocs, Helm Charts, and Kubernetes. New project areas that will be added include Permissions and Software Templates. New project areas can be proposed by nominating a project area maintainer for the area. The new model also adds an organization member role for contributors who want to take a more active role in the Backstage community. You can open an issue to become an organization member [here](https://github.com/backstage/community/issues/new/choose). +Core maintainers [Ben Lambert](https://github.com/benjdlambert) and [Patrik Oldsberg](https://github.com/Rugvip) took center stage on Wednesday for the Backstage Maintainer Track: State of Backstage in 2023 talk. Backstage has officially hit over 1,000 adopters and 1,000 contributors – so it’s apt timing to modernize the governance model for the project. Taking pointers from the [CNCF Contributor Ladder Governance Template](https://contribute.cncf.io/maintainers/templates/), a new [Backstage Governance Model](https://github.com/backstage/community/blob/main/GOVERNANCE.md) is now in effect! Patrik walked us through the new ladder model which introduces a number of changes, one being the addition of [project area maintainers](https://github.com/backstage/community/blob/main/GOVERNANCE.md#project-area-maintainer). This role lets members of the community take increased ownership over a specific area of interest, like Catalog, Discoverability, TechDocs, Helm Charts, and Kubernetes. New project areas that will be added include Permissions and Software Templates. New project areas can be proposed by nominating a project area maintainer for the area. The new model also adds an organization member role for contributors who want to take a more active role in the Backstage community. You can open an issue to become an organization member [here](https://github.com/backstage/community/issues/new/choose). ![Contributor ladder](assets/2023-04-26/contributor_ladder.png) diff --git a/microsite/blog/2024-04-19-community-plugins.mdx b/microsite/blog/2024-04-19-community-plugins.mdx index bb8b9c2313..49d1765bce 100644 --- a/microsite/blog/2024-04-19-community-plugins.mdx +++ b/microsite/blog/2024-04-19-community-plugins.mdx @@ -14,7 +14,7 @@ For those who depended on these plugins, migrating is as simple as `yarn backsta ## The community plugins repo -Some of you who have been around a while, or have seen our [Maintainer Track talks](https://www.youtube.com/watch?v=ONMBYnhxnNU) at KubeCon, might have seen [this RFC](https://github.com/backstage/backstage/issues/20266) which outlines some issues with the scale of the `backstage/backstage` monorepo, and us as maintainers being the de facto owners of all plugins without a [project area](https://github.com/backstage/backstage/blob/master/GOVERNANCE.md#project-area) or [plugin maintainer](https://github.com/backstage/backstage/blob/master/GOVERNANCE.md#project-area-maintainer). +Some of you who have been around a while, or have seen our [Maintainer Track talks](https://www.youtube.com/watch?v=ONMBYnhxnNU) at KubeCon, might have seen [this RFC](https://github.com/backstage/backstage/issues/20266) which outlines some issues with the scale of the `backstage/backstage` monorepo, and us as maintainers being the de facto owners of all plugins without a [project area](https://github.com/backstage/community/blob/main/GOVERNANCE.md#project-area) or [plugin maintainer](https://github.com/backstage/community/blob/main/GOVERNANCE.md#project-area-maintainer). There was some great discussion in this issue, and some great ideas. One of the ideas was to create a dedicated home for community plugins, with all the burden of release tooling and workspace tooling already set up, which is a pretty big barrier for people wanting to create plugins for Backstage in their own organization or personal account. These plugins would then have the ability to release independently of the main monorepo, and have their own release cadence, which is something that we've been looking at exploring for a while. From d4a3b10f02bae9888166cd386d1d875c364cdee2 Mon Sep 17 00:00:00 2001 From: secustor Date: Tue, 30 Apr 2024 14:00:43 +0200 Subject: [PATCH 56/90] feat: expose DD RUM sessionSampleRate and sessionReplaySampleRate Signed-off-by: secustor --- app-config.yaml | 3 + docs/integrations/datadog-rum/installation.md | 2 + packages/app-next/public/index.html | 5 +- packages/app/public/index.html | 5 +- packages/cli/package.json | 62 +++++++++++-------- 5 files changed, 49 insertions(+), 28 deletions(-) diff --git a/app-config.yaml b/app-config.yaml index 9e5b5c468c..cb70170541 100644 --- a/app-config.yaml +++ b/app-config.yaml @@ -9,6 +9,9 @@ app: # applicationId: qwerty # site: # datadoghq.eu default = datadoghq.com # env: # optional + # sessionSampleRate: 100 + # sessionReplaySampleRate: 0 + support: url: https://github.com/backstage/backstage/issues # Used by common ErrorPage items: # Used by common SupportButton component diff --git a/docs/integrations/datadog-rum/installation.md b/docs/integrations/datadog-rum/installation.md index 32eb2b3de8..4dc6c24134 100644 --- a/docs/integrations/datadog-rum/installation.md +++ b/docs/integrations/datadog-rum/installation.md @@ -22,6 +22,8 @@ app: applicationId: qwerty # site: datadoghq.eu # env: 'staging' + # sessionSampleRate: 100 + # sessionReplaySampleRate: 0 ``` If your [`app-config.yaml`](https://github.com/backstage/backstage/blob/e0506af8fc54074a160fb91c83d6cae8172d3bb3/app-config.yaml#L5) file does not have this configuration, you may have to adjust your [`packages/app/public/index.html`](https://github.com/backstage/backstage/blob/e0506af8fc54074a160fb91c83d6cae8172d3bb3/packages/app/public/index.html#L69) to include the Datadog RUM `init()` section manually. diff --git a/packages/app-next/public/index.html b/packages/app-next/public/index.html index 1bbfda590d..63ab0bec0c 100644 --- a/packages/app-next/public/index.html +++ b/packages/app-next/public/index.html @@ -72,7 +72,10 @@ site: '<%= config.getOptionalString("app.datadogRum.site") || "datadoghq.com" %>', service: 'backstage', env: '<%= config.getString("app.datadogRum.env") %>', - sampleRate: 100, + sampleRate: + '<%= config.getOptionalNumber("app.datadogRum.sessionSampleRate") || 100 %>', + sessionReplaySampleRate: + '<%= config.getOptionalNumber("app.datadogRum.sessionReplaySampleRate") || 0 %>', trackInteractions: true, }); }); diff --git a/packages/app/public/index.html b/packages/app/public/index.html index 631666da28..9a8267e98e 100644 --- a/packages/app/public/index.html +++ b/packages/app/public/index.html @@ -72,7 +72,10 @@ site: '<%= config.getOptionalString("app.datadogRum.site") || "datadoghq.com" %>', service: 'backstage', env: '<%= config.getString("app.datadogRum.env") %>', - sampleRate: 100, + sampleRate: + '<%= config.getOptionalNumber("app.datadogRum.sessionSampleRate") || 100 %>', + sessionReplaySampleRate: + '<%= config.getOptionalNumber("app.datadogRum.sessionReplaySampleRate") || 0 %>', trackInteractions: true, }); }); diff --git a/packages/cli/package.json b/packages/cli/package.json index ecdfa590d6..c8baae4d4d 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,34 +1,46 @@ { "name": "@backstage/cli", - "description": "CLI for developing Backstage plugins and apps", "version": "0.26.5-next.0", - "publishConfig": { - "access": "public" - }, + "description": "CLI for developing Backstage plugins and apps", "backstage": { "role": "cli" }, + "publishConfig": { + "access": "public" + }, + "keywords": [ + "backstage" + ], "homepage": "https://backstage.io", "repository": { "type": "git", "url": "https://github.com/backstage/backstage", "directory": "packages/cli" }, - "keywords": [ - "backstage" - ], "license": "Apache-2.0", "main": "dist/index.cjs.js", - "scripts": { - "build": "backstage-cli package build", - "lint": "backstage-cli package lint", - "test": "backstage-cli package test", - "clean": "backstage-cli package clean", - "start": "nodemon --" - }, "bin": { "backstage-cli": "bin/backstage-cli" }, + "files": [ + "asset-types", + "templates", + "config", + "bin", + "dist/**/*.js" + ], + "scripts": { + "build": "backstage-cli package build", + "clean": "backstage-cli package clean", + "lint": "backstage-cli package lint", + "start": "nodemon --", + "test": "backstage-cli package test" + }, + "nodemonConfig": { + "exec": "bin/backstage-cli", + "ext": "ts", + "watch": "./src" + }, "dependencies": { "@backstage/catalog-model": "workspace:^", "@backstage/cli-common": "workspace:^", @@ -197,18 +209,6 @@ "optional": true } }, - "files": [ - "asset-types", - "templates", - "config", - "bin", - "dist/**/*.js" - ], - "nodemonConfig": { - "watch": "./src", - "exec": "bin/backstage-cli", - "ext": "ts" - }, "configSchema": { "$schema": "https://backstage.io/schema/config-v1", "title": "@backstage/cli", @@ -248,6 +248,16 @@ "type": "string", "visibility": "frontend", "description": "site for Datadog RUM events" + }, + "sessionSampleRate": { + "type": "number", + "visibility": "frontend", + "description": "sample rate of Datadog RUM events" + }, + "sessionReplaySampleRate": { + "type": "number", + "visibility": "frontend", + "description": "sample rate of session replays based upon already sampled Datadog RUM events" } }, "required": [ From 84cdb92346785ceffda429fa64e35d7a6bce94cf Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 30 Apr 2024 13:20:34 -0600 Subject: [PATCH 57/90] integration support for harness - fixed comments p2 Signed-off-by: Calvin Lee --- packages/integration/package.json | 1 - packages/integration/src/harness/core.test.ts | 16 ++++++++-------- packages/integration/src/harness/core.ts | 4 ---- 3 files changed, 8 insertions(+), 13 deletions(-) diff --git a/packages/integration/package.json b/packages/integration/package.json index b57b94f982..bbd0eecbe7 100644 --- a/packages/integration/package.json +++ b/packages/integration/package.json @@ -50,7 +50,6 @@ "devDependencies": { "@backstage/cli": "workspace:^", "@backstage/config-loader": "workspace:^", - "@backstage/test-utils": "workspace:^", "@types/luxon": "^3.0.0", "msw": "^1.0.0" }, diff --git a/packages/integration/src/harness/core.test.ts b/packages/integration/src/harness/core.test.ts index abfbb5d4c2..d3d402806a 100644 --- a/packages/integration/src/harness/core.test.ts +++ b/packages/integration/src/harness/core.test.ts @@ -15,7 +15,7 @@ */ import { setupServer } from 'msw/node'; -import { setupRequestMockHandlers } from '@backstage/test-utils'; +import { setupRequestMockHandlers } from '../helpers'; import { HarnessIntegrationConfig } from './config'; import { getHarnessEditContentsUrl, @@ -59,26 +59,26 @@ describe('Harness code core', () => { }); }); - describe('getGerritRequestOptions', () => { + describe('getHarnessRequestOptions', () => { it('adds token header when only a token is specified', () => { const authRequest: HarnessIntegrationConfig = { - host: 'gerrit.com', + host: 'app.harness.io', token: 'P', }; const anonymousRequest: HarnessIntegrationConfig = { - host: 'gerrit.com', + host: 'app.harness.io', }; expect( (getHarnessRequestOptions(authRequest).headers as any).Authorization, ).toEqual('Bearer P'); - expect( - getHarnessRequestOptions(anonymousRequest).headers, - ).toBeUndefined(); + expect(getHarnessRequestOptions(anonymousRequest).headers).toStrictEqual( + {}, + ); }); it('adds basic auth when apikey and token are specified', () => { const authRequest: HarnessIntegrationConfig = { - host: 'gerrit.com', + host: 'app.harness.io', token: 'P', apiKey: 'a', }; diff --git a/packages/integration/src/harness/core.ts b/packages/integration/src/harness/core.ts index 8eba850ec2..3a61905e27 100644 --- a/packages/integration/src/harness/core.ts +++ b/packages/integration/src/harness/core.ts @@ -120,10 +120,6 @@ export function getHarnessRequestOptions(config: HarnessIntegrationConfig): { const headers: Record = {}; const { token, apiKey } = config; - if (!token) { - return headers; - } - if (apiKey) { headers['x-api-key'] = apiKey; } else if (token) { From 677b10650b806d743e0f6ad1b2718fb11d8cf3c5 Mon Sep 17 00:00:00 2001 From: Calvin Lee Date: Tue, 30 Apr 2024 13:23:30 -0600 Subject: [PATCH 58/90] integration support for harness - fixed comments p2 Signed-off-by: Calvin Lee --- yarn.lock | 1 - 1 file changed, 1 deletion(-) diff --git a/yarn.lock b/yarn.lock index 60b03d4399..3cf5a79175 100644 --- a/yarn.lock +++ b/yarn.lock @@ -4368,7 +4368,6 @@ __metadata: "@backstage/config": "workspace:^" "@backstage/config-loader": "workspace:^" "@backstage/errors": "workspace:^" - "@backstage/test-utils": "workspace:^" "@octokit/auth-app": ^4.0.0 "@octokit/rest": ^19.0.3 "@types/luxon": ^3.0.0 From 12a5feff7a9fd4c6e3c844f25da850aff8de77d6 Mon Sep 17 00:00:00 2001 From: Chap Ambrose Date: Tue, 30 Apr 2024 16:52:20 -0500 Subject: [PATCH 59/90] seperate ensureSchemaExists config Signed-off-by: Chap Ambrose --- .../src/database/connectors/postgres.ts | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/packages/backend-common/src/database/connectors/postgres.ts b/packages/backend-common/src/database/connectors/postgres.ts index 1ddc096e05..1aa4d2afa9 100644 --- a/packages/backend-common/src/database/connectors/postgres.ts +++ b/packages/backend-common/src/database/connectors/postgres.ts @@ -321,7 +321,7 @@ export class PgConnector implements Connector { let schemaOverrides; if (this.getPluginDivisionModeConfig() === 'schema') { schemaOverrides = this.getSchemaOverrides(pluginId); - if (this.getEnsureExistsConfig(pluginId)) { + if (this.getEnsureSchemaExistsConfig(pluginId)) { try { await pgConnector.ensureSchemaExists!(pluginConfig, pluginId); } catch (error) { @@ -437,6 +437,15 @@ export class PgConnector implements Connector { ); } + private getEnsureSchemaExistsConfig(pluginId: string): boolean { + const baseConfig = + this.config.getOptionalBoolean('ensureSchemaExists') ?? true; + return ( + this.config.getOptionalBoolean(`${pluginPath(pluginId)}.ensureExists`) ?? + baseConfig + ); + } + private getPluginDivisionModeConfig(): string { return this.config.getOptionalString('pluginDivisionMode') ?? 'database'; } From 86ae51bb4ad512d9c569e192fcf5ccaa0cf8b698 Mon Sep 17 00:00:00 2001 From: Chap Ambrose Date: Wed, 1 May 2024 08:38:17 -0500 Subject: [PATCH 60/90] fixed getEnsureSchemaExistsConfig Signed-off-by: Chap Ambrose --- packages/backend-common/src/database/connectors/postgres.ts | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/packages/backend-common/src/database/connectors/postgres.ts b/packages/backend-common/src/database/connectors/postgres.ts index 1aa4d2afa9..126c8cb89c 100644 --- a/packages/backend-common/src/database/connectors/postgres.ts +++ b/packages/backend-common/src/database/connectors/postgres.ts @@ -441,8 +441,9 @@ export class PgConnector implements Connector { const baseConfig = this.config.getOptionalBoolean('ensureSchemaExists') ?? true; return ( - this.config.getOptionalBoolean(`${pluginPath(pluginId)}.ensureExists`) ?? - baseConfig + this.config.getOptionalBoolean( + `${pluginPath(pluginId)}.getEnsureSchemaExistsConfig`, + ) ?? baseConfig ); } From ccc8851bd01a9f05bc4ce8e71f11a417c90cfdb4 Mon Sep 17 00:00:00 2001 From: Chap Ambrose Date: Wed, 1 May 2024 09:05:30 -0500 Subject: [PATCH 61/90] add changeset Signed-off-by: Chap Ambrose --- .changeset/heavy-trainers-fly.md | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 .changeset/heavy-trainers-fly.md diff --git a/.changeset/heavy-trainers-fly.md b/.changeset/heavy-trainers-fly.md new file mode 100644 index 0000000000..db09f63496 --- /dev/null +++ b/.changeset/heavy-trainers-fly.md @@ -0,0 +1,5 @@ +--- +'@backstage/backend-common': patch +--- + +add ensureSchemaExists backend database config From 0b8b8e80c8a6bfe74420d40c0f91fa6dcdcfeefe Mon Sep 17 00:00:00 2001 From: Chap Ambrose Date: Wed, 1 May 2024 10:07:08 -0500 Subject: [PATCH 62/90] set ensureSchemaExists to false to match current behavior Signed-off-by: Chap Ambrose --- .changeset/heavy-trainers-fly.md | 2 +- .../backend-common/src/database/connectors/postgres.ts | 7 +++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/.changeset/heavy-trainers-fly.md b/.changeset/heavy-trainers-fly.md index db09f63496..48560b7ddd 100644 --- a/.changeset/heavy-trainers-fly.md +++ b/.changeset/heavy-trainers-fly.md @@ -2,4 +2,4 @@ '@backstage/backend-common': patch --- -add ensureSchemaExists backend database config +Added config prop `ensureSchemaExists` to support postgres instances where user can create schemas but not databases. diff --git a/packages/backend-common/src/database/connectors/postgres.ts b/packages/backend-common/src/database/connectors/postgres.ts index 126c8cb89c..b81bfd5b50 100644 --- a/packages/backend-common/src/database/connectors/postgres.ts +++ b/packages/backend-common/src/database/connectors/postgres.ts @@ -321,7 +321,10 @@ export class PgConnector implements Connector { let schemaOverrides; if (this.getPluginDivisionModeConfig() === 'schema') { schemaOverrides = this.getSchemaOverrides(pluginId); - if (this.getEnsureSchemaExistsConfig(pluginId)) { + if ( + this.getEnsureSchemaExistsConfig(pluginId) || + this.getEnsureExistsConfig(pluginId) + ) { try { await pgConnector.ensureSchemaExists!(pluginConfig, pluginId); } catch (error) { @@ -439,7 +442,7 @@ export class PgConnector implements Connector { private getEnsureSchemaExistsConfig(pluginId: string): boolean { const baseConfig = - this.config.getOptionalBoolean('ensureSchemaExists') ?? true; + this.config.getOptionalBoolean('ensureSchemaExists') ?? false; return ( this.config.getOptionalBoolean( `${pluginPath(pluginId)}.getEnsureSchemaExistsConfig`, From d541ff686f828c9b7a18e447e2dc81359f5918fe Mon Sep 17 00:00:00 2001 From: Heikki Hellgren Date: Thu, 2 May 2024 08:11:35 +0300 Subject: [PATCH 63/90] fix: email processor esm issue with p-throttle + config read Signed-off-by: Heikki Hellgren --- .changeset/cyan-eagles-hammer.md | 6 ++++++ .github/renovate.json5 | 4 ++++ .../notifications-backend-module-email/package.json | 2 +- .../processor/NotificationsEmailProcessor.test.ts | 12 ++++++------ .../src/processor/NotificationsEmailProcessor.ts | 2 +- yarn.lock | 10 +++++----- 6 files changed, 23 insertions(+), 13 deletions(-) create mode 100644 .changeset/cyan-eagles-hammer.md diff --git a/.changeset/cyan-eagles-hammer.md b/.changeset/cyan-eagles-hammer.md new file mode 100644 index 0000000000..dfe6f0bc62 --- /dev/null +++ b/.changeset/cyan-eagles-hammer.md @@ -0,0 +1,6 @@ +--- +'@backstage/plugin-notifications-backend-module-email': patch +'@backstage/plugin-notifications-backend': patch +--- + +Fixed email processor `esm` issue and config reading diff --git a/.github/renovate.json5 b/.github/renovate.json5 index a9cb54912a..1c8d611c49 100644 --- a/.github/renovate.json5 +++ b/.github/renovate.json5 @@ -62,6 +62,10 @@ matchPackageNames: ['p-limit'], allowedVersions: '<4.0.0', }, + { + matchPackageNames: ['p-throttle'], + allowedVersions: '<4.0.0', + }, { matchPackageNames: ['p-queue'], allowedVersions: '<7.0.0', diff --git a/plugins/notifications-backend-module-email/package.json b/plugins/notifications-backend-module-email/package.json index 94b5e10578..bb99d9cb97 100644 --- a/plugins/notifications-backend-module-email/package.json +++ b/plugins/notifications-backend-module-email/package.json @@ -45,7 +45,7 @@ "@backstage/types": "workspace:^", "lodash": "^4.17.21", "nodemailer": "^6.9.13", - "p-throttle": "^6.1.0" + "p-throttle": "^4.1.1" }, "devDependencies": { "@backstage/backend-test-utils": "workspace:^", diff --git a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts index 3af1e498ca..11eef9b999 100644 --- a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts +++ b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts @@ -52,7 +52,7 @@ describe('NotificationsEmailProcessor', () => { notifications: { processors: { email: { - transport: { + transportConfig: { transport: 'smtp', hostname: 'localhost', port: 465, @@ -98,7 +98,7 @@ describe('NotificationsEmailProcessor', () => { notifications: { processors: { email: { - transport: { + transportConfig: { transport: 'ses', region: 'us-west-2', }, @@ -138,7 +138,7 @@ describe('NotificationsEmailProcessor', () => { notifications: { processors: { email: { - transport: { + transportConfig: { transport: 'sendmail', path: '/usr/local/bin/sendmail', }, @@ -189,7 +189,7 @@ describe('NotificationsEmailProcessor', () => { notifications: { processors: { email: { - transport: { + transportConfig: { transport: 'sendmail', path: '/usr/local/bin/sendmail', }, @@ -246,7 +246,7 @@ describe('NotificationsEmailProcessor', () => { notifications: { processors: { email: { - transport: { + transportConfig: { transport: 'sendmail', path: '/usr/local/bin/sendmail', }, @@ -306,7 +306,7 @@ describe('NotificationsEmailProcessor', () => { notifications: { processors: { email: { - transport: { + transportConfig: { transport: 'sendmail', path: '/usr/local/bin/sendmail', }, diff --git a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts index 5387d09c3d..ce944d9592 100644 --- a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts +++ b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts @@ -62,7 +62,7 @@ export class NotificationsEmailProcessor implements NotificationProcessor { const emailProcessorConfig = config.getConfig( 'notifications.processors.email', ); - this.transportConfig = emailProcessorConfig.getConfig('transport'); + this.transportConfig = emailProcessorConfig.getConfig('transportConfig'); this.broadcastConfig = emailProcessorConfig.getOptionalConfig('broadcastConfig'); this.sender = emailProcessorConfig.getString('sender'); diff --git a/yarn.lock b/yarn.lock index 3cf5a79175..6b4def9d8e 100644 --- a/yarn.lock +++ b/yarn.lock @@ -6193,7 +6193,7 @@ __metadata: "@types/nodemailer": ^6.4.14 lodash: ^4.17.21 nodemailer: ^6.9.13 - p-throttle: ^6.1.0 + p-throttle: ^4.1.1 languageName: unknown linkType: soft @@ -33449,10 +33449,10 @@ __metadata: languageName: node linkType: hard -"p-throttle@npm:^6.1.0": - version: 6.1.0 - resolution: "p-throttle@npm:6.1.0" - checksum: c1947cca8844564c3d86f8c09067add5e7398b87898cb1f1aea5c48d2c211590d039a316d28a4b0d95364ffa0213c01dff6bf71175c468eab0e381f77715dbdb +"p-throttle@npm:^4.1.1": + version: 4.1.1 + resolution: "p-throttle@npm:4.1.1" + checksum: fe8709f3c3b1da7c033479375c2c302e80c1a5d86449013afa7cd46d1dc210bc824a7e4a9d088e66d31987d00878c2b5491bb2fe76246d4d2fc9a1636f5f8298 languageName: node linkType: hard From 6a8e728cf2686dbde09c0b58eb66e53ccbf76c4f Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Thu, 2 May 2024 13:38:53 +0200 Subject: [PATCH 64/90] docs/architecture-overview: update to include new backend system Signed-off-by: Patrik Oldsberg --- .../package-architecture.drawio.svg | 663 ++++++++++-------- 1 file changed, 369 insertions(+), 294 deletions(-) diff --git a/docs/assets/architecture-overview/package-architecture.drawio.svg b/docs/assets/architecture-overview/package-architecture.drawio.svg index df8c0f8820..89fab2647d 100644 --- a/docs/assets/architecture-overview/package-architecture.drawio.svg +++ b/docs/assets/architecture-overview/package-architecture.drawio.svg @@ -1,69 +1,66 @@ - + - - - - - - - - - - + + + + + + + + + - +
-
-
+
+
app
- + app - - - - - - + + + + - +
-
-
+
+
backend
- + backend - - - - - - + + + + + + - +
-
-
+
+
plugin-<plugin-id>
@@ -74,19 +71,19 @@ - - - - - - + + + + + + - +
-
-
+
+
plugin-<plugin-id>-backend
@@ -97,69 +94,9 @@ - - - - - - Backend Libraries - - - - - - -
-
-
- @backstage/backend-common -
-
-
- - - @backstage/backend-common - - - - - - - -
-
-
- @backstage/backend-test-utils -
-
-
- - - @backstage/backend-test-utils - - - - - - - -
-
-
- @backstage/backend-tasks -
-
-
- - - @backstage/backend-tasks - - - - - - - + + + Common Libraries @@ -167,16 +104,16 @@ - +
-
-
+
+
@backstage/catalog-client
- + @backstage/catalog-client @@ -184,16 +121,16 @@ - +
-
-
+
+
@backstage/types
- + @backstage/types @@ -201,16 +138,16 @@ - +
-
-
+
+
@backstage/config
- + @backstage/config @@ -218,16 +155,16 @@ - +
-
-
+
+
@backstage/errors
- + @backstage/errors @@ -235,16 +172,16 @@ - +
-
-
+
+
@backstage/catalog-model
- + @backstage/catalog-model @@ -252,24 +189,23 @@ - +
-
-
+
+
@backstage/integration
- + @backstage/integration - - - - + + + Frontend App Core @@ -277,16 +213,16 @@ - +
-
-
+
+
@backstage/core-app-api
- + @backstage/core-app-api @@ -294,36 +230,36 @@ - +
-
-
+
+
@backstage/app-defaults
- + @backstage/app-defaults - - - - - - - - - + + + + + + + + + - +
-
-
+
+
plugin-<plugin-id>-backend-module-<module-id>
@@ -334,17 +270,17 @@ - - - - + + + + - +
-
-
+
+
plugin-<plugin-id>-module-<module-id>
@@ -355,10 +291,9 @@ - - - - + + + Common Tooling @@ -366,28 +301,27 @@ - +
-
-
+
+
@backstage/cli
- + @backstage/cli - - - - - - - - + + + + + + + External Plugin Libraries @@ -395,16 +329,16 @@ - +
-
-
+
+
plugin-<other-plugin-id>-react
- + plugin-<other-plugin-id>-react @@ -412,16 +346,16 @@ - +
-
-
+
+
plugin-<other-plugin-id>-common
- + plugin-<other-plugin-id>-common @@ -429,30 +363,29 @@ - +
-
-
+
+
plugin-<other-plugin-id>-node
- + plugin-<other-plugin-id>-node - - - - - - - - - - + + + + + + + + + Plugin Libraries @@ -460,10 +393,10 @@ - +
-
-
+
+
plugin-<plugin-id>-react
@@ -477,10 +410,10 @@ - +
-
-
+
+
plugin-<plugin-id>-common
@@ -494,10 +427,10 @@ - +
-
-
+
+
plugin-<plugin-id>-node
@@ -508,13 +441,11 @@ - - - - - - - + + + + + Frontend Plugin Core @@ -522,16 +453,16 @@ - +
-
-
+
+
@backstage/core-plugin-api
- + @backstage/core-plugin-api @@ -539,16 +470,16 @@ - +
-
-
+
+
@backstage/test-utils
- + @backstage/test-utils @@ -556,24 +487,23 @@ - +
-
-
+
+
@backstage/dev-utils
- + @backstage/dev-utils - - - - + + + Frontend Libraries @@ -581,16 +511,16 @@ - +
-
-
+
+
@backstage/integration-react
- + @backstage/integration-react @@ -598,16 +528,16 @@ - +
-
-
+
+
@backstage/core-components
- + @backstage/core-components @@ -615,131 +545,276 @@ - +
-
-
+
+
@backstage/theme
- + @backstage/theme - - - - + + + + - -
-
-
+ +
+
+
Frontend Package
- + Frontend Package - -
-
-
+ +
+
+
Isomorphic Package
- + Isomorphic Package - -
-
-
+ +
+
+
Backend Package
- + Backend Package - + - -
-
-
+ +
+
+
CLI Package
- + CLI Package - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + - -
-
-
+ +
+
+
Compatibility
- + Compatibility - - - - + + + + + + + + Backend App Core + + + + + + +
+
+
+ @backstage/backend-app-api +
+
+
+ + + @backstage/backend-app-api + + + + + + + +
+
+
+ @backstage/backend-defaults +
+
+
+ + + @backstage/backend-defaults + + + + + + + + + + + + Backend Plugin Core + + + + + + +
+
+
+ @backstage/backend-plugin-api +
+
+
+ + + @backstage/backend-plugin-api + + + + + + + +
+
+
+ @backstage/backend-test-utils +
+
+
+ + + @backstage/backend-test-utils + + + + + + + +
+
+
+ @backstage/backend-dev-utils +
+
+
+ + + @backstage/backend-dev-utils + + + + + + + + Backend Libraries + + + + + + +
+
+
+ @backstage/backend-tasks +
+
+
+ + + @backstage/backend-tasks + + + + + + + +
+
+
+ @backstage/backend-openapi-utils +
+
+
+ + + @backstage/backend-openapi-utils + + + + + - Viewer does not support full SVG 1.1 + Text is not SVG - cannot display From ca7ba6694d65392e36a04fa4ffe61ae2dc6e29ec Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Thu, 2 May 2024 13:39:11 +0200 Subject: [PATCH 65/90] docs/versioning-policy: update to include new backend system Signed-off-by: Patrik Oldsberg --- docs/overview/versioning-policy.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/overview/versioning-policy.md b/docs/overview/versioning-policy.md index 0b1bc975bd..2ae73e1383 100644 --- a/docs/overview/versioning-policy.md +++ b/docs/overview/versioning-policy.md @@ -81,19 +81,21 @@ In order for Backstage to function properly the following versioning rules must be followed. The rules are referring to the [Package Architecture](https://backstage.io/docs/overview/architecture-overview#package-architecture). -- The versions of all the packages in the `Frontend App Core` must be from the - same release, and it is recommended to keep `Common Tooling` on that release - too. -- The Backstage dependencies of any given plugin should be from the same - release. This includes the packages from `Common Libraries`, - `Frontend Plugin Core`, and `Frontend Libraries`, or alternatively the - `Backend Libraries`. -- There must be no package that is from a newer release than the - `Frontend App Core` packages in the app. +- The versions of all packages for each of the "App Core" groups must be from the + same Backstage release. +- For each frontend and backend setup, the "App Core" packages must be ahead of or on the same Backstage release as the "Plugin Core" packages, including transitive dependencies of all installed plugins and modules. +- For any given plugin, the versions of all packages from the "Plugin Core" and + "Library" groups must be from the same Backstage release. - Frontend plugins with a corresponding backend plugin should be from the same release. The update to the backend plugin **MUST** be deployed before or together with the update to the frontend plugin. +It is allowed and often expected that the "Plugin Core" and "Library" packages +are from older releases than the "App Core" packages. It is also allowed to have +duplicate installations of the "Plugin Core" and "Library" packages. This is all +to make sure that upgrading Backstage is as smooth as possible and allows for +more flexibility across the entire plugin ecosystem. + ## Package Versioning Policy Every individual package is versioned according to [semver](https://semver.org). From 483a4f9425336091f53f79a6d6037a12dfba3f01 Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Thu, 2 May 2024 13:59:48 +0200 Subject: [PATCH 66/90] microsite/data: update plugin link Signed-off-by: Patrik Oldsberg --- microsite/data/plugins/betterscan.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/microsite/data/plugins/betterscan.yaml b/microsite/data/plugins/betterscan.yaml index 92d1eaa7b2..852ae0081c 100644 --- a/microsite/data/plugins/betterscan.yaml +++ b/microsite/data/plugins/betterscan.yaml @@ -4,7 +4,7 @@ author: Marcin Kozlowski authorUrl: https://betterscan.io category: Security description: View security scanned vulnerabilities in Code and Cloud scanned using Open Source and proprietary scanners directly in Backstage. -documentation: https://github.com/marcinguy/betterscan-ce +documentation: https://www.npmjs.com/package/@marcinguy/backstage-plugin-betterscan iconUrl: https://uploads-ssl.webflow.com/6339e3b81867539b5fe2498d/633a1643dcb06d3029867161_g4.svg npmPackageName: '@marcinguy/backstage-plugin-betterscan' addedDate: '2022-12-08' From 2a6f10d77a4d7b86f43a5d6915959093c6572970 Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Thu, 2 May 2024 14:31:05 +0200 Subject: [PATCH 67/90] cli: only warn when bump fails + fix forbidden duplicate filter Signed-off-by: Patrik Oldsberg --- .changeset/kind-toes-scream.md | 5 ++ .../cli/src/commands/versions/bump.test.ts | 65 +++++++------------ packages/cli/src/commands/versions/bump.ts | 26 ++++---- packages/cli/src/commands/versions/lint.ts | 5 +- 4 files changed, 41 insertions(+), 60 deletions(-) create mode 100644 .changeset/kind-toes-scream.md diff --git a/.changeset/kind-toes-scream.md b/.changeset/kind-toes-scream.md new file mode 100644 index 0000000000..d634f17ce4 --- /dev/null +++ b/.changeset/kind-toes-scream.md @@ -0,0 +1,5 @@ +--- +'@backstage/cli': patch +--- + +The `versions:bump` command will no longer exit with a non-zero status if the version bump fails due to forbidden duplicate package installations. It will now also provide more information about how to troubleshoot such an error. The set of forbidden duplicates has also been expanded to include all `@backstage/*-app-api` packages. diff --git a/packages/cli/src/commands/versions/bump.test.ts b/packages/cli/src/commands/versions/bump.test.ts index c8174639a6..0dfe2f61c3 100644 --- a/packages/cli/src/commands/versions/bump.test.ts +++ b/packages/cli/src/commands/versions/bump.test.ts @@ -894,30 +894,19 @@ describe('bump', () => { newVersions: [], newRanges: [ { - name: 'first-duplicate', - oldRange: 'first-duplicate', - newRange: 'first-duplicate', - oldVersion: '1.0.0', - newVersion: '2.0.0', - }, - { - name: 'second-duplicate', - oldRange: 'second-duplicate', - newRange: 'second-duplicate', - oldVersion: '1.0.0', - newVersion: '2.0.0', - }, - { - name: 'third-duplicate', - oldRange: 'third-duplicate', - newRange: 'third-duplicate', + name: '@backstage/backend-app-api', + oldRange: '^1.0.0', + newRange: '^2.0.0', oldVersion: '1.0.0', newVersion: '2.0.0', }, ], }); mockDir.setContent({ - 'yarn.lock': lockfileMock, + 'yarn.lock': `${HEADER} +"@backstage/backend-app-api@^1.0.0": + version "1.0.0" +`, 'package.json': JSON.stringify({ workspaces: { packages: ['packages/*'], @@ -928,16 +917,7 @@ describe('bump', () => { 'package.json': JSON.stringify({ name: 'a', dependencies: { - '@backstage/core': '^1.0.5', - }, - }), - }, - b: { - 'package.json': JSON.stringify({ - name: 'b', - dependencies: { - '@backstage/core': '^1.0.3', - '@backstage/theme': '^1.0.0', + '@backstage/backend-app-api': '^1.0.0', }, }), }, @@ -952,7 +932,12 @@ describe('bump', () => { res( ctx.status(200), ctx.json({ - packages: [], + packages: [ + { + name: '@backstage/backend-app-api', + version: '2.0.0', + }, + ], }), ), ), @@ -962,24 +947,20 @@ describe('bump', () => { }); expectLogsToMatch(logs, [ 'Using default pattern glob @backstage/*', - 'Checking for updates of @backstage/core', - 'Checking for updates of @backstage/theme', - 'Checking for updates of @backstage/core-api', + 'Checking for updates of @backstage/backend-app-api', + 'Checking for updates of @backstage/backend-app-api', 'Some packages are outdated, updating', - 'unlocking @backstage/core@^1.0.3 ~> 1.0.6', - 'unlocking @backstage/core-api@^1.0.6 ~> 1.0.7', - 'unlocking @backstage/core-api@^1.0.3 ~> 1.0.7', - 'bumping @backstage/core in a to ^1.0.6', - 'bumping @backstage/core in b to ^1.0.6', - 'bumping @backstage/theme in b to ^2.0.0', + 'bumping @backstage/backend-app-api in a to ^2.0.0', 'Running yarn install to install new versions', 'Checking for moved packages to the @backstage-community namespace...', '⚠️ The following packages may have breaking changes:', - ' @backstage/theme : 1.0.0 ~> 2.0.0', - ' https://github.com/backstage/backstage/blob/master/packages/theme/CHANGELOG.md', + ' @backstage/backend-app-api : 1.0.0 ~> 2.0.0', + ' https://github.com/backstage/backstage/blob/master/packages/backend-app-api/CHANGELOG.md', 'Version bump complete!', - 'The following packages have duplicates but have been allowed:', - 'first-duplicate, second-duplicate, third-duplicate', + ' ⚠️ Warning! ⚠️', + ' The below package(s) have incompatible duplicate installations, likely due to a bad dependency in a plugin.', + ' You can investigate this by running `yarn why `, and report the issue to the plugin maintainers.', + ' @backstage/backend-app-api', ]); }); }); diff --git a/packages/cli/src/commands/versions/bump.ts b/packages/cli/src/commands/versions/bump.ts index aad5a7831f..1544240704 100644 --- a/packages/cli/src/commands/versions/bump.ts +++ b/packages/cli/src/commands/versions/bump.ts @@ -331,24 +331,22 @@ export default async (opts: OptionValues) => { forbiddenDuplicatesFilter(name), ); if (forbiddenNewRanges.length > 0) { - throw new Error( - `Version bump failed for ${forbiddenNewRanges - .map(i => i.name) - .join(', ')}`, - ); - } - - const allowedDuplicates = result.newRanges.filter( - ({ name }) => !forbiddenDuplicatesFilter(name), - ); - - if (allowedDuplicates.length > 0) { + console.log(chalk.yellow(' ⚠️ Warning! ⚠️')); + console.log(); console.log( chalk.yellow( - 'The following packages have duplicates but have been allowed:', + ' The below package(s) have incompatible duplicate installations, likely due to a bad dependency in a plugin.', ), ); - console.log(chalk.yellow(allowedDuplicates.map(i => i.name).join(', '))); + console.log( + chalk.yellow( + ' You can investigate this by running `yarn why `, and report the issue to the plugin maintainers.', + ), + ); + console.log(); + for (const { name } of forbiddenNewRanges) { + console.log(chalk.yellow(` ${name}`)); + } } }; diff --git a/packages/cli/src/commands/versions/lint.ts b/packages/cli/src/commands/versions/lint.ts index 9466fd0ab7..bfb2a752a0 100644 --- a/packages/cli/src/commands/versions/lint.ts +++ b/packages/cli/src/commands/versions/lint.ts @@ -27,10 +27,7 @@ export const includedFilter = (name: string) => INCLUDED.some(pattern => pattern.test(name)); // Packages that are not allowed to have any duplicates -const FORBID_DUPLICATES = [ - /^@backstage\/core-app-api$/, - /^@backstage\/plugin-/, -]; +const FORBID_DUPLICATES = [/^@backstage\/\w+-app-api$/, /^@backstage\/plugin-/]; // There are some packages that ARE explicitly allowed to have duplicates since // they handle that appropriately. This takes precedence over FORBID_DUPLICATES From e538b100433070d3dd6f06c3a7a115047ac3c02b Mon Sep 17 00:00:00 2001 From: Heikki Hellgren Date: Tue, 30 Apr 2024 09:50:26 +0300 Subject: [PATCH 68/90] feat: support relative notification links sent via email processor Signed-off-by: Heikki Hellgren --- .changeset/giant-donkeys-talk.md | 5 + .../NotificationsEmailProcessor.test.ts | 225 ++++++++++++------ .../processor/NotificationsEmailProcessor.ts | 38 ++- 3 files changed, 188 insertions(+), 80 deletions(-) create mode 100644 .changeset/giant-donkeys-talk.md diff --git a/.changeset/giant-donkeys-talk.md b/.changeset/giant-donkeys-talk.md new file mode 100644 index 0000000000..62af2eb7e8 --- /dev/null +++ b/.changeset/giant-donkeys-talk.md @@ -0,0 +1,5 @@ +--- +'@backstage/plugin-notifications-backend-module-email': patch +--- + +Support relative links in notifications sent via email diff --git a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts index 11eef9b999..cdc7b9059a 100644 --- a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts +++ b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts @@ -30,6 +30,36 @@ jest.mock('nodemailer', () => ({ createTransport: jest.fn(), })); +const DEFAULT_ENTITIES_RESPONSE = { + items: [ + { + kind: 'User', + spec: { + profile: { + email: 'mock@backstage.io', + }, + }, + }, + ], +}; + +const DEFAULT_SENDMAIL_CONFIG = { + app: { + baseUrl: 'https://example.org', + }, + notifications: { + processors: { + email: { + transportConfig: { + transport: 'sendmail', + path: '/usr/local/bin/sendmail', + }, + sender: 'backstage@backstage.io', + }, + }, + }, +}; + describe('NotificationsEmailProcessor', () => { const logger = mockServices.logger.mock(); const auth = mockServices.auth(); @@ -49,6 +79,10 @@ describe('NotificationsEmailProcessor', () => { const processor = new NotificationsEmailProcessor( logger, new ConfigReader({ + app: { + baseUrl: 'http://localhost:3000', + externalBaseUrl: 'https://example.org', + }, notifications: { processors: { email: { @@ -95,6 +129,10 @@ describe('NotificationsEmailProcessor', () => { const processor = new NotificationsEmailProcessor( logger, new ConfigReader({ + app: { + baseUrl: 'http://localhost:3000', + externalBaseUrl: 'https://example.org', + }, notifications: { processors: { email: { @@ -135,6 +173,10 @@ describe('NotificationsEmailProcessor', () => { const processor = new NotificationsEmailProcessor( logger, new ConfigReader({ + app: { + baseUrl: 'http://localhost:3000', + externalBaseUrl: 'https://example.org', + }, notifications: { processors: { email: { @@ -175,29 +217,10 @@ describe('NotificationsEmailProcessor', () => { it('should send user email', async () => { (createTransport as jest.Mock).mockReturnValue(mockTransport); - getEntityRefMock.mockResolvedValue({ - kind: 'User', - spec: { - profile: { - email: 'mock@backstage.io', - }, - }, - }); + getEntityRefMock.mockResolvedValue(DEFAULT_ENTITIES_RESPONSE.items[0]); const processor = new NotificationsEmailProcessor( logger, - new ConfigReader({ - notifications: { - processors: { - email: { - transportConfig: { - transport: 'sendmail', - path: '/usr/local/bin/sendmail', - }, - sender: 'backstage@backstage.io', - }, - }, - }, - }), + mockServices.rootConfig({ data: DEFAULT_SENDMAIL_CONFIG }), mockCatalogClient as unknown as CatalogClient, auth, ); @@ -218,41 +241,29 @@ describe('NotificationsEmailProcessor', () => { expect(sendmailMock).toHaveBeenCalledWith({ from: 'backstage@backstage.io', - html: '

', + html: '

https://example.org/notifications

', replyTo: undefined, subject: 'notification', - text: '', + text: 'https://example.org/notifications', to: 'mock@backstage.io', }); }); it('should send email to all', async () => { (createTransport as jest.Mock).mockReturnValue(mockTransport); - getEntitiesMock.mockResolvedValue({ - items: [ - { - kind: 'User', - spec: { - profile: { - email: 'mock@backstage.io', - }, - }, - }, - ], - }); + getEntitiesMock.mockResolvedValue(DEFAULT_ENTITIES_RESPONSE); const processor = new NotificationsEmailProcessor( logger, - new ConfigReader({ - notifications: { - processors: { - email: { - transportConfig: { - transport: 'sendmail', - path: '/usr/local/bin/sendmail', - }, - sender: 'backstage@backstage.io', - broadcastConfig: { - receiver: 'users', + mockServices.rootConfig({ + data: { + ...DEFAULT_SENDMAIL_CONFIG, + notifications: { + processors: { + email: { + ...DEFAULT_SENDMAIL_CONFIG.notifications.processors.email, + broadcastConfig: { + receiver: 'users', + }, }, }, }, @@ -278,42 +289,30 @@ describe('NotificationsEmailProcessor', () => { expect(sendmailMock).toHaveBeenCalledWith({ from: 'backstage@backstage.io', - html: '

', + html: '

https://example.org/notifications

', replyTo: undefined, subject: 'notification', - text: '', + text: 'https://example.org/notifications', to: 'mock@backstage.io', }); }); it('should send email to configured addresses', async () => { (createTransport as jest.Mock).mockReturnValue(mockTransport); - getEntitiesMock.mockResolvedValue({ - items: [ - { - kind: 'User', - spec: { - profile: { - email: 'mock@backstage.io', - }, - }, - }, - ], - }); + getEntitiesMock.mockResolvedValue(DEFAULT_ENTITIES_RESPONSE); const processor = new NotificationsEmailProcessor( logger, - new ConfigReader({ - notifications: { - processors: { - email: { - transportConfig: { - transport: 'sendmail', - path: '/usr/local/bin/sendmail', - }, - sender: 'backstage@backstage.io', - broadcastConfig: { - receiver: 'config', - receiverEmails: ['broadcast@backstage.io'] as JsonArray, + mockServices.rootConfig({ + data: { + ...DEFAULT_SENDMAIL_CONFIG, + notifications: { + processors: { + email: { + ...DEFAULT_SENDMAIL_CONFIG.notifications.processors.email, + broadcastConfig: { + receiver: 'config', + receiverEmails: ['broadcast@backstage.io'] as JsonArray, + }, }, }, }, @@ -339,11 +338,89 @@ describe('NotificationsEmailProcessor', () => { expect(sendmailMock).toHaveBeenCalledWith({ from: 'backstage@backstage.io', - html: '

', + html: '

https://example.org/notifications

', replyTo: undefined, subject: 'notification', - text: '', + text: 'https://example.org/notifications', to: 'broadcast@backstage.io', }); }); + + it('should send email with relative link to given address', async () => { + (createTransport as jest.Mock).mockReturnValue(mockTransport); + getEntityRefMock.mockResolvedValue(DEFAULT_ENTITIES_RESPONSE.items[0]); + const processor = new NotificationsEmailProcessor( + logger, + mockServices.rootConfig({ + data: DEFAULT_SENDMAIL_CONFIG, + }), + mockCatalogClient as unknown as CatalogClient, + auth, + ); + + await processor.postProcess( + { + origin: 'plugin', + id: '1234', + user: 'user:default/mock', + created: new Date(), + payload: { + title: 'notification', + link: 'catalog/user/default/john.doe', + }, + }, + { + recipients: { type: 'entity', entityRef: 'user:default/mock' }, + payload: { title: 'notification' }, + }, + ); + + expect(sendmailMock).toHaveBeenCalledWith({ + from: 'backstage@backstage.io', + html: '

https://example.org/catalog/user/default/john.doe

', + replyTo: undefined, + subject: 'notification', + text: 'https://example.org/catalog/user/default/john.doe', + to: 'mock@backstage.io', + }); + }); + + it('should send email with absolute link to given address', async () => { + (createTransport as jest.Mock).mockReturnValue(mockTransport); + getEntityRefMock.mockResolvedValue(DEFAULT_ENTITIES_RESPONSE.items[0]); + const processor = new NotificationsEmailProcessor( + logger, + mockServices.rootConfig({ + data: DEFAULT_SENDMAIL_CONFIG, + }), + mockCatalogClient as unknown as CatalogClient, + auth, + ); + + await processor.postProcess( + { + origin: 'plugin', + id: '1234', + user: 'user:default/mock', + created: new Date(), + payload: { + title: 'notification', + link: 'https://backstage.io', + }, + }, + { + recipients: { type: 'entity', entityRef: 'user:default/mock' }, + payload: { title: 'notification' }, + }, + ); + + expect(sendmailMock).toHaveBeenCalledWith({ + from: 'backstage@backstage.io', + html: '

https://backstage.io/

', + replyTo: undefined, + subject: 'notification', + text: 'https://backstage.io/', + to: 'mock@backstage.io', + }); + }); }); diff --git a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts index ce944d9592..ecf4eb7773 100644 --- a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts +++ b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts @@ -50,6 +50,7 @@ export class NotificationsEmailProcessor implements NotificationProcessor { private readonly cacheTtl: number; private readonly concurrencyLimit: number; private readonly throttleInterval: number; + private readonly frontendBaseUrl: string; constructor( private readonly logger: LoggerService, @@ -78,6 +79,7 @@ export class NotificationsEmailProcessor implements NotificationProcessor { this.cacheTtl = cacheConfig ? durationToMilliseconds(readDurationFromConfig(cacheConfig)) : 3_600_000; + this.frontendBaseUrl = config.getString('app.baseUrl'); } private async getTransporter() { @@ -215,20 +217,44 @@ export class NotificationsEmailProcessor implements NotificationProcessor { ); } - private async sendPlainEmail(notification: Notification, emails: string[]) { + private getNotificationLink(notification: Notification) { + if (notification.payload.link) { + try { + const url = new URL(notification.payload.link, this.frontendBaseUrl); + return url.toString(); + } catch (_e) { + // noop: fallback to relative URL + } + return notification.payload.link; + } + return `${this.frontendBaseUrl}/notifications`; + } + + private getHtmlContent(notification: Notification) { const contentParts: string[] = []; if (notification.payload.description) { contentParts.push(`${notification.payload.description}`); } - if (notification.payload.link) { - contentParts.push(`${notification.payload.link}`); - } + const link = this.getNotificationLink(notification); + contentParts.push(`${link}`); + return `

${contentParts.join('
')}

`; + } + private getTextContent(notification: Notification) { + const contentParts: string[] = []; + if (notification.payload.description) { + contentParts.push(notification.payload.description); + } + contentParts.push(this.getNotificationLink(notification)); + return contentParts.join('\n\n'); + } + + private async sendPlainEmail(notification: Notification, emails: string[]) { const mailOptions = { from: this.sender, subject: notification.payload.title, - html: `

${contentParts.join('
')}

`, - text: contentParts.join('\n\n'), + html: this.getHtmlContent(notification), + text: this.getTextContent(notification), replyTo: this.replyTo, }; From 79bb100f0526888566b93b270ffb1b0ac0c9cced Mon Sep 17 00:00:00 2001 From: Heikki Hellgren Date: Tue, 30 Apr 2024 13:24:20 +0300 Subject: [PATCH 69/90] fix: ensure proper slashes in the email url Signed-off-by: Heikki Hellgren --- .../NotificationsEmailProcessor.test.ts | 26 +++++++++++++++++++ .../processor/NotificationsEmailProcessor.ts | 8 +++++- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts index cdc7b9059a..79d1110968 100644 --- a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts +++ b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.test.ts @@ -383,6 +383,32 @@ describe('NotificationsEmailProcessor', () => { text: 'https://example.org/catalog/user/default/john.doe', to: 'mock@backstage.io', }); + + await processor.postProcess( + { + origin: 'plugin', + id: '1234', + user: 'user:default/mock', + created: new Date(), + payload: { + title: 'notification', + link: '/catalog/user/default/jane.doe', + }, + }, + { + recipients: { type: 'entity', entityRef: 'user:default/mock' }, + payload: { title: 'notification' }, + }, + ); + + expect(sendmailMock).toHaveBeenCalledWith({ + from: 'backstage@backstage.io', + html: '

https://example.org/catalog/user/default/jane.doe

', + replyTo: undefined, + subject: 'notification', + text: 'https://example.org/catalog/user/default/jane.doe', + to: 'mock@backstage.io', + }); }); it('should send email with absolute link to given address', async () => { diff --git a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts index ecf4eb7773..c06d28f903 100644 --- a/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts +++ b/plugins/notifications-backend-module-email/src/processor/NotificationsEmailProcessor.ts @@ -219,8 +219,14 @@ export class NotificationsEmailProcessor implements NotificationProcessor { private getNotificationLink(notification: Notification) { if (notification.payload.link) { + const stripLeadingSlash = (s: string) => s.replace(/^\//, ''); + const ensureTrailingSlash = (s: string) => s.replace(/\/?$/, '/'); + try { - const url = new URL(notification.payload.link, this.frontendBaseUrl); + const url = new URL( + stripLeadingSlash(notification.payload.link), + ensureTrailingSlash(this.frontendBaseUrl), + ); return url.toString(); } catch (_e) { // noop: fallback to relative URL From 8e9727b825d401bbfbafd22785335af20a41bad7 Mon Sep 17 00:00:00 2001 From: Chap Ambrose Date: Thu, 2 May 2024 08:22:18 -0500 Subject: [PATCH 70/90] add ensureSchemaExists to config schema Signed-off-by: Chap Ambrose --- packages/backend-common/config.d.ts | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/packages/backend-common/config.d.ts b/packages/backend-common/config.d.ts index 700d9a6c6c..bdbec3a352 100644 --- a/packages/backend-common/config.d.ts +++ b/packages/backend-common/config.d.ts @@ -111,6 +111,13 @@ export interface Config { * Defaults to true if unspecified. */ ensureExists?: boolean; + /** + * Whether to ensure the given database schema exists by creating it if it does not. + * Defaults to false if unspecified. + * + * * NOTE: Currently only supported by the `pg` client when pluginDivisionMode: schema + */ + ensureSchemaExists?: boolean; /** * How plugins databases are managed/divided in the provided database instance. * @@ -147,6 +154,13 @@ export interface Config { * Defaults to base config if unspecified. */ ensureExists?: boolean; + /** + * Whether to ensure the given database schema exists by creating it if it does not. + * Defaults to false if unspecified. + * + * * NOTE: Currently only supported by the `pg` client when pluginDivisionMode: schema + */ + ensureSchemaExists?: boolean; /** * Arbitrary config object to pass to knex when initializing * (https://knexjs.org/#Installation-client). Most notable is the From 99fe60b13e017bdb73a073b554e1891a3e76d12c Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Thu, 2 May 2024 16:01:08 +0200 Subject: [PATCH 71/90] Apply suggestions from code review Signed-off-by: Patrik Oldsberg --- packages/backend-common/config.d.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/backend-common/config.d.ts b/packages/backend-common/config.d.ts index bdbec3a352..c6263cde97 100644 --- a/packages/backend-common/config.d.ts +++ b/packages/backend-common/config.d.ts @@ -115,7 +115,7 @@ export interface Config { * Whether to ensure the given database schema exists by creating it if it does not. * Defaults to false if unspecified. * - * * NOTE: Currently only supported by the `pg` client when pluginDivisionMode: schema + * NOTE: Currently only supported by the `pg` client when pluginDivisionMode: schema */ ensureSchemaExists?: boolean; /** @@ -158,7 +158,7 @@ export interface Config { * Whether to ensure the given database schema exists by creating it if it does not. * Defaults to false if unspecified. * - * * NOTE: Currently only supported by the `pg` client when pluginDivisionMode: schema + * NOTE: Currently only supported by the `pg` client when pluginDivisionMode: schema */ ensureSchemaExists?: boolean; /** From 8ade1283af574c503d1bf98db642eaa39f6142bb Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Thu, 2 May 2024 19:48:57 +0000 Subject: [PATCH 72/90] chore(deps): update actions/checkout action to v4.1.4 Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> --- .github/workflows/api-breaking-changes.yml | 2 +- .github/workflows/automate_changeset_feedback.yml | 2 +- .github/workflows/automate_merge_message.yml | 2 +- .github/workflows/ci.yml | 6 +++--- .github/workflows/deploy_docker-image.yml | 2 +- .github/workflows/deploy_microsite.yml | 2 +- .github/workflows/deploy_nightly.yml | 2 +- .github/workflows/deploy_packages.yml | 4 ++-- .github/workflows/scorecard.yml | 2 +- .github/workflows/sync_code-formatting.yml | 2 +- .github/workflows/sync_dependabot-changesets.yml | 2 +- .github/workflows/sync_release-manifest.yml | 4 ++-- .github/workflows/sync_renovate-changesets.yml | 2 +- .github/workflows/sync_snyk-github-issues.yml | 2 +- .github/workflows/sync_snyk-monitor.yml | 2 +- .github/workflows/sync_version-packages.yml | 2 +- .github/workflows/uffizzi-build.yml | 4 ++-- .github/workflows/verify_accessibility.yml | 2 +- .github/workflows/verify_codeql.yml | 2 +- .github/workflows/verify_docs-quality.yml | 2 +- .github/workflows/verify_e2e-kubernetes.yml | 2 +- .github/workflows/verify_e2e-linux.yml | 2 +- .github/workflows/verify_e2e-techdocs.yml | 2 +- .github/workflows/verify_e2e-windows.yml | 2 +- .github/workflows/verify_fossa.yml | 2 +- .github/workflows/verify_microsite.yml | 2 +- .github/workflows/verify_microsite_accessibility.yml | 2 +- .github/workflows/verify_storybook.yml | 2 +- .github/workflows/verify_windows.yml | 2 +- 29 files changed, 34 insertions(+), 34 deletions(-) diff --git a/.github/workflows/api-breaking-changes.yml b/.github/workflows/api-breaking-changes.yml index 07206718bb..901516305d 100644 --- a/.github/workflows/api-breaking-changes.yml +++ b/.github/workflows/api-breaking-changes.yml @@ -18,7 +18,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: # Fetch the commit that's merged into the base rather than the target ref # This will let us diff only the contents of the PR, without fetching more history diff --git a/.github/workflows/automate_changeset_feedback.yml b/.github/workflows/automate_changeset_feedback.yml index 4d965499d2..fefc73bd9f 100644 --- a/.github/workflows/automate_changeset_feedback.yml +++ b/.github/workflows/automate_changeset_feedback.yml @@ -27,7 +27,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: # Fetch the commit that's merged into the base rather than the target ref # This will let us diff only the contents of the PR, without fetching more history diff --git a/.github/workflows/automate_merge_message.yml b/.github/workflows/automate_merge_message.yml index 38243ee0ac..d6520d16a6 100644 --- a/.github/workflows/automate_merge_message.yml +++ b/.github/workflows/automate_merge_message.yml @@ -28,7 +28,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: ref: '${{ github.event.pull_request.merge_commit_sha }}' diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fdd31b2d57..c819ae8130 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -32,7 +32,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js ${{ matrix.node-version }} uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 @@ -68,7 +68,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js ${{ matrix.node-version }} uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 @@ -197,7 +197,7 @@ jobs: INTEGRATION_TEST_AZURE_TOKEN: ${{ secrets.INTEGRATION_TEST_AZURE_TOKEN }} steps: - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: fetch master branch run: git fetch origin master diff --git a/.github/workflows/deploy_docker-image.yml b/.github/workflows/deploy_docker-image.yml index 020254c872..8c263fdd4f 100644 --- a/.github/workflows/deploy_docker-image.yml +++ b/.github/workflows/deploy_docker-image.yml @@ -25,7 +25,7 @@ jobs: egress-policy: audit - name: checkout - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: path: backstage ref: ${{ github.event.client_payload.version && env.RELEASE_VERSION || github.ref }} diff --git a/.github/workflows/deploy_microsite.yml b/.github/workflows/deploy_microsite.yml index 6864ec67dc..b955b39b59 100644 --- a/.github/workflows/deploy_microsite.yml +++ b/.github/workflows/deploy_microsite.yml @@ -28,7 +28,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js 18.x uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 diff --git a/.github/workflows/deploy_nightly.yml b/.github/workflows/deploy_nightly.yml index 07c3db0f11..a3a1b5e283 100644 --- a/.github/workflows/deploy_nightly.yml +++ b/.github/workflows/deploy_nightly.yml @@ -19,7 +19,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js 18.x uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 diff --git a/.github/workflows/deploy_packages.yml b/.github/workflows/deploy_packages.yml index f9ef4c67eb..0c44d32384 100644 --- a/.github/workflows/deploy_packages.yml +++ b/.github/workflows/deploy_packages.yml @@ -65,7 +65,7 @@ jobs: INTEGRATION_TEST_AZURE_TOKEN: ${{ secrets.INTEGRATION_TEST_AZURE_TOKEN }} steps: - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js ${{ matrix.node-version }} uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 @@ -148,7 +148,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js ${{ matrix.node-version }} uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 67fdff4020..5acd11c9b0 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -34,7 +34,7 @@ jobs: egress-policy: audit - name: 'Checkout code' - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: persist-credentials: false diff --git a/.github/workflows/sync_code-formatting.yml b/.github/workflows/sync_code-formatting.yml index 5dbd0361c1..a59eea9c9b 100644 --- a/.github/workflows/sync_code-formatting.yml +++ b/.github/workflows/sync_code-formatting.yml @@ -14,7 +14,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: # Fetch changes to previous commit - required for 'only_changed' in Prettier action fetch-depth: 0 diff --git a/.github/workflows/sync_dependabot-changesets.yml b/.github/workflows/sync_dependabot-changesets.yml index 8b3be335ed..88997ea5dd 100644 --- a/.github/workflows/sync_dependabot-changesets.yml +++ b/.github/workflows/sync_dependabot-changesets.yml @@ -16,7 +16,7 @@ jobs: egress-policy: audit - name: Checkout - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: fetch-depth: 2 ref: ${{ github.head_ref }} diff --git a/.github/workflows/sync_release-manifest.yml b/.github/workflows/sync_release-manifest.yml index b0a2499d43..1549ec0e90 100644 --- a/.github/workflows/sync_release-manifest.yml +++ b/.github/workflows/sync_release-manifest.yml @@ -21,7 +21,7 @@ jobs: run: npm install semver@7.3.5 fs-extra@10.0.0 @manypkg/get-packages@1.1.1 - name: Checkout - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: path: backstage # 'v' prefix is added here for the tag, we keep it out of the manifest logic @@ -29,7 +29,7 @@ jobs: # Checkout backstage/versions into /backstage/versions, which is where store the output - name: Checkout versions - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: repository: backstage/versions path: backstage/versions diff --git a/.github/workflows/sync_renovate-changesets.yml b/.github/workflows/sync_renovate-changesets.yml index d02e35483c..e95a68ac2f 100644 --- a/.github/workflows/sync_renovate-changesets.yml +++ b/.github/workflows/sync_renovate-changesets.yml @@ -16,7 +16,7 @@ jobs: egress-policy: audit - name: Checkout - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: fetch-depth: 2 ref: ${{ github.head_ref }} diff --git a/.github/workflows/sync_snyk-github-issues.yml b/.github/workflows/sync_snyk-github-issues.yml index d5a4048044..bebbbc4fbe 100644 --- a/.github/workflows/sync_snyk-github-issues.yml +++ b/.github/workflows/sync_snyk-github-issues.yml @@ -16,7 +16,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js 18.x uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 diff --git a/.github/workflows/sync_snyk-monitor.yml b/.github/workflows/sync_snyk-monitor.yml index e2507996b1..cb3efcfac3 100644 --- a/.github/workflows/sync_snyk-monitor.yml +++ b/.github/workflows/sync_snyk-monitor.yml @@ -29,7 +29,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: Monitor and Synchronize Snyk Policies uses: snyk/actions/node@8349f9043a8b7f0f3ee8885bf28f0b388d2446e8 # master with: diff --git a/.github/workflows/sync_version-packages.yml b/.github/workflows/sync_version-packages.yml index b2ad80b6da..82a0eae7be 100644 --- a/.github/workflows/sync_version-packages.yml +++ b/.github/workflows/sync_version-packages.yml @@ -18,7 +18,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: fetch-depth: 20000 fetch-tags: true diff --git a/.github/workflows/uffizzi-build.yml b/.github/workflows/uffizzi-build.yml index 4a52af21bc..11a0054703 100644 --- a/.github/workflows/uffizzi-build.yml +++ b/.github/workflows/uffizzi-build.yml @@ -31,7 +31,7 @@ jobs: egress-policy: audit - name: checkout - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: setup-node uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 @@ -89,7 +89,7 @@ jobs: egress-policy: audit - name: Checkout git repo - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: Render Compose File run: | # update image after the build above diff --git a/.github/workflows/verify_accessibility.yml b/.github/workflows/verify_accessibility.yml index 1c25d709cb..b4615fee0a 100644 --- a/.github/workflows/verify_accessibility.yml +++ b/.github/workflows/verify_accessibility.yml @@ -24,7 +24,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: Use Node.js 18.x uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 with: diff --git a/.github/workflows/verify_codeql.yml b/.github/workflows/verify_codeql.yml index 920c9bab17..60cb89a045 100644 --- a/.github/workflows/verify_codeql.yml +++ b/.github/workflows/verify_codeql.yml @@ -47,7 +47,7 @@ jobs: egress-policy: audit - name: Checkout repository - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: # We must fetch at least the immediate parents so that if this is # a pull request then we can checkout the head. diff --git a/.github/workflows/verify_docs-quality.yml b/.github/workflows/verify_docs-quality.yml index 3550df6fc0..2eb7608ccd 100644 --- a/.github/workflows/verify_docs-quality.yml +++ b/.github/workflows/verify_docs-quality.yml @@ -16,7 +16,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 # Vale does not support file excludes, so we use the script to generate a list of files instead # The action also does not allow args or a local config file to be passed in, so the files array diff --git a/.github/workflows/verify_e2e-kubernetes.yml b/.github/workflows/verify_e2e-kubernetes.yml index 6ff41c8b74..3c425c25e9 100644 --- a/.github/workflows/verify_e2e-kubernetes.yml +++ b/.github/workflows/verify_e2e-kubernetes.yml @@ -26,7 +26,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js ${{ matrix.node-version }} uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 diff --git a/.github/workflows/verify_e2e-linux.yml b/.github/workflows/verify_e2e-linux.yml index a750dfea3b..75c7a800d8 100644 --- a/.github/workflows/verify_e2e-linux.yml +++ b/.github/workflows/verify_e2e-linux.yml @@ -45,7 +45,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: Configure Git run: | diff --git a/.github/workflows/verify_e2e-techdocs.yml b/.github/workflows/verify_e2e-techdocs.yml index 3bcda5f2b6..d5f2594285 100644 --- a/.github/workflows/verify_e2e-techdocs.yml +++ b/.github/workflows/verify_e2e-techdocs.yml @@ -34,7 +34,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0 with: python-version: '3.9' diff --git a/.github/workflows/verify_e2e-windows.yml b/.github/workflows/verify_e2e-windows.yml index 078ddd6855..a6fa469666 100644 --- a/.github/workflows/verify_e2e-windows.yml +++ b/.github/workflows/verify_e2e-windows.yml @@ -42,7 +42,7 @@ jobs: git config --global core.autocrlf false git config --global core.eol lf - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: Configure Git run: | diff --git a/.github/workflows/verify_fossa.yml b/.github/workflows/verify_fossa.yml index 801c5fbd28..ccc9e4509a 100644 --- a/.github/workflows/verify_fossa.yml +++ b/.github/workflows/verify_fossa.yml @@ -19,7 +19,7 @@ jobs: egress-policy: audit - name: Checkout - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: Install Fossa run: "curl -H 'Cache-Control: no-cache' https://raw.githubusercontent.com/fossas/fossa-cli/master/install.sh | bash" diff --git a/.github/workflows/verify_microsite.yml b/.github/workflows/verify_microsite.yml index 68378aa086..db6a3e2d40 100644 --- a/.github/workflows/verify_microsite.yml +++ b/.github/workflows/verify_microsite.yml @@ -28,7 +28,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js 18.x uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 diff --git a/.github/workflows/verify_microsite_accessibility.yml b/.github/workflows/verify_microsite_accessibility.yml index e0fea1a72c..19417c5cee 100644 --- a/.github/workflows/verify_microsite_accessibility.yml +++ b/.github/workflows/verify_microsite_accessibility.yml @@ -19,7 +19,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: Use Node.js 18.x uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 diff --git a/.github/workflows/verify_storybook.yml b/.github/workflows/verify_storybook.yml index 940c136797..6a37f5abb6 100644 --- a/.github/workflows/verify_storybook.yml +++ b/.github/workflows/verify_storybook.yml @@ -32,7 +32,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 with: fetch-depth: 0 # Required to retrieve git history diff --git a/.github/workflows/verify_windows.yml b/.github/workflows/verify_windows.yml index 32b0452eec..c7ff8f5ca9 100644 --- a/.github/workflows/verify_windows.yml +++ b/.github/workflows/verify_windows.yml @@ -33,7 +33,7 @@ jobs: with: egress-policy: audit - - uses: actions/checkout@1d96c772d19495a3b5c517cd2bc0cb401ea0529f # v4.1.3 + - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4 - name: use node.js ${{ matrix.node-version }} uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2 From 79a6358f50854b626d06598cbae0384e44c0b68f Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Fri, 3 May 2024 10:40:49 +0200 Subject: [PATCH 73/90] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Fredrik Adelöw Signed-off-by: Patrik Oldsberg --- .../building-apps/02-configuring-extensions.md | 2 +- docs/frontend-system/building-apps/08-migrating.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/frontend-system/building-apps/02-configuring-extensions.md b/docs/frontend-system/building-apps/02-configuring-extensions.md index 5f090e23cd..4052ef47f0 100644 --- a/docs/frontend-system/building-apps/02-configuring-extensions.md +++ b/docs/frontend-system/building-apps/02-configuring-extensions.md @@ -6,7 +6,7 @@ sidebar_label: Configuring Extensions description: Documentation for how to configure extensions in a Backstage app --- -All extensions in a Backstage app can be configured through static configuration. This configuration is all done under a the `app.extensions` configuration key. For more general information on how to write configuration for Backstage, see the section on [writing configuration](../../conf/writing.md). +All extensions in a Backstage app can be configured through static configuration. This configuration is all done under the `app.extensions` configuration key. For more general information on how to write configuration for Backstage, see the section on [writing configuration](../../conf/writing.md). ## Extension Configuration Schema diff --git a/docs/frontend-system/building-apps/08-migrating.md b/docs/frontend-system/building-apps/08-migrating.md index 8bcc1f973f..dcc58325cf 100644 --- a/docs/frontend-system/building-apps/08-migrating.md +++ b/docs/frontend-system/building-apps/08-migrating.md @@ -117,7 +117,7 @@ You can then also add any additional extensions that you may need to create as p [Utility API](../utility-apis/01-index.md) factories are now installed as extensions instead. Pass the existing factory to `createApiExtension` and install it in the app. For more information, see the section on [configuring Utility APIs](../utility-apis/04-configuring.md). -For example, the following apis configuration: +For example, the following `apis` configuration: ```ts const app = createApp({ @@ -151,7 +151,7 @@ Icons are currently installed through the usual options to `createApp`, but will Plugins are now passed through the `features` options instead. -For example, the following plugins configuration: +For example, the following `plugins` configuration: ```tsx import { homePlugin } from '@backstage/plugin-home'; @@ -163,7 +163,7 @@ createApp({ }); ``` -Can be converted to the following features configuration: +Can be converted to the following `features` configuration: ```tsx // plugins are now default exported via alpha subpath From ae500129a9146c600453de465103f917b5fbbcb4 Mon Sep 17 00:00:00 2001 From: Nitin Ramnani Date: Fri, 3 May 2024 14:49:01 +0530 Subject: [PATCH 74/90] Added stories for multiple components Signed-off-by: Nitin Ramnani --- .../HeaderIconLinkRow.stories.tsx | 51 ++++++++++++++++ .../ResponseErrorPanel.stories.tsx | 38 ++++++++++++ .../layout/BottomLink/BottomLink.stories.tsx | 33 +++++++++++ .../ContentHeader/ContentHeader.stories.tsx | 58 +++++++++++++++++++ 4 files changed, 180 insertions(+) create mode 100644 packages/core-components/src/components/HeaderIconLinkRow/HeaderIconLinkRow.stories.tsx create mode 100644 packages/core-components/src/components/ResponseErrorPanel/ResponseErrorPanel.stories.tsx create mode 100644 packages/core-components/src/layout/BottomLink/BottomLink.stories.tsx create mode 100644 packages/core-components/src/layout/ContentHeader/ContentHeader.stories.tsx diff --git a/packages/core-components/src/components/HeaderIconLinkRow/HeaderIconLinkRow.stories.tsx b/packages/core-components/src/components/HeaderIconLinkRow/HeaderIconLinkRow.stories.tsx new file mode 100644 index 0000000000..e6891081e3 --- /dev/null +++ b/packages/core-components/src/components/HeaderIconLinkRow/HeaderIconLinkRow.stories.tsx @@ -0,0 +1,51 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import React from 'react'; +import { HeaderIconLinkRow } from '../HeaderIconLinkRow'; +import { IconLinkVerticalProps } from './IconLinkVertical'; + + +type Props = { + links: IconLinkVerticalProps[]; + }; + +export default { + title: 'Data Display/HeaderIconLinkRow', + component: HeaderIconLinkRow, +}; + + + +export const Default = (args:Props) => +Default.args = { + links: [ + { + color: 'primary', + disabled: false, + href: "https://google.com", + label: "primary", + title: "title" + }, + { + color: 'secondary', + disabled: false, + href: "https://google.com", + label: "secondary", + title: "title-2" + }, + ] +}; \ No newline at end of file diff --git a/packages/core-components/src/components/ResponseErrorPanel/ResponseErrorPanel.stories.tsx b/packages/core-components/src/components/ResponseErrorPanel/ResponseErrorPanel.stories.tsx new file mode 100644 index 0000000000..62eb523a02 --- /dev/null +++ b/packages/core-components/src/components/ResponseErrorPanel/ResponseErrorPanel.stories.tsx @@ -0,0 +1,38 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import React from 'react'; +import { ResponseErrorPanel } from '../ResponseErrorPanel'; +import { ErrorPanelProps } from '../ErrorPanel'; + +export default { + title: 'Data Display/ResponseErrorPanel', + component: ResponseErrorPanel, +}; + +export const Default = (args:ErrorPanelProps) => +Default.args = { + error: new Error('Error message from error object'), + defaultExpanded: false +}; + + +export const WithTitle = (args:ErrorPanelProps) => +WithTitle.args = { + error: new Error('test'), + defaultExpanded: false, + title:"Title prop is passed" +}; diff --git a/packages/core-components/src/layout/BottomLink/BottomLink.stories.tsx b/packages/core-components/src/layout/BottomLink/BottomLink.stories.tsx new file mode 100644 index 0000000000..84f268681d --- /dev/null +++ b/packages/core-components/src/layout/BottomLink/BottomLink.stories.tsx @@ -0,0 +1,33 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + + +import React from 'react'; +import { BottomLink } from '../BottomLink'; + +export default { + title: 'Layout/BottomLink', + component: BottomLink, +}; + +export const Default = (args:{ + link:string + title:string +}) => +Default.args = { + link: 'https://google.com', + title: 'This is bottom link' +}; diff --git a/packages/core-components/src/layout/ContentHeader/ContentHeader.stories.tsx b/packages/core-components/src/layout/ContentHeader/ContentHeader.stories.tsx new file mode 100644 index 0000000000..ce6365c28e --- /dev/null +++ b/packages/core-components/src/layout/ContentHeader/ContentHeader.stories.tsx @@ -0,0 +1,58 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import React , {ReactNode} from 'react'; +import { ContentHeader } from '../ContentHeader'; + +export default { + title: 'Layout/ContentHeader', + component: ContentHeader, +}; + +type ContentHeaderProps = { + title?: string + titleComponent?: ReactNode; + description?: string; + textAlign?: 'left' | 'right' | 'center'; + }; + +export const Default = (args:ContentHeaderProps) =>
Child of Content Header
 +Default.args = { + title: 'This is Content Header default aligned', + description:'This is description' +}; + + +export const Left = (args:ContentHeaderProps) =>
Child of Content Header
 +Left.args = { + title: 'This is Content Header left aligned', + description:'This is description', + textAlign: 'left' +}; + +export const Right = (args:ContentHeaderProps) =>
Child of Content Header
 +Right.args = { + title: 'This is Content Header right aligned', + description:'This is description', + textAlign: 'right' +}; + +export const Center = (args:ContentHeaderProps) =>
Child of Content Header
 +Center.args = { + title: 'This is Content Header center aligned', + description:'This is description', + textAlign: 'center' +}; From baf298f57ec2a6fe1b43dd584cf78f2e0c959467 Mon Sep 17 00:00:00 2001 From: Nitin Ramnani Date: Fri, 3 May 2024 14:52:20 +0530 Subject: [PATCH 75/90] Added stories for multiple components Signed-off-by: Nitin Ramnani --- .../HeaderIconLinkRow.stories.tsx | 49 +++++++-------- .../ResponseErrorPanel.stories.tsx | 23 ++++--- .../layout/BottomLink/BottomLink.stories.tsx | 16 +++-- .../ContentHeader/ContentHeader.stories.tsx | 63 ++++++++++++------- 4 files changed, 82 insertions(+), 69 deletions(-) diff --git a/packages/core-components/src/components/HeaderIconLinkRow/HeaderIconLinkRow.stories.tsx b/packages/core-components/src/components/HeaderIconLinkRow/HeaderIconLinkRow.stories.tsx index e6891081e3..e472ab5599 100644 --- a/packages/core-components/src/components/HeaderIconLinkRow/HeaderIconLinkRow.stories.tsx +++ b/packages/core-components/src/components/HeaderIconLinkRow/HeaderIconLinkRow.stories.tsx @@ -18,34 +18,31 @@ import React from 'react'; import { HeaderIconLinkRow } from '../HeaderIconLinkRow'; import { IconLinkVerticalProps } from './IconLinkVertical'; - type Props = { - links: IconLinkVerticalProps[]; - }; - -export default { - title: 'Data Display/HeaderIconLinkRow', - component: HeaderIconLinkRow, + links: IconLinkVerticalProps[]; }; +export default { + title: 'Data Display/HeaderIconLinkRow', + component: HeaderIconLinkRow, +}; - -export const Default = (args:Props) => +export const Default = (args: Props) => ; Default.args = { - links: [ - { - color: 'primary', - disabled: false, - href: "https://google.com", - label: "primary", - title: "title" - }, - { - color: 'secondary', - disabled: false, - href: "https://google.com", - label: "secondary", - title: "title-2" - }, - ] -}; \ No newline at end of file + links: [ + { + color: 'primary', + disabled: false, + href: 'https://google.com', + label: 'primary', + title: 'title', + }, + { + color: 'secondary', + disabled: false, + href: 'https://google.com', + label: 'secondary', + title: 'title-2', + }, + ], +}; diff --git a/packages/core-components/src/components/ResponseErrorPanel/ResponseErrorPanel.stories.tsx b/packages/core-components/src/components/ResponseErrorPanel/ResponseErrorPanel.stories.tsx index 62eb523a02..8d03e7359f 100644 --- a/packages/core-components/src/components/ResponseErrorPanel/ResponseErrorPanel.stories.tsx +++ b/packages/core-components/src/components/ResponseErrorPanel/ResponseErrorPanel.stories.tsx @@ -19,20 +19,23 @@ import { ResponseErrorPanel } from '../ResponseErrorPanel'; import { ErrorPanelProps } from '../ErrorPanel'; export default { - title: 'Data Display/ResponseErrorPanel', - component: ResponseErrorPanel, + title: 'Data Display/ResponseErrorPanel', + component: ResponseErrorPanel, }; -export const Default = (args:ErrorPanelProps) => +export const Default = (args: ErrorPanelProps) => ( + +); Default.args = { - error: new Error('Error message from error object'), - defaultExpanded: false + error: new Error('Error message from error object'), + defaultExpanded: false, }; - -export const WithTitle = (args:ErrorPanelProps) => +export const WithTitle = (args: ErrorPanelProps) => ( + +); WithTitle.args = { - error: new Error('test'), - defaultExpanded: false, - title:"Title prop is passed" + error: new Error('test'), + defaultExpanded: false, + title: 'Title prop is passed', }; diff --git a/packages/core-components/src/layout/BottomLink/BottomLink.stories.tsx b/packages/core-components/src/layout/BottomLink/BottomLink.stories.tsx index 84f268681d..e05af86993 100644 --- a/packages/core-components/src/layout/BottomLink/BottomLink.stories.tsx +++ b/packages/core-components/src/layout/BottomLink/BottomLink.stories.tsx @@ -14,20 +14,18 @@ * limitations under the License. */ - import React from 'react'; import { BottomLink } from '../BottomLink'; export default { - title: 'Layout/BottomLink', - component: BottomLink, + title: 'Layout/BottomLink', + component: BottomLink, }; -export const Default = (args:{ - link:string - title:string -}) => +export const Default = (args: { link: string; title: string }) => ( + +); Default.args = { - link: 'https://google.com', - title: 'This is bottom link' + link: 'https://google.com', + title: 'This is bottom link', }; diff --git a/packages/core-components/src/layout/ContentHeader/ContentHeader.stories.tsx b/packages/core-components/src/layout/ContentHeader/ContentHeader.stories.tsx index ce6365c28e..27df00e72c 100644 --- a/packages/core-components/src/layout/ContentHeader/ContentHeader.stories.tsx +++ b/packages/core-components/src/layout/ContentHeader/ContentHeader.stories.tsx @@ -14,45 +14,60 @@ * limitations under the License. */ -import React , {ReactNode} from 'react'; +import React, { ReactNode } from 'react'; import { ContentHeader } from '../ContentHeader'; export default { - title: 'Layout/ContentHeader', - component: ContentHeader, + title: 'Layout/ContentHeader', + component: ContentHeader, }; type ContentHeaderProps = { - title?: string - titleComponent?: ReactNode; - description?: string; - textAlign?: 'left' | 'right' | 'center'; - }; + title?: string; + titleComponent?: ReactNode; + description?: string; + textAlign?: 'left' | 'right' | 'center'; +}; -export const Default = (args:ContentHeaderProps) =>
Child of Content Header
 +export const Default = (args: ContentHeaderProps) => ( + +
Child of Content Header
+ +); Default.args = { - title: 'This is Content Header default aligned', - description:'This is description' + title: 'This is Content Header default aligned', + description: 'This is description', }; - -export const Left = (args:ContentHeaderProps) =>
Child of Content Header
 +export const Left = (args: ContentHeaderProps) => ( + +
Child of Content Header
+ +); Left.args = { - title: 'This is Content Header left aligned', - description:'This is description', - textAlign: 'left' + title: 'This is Content Header left aligned', + description: 'This is description', + textAlign: 'left', }; -export const Right = (args:ContentHeaderProps) =>
Child of Content Header
 +export const Right = (args: ContentHeaderProps) => ( + +
Child of Content Header
+ +); Right.args = { - title: 'This is Content Header right aligned', - description:'This is description', - textAlign: 'right' + title: 'This is Content Header right aligned', + description: 'This is description', + textAlign: 'right', }; -export const Center = (args:ContentHeaderProps) =>
Child of Content Header
 +export const Center = (args: ContentHeaderProps) => ( + +
Child of Content Header
+ +); Center.args = { - title: 'This is Content Header center aligned', - description:'This is description', - textAlign: 'center' + title: 'This is Content Header center aligned', + description: 'This is description', + textAlign: 'center', }; From 42eaf63a7525f7bce09838f78d29abfc196680ca Mon Sep 17 00:00:00 2001 From: Heikki Hellgren Date: Fri, 3 May 2024 12:27:52 +0300 Subject: [PATCH 76/90] feat: increase default and allow changing snackbar auto hide duration Signed-off-by: Heikki Hellgren --- .changeset/perfect-beers-explode.md | 5 +++++ plugins/notifications/api-report.md | 1 + .../NotificationsSideBarItem/NotificationsSideBarItem.tsx | 5 +++++ 3 files changed, 11 insertions(+) create mode 100644 .changeset/perfect-beers-explode.md diff --git a/.changeset/perfect-beers-explode.md b/.changeset/perfect-beers-explode.md new file mode 100644 index 0000000000..3045ca7f12 --- /dev/null +++ b/.changeset/perfect-beers-explode.md @@ -0,0 +1,5 @@ +--- +'@backstage/plugin-notifications': patch +--- + +Increase default and allow modifying notification snackbar auto hide duration diff --git a/plugins/notifications/api-report.md b/plugins/notifications/api-report.md index 3699aa1bb0..9240bdb1c2 100644 --- a/plugins/notifications/api-report.md +++ b/plugins/notifications/api-report.md @@ -102,6 +102,7 @@ export const NotificationsSidebarItem: (props?: { webNotificationsEnabled?: boolean; titleCounterEnabled?: boolean; snackbarEnabled?: boolean; + snackbarAutoHideDuration?: number | null; className?: string; icon?: IconComponent; text?: string; diff --git a/plugins/notifications/src/components/NotificationsSideBarItem/NotificationsSideBarItem.tsx b/plugins/notifications/src/components/NotificationsSideBarItem/NotificationsSideBarItem.tsx index 000eabcae6..7b55c63747 100644 --- a/plugins/notifications/src/components/NotificationsSideBarItem/NotificationsSideBarItem.tsx +++ b/plugins/notifications/src/components/NotificationsSideBarItem/NotificationsSideBarItem.tsx @@ -83,6 +83,7 @@ export const NotificationsSidebarItem = (props?: { webNotificationsEnabled?: boolean; titleCounterEnabled?: boolean; snackbarEnabled?: boolean; + snackbarAutoHideDuration?: number | null; className?: string; icon?: IconComponent; text?: string; @@ -93,6 +94,7 @@ export const NotificationsSidebarItem = (props?: { webNotificationsEnabled = false, titleCounterEnabled = true, snackbarEnabled = true, + snackbarAutoHideDuration = 10000, icon = NotificationsIcon, text = 'Notifications', ...restProps @@ -100,6 +102,7 @@ export const NotificationsSidebarItem = (props?: { webNotificationsEnabled: false, titleCounterEnabled: true, snackbarEnabled: true, + snackbarAutoHideDuration: 10000, }; const { loading, error, value, retry } = useNotificationsApi(api => @@ -196,6 +199,7 @@ export const NotificationsSidebarItem = (props?: { variant: notification.payload.severity, anchorOrigin: { vertical: 'bottom', horizontal: 'right' }, action, + autoHideDuration: snackbarAutoHideDuration, } as OptionsWithExtraProps); } }) @@ -216,6 +220,7 @@ export const NotificationsSidebarItem = (props?: { sendWebNotification, webNotificationsEnabled, snackbarEnabled, + snackbarAutoHideDuration, notificationsApi, alertApi, getSnackbarProperties, From 598e8e51dd3430702f05c0f89c33215a0c49ebf3 Mon Sep 17 00:00:00 2001 From: Patrik Oldsberg Date: Fri, 3 May 2024 14:09:58 +0200 Subject: [PATCH 77/90] root: patch changesets to handle workspace ranges differently Signed-off-by: Patrik Oldsberg --- ...le-release-plan-npm-6.0.0-f7b3005037.patch | 32 +++++++++++++++++++ package.json | 1 + yarn.lock | 16 +++++++++- 3 files changed, 48 insertions(+), 1 deletion(-) create mode 100644 .yarn/patches/@changesets-assemble-release-plan-npm-6.0.0-f7b3005037.patch diff --git a/.yarn/patches/@changesets-assemble-release-plan-npm-6.0.0-f7b3005037.patch b/.yarn/patches/@changesets-assemble-release-plan-npm-6.0.0-f7b3005037.patch new file mode 100644 index 0000000000..045e983dbe --- /dev/null +++ b/.yarn/patches/@changesets-assemble-release-plan-npm-6.0.0-f7b3005037.patch @@ -0,0 +1,32 @@ +diff --git a/dist/changesets-assemble-release-plan.cjs.js b/dist/changesets-assemble-release-plan.cjs.js +index ee5c0f67fabadeb112e9f238d8b144a4d125830f..9b0e1a156dd88cee35f82faf718d82a8a8f80325 100644 +--- a/dist/changesets-assemble-release-plan.cjs.js ++++ b/dist/changesets-assemble-release-plan.cjs.js +@@ -179,12 +179,23 @@ function getDependencyVersionRanges(dependentPkgJSON, dependencyRelease) { + if (!versionRange) continue; + + if (versionRange.startsWith("workspace:")) { ++ // intentionally keep other workspace ranges untouched ++ // this has to be fixed but this should only be done when adding appropriate tests ++ let workspaceRange = versionRange.replace(/^workspace:/, ""); ++ switch (workspaceRange) { ++ case "*": ++ // workspace:* actually means the current exact version, and not a wildcard similar to a reguler * range ++ workspaceRange = dependencyRelease.oldVersion; ++ break; ++ case "~": ++ case "^": ++ // Use ^oldVersion for workspace:^ or ~oldVersion for workspace:~. ++ // The version range might have changed in dependent package, but that should have its own changeset bumping that package. ++ workspaceRange += dependencyRelease.oldVersion; ++ } + dependencyVersionRanges.push({ + depType: type, +- versionRange: // intentionally keep other workspace ranges untouched +- // this has to be fixed but this should only be done when adding appropriate tests +- versionRange === "workspace:*" ? // workspace:* actually means the current exact version, and not a wildcard similar to a reguler * range +- dependencyRelease.oldVersion : versionRange.replace(/^workspace:/, "") ++ versionRange: workspaceRange, + }); + } else { + dependencyVersionRanges.push({ diff --git a/package.json b/package.json index 5d9fbef716..768448402f 100644 --- a/package.json +++ b/package.json @@ -84,6 +84,7 @@ }, "prettier": "@spotify/prettier-config", "resolutions": { + "@changesets/assemble-release-plan@^6.0.0": "patch:@changesets/assemble-release-plan@npm%3A6.0.0#./.yarn/patches/@changesets-assemble-release-plan-npm-6.0.0-f7b3005037.patch", "@material-ui/pickers@^3.2.10": "patch:@material-ui/pickers@npm%3A3.3.11#./.yarn/patches/@material-ui-pickers-npm-3.3.11-1c8f68ea20.patch", "@material-ui/pickers@^3.3.10": "patch:@material-ui/pickers@npm%3A3.3.11#./.yarn/patches/@material-ui-pickers-npm-3.3.11-1c8f68ea20.patch", "@types/react": "^18", diff --git a/yarn.lock b/yarn.lock index c07c71c8b3..dbc233dd64 100644 --- a/yarn.lock +++ b/yarn.lock @@ -7817,7 +7817,7 @@ __metadata: languageName: node linkType: hard -"@changesets/assemble-release-plan@npm:^6.0.0": +"@changesets/assemble-release-plan@npm:6.0.0": version: 6.0.0 resolution: "@changesets/assemble-release-plan@npm:6.0.0" dependencies: @@ -7831,6 +7831,20 @@ __metadata: languageName: node linkType: hard +"@changesets/assemble-release-plan@patch:@changesets/assemble-release-plan@npm%3A6.0.0#./.yarn/patches/@changesets-assemble-release-plan-npm-6.0.0-f7b3005037.patch::locator=root%40workspace%3A.": + version: 6.0.0 + resolution: "@changesets/assemble-release-plan@patch:@changesets/assemble-release-plan@npm%3A6.0.0#./.yarn/patches/@changesets-assemble-release-plan-npm-6.0.0-f7b3005037.patch::version=6.0.0&hash=43c5e4&locator=root%40workspace%3A." + dependencies: + "@babel/runtime": ^7.20.1 + "@changesets/errors": ^0.2.0 + "@changesets/get-dependents-graph": ^2.0.0 + "@changesets/types": ^6.0.0 + "@manypkg/get-packages": ^1.1.3 + semver: ^7.5.3 + checksum: b7a68e28d03379bdc2a1d7171963990e1b88d0e5efca5f5ed490c0f583d66d0ccbd4c04ebf4e5cd1c38e1346ff0e5c4c16e1b4973cdd97fabdbad0c6996fe016 + languageName: node + linkType: hard + "@changesets/changelog-git@npm:^0.2.0": version: 0.2.0 resolution: "@changesets/changelog-git@npm:0.2.0" From ece858b60957a8372aa89eab2abe12e46f541bb1 Mon Sep 17 00:00:00 2001 From: Jack Palmer Date: Fri, 3 May 2024 14:20:46 +0100 Subject: [PATCH 78/90] fix: Add config.d.ts for auth-backend-module-github-provider Signed-off-by: Jack Palmer --- .../config.d.ts | 34 +++++++++++++++++++ .../package.json | 30 ++++++++-------- 2 files changed, 50 insertions(+), 14 deletions(-) create mode 100644 plugins/auth-backend-module-github-provider/config.d.ts diff --git a/plugins/auth-backend-module-github-provider/config.d.ts b/plugins/auth-backend-module-github-provider/config.d.ts new file mode 100644 index 0000000000..c8af7eb5db --- /dev/null +++ b/plugins/auth-backend-module-github-provider/config.d.ts @@ -0,0 +1,34 @@ +/* + * Copyright 2020 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +export interface Config { + auth?: { + providers?: { + /** @visibility frontend */ + github?: { + [authEnv: string]: { + clientId: string; + /** + * @visibility secret + */ + clientSecret: string; + callbackUrl?: string; + enterpriseInstanceUrl?: string; + }; + }; + }; + }; +} diff --git a/plugins/auth-backend-module-github-provider/package.json b/plugins/auth-backend-module-github-provider/package.json index 65217f0eeb..0e8db4d1a5 100644 --- a/plugins/auth-backend-module-github-provider/package.json +++ b/plugins/auth-backend-module-github-provider/package.json @@ -1,10 +1,10 @@ { "name": "@backstage/plugin-auth-backend-module-github-provider", - "description": "The github-provider backend module for the auth plugin.", "version": "0.1.15-next.1", - "main": "src/index.ts", - "types": "src/index.ts", - "license": "Apache-2.0", + "description": "The github-provider backend module for the auth plugin.", + "backstage": { + "role": "backend-plugin-module" + }, "publishConfig": { "access": "public", "main": "dist/index.cjs.js", @@ -15,17 +15,21 @@ "url": "https://github.com/backstage/backstage", "directory": "plugins/auth-backend-module-github-provider" }, - "backstage": { - "role": "backend-plugin-module" - }, + "license": "Apache-2.0", + "main": "src/index.ts", + "types": "src/index.ts", + "files": [ + "dist", + "config.d.ts" + ], "scripts": { - "start": "backstage-cli package start", "build": "backstage-cli package build", - "lint": "backstage-cli package lint", - "test": "backstage-cli package test", "clean": "backstage-cli package clean", + "lint": "backstage-cli package lint", "prepack": "backstage-cli package prepack", - "postpack": "backstage-cli package postpack" + "postpack": "backstage-cli package postpack", + "start": "backstage-cli package start", + "test": "backstage-cli package test" }, "dependencies": { "@backstage/backend-plugin-api": "workspace:^", @@ -39,7 +43,5 @@ "@backstage/plugin-auth-backend": "workspace:^", "supertest": "^6.3.3" }, - "files": [ - "dist" - ] + "configSchema": "config.d.ts" } From e4fb486a3edc86617eaccd95f601323e0b983d0a Mon Sep 17 00:00:00 2001 From: Jack Palmer Date: Fri, 3 May 2024 14:23:11 +0100 Subject: [PATCH 79/90] chore: add changeset Signed-off-by: Jack Palmer --- .changeset/blue-balloons-draw.md | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 .changeset/blue-balloons-draw.md diff --git a/.changeset/blue-balloons-draw.md b/.changeset/blue-balloons-draw.md new file mode 100644 index 0000000000..8208af5105 --- /dev/null +++ b/.changeset/blue-balloons-draw.md @@ -0,0 +1,5 @@ +--- +'@backstage/plugin-auth-backend-module-github-provider': patch +--- + +fix: Add missing config.d.ts for auth-backend-module-github-provider From 8f6a945b0ee0b33bc32dbce0bccd5e53c68ef0d6 Mon Sep 17 00:00:00 2001 From: Jack Palmer Date: Fri, 3 May 2024 14:24:23 +0100 Subject: [PATCH 80/90] fix: Typo in copyright Signed-off-by: Jack Palmer --- plugins/auth-backend-module-github-provider/config.d.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/auth-backend-module-github-provider/config.d.ts b/plugins/auth-backend-module-github-provider/config.d.ts index c8af7eb5db..e79711310d 100644 --- a/plugins/auth-backend-module-github-provider/config.d.ts +++ b/plugins/auth-backend-module-github-provider/config.d.ts @@ -1,5 +1,5 @@ /* - * Copyright 2020 The Backstage Authors + * Copyright 2024 The Backstage Authors * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. From 4d15444275f26bedf4037bad5c93f73c7aa48cd5 Mon Sep 17 00:00:00 2001 From: Vincenzo Scamporlino Date: Fri, 3 May 2024 16:47:25 +0200 Subject: [PATCH 81/90] cli: fix repo fix workspace path on windows Signed-off-by: Vincenzo Scamporlino --- packages/cli/src/commands/repo/fix.ts | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/packages/cli/src/commands/repo/fix.ts b/packages/cli/src/commands/repo/fix.ts index b975203a73..049f5a8416 100644 --- a/packages/cli/src/commands/repo/fix.ts +++ b/packages/cli/src/commands/repo/fix.ts @@ -22,11 +22,7 @@ import { } from '@backstage/cli-node'; import { OptionValues } from 'commander'; import fs from 'fs-extra'; -import { - resolve as resolvePath, - join as joinPath, - relative as relativePath, -} from 'path'; +import { resolve as resolvePath, posix, relative as relativePath } from 'path'; import { paths } from '../../lib/paths'; /** @@ -205,7 +201,7 @@ export function createRepositoryFieldFixer() { const rootDir = rootRepoField.directory || ''; return (pkg: FixablePackage) => { - const expectedPath = joinPath( + const expectedPath = posix.join( rootDir, relativePath(paths.targetRoot, pkg.dir), ); From c52052bdceda72743e2d0305d1f6aff3c2d6728c Mon Sep 17 00:00:00 2001 From: Jack Palmer Date: Fri, 3 May 2024 15:49:28 +0100 Subject: [PATCH 82/90] fox: Add config.d.ts for aws-alb and rename iss to issuer Signed-off-by: Jack Palmer --- .../config.d.ts | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 plugins/auth-backend-module-aws-alb-provider/config.d.ts diff --git a/plugins/auth-backend-module-aws-alb-provider/config.d.ts b/plugins/auth-backend-module-aws-alb-provider/config.d.ts new file mode 100644 index 0000000000..1978e5a4df --- /dev/null +++ b/plugins/auth-backend-module-aws-alb-provider/config.d.ts @@ -0,0 +1,27 @@ +/* + * Copyright 2024 The Backstage Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +export interface Config { + auth?: { + providers?: { + /** @visibility frontend */ + awsalb?: { + issuer?: string; + region: string; + }; + }; + }; +} From 9f974a05da653abd5a24ea43f7f496e43291066b Mon Sep 17 00:00:00 2001 From: Jack Palmer Date: Fri, 3 May 2024 15:49:53 +0100 Subject: [PATCH 83/90] fix: Tidy auth-backend config.d.ts Signed-off-by: Jack Palmer --- plugins/auth-backend/config.d.ts | 42 -------------------------------- 1 file changed, 42 deletions(-) diff --git a/plugins/auth-backend/config.d.ts b/plugins/auth-backend/config.d.ts index f0ceaa4224..a7ea18ed1d 100644 --- a/plugins/auth-backend/config.d.ts +++ b/plugins/auth-backend/config.d.ts @@ -89,29 +89,6 @@ export interface Config { * @additionalProperties true */ providers?: { - /** @visibility frontend */ - google?: { - [authEnv: string]: { - clientId: string; - /** - * @visibility secret - */ - clientSecret: string; - callbackUrl?: string; - }; - }; - /** @visibility frontend */ - github?: { - [authEnv: string]: { - clientId: string; - /** - * @visibility secret - */ - clientSecret: string; - callbackUrl?: string; - enterpriseInstanceUrl?: string; - }; - }; /** @visibility frontend */ saml?: { entryPoint: string; @@ -137,20 +114,6 @@ export interface Config { acceptedClockSkewMs?: number; }; /** @visibility frontend */ - oauth2?: { - [authEnv: string]: { - clientId: string; - /** - * @visibility secret - */ - clientSecret: string; - authorizationUrl: string; - tokenUrl: string; - scope?: string; - disableRefresh?: boolean; - }; - }; - /** @visibility frontend */ auth0?: { [authEnv: string]: { clientId: string; @@ -177,11 +140,6 @@ export interface Config { callbackUrl?: string; }; }; - /** @visibility frontend */ - awsalb?: { - iss?: string; - region: string; - }; /** * The backstage token expiration. */ From ed4cd84b10ae123882b0cf549ea9f541ae7d843d Mon Sep 17 00:00:00 2001 From: Jack Palmer Date: Fri, 3 May 2024 15:51:05 +0100 Subject: [PATCH 84/90] chore: Update changelog Signed-off-by: Jack Palmer --- .changeset/blue-balloons-draw.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.changeset/blue-balloons-draw.md b/.changeset/blue-balloons-draw.md index 8208af5105..d17760a92b 100644 --- a/.changeset/blue-balloons-draw.md +++ b/.changeset/blue-balloons-draw.md @@ -3,3 +3,5 @@ --- fix: Add missing config.d.ts for auth-backend-module-github-provider +fix: Add missing config.d.ts for auth-backend-module-aws-alb-provider +fix: Remove duplicate provider config from auth-backend From cc3c51833bf48731751d435b21a341964deb2ad4 Mon Sep 17 00:00:00 2001 From: Vincenzo Scamporlino Date: Fri, 3 May 2024 16:54:18 +0200 Subject: [PATCH 85/90] cli: repo fix changeset Signed-off-by: Vincenzo Scamporlino --- .changeset/slimy-kids-behave.md | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 .changeset/slimy-kids-behave.md diff --git a/.changeset/slimy-kids-behave.md b/.changeset/slimy-kids-behave.md new file mode 100644 index 0000000000..5e635878ff --- /dev/null +++ b/.changeset/slimy-kids-behave.md @@ -0,0 +1,5 @@ +--- +'@backstage/cli': patch +--- + +Fixed an issue causing the `repo fix` command to set an incorrect `workspace` property using Windows From 4a0577e0ea14c7b490e7ef0d0a459a8b375e292b Mon Sep 17 00:00:00 2001 From: Jack Palmer Date: Fri, 3 May 2024 15:54:41 +0100 Subject: [PATCH 86/90] chore: Fix changelog Signed-off-by: Jack Palmer --- .changeset/blue-balloons-draw.md | 7 ------- .changeset/little-rockets-live.md | 7 +++++++ 2 files changed, 7 insertions(+), 7 deletions(-) delete mode 100644 .changeset/blue-balloons-draw.md create mode 100644 .changeset/little-rockets-live.md diff --git a/.changeset/blue-balloons-draw.md b/.changeset/blue-balloons-draw.md deleted file mode 100644 index d17760a92b..0000000000 --- a/.changeset/blue-balloons-draw.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -'@backstage/plugin-auth-backend-module-github-provider': patch ---- - -fix: Add missing config.d.ts for auth-backend-module-github-provider -fix: Add missing config.d.ts for auth-backend-module-aws-alb-provider -fix: Remove duplicate provider config from auth-backend diff --git a/.changeset/little-rockets-live.md b/.changeset/little-rockets-live.md new file mode 100644 index 0000000000..767fda917e --- /dev/null +++ b/.changeset/little-rockets-live.md @@ -0,0 +1,7 @@ +--- +'@backstage/plugin-auth-backend-module-aws-alb-provider': patch +'@backstage/plugin-auth-backend-module-github-provider': patch +'@backstage/plugin-auth-backend': patch +--- + +fix: Move config declarations to appropriate auth backend modules From a8ea3c5918f8b7c5e54cad5f11aaadf11c856079 Mon Sep 17 00:00:00 2001 From: Jeeva Ramanathan Date: Fri, 3 May 2024 20:34:59 +0530 Subject: [PATCH 87/90] Check pipeline Signed-off-by: Jeeva Ramanathan From 073b2ebf638851d28dc8f1434d0b36f94eebfaf6 Mon Sep 17 00:00:00 2001 From: Beth Griggs Date: Fri, 3 May 2024 15:21:32 +0100 Subject: [PATCH 88/90] chore(test): increase test coverage of WinstonLogger Signed-off-by: Beth Griggs --- .../src/logging/WinstonLogger.test.ts | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/packages/backend-app-api/src/logging/WinstonLogger.test.ts b/packages/backend-app-api/src/logging/WinstonLogger.test.ts index c7173b1cbe..bec6a9fc0f 100644 --- a/packages/backend-app-api/src/logging/WinstonLogger.test.ts +++ b/packages/backend-app-api/src/logging/WinstonLogger.test.ts @@ -22,6 +22,17 @@ function msg(info: TransformableInfo): TransformableInfo { } describe('WinstonLogger', () => { + it('creates a winston logger instance with default options', () => { + const logger = WinstonLogger.create({}); + expect(logger).toBeInstanceOf(WinstonLogger); + }); + + it('creates a child logger', () => { + const logger = WinstonLogger.create({}); + const childLogger = logger.child({ plugin: 'test-plugin' }); + expect(childLogger).toBeInstanceOf(WinstonLogger); + }); + it('redacter should redact and escape regex', () => { const redacter = WinstonLogger.redacter(); const log = { @@ -47,4 +58,24 @@ describe('WinstonLogger', () => { }), ); }); + + it('redacter should redact nested object', () => { + const redacter = WinstonLogger.redacter(); + const log = { + level: 'error', + message: { + nested: 'hello (world) from nested object', + }, + }; + + redacter.add(['hello']); + expect(redacter.format.transform(msg(log))).toEqual( + msg({ + ...log, + message: { + nested: '[REDACTED] (world) from nested', + }, + }), + ); + }); }); From 0cda20fa31a4a92ad69a3a3e87d93d4ee64bb0df Mon Sep 17 00:00:00 2001 From: Bethany Griggs Date: Fri, 3 May 2024 18:08:37 +0100 Subject: [PATCH 89/90] fixup! typo Signed-off-by: Bethany Griggs --- packages/backend-app-api/src/logging/WinstonLogger.test.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/backend-app-api/src/logging/WinstonLogger.test.ts b/packages/backend-app-api/src/logging/WinstonLogger.test.ts index bec6a9fc0f..d025719d9a 100644 --- a/packages/backend-app-api/src/logging/WinstonLogger.test.ts +++ b/packages/backend-app-api/src/logging/WinstonLogger.test.ts @@ -73,7 +73,7 @@ describe('WinstonLogger', () => { msg({ ...log, message: { - nested: '[REDACTED] (world) from nested', + nested: '[REDACTED] (world) from nested object', }, }), ); From b741e21fa704ff78675641ebc506da6717164fb1 Mon Sep 17 00:00:00 2001 From: Tharun Paul <55498156+paul-tharun@users.noreply.github.com> Date: Sat, 4 May 2024 23:08:10 +0530 Subject: [PATCH 90/90] Fix hyperlink to what-is-a-plugin link Signed-off-by: Tharun Paul <55498156+paul-tharun@users.noreply.github.com> --- docs/faq/product.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/faq/product.md b/docs/faq/product.md index 79465cf53d..edbb1ffb4d 100644 --- a/docs/faq/product.md +++ b/docs/faq/product.md @@ -37,7 +37,7 @@ more, read our blog post, Yes, we've already started releasing open source versions of some of the plugins we use here, and we'll continue to do so. -[Plugins](#what-is-a-plugin-in-backstage) are the building blocks of +[Plugins](technical.md#what-is-a-plugin-in-backstage) are the building blocks of functionality in Backstage. We have over 120 plugins inside Spotify — many of those are specialized for our use, so will remain internal and proprietary to us. But we estimate that about a third of our existing plugins make good open