From 2e14b0e9242ea5ae09594a686bdf40b844858c93 Mon Sep 17 00:00:00 2001 From: Andre Wanlin Date: Thu, 2 May 2024 08:46:27 -0500 Subject: [PATCH] Changes based on feedback Signed-off-by: Andre Wanlin --- docs/integrations/github/github-apps.md | 43 +++++++++-- docs/integrations/github/org.md | 99 ++++++++++++++----------- 2 files changed, 93 insertions(+), 49 deletions(-) diff --git a/docs/integrations/github/github-apps.md b/docs/integrations/github/github-apps.md index 604713a490..6e3d028e6d 100644 --- a/docs/integrations/github/github-apps.md +++ b/docs/integrations/github/github-apps.md @@ -31,7 +31,7 @@ A GitHub app created with the cli will have read access by default. You have to manually update the GitHub App settings in GitHub to grant the app more permissions if needed. -### Using the CLI (public GitHub only) +## Using the CLI (public GitHub only) You can use the `backstage-cli` to create a GitHub App using a manifest file that we provide. This gives us a way to automate some of the work required to @@ -53,7 +53,7 @@ Note that the created app will have a webhook that is disabled by default and points to `smee.io`, which is intended for local development. There's also currently no part of Backstage that makes use of the webhook. -### GitHub Enterprise +## GitHub Enterprise You have to create the GitHub Application manually using these [instructions](https://docs.github.com/en/free-pro-team@latest/developers/apps/creating-a-github-app) @@ -76,7 +76,7 @@ privateKey: | -----END RSA PRIVATE KEY----- ``` -### Including in Integrations Config +## Including in Integrations Config Once the credentials are stored in a YAML file generated by `create-github-app`, or manually by following the [GitHub Enterprise](#github-enterprise) @@ -95,7 +95,25 @@ integrations: - $include: example-backstage-app-credentials.yaml ``` -### Limiting the GitHub App installations +Alternatively you can use environment variables as well: + +```yaml +integrations: + github: + - host: github.com + apps: + - appId: ${AUTH_ORG_APP_ID} + clientId: ${AUTH_ORG_CLIENT_ID} + clientSecret: ${AUTH_ORG_CLIENT_SECRET} + privateKey: ${AUTH_ORG1_PRIVATE_KEY} + webhookSecret: ${AUTH_ORG_WEBHOOK_SECRET} +``` + +:::Note +Note that in both examples above `apps` is an array which means you can add multiple GitHub Apps using `$include` or environment variables as long as they are each for a different GitHub Org as mentioned under the [Caveats](#caveats) section +::: + +## Limiting the GitHub App installations If you want to limit the GitHub app installations visible to backstage you may optionally include the `allowedInstallationOwners` option. If you configure @@ -117,7 +135,7 @@ privateKey: | This will result in backstage preventing the use of any installation that is not within the allow list. -### App permissions +## App permissions When creating a GitHub App, you must select permissions to define the level of access for the app. The permissions required vary depending on your use of the @@ -140,7 +158,20 @@ integration: - `Secrets`: `Read & write` (if templates include GitHub Action Repository Secrets) - `Environments`: `Read & write` (if templates include GitHub Environments) -### Troubleshooting +## Updating Permissions + +There may be times where you need to update the permissions for your GitHub App, to easily get at the GitHub App you can find it at this URL: + +```sh +https://github.com/organizations/{ORG}/settings/apps/{APP_NAME}/permissions +``` + +**Please note that when you change permissions, the app owner will get an email +that must be approved first before the changes are applied.** + +![email](../../assets/integrations/github/email.png) + +## Troubleshooting `HttpError: This endpoint requires you to be authenticated.` diff --git a/docs/integrations/github/org.md b/docs/integrations/github/org.md index a9936ff558..0b33cfa184 100644 --- a/docs/integrations/github/org.md +++ b/docs/integrations/github/org.md @@ -21,6 +21,13 @@ entities that mirror your org setup. > provide authentication. See the > [GitHub auth provider](../../auth/github/provider.md) for that. +## Permissions + +Prior to installing the GitHub Org provider you should confirm you have the right permissions: + +- Personal Access Token permissions are listed in the [GitHub Locations](./locations.md#token-scopes) documentation +- GitHub App(s) permissions are listed in the [GitHub Apps](./github-apps.md#app-permissions) documentation + ## Installation You will have to add the GitHub Org provider to your backend as it is not installed by default, therefore you have to add a @@ -32,7 +39,22 @@ package. yarn --cwd packages/backend add @backstage/plugin-catalog-backend-module-github-org ``` -And then update your backend by adding the following line: +Next add the basic configuration to `app-config.yaml` + +```yaml title="app-config.yaml" +catalog: + providers: + githubOrg: + id: github + githubUrl: https://github.com + orgs: ['organization-1', 'organization-2', 'organization-3'] + schedule: + initialDelay: { seconds: 30 } + frequency: { hours: 1 } + timeout: { minutes: 50 } +``` + +Finally, update your backend by adding the following line: ```ts title="packages/backend/src/index.ts" backend.add(import('@backstage/plugin-catalog-backend/alpha')); @@ -40,7 +62,38 @@ backend.add(import('@backstage/plugin-catalog-backend/alpha')); backend.add(import('@backstage/plugin-catalog-backend-module-github-org')); ``` -## Events Support +### Configuration Details + +In the installation steps above we included an simple example of the needed configuration. The section goes into more details about the various configuration options. + +```yaml title="app-config.yaml" +catalog: + providers: + githubOrg: + - id: github + githubUrl: https://github.com + orgs: ['organization-1', 'organization-2', 'organization-3'] + schedule: + initialDelay: { seconds: 30 } + frequency: { hours: 1 } + timeout: { minutes: 50 } + - id: ghe + githubUrl: https://ghe.mycompany.com + orgs: ['internal-1', 'internal-2', 'internal-3'] + schedule: + initialDelay: { seconds: 30 } + frequency: { hours: 1 } + timeout: { minutes: 50 } +``` + +Directly under the `githubOrg` is a list of configurations, each entry is a structure with the following elements: + +- `id`: A stable id for this provider. Entities from this provider will be associated with this ID, so you should take care not to change it over time since that may lead to orphaned entities and/or conflicts. +- `githubUrl`: The target that this provider should consume +- `orgs` (optional): The list of the GitHub orgs to consume. By default wil consume all accessible orgs on the given GitHub instance (support for GitHub App integration only). +- `schedule`: The refresh schedule to use, matches the structure of [`TaskScheduleDefinitionConfig`](https://backstage.io/docs/reference/backend-tasks.taskscheduledefinitionconfig/) + +### Events Support The catalog module for GitHub Org comes with events support enabled. This will make it subscribe to its relevant topics and expects these events to be published via the `EventsService`. @@ -67,44 +120,7 @@ You can decide between the following options (extensible): You can check the official docs to [configure your webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks) and to [secure your request](https://docs.github.com/en/developers/webhooks-and-events/webhooks/securing-your-webhooks). The webhook will need to be configured to forward `organization`,`team` and `membership` events. -## Configuration - -As mentioned above, you also must have some configuration in your app-config -that describes the targets that you want to import. This lets the entity -provider know what authorization to use, and what the API endpoints are. You may -or may not have such an entry already added since before: - -```yaml -integrations: - github: - # example for public github - - host: github.com - token: ${GITHUB_TOKEN} - # example for a private GitHub Enterprise instance - - host: ghe.example.net - apiBaseUrl: https://ghe.example.net/api/v3 - token: ${GHE_TOKEN} -``` - -These examples use `${}` placeholders to reference environment variables. This -is often suitable for production setups, but also means that you will have to -supply those variables to the backend as it starts up. If you want, for local -development in particular, you can experiment first by putting the actual tokens -in a mirrored config directly in your `app-config.local.yaml` as well. - -If Backstage is configured to use GitHub Apps authentication you must grant -`Read-Only` access for `Members` under `Organization` in order to ingest users -correctly. You can modify the app's permissions under the organization settings, -`https://github.com/organizations/{ORG}/settings/apps/{APP_NAME}/permissions`. - -![permissions](../../assets/integrations/github/permissions.png) - -**Please note that when you change permissions, the app owner will get an email -that must be approved first before the changes are applied.** - -![email](../../assets/integrations/github/email.png) - -### Custom Transformers +## Custom Transformers You can inject your own transformation logic to help map from GH API responses into backstage entities. You can do this on the user and team requests to @@ -227,9 +243,6 @@ export const myVerifiedUserTransformer: UserTransformer = async (user, ctx) => { This example assumes you have implemented the custom transformer following the [Custom Transformers](#custom-transformers) and [Transformer Examples](#transformer-examples) documentation in the sections above. -Once you have imported the emails you can resolve users in your [sign-in -resolver](../../auth/github/provider.md) using the catalog entity search via email - Once you have imported the emails you can resolve users by building a [Custom Resolver](../../auth/identity-resolver.md#building-custom-resolvers). In this custom resolver you can then use this example to properly match the user: ```ts