Merge branch 'master' into identity-api-client-interface
This commit is contained in:
Brian Fletcher
@@ -57,11 +57,16 @@ This is an array used to determine where to retrieve cluster configuration from.
|
||||
|
||||
Valid cluster locator methods are:
|
||||
|
||||
- [`catalog`](#catalog)
|
||||
- [`localKubectlProxy`](#localKubectlProxy)
|
||||
- [`config`](#config)
|
||||
- [`gke`](#gke)
|
||||
- [custom `KubernetesClustersSupplier`](#custom-kubernetesclusterssupplier)
|
||||
|
||||
#### `catalog`
|
||||
|
||||
This cluster locator method will read cluster information from the catalog.
|
||||
|
||||
#### `localKubectlProxy`
|
||||
|
||||
This cluster locator method will assume a locally running [`kubectl proxy`](https://kubernetes.io/docs/tasks/extend-kubernetes/http-proxy-access-api/#using-kubectl-to-start-a-proxy-server) process using the default port (8001).
|
||||
|
||||
@@ -112,6 +112,8 @@ of the `SearchType` component.
|
||||
</Paper>
|
||||
```
|
||||
|
||||
> Check out the documentation around [integrating search into plugins](../../plugins/integrating-search-into-plugins.md#create-a-collator) for how to create your own collator.
|
||||
|
||||
## How to limit what can be searched in the Software Catalog
|
||||
|
||||
The Software Catalog includes a wealth of information about the components,
|
||||
|
||||
@@ -93,8 +93,12 @@ spec:
|
||||
|
||||
# some outputs which are saved along with the job for use in the frontend
|
||||
output:
|
||||
remoteUrl: ${{ steps.publish.output.remoteUrl }}
|
||||
entityRef: ${{ steps.register.output.entityRef }}
|
||||
links:
|
||||
- title: Repository
|
||||
url: ${{ steps.publish.output.remoteUrl }}
|
||||
- title: Open in catalog
|
||||
icon: catalog
|
||||
entityRef: ${{ steps.register.output.entityRef }}
|
||||
```
|
||||
|
||||
Let's dive in and pick apart what each of these sections do and what they are.
|
||||
@@ -495,8 +499,12 @@ The main two that are used are the following:
|
||||
|
||||
```yaml
|
||||
output:
|
||||
remoteUrl: ${{ steps.publish.output.remoteUrl }} # link to the remote repository
|
||||
entityRef: ${{ steps.register.output.entityRef }} # link to the entity that has been ingested to the catalog
|
||||
links:
|
||||
- title: Repository
|
||||
url: ${{ steps.publish.output.remoteUrl }} # link to the remote repository
|
||||
- title: Open in catalog
|
||||
icon: catalog
|
||||
entityRef: ${{ steps.register.output.entityRef }} # link to the entity that has been ingested to the catalog
|
||||
```
|
||||
|
||||
## The templating syntax
|
||||
|
||||
@@ -66,6 +66,7 @@ See [TechDocs Architecture](architecture.md) to get an overview of where the bel
|
||||
| GitHub Enterprise | Yes ✅ |
|
||||
| Bitbucket | Yes ✅ |
|
||||
| Azure DevOps | Yes ✅ |
|
||||
| Gerrit | Yes ✅ |
|
||||
| GitLab | Yes ✅ |
|
||||
| GitLab Enterprise | Yes ✅ |
|
||||
|
||||
|
||||
@@ -3,12 +3,239 @@ id: org
|
||||
title: Microsoft Azure Active Directory Organizational Data
|
||||
sidebar_label: Org Data
|
||||
# prettier-ignore
|
||||
description: Importing users and groups from a Microsoft Azure Active Directory into Backstage
|
||||
description: Importing users and groups from Microsoft Azure Active Directory into Backstage
|
||||
---
|
||||
|
||||
The Backstage catalog can be set up to ingest organizational data - users and
|
||||
teams - directly from a tenant in Microsoft Azure Active Directory via the
|
||||
Microsoft Graph API.
|
||||
|
||||
More details on this are available in the
|
||||
[README of the `@backstage/plugin-catalog-backend-module-msgraph` package](https://github.com/backstage/backstage/blob/master/plugins/catalog-backend-module-msgraph/README.md).
|
||||
## Installation
|
||||
|
||||
The package is not installed by default, therefore you have to add `@backstage/plugin-catalog-backend-module-msgraph` to your backend package.
|
||||
|
||||
```bash
|
||||
# From your Backstage root directory
|
||||
yarn add --cwd packages/backend @backstage/plugin-catalog-backend-module-msgraph
|
||||
```
|
||||
|
||||
Next add the basic configuration to `app-config.yaml`
|
||||
|
||||
```yaml
|
||||
catalog:
|
||||
providers:
|
||||
microsoftGraphOrg:
|
||||
default:
|
||||
tenantId: ${AZURE_TENANT_ID}
|
||||
user:
|
||||
filter: accountEnabled eq true and userType eq 'member'
|
||||
group:
|
||||
filter: >
|
||||
securityEnabled eq false
|
||||
and mailEnabled eq true
|
||||
and groupTypes/any(c:c+eq+'Unified')
|
||||
```
|
||||
|
||||
Finally, register the plugin in `catalog.ts`.
|
||||
For large organizations, this plugin can take a long time, so be careful setting low frequency / timeouts.
|
||||
|
||||
```diff
|
||||
// packages/backend/src/plugins/catalog.ts
|
||||
+import { MicrosoftGraphOrgEntityProvider } from '@backstage/plugin-catalog-backend-module-msgraph';
|
||||
|
||||
export default async function createPlugin(
|
||||
env: PluginEnvironment,
|
||||
): Promise<Router> {
|
||||
const builder = await CatalogBuilder.create(env);
|
||||
|
||||
+ builder.addEntityProvider(
|
||||
+ MicrosoftGraphOrgEntityProvider.fromConfig(env.config, {
|
||||
+ logger: env.logger,
|
||||
+ schedule: env.scheduler.createScheduledTaskRunner({
|
||||
+ frequency: { hours: 1 },
|
||||
+ timeout: { minutes: 50 },
|
||||
+ initialDelay: { seconds: 15}
|
||||
+ }),
|
||||
+ }),
|
||||
+ );
|
||||
```
|
||||
|
||||
## Authenticating with Microsoft Graph
|
||||
|
||||
### Local Development
|
||||
|
||||
For a local dev environment, it's recommended you have the Azure CLI or Azure PowerShell installed, and are logged in to those.
|
||||
Alternatively you can use VSCode with the Azure extension if you install `@azure/identity-vscode`.
|
||||
When these are set up, the plugin will authenticate with the Microsoft Graph API without you needing to configure any credentials, or granting any special permissions.
|
||||
If you can't do this, you'll have to create an App Registration.
|
||||
|
||||
### App Registration
|
||||
|
||||
If none of the other authentication methods work, you can create an app registration in the azure portal.
|
||||
By default the graph plugin requires the following Application permissions (not Delegated) for Microsoft Graph:
|
||||
|
||||
- `GroupMember.Read.All`
|
||||
- `User.Read.All`
|
||||
|
||||
If your organization required Admin Consent for these permissions, that will need to be granted.
|
||||
|
||||
When authenticating with a ClientId/ClientSecret, you can either set the `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and `AZURE_CLIENT_SECRET` environment variables, or specify the values in configuration
|
||||
|
||||
```yaml
|
||||
microsoftGraphOrg:
|
||||
default:
|
||||
##...
|
||||
clientId: 9ef1aac6-b454-4e69-9cf5-7199df049281
|
||||
clientSecret: REDACTED
|
||||
```
|
||||
|
||||
To authenticate with a certificate rather than a client secret, you can set the `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and `AZURE_CLIENT_CERTIFICATE_PATH` environments
|
||||
|
||||
### Managed Identity
|
||||
|
||||
If deploying to resources that supports Managed Identity, and has identities configured (e.g. Azure App Services, Azure Container Apps), Managed Identity should be picked up without any additional configuration.
|
||||
If your app has multiple managed identities, you may need to set the `AZURE_CLIENT_ID` environment variable to tell Azure Identity which identity to use.
|
||||
|
||||
You will need to grant the same permissions to your identity as described for App Registrations above.
|
||||
|
||||
## Filtering imported Users and Groups
|
||||
|
||||
By default, the plugin will import all users and groups from your directory.
|
||||
This can be customized through filters and search queries.
|
||||
|
||||
### Groups
|
||||
|
||||
A smaller set of groups can be obtained by configuring a search query or a filter.
|
||||
If both `filter` and `search` are provided, then groups must match both to be ingested.
|
||||
|
||||
```yaml
|
||||
microsoftGraphOrg:
|
||||
providerId:
|
||||
group:
|
||||
filter: securityEnabled eq false and mailEnabled eq true and groupTypes/any(c:c+eq+'Unified')
|
||||
search: '"description:One" AND ("displayName:Video" OR "displayName:Drive")'
|
||||
```
|
||||
|
||||
In addition to these groups, one additional group will be created for your organization.
|
||||
All imported groups will be a child of this group.
|
||||
|
||||
### Users
|
||||
|
||||
There are two modes for importing users - You can import all user objects matching a `filter`.
|
||||
|
||||
```yaml
|
||||
microsoftGraphOrg:
|
||||
providerId:
|
||||
user:
|
||||
filter: accountEnabled eq true and userType eq 'member'
|
||||
```
|
||||
|
||||
Alternatively you can import users that are members of specific groups.
|
||||
For each group matching the `search` and `filter` query, each group member will be imported.
|
||||
Only direct group members will be imported, not transient users.
|
||||
|
||||
```yaml
|
||||
microsoftGraphOrg:
|
||||
providerId:
|
||||
userGroupMember:
|
||||
filter: "displayName eq 'Backstage Users'"
|
||||
search: '"description:One" AND ("displayName:Video" OR "displayName:Drive")'
|
||||
```
|
||||
|
||||
## Customizing Transformation
|
||||
|
||||
Ingested entities can be customized by providing custom transformers.
|
||||
These can be used to completely replace the built in logic, or used to tweak it by using the default transformers (`defaultGroupTransformer`, `defaultUserTransformer` and `defaultOrganizationTransformer`
|
||||
Entities can also be excluded from backstage by returning `undefined`.
|
||||
|
||||
These Transformers are be registered when configuring `MicrosoftGraphOrgEntityProvider`
|
||||
|
||||
```diff
|
||||
builder.addEntityProvider(
|
||||
MicrosoftGraphOrgEntityProvider.fromConfig(env.config, {
|
||||
// ...
|
||||
+ groupTransformer: myGroupTransformer,
|
||||
+ userTransformer: myUserTransformer,
|
||||
+ organizationTransformer: myOrganizationTransformer,
|
||||
}),
|
||||
);
|
||||
```
|
||||
|
||||
When using custom transformers, you may want to customize the data returned.
|
||||
Several configuration options can be provided to tweak the Microsoft Graph query to get the data you need
|
||||
|
||||
```yaml
|
||||
microsoftGraphOrg:
|
||||
providerId:
|
||||
user:
|
||||
expand: manager
|
||||
group:
|
||||
expand: member
|
||||
select: ['id', 'displayName', 'description']
|
||||
```
|
||||
|
||||
The following provides an example of each kind of transformer
|
||||
|
||||
```ts
|
||||
import * as MicrosoftGraph from '@microsoft/microsoft-graph-types';
|
||||
import {
|
||||
defaultGroupTransformer,
|
||||
defaultUserTransformer,
|
||||
defaultOrganizationTransformer,
|
||||
} from '@backstage/plugin-catalog-backend-module-msgraph';
|
||||
import { GroupEntity, UserEntity } from '@backstage/catalog-model';
|
||||
|
||||
// This group transformer completely replaces the built in logic with custom logic.
|
||||
export async function myGroupTransformer(
|
||||
group: MicrosoftGraph.Group,
|
||||
groupPhoto?: string,
|
||||
): Promise<GroupEntity | undefined> {
|
||||
return {
|
||||
apiVersion: 'backstage.io/v1alpha1',
|
||||
kind: 'Group',
|
||||
metadata: {
|
||||
name: group.id!,
|
||||
annotations: {},
|
||||
},
|
||||
spec: {
|
||||
type: 'aad',
|
||||
children: [],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// This user transformer makes use of the built in logic, but also sets the description field
|
||||
export async function myUserTransformer(
|
||||
graphUser: MicrosoftGraph.User,
|
||||
userPhoto?: string,
|
||||
): Promise<UserEntity | undefined> {
|
||||
const backstageUser = await defaultUserTransformer(graphUser, userPhoto);
|
||||
|
||||
if (backstageUser) {
|
||||
backstageUser.metadata.description = 'Loaded from Azure Active Directory';
|
||||
}
|
||||
|
||||
return backstageUser;
|
||||
}
|
||||
|
||||
// Example organization transformer that removes the organization group completely
|
||||
export async function myOrganizationTransformer(
|
||||
graphOrganization: MicrosoftGraph.Organization,
|
||||
): Promise<GroupEntity | undefined> {
|
||||
return undefined;
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Authentication / Token Errors
|
||||
|
||||
See [Troubleshooting Azure Identity Authentication Issues](https://aka.ms/azsdk/js/identity/troubleshoot)
|
||||
|
||||
### Error while reading users from Microsoft Graph: Authorization_RequestDenied - Insufficient privileges to complete the operation.
|
||||
|
||||
- Make sure you've granted all the required permissions to your application registration or managed identity
|
||||
- Make sure the permissions are `Application` permissions rather than `Delegated`
|
||||
- If your organization has configured "Admin consent" to be required, make sure this has been granted for your application permissions
|
||||
- If your group queries are returning Microsoft Teams groups, you may need to grant addition permissions (e.g. `Team.ReadBasic.All`, `TeamMember.Read.All`)
|
||||
- If you've added additional `select` or `expand` fields, those may need additional permissions granted
|
||||
|
||||
@@ -24,6 +24,8 @@ app:
|
||||
# env: 'staging'
|
||||
```
|
||||
|
||||
If your [`app-config.yaml`](https://github.com/backstage/backstage/blob/master/app-config.yaml#L5) file does not have this configuration, you may have to adjust your [`packages/app/public/index.html`](https://github.com/backstage/backstage/blob/master/packages/app/public/index.html#L64) to include the Datadog RUM `init()` section manually.
|
||||
|
||||
The `clientToken` and `applicationId` are generated from the Datadog RUM page
|
||||
following
|
||||
[these instructions](https://docs.datadoghq.com/real_user_monitoring/browser/).
|
||||
|
||||
@@ -74,7 +74,7 @@ The extension type is a simple one:
|
||||
|
||||
```ts
|
||||
export type Extension<T> = {
|
||||
expose(plugin: BackstagePlugin<any, any>): T;
|
||||
expose(plugin: BackstagePlugin): T;
|
||||
};
|
||||
```
|
||||
|
||||
|
||||
@@ -14,7 +14,139 @@ Search Platform in your plugin.
|
||||
|
||||
## Providing data to the search platform
|
||||
|
||||
> A guide on how to create collators is coming soon!
|
||||
### Create a collator
|
||||
|
||||
> Knowing what a [collator](../features/search/concepts.md#collators) is will help you as you build it out.
|
||||
|
||||
Imagine you have a plugin that is responsible for storing FAQ snippets in a database. You want other engineers to be able to easily find your questions and answers. So that means you want them to be indexed by the search platform. Lets say the FAQ snippets can be viewed at a URL like `backstage.example.biz/faq-snippets`.
|
||||
|
||||
The search platform provides an interface (`DocumentCollatorFactory` from package `@backstage/plugin-search-common`) that allows you to do exactly that. It works by registering each of your entries as a "document" that later represents one search result each.
|
||||
|
||||
> You can always look at a working example, e.g. [StackOverflowQuestionsCollatorFactory](https://github.com/backstage/backstage/blob/master/plugins/stack-overflow-backend/src/search/StackOverflowQuestionsCollatorFactory.ts), if you are unsure or want to follow best practices.
|
||||
|
||||
#### 1. Install collator interface dependencies
|
||||
|
||||
We will need the interface `DocumentCollatorFactory` from package `@backstage/plugin-search-common`, so let's add it to your plugins dependencies:
|
||||
|
||||
```sh
|
||||
# navigate to the plugin directory
|
||||
# (for this tutorial our plugin lives in the backstage repo, if your plugin lives in a separate repo you need to clone that first)
|
||||
cd plugins/faq-snippets
|
||||
|
||||
# Create a new branch using Git command-line
|
||||
git checkout -b tutorials/new-faq-snippets-collator
|
||||
|
||||
# Install the package containing the interface
|
||||
yarn add @backstage/plugin-search-common
|
||||
```
|
||||
|
||||
#### 2. Define your document type
|
||||
|
||||
Before we can start generating documents from our FAQ entries, we first have to define a document type containing all necessary information we need to later display our entry as search result. The package `@backstage/plugin-search-common` we installed earlier contains a type `IndexableDocument` that we can extend.
|
||||
|
||||
Create a new file `plugins/faq-snippets/src/search/collators/FaqSnippetDocument.ts` and paste the following below:
|
||||
|
||||
```ts
|
||||
import { IndexableDocument } from '@backstage/plugin-search-common';
|
||||
|
||||
export interface FaqSnippetDocument extends IndexableDocument {
|
||||
answered_by: string;
|
||||
}
|
||||
```
|
||||
|
||||
#### 3. Use Backstage App configuration
|
||||
|
||||
Your new collator could benefit from using configuration directly from the Backstage `app-config.yaml` file which is located on the project's root folder:
|
||||
|
||||
```yaml
|
||||
faq:
|
||||
baseUrl: https://backstage.example.biz/faq-snippets
|
||||
```
|
||||
|
||||
#### 4. Implement your collator
|
||||
|
||||
Imagine your FAQs can be retrieved at the URL `https://backstage.example.biz/faq-snippets` with following JSON response format:
|
||||
|
||||
```json
|
||||
{
|
||||
"items": [
|
||||
{
|
||||
"id": 42,
|
||||
"question": "What is The Answer to the Ultimate Question of Life, the Universe, and Everything?",
|
||||
"answer": "Forty-two",
|
||||
"user": "Deep Thought"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Below we provide an example implementation of how the FAQ collator factory could look like using our new document type, placed in the `plugins/faq-snippets/src/search/collators/FaqCollatorFactory.ts` file:
|
||||
|
||||
```ts
|
||||
import fetch from 'cross-fetch';
|
||||
import { Logger } from 'winston';
|
||||
import { Config } from '@backstage/config';
|
||||
import { Readable } from 'stream';
|
||||
import { DocumentCollatorFactory } from '@backstage/plugin-search-common';
|
||||
|
||||
import { FaqDocument } from './FaqDocument';
|
||||
|
||||
export type FaqCollatorFactoryOptions = {
|
||||
baseUrl?: string;
|
||||
logger: Logger;
|
||||
};
|
||||
|
||||
export class FaqCollatorFactory implements DocumentCollatorFactory {
|
||||
private readonly baseUrl: string | undefined;
|
||||
private readonly logger: Logger;
|
||||
public readonly type: string = 'faq-snippets';
|
||||
|
||||
private constructor(options: FaqCollatorFactoryOptions) {
|
||||
this.baseUrl = options.baseUrl;
|
||||
this.logger = options.logger;
|
||||
}
|
||||
|
||||
static fromConfig(config: Config, options: FaqCollatorFactoryOptions) {
|
||||
const baseUrl =
|
||||
config.getOptionalString('faq.baseUrl') ||
|
||||
'https://backstage.example.biz/faq-snippets';
|
||||
return new FaqCollatorFactory({ ...options, baseUrl });
|
||||
}
|
||||
|
||||
async getCollator() {
|
||||
return Readable.from(this.execute());
|
||||
}
|
||||
|
||||
async *execute(): AsyncGenerator<FaqDocument> {
|
||||
if (!this.baseUrl) {
|
||||
this.logger.error(`No faq.baseUrl configured in your app-config.yaml`);
|
||||
return;
|
||||
}
|
||||
|
||||
const response = await fetch(this.baseUrl);
|
||||
const data = await response.json();
|
||||
|
||||
for (const faq of data.items) {
|
||||
yield {
|
||||
title: faq.question,
|
||||
location: `/faq-snippets/${faq.id}`,
|
||||
text: faq.answer,
|
||||
answered_by: faq.user,
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 5. Test your collator
|
||||
|
||||
To verify your implementation works as expected make sure to add tests for it. For your convenience, there is the [`TestPipeline`](https://backstage.io/docs/reference/plugin-search-backend-node.testpipeline) utility that emulates a pipeline into which you can integrate your custom collator.
|
||||
|
||||
Look at [DefaultTechDocsCollatorFactory test](https://github.com/backstage/backstage/blob/master/plugins/techdocs-backend/src/search/DefaultTechDocsCollatorFactory.test.ts), for an example.
|
||||
|
||||
#### 6. Make your plugins collator discoverable for others
|
||||
|
||||
If you want to make your collator discoverable for other adopters, add it to the list of [plugins integrated to search](https://backstage.io/docs/features/search/search-overview#plugins-integrated-with-backstage-search).
|
||||
|
||||
## Building a search experience into your plugin
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user