Merge branch 'master' into feature/EKSCatalog

Signed-off-by: Jonah Back <jonah@jonahback.com>
2022-07-05 10:26:19 -07:00
parent f6216b850e 74f72e5341
commit 621fa1e5a8
597 changed files with 17151 additions and 4035 deletions
@@ -59,10 +59,38 @@ configured and make the following changes to your backend:

 // Initialize a connection to a search engine.
 const searchEngine = (await PgSearchEngine.supported(env.database))
-  ? await PgSearchEngine.from({ database: env.database })
+  ? await PgSearchEngine.fromConfig(env.config, { database: env.database })
  : new LunrSearchEngine({ logger: env.logger });
 ```

+## Optional Configuration
+
+The following is an example of the optional configuration that can be applied when using Postgres as the search backend. Currently this is mostly for just the highlight feature:
+
+```yaml
+search:
+  pg:
+    highlightOptions:
+      useHighlight: true # Used to enable to disable the highlight feature. The default value is true
+      maxWord: 35 # Used to set the longest headlines to output. The default value is 35.
+      minWord: 15 # Used to set the shortest headlines to output. The default value is 15.
+      shortWord: 3 # Words of this length or less will be dropped at the start and end of a headline, unless they are query terms. The default value of three (3) eliminates common English articles.
+      highlightAll: false # If true the whole document will be used as the headline, ignoring the preceding three parameters. The default is false.
+      maxFragments: 0 # Maximum number of text fragments to display. The default value of zero selects a non-fragment-based headline generation method. A value greater than zero selects fragment-based headline generation (see the linked documentation above for more details).
+      fragmentDelimiter: ' ... ' # Delimiter string used to concatenate fragments. Defaults to " ... ".
+```
+
+**Note:** the highlight search term feature uses `ts_headline` which has been known to potentially impact performance. You only need this minimal config to disable it should you have issues:
+
+```yaml
+search:
+  pg:
+    highlightOptions:
+      useHighlight: false
+```
+
+The Postgres documentation on [Highlighting Results](https://www.postgresql.org/docs/current/textsearch-controls.html#TEXTSEARCH-HEADLINE) has more details.
+
 ## ElasticSearch

 Backstage supports ElasticSearch search engine connections, indexing and
@@ -80,15 +108,16 @@ const searchEngine = await ElasticSearchSearchEngine.initialize({
 const indexBuilder = new IndexBuilder({ logger: env.logger, searchEngine });
 ```

-For the engine to be available, your backend package needs a dependency into
+For the engine to be available, your backend package needs a dependency on
 package `@backstage/plugin-search-backend-module-elasticsearch`.

 ElasticSearch needs some additional configuration before it is ready to use
 within your instance. The configuration options are documented in the
 [configuration schema definition file.](https://github.com/backstage/backstage/blob/master/plugins/search-backend-module-elasticsearch/config.d.ts)

-The underlying functionality is using official ElasticSearch client version 7.x,
-meaning that ElasticSearch version 7 is the only one confirmed to be supported.
+The underlying functionality uses either the official ElasticSearch client
+version 7.x (meaning that ElasticSearch version 7 is the only one confirmed to
+be supported), or the OpenSearch client, when the `aws` provider is configured.

 Should you need to create your own bespoke search experiences that require more
 than just a query translator (such as faceted search or Relay pagination), you
@@ -97,9 +126,19 @@ search clients. The version of the client need not be the same as one used
 internally by the elastic search engine plugin. For example:

 ```typescript
-import { Client } from '@elastic/elastic-search';
+import { isOpenSearchCompatible } from '@backstage/plugin-search-backend-module-elasticsearch';
+import { Client as ElasticClient } from '@elastic/elastic-search';
+import { Client as OpenSearchClient } from '@opensearch-project/opensearch';

-const client = searchEngine.newClient(options => new Client(options));
+const client = searchEngine.newClient(options => {
+  // In reality, you would only import / instantiate one of the following, but
+  // for illustrative purposes, here are both:
+  if (isOpenSearchCompatible(options)) {
+    return new OpenSearchClient(options);
+  } else {
+    return new ElasticClient(options);
+  }
+});
 ```

 #### Set custom index template
@@ -106,8 +106,7 @@ with [Static Location Configuration](#static-location-configuration) or a
 discovery processor like
 [GitHub Discovery](../../integrations/github/discovery.md). To enforce usage of
 processors to locate entities we can configure the catalog into `readonly` mode.
-This configuration disables the mutating backend catalog APIs and disallows
-users from registering new entities at run-time.
+This configuration disables registering and deleting locations with the catalog APIs.

 ```yaml
 catalog:
@@ -117,6 +116,8 @@ catalog:
 > **Note that any plugin relying on the catalog API for creating, updating and
 > deleting entities will not work in this mode.**

+Deleting an entity by UUID, `DELETE /entities/by-uid/:uid`, is allowed when using this mode. It may be rediscovered as noted in [explicit deletion](life-of-an-entity.md#explicit-deletion).
+
 A common use case for this configuration is when organizations have a remote
 source that should be mirrored into Backstage. To make Backstage a mirror of
 this remote source, users cannot also register new entities with e.g. the
@@ -86,6 +86,13 @@ contains more information about the required fields.
 Once we have a `template.yaml` ready, we can then add it to the software catalog
 for use by the scaffolder.

+> Note: When you add or modify a template, you will need to refresh the location entity.
+> Otherwise, Backstage won't display the template in the available templates,
+> or it will keep showing the old template. You can refresh the location instance by
+> going into `Catalog` web page, choosing `Locations` instead of `Components`, and selecting the correct location entity.
+> From there, you can click on the refresh icon representing "Scheduled entity refresh" action.
+> Afterwards, you should see your template updated.
+
 You can add the template files to the catalog through
 [static location configuration](../software-catalog/configuration.md#static-location-configuration),
 for example:
@@ -97,6 +104,8 @@ catalog:
      target: https://github.com/backstage/software-templates/blob/main/scaffolder-templates/react-ssr-template/template.yaml
      rules:
        - allow: [Template]
+    - type: file
+      target: template.yaml # Backstage will expect the file to be in packages/backend/template.yaml
 ```

 Or you can add the template using the `catalog-import` plugin, which unless
@@ -9,8 +9,9 @@ by writing custom actions which can be used along side our
 [built-in actions](./builtin-actions.md).

 > Note: When adding custom actions, the actions array will **replace the
-> built-in actions too**. To ensure you can continue to include the builtin
-> actions, see below to include them during registration of your action.
+> built-in actions too**. Meaning, you will no longer be able to use them.
+> If you want to continue using the builtin actions, include them in the actions
+> array when registering your custom actions, as seen below.

 ## Writing your Custom Action

@@ -122,27 +123,32 @@ will set the available actions that the scaffolder has access to.
 ```ts
 import { createBuiltinActions } from '@backstage/plugin-scaffolder-backend';
 import { ScmIntegrations } from '@backstage/integration';
+import { createNewFileAction } from './actions/custom';

-const integrations = ScmIntegrations.fromConfig(env.config);
+export default async function createPlugin(
+  env: PluginEnvironment,
+): Promise<Router> {
+  const catalogClient = new CatalogClient({ discoveryApi: env.discovery });
+  const integrations = ScmIntegrations.fromConfig(env.config);

-const builtInActions = createBuiltinActions({
-  containerRunner,
-  integrations,
-  catalogClient,
-  config: env.config,
-  reader: env.reader,
-});
+  const builtInActions = createBuiltinActions({
+    integrations,
+    catalogClient,
+    config: env.config,
+    reader: env.reader,
+  });

-const actions = [...builtInActions, createNewFileAction()];
-return await createRouter({
-  containerRunner,
-  catalogClient,
-  actions,
-  logger: env.logger,
-  config: env.config,
-  database: env.database,
-  reader: env.reader,
-});
+  const actions = [...builtInActions, createNewFileAction()];
+
+  return createRouter({
+    actions,
+    catalogClient: catalogClient,
+    logger: env.logger,
+    config: env.config,
+    database: env.database,
+    reader: env.reader,
+  });
+}
 ```

 ## List of custom action packages
@@ -320,6 +320,30 @@ The `allowedHosts` part should be set to where you wish to enable this template
 to publish to. And it can be any host that is listed in your `integrations`
 config in `app-config.yaml`.

+Besides specifying `allowedHosts` you can also restrict the template to publish to
+repositories owned by specific users/groups/namespaces by setting the `allowedOwners`
+option. With the `allowedRepos` option you are able to narrow it down further to a
+specific set of repository names. A full example could look like this:
+
+```yaml
+- title: Choose a location
+  required:
+    - repoUrl
+  properties:
+    repoUrl:
+      title: Repository Location
+      type: string
+      ui:field: RepoUrlPicker
+      ui:options:
+        allowedHosts:
+          - github.com
+        allowedOwners:
+          - backstage
+          - someGithubUser
+        allowedRepos:
+          - backstage
+```
+
 The `RepoUrlPicker` is a custom field that we provide part of the
 `plugin-scaffolder`. You can provide your own custom fields by
 [writing your own Custom Field Extensions](./writing-custom-field-extensions.md)
@@ -485,6 +509,64 @@ the value of `firstName` from the parameters). This is great for passing the
 values from the form into different steps and reusing these input variables.
 These template strings preserve the type of the parameter.

+The `${{ parameters.firstName }}` pattern will work only in the template file.
+If you want to start using values provided from the UI in your code, you will have to use
+the `${{ values.firstName }}` pattern. Additionally, you have to pass
+the parameters from the UI to the input of the `fetch:template` step.
+
+```yaml
+apiVersion: scaffolder.backstage.io/v1beta3
+kind: Template
+metadata:
+  name: v1beta3-demo
+  title: Test Action
+  description: scaffolder v1beta3 template demo
+spec:
+  owner: backstage/techdocs-core
+  type: service
+  parameters:
+    - title: Fill in some steps
+      required:
+        - name
+      properties:
+        name:
+          title: Name
+          type: string
+          description: Unique name of your project
+        urlParameter:
+          title: URL endpoint
+          type: string
+          description: URL endpoint at which the component can be reached
+          default: 'https://www.example.com'
+        enabledDB:
+          title: Enable Database
+          type: boolean
+          default: false
+  ...
+  steps:
+    - id: fetch-base
+      name: Fetch Base
+      action: fetch:template
+      input:
+        url: ./template
+        values:
+          name: ${{ parameters.name }}
+          url: ${{ parameters.urlParameter }}
+          enabledDB: ${{ parameters.enabledDB }}
+```
+
+Afterwards, if you are using the builtin templating action, you can start using
+the variables in your code. You can use also any other templating functions from
+[Nunjucks](https://mozilla.github.io/nunjucks/templating.html#tags) as well.
+
+```bash
+#!/bin/bash
+echo "Hi my name is ${{ values.name }}, and you can fine me at ${{ values.url }}!"
+{% if values.enabledDB %}
+echo "You have enabled your database!"
+{% endif %}
+```
+
 As you can see above in the `Outputs` section, `actions` and `steps` can also
 output things. You can grab that output using `steps.$stepId.output.$property`.

@@ -52,6 +52,8 @@ Addons are rendered in the order in which they are registered.

 ## Installing and using Addons

+To start using Addons you need to add the `@backstage/plugin-techdocs-module-addons-contrib` package to your app. You can do that by running this command from the root of your project: `yarn add --cwd packages/app @backstage/plugin-techdocs-module-addons-contrib`
+
 Addons can be installed and configured in much the same way as extensions for
 other Backstage plugins: by adding them underneath an extension registry
 component (`<TechDocsAddons>`) under the route representing the TechDocs Reader
@@ -74,6 +76,18 @@ import { ReportIssue } from '@backstage/plugin-techdocs-module-addons-contrib';
 </Route>;
 ```

+If you are using a custom [TechDocs reader page](./how-to-guides.md#how-to-customize-the-techdocs-reader-page) your setup will be very similar, here's an example:
+
+```ts
+<Route path="/docs/:namespace/:kind/:name/*" element={<TechDocsReaderPage />}>
+  <TechDocsAddons>
+    <ReportIssue />
+    {/* Other addons can be added here. */}
+  </TechDocsAddons>
+  {techDocsPage} // This is your custom TechDocs reader page
+</Route>
+```
+
 The process for configuring Addons on the documentation tab on the entity page
 is very similar; instead of adding the `<TechDocsAddons>` registry under a
 `<Route>`, you'd add it as a child of `<EntityTechdocsContent />`:
@@ -160,7 +160,8 @@ techdocs:

  # techdocs.cache is optional, and is only recommended when you've configured
  # an external techdocs.publisher.type above. Also requires backend.cache to
-  # be configured with a valid cache store.
+  # be configured with a valid cache store. Configure techdocs.cache.ttl to
+  # enable caching of techdocs assets.
  cache:
    # Represents the number of milliseconds a statically built asset should
    # stay cached. Cache invalidation is handled automatically by the frontend,