WIP documentation for the V1 Search Milestone

Signed-off-by: Eric Peterson <ericpeterson@spotify.com>
2021-05-18 14:54:01 +02:00
parent e3c18ec501
commit f86171027e
2 changed files with 243 additions and 0 deletions
@@ -0,0 +1,88 @@
+---
+id: concepts
+title: Search Concepts
+description: Documentation on Backstage Search Concepts
+---
+
+# Search Concepts
+
+Backstage Search is _blah_.
+
+To get started, you should get familiar with these core concepts:
+
+- Search Engines
+- Query Translators
+- Documents and Indices
+- Collators
+- Decorators
+- The Scheduler
+
+### Search Engines
+
+Backstage Search isn't a search engine itself, rather, it provides an interface
+between your Backstage instance and a Search Engine of your choice. More
+concretely, a `SearchEngine` is an interface whose concrete implementations
+facilitate communication with different search engines (like ElasticSearch,
+Solr, Algolia, etc). This abstraction exists in order to support your
+organization's needs.
+
+Out of the box, Backstage Search comes pre-packaged with an in-memory search
+engine implementation built on top of Lunr.
+
+### Query Translators
+
+Because you can bring your own search engine, and because search engines have
+very unique and robust query languages themselves, there needs to be a
+translation layer between an abstract search query (containing search terms,
+filters, and document types) into a concrete search query that is specific to a
+search engine.
+
+Search Engines come pre-packaged with simple translators that do rudimentary
+transformations of search terms and filters, but you may want to provide your
+own to help tune seearch results in the context of your organization.
+
+### Documents and Indices
+
+"Document" is an abstract concept representing something that can be found by
+searching for it. A document can represent a software entity, a TechDocs page,
+etc. Documents are made up of metadata fields, at a minimum including a title,
+text, and location (as in a URL).
+
+An index is a collection of such documents of a given type.
+
+### Collators
+
+You need to be able to search something! Collators are the way to define what
+can be searched. Specifically, they're classes which return documents conforming
+to a minimum set of fields (including a document title, location, and text), but
+which can contain any other fields as defined by the collator itself. One
+collator is responsible for defining and collecting documents of a type.
+
+Some plugins, like the Catalog Backend, provide so-called "default" collators
+which you can use out-of-the-box to start searching across Backstage quickly.
+
+### Decorators
+
+Sometimes you want to add extra information to a set of documents in your search
+index that the collator may not be aware of. For example, the Software Catalog
+knows about software entities, but it may not know about their usage or quality.
+
+Decorators are classes which can add extra fields to pre-collated documents.
+This extra metadata could then be used to bias search results or otherwise
+improve the search experience in your Backstage instance.
+
+### The Scheduler
+
+There are many ways a search index could be built and maintained, but Backstage
+Search chooses to completely rebuild indices on a schedule. Different collators
+can be configured to refresh at different intervals, depending on how often the
+source information is updated.
+
+TODO: There should probably be some front-end concepts here too?
+
+- Search Page
+- Search Components
+- Search Context
+
+[Primarily solves for “As an App Integrator, I should know how to add do uments
+to the search index”]
@@ -0,0 +1,155 @@
+---
+id: getting-started
+title: Getting Started with Search
+description: How to set up and install Backstage Search
+---
+
+# Getting Started
+
+Search functions as a plugin to Backstage, so you will need to use Backstage to
+use Search.
+
+If you haven't setup Backstage already, start
+[here](../../getting-started/index.md).
+
+> If you used `npx @backstage/create-app`, Search may already be present.
+>
+> You should skip to [`Customizing Search`](#customizing-search) below.
+
+## Adding Search to the Frontend
+
+```bash
+# From your Backstage root directory
+cd packages/app
+yarn add @backstage/plugin-search
+```
+
+TODO: Add frontend instructions here.
+
+## Adding Search to the Backend
+
+Add the following plugins into your backend app:
+
+```bash
+# From your Backstage root directory
+cd packages/backend
+yarn add @backstage/plugin-search-backend @backstage/plugin-search-backend-node
+```
+
+Create a `packages/backend/src/plugins/search.ts` file containing the following
+code:
+
+```typescript
+import { useHotCleanup } from '@backstage/backend-common';
+import { createRouter } from '@backstage/plugin-search-backend';
+import {
+  IndexBuilder,
+  LunrSearchEngine,
+} from '@backstage/plugin-search-backend-node';
+import { PluginEnvironment } from '../types';
+import { DefaultCatalogCollator } from '@backstage/plugin-catalog-backend';
+
+export default async function createPlugin({
+  logger,
+  discovery,
+}: PluginEnvironment) {
+  const searchEngine = new LunrSearchEngine({ logger });
+  const indexBuilder = new IndexBuilder({ logger, searchEngine });
+
+  indexBuilder.addCollator({
+    type: 'software-catalog',
+    defaultRefreshIntervalSeconds: 600,
+    collator: new DefaultCatalogCollator({ discovery }),
+  });
+
+  const { scheduler } = await indexBuilder.build();
+
+  scheduler.start();
+  useHotCleanup(module, () => scheduler.stop());
+
+  return await createRouter({
+    engine: indexBuilder.getSearchEngine(),
+    logger,
+  });
+}
+```
+
+Make the following modifications to your `packages/backend/src/index.ts` file:
+
+Import the `plugins/search` file you created above:
+
+```typescript
+import search from './plugins/search';
+```
+
+Set up an environment for search:
+
+```typescript
+const searchEnv = useHotMemoize(module, () => createEnv('search'));
+```
+
+Register the search service with the router:
+
+```typescript
+apiRouter.use('/search', await search(searchEnv));
+```
+
+## Customizing Search
+
+Backstage Search isn't a search engine itself, rather, it provides an interface
+between your Backstage instance and a Search Engine of your choice. Currently,
+we only support one, in-memory search Engine called Lunr. It can be instantiated
+like this:
+
+```typescript
+const searchEngine = new LunrSearchEngine({ logger });
+const indexBuilder = new IndexBuilder({ logger, searchEngine });
+```
+
+Backstage Search can be used to power search of anything! Plugins like the
+Catalog offer default "collators" which are responsible for providing documents
+to be indexed. You can register any number of collators with the `IndexBuilder`
+like this:
+
+```typescript
+const indexBuilder = new IndexBuilder({ logger, searchEngine });
+
+indexBuilder.addCollator({
+  type: 'software-catalog',
+  defaultRefreshIntervalSeconds: 600,
+  collator: new DefaultCatalogCollator({ discovery }),
+});
+
+indexBuilder.addCollator({
+  type: 'my-custom-stuff',
+  defaultRefreshIntervalSeconds: 3600,
+  collator: new MyCustomCollator(),
+});
+```
+
+Backstage Search builds and maintains its index on a schedule. You can change
+how often the indexes are rebuilt for a given type of document. You may want to
+do this if your documents are updated more or less frequently. You can do so by
+modifying its `defaultRefreshIntervalSeconds` value, like this:
+
+```typescript {3}
+indexBuilder.addCollator({
+  type: 'software-catalog',
+  defaultRefreshIntervalSeconds: 600,
+  collator: new DefaultCatalogCollator({ discovery }),
+});
+```
+
+---
+
+[Primarily solves for “As an App Integrator, I should be able to install out-of
+the-box Search”]
+
+Backend: Different for new Backstage instances (created via create-app) vs.
+people enabling search on existing Backstage instances (see catalog install for
+example).
+
+Frontend: Search Page Route…? Customizing result components…?
+
+Conceptually… What’s the difference between “out-of-the-box” or “basic” and the
+recommended deployment (which will come)