Merge branch 'master' of github.com:spotify/backstage into ndudnik/validate-locations

2020-05-25 18:35:12 +02:00
parent 8e4e6d0aba e1f31ed95a
commit ca63580518
38 changed files with 1238 additions and 512 deletions
@@ -1,6 +1,5 @@
 .git
 node_modules
 packages/*/node_modules
-packages/*/dist
 plugins/*/node_modules
 plugins/*/dist
@@ -7,6 +7,7 @@ The value of Backstage grows with every new plugin that gets added. Here is a co
 - [Development Environment](development-environment.md)
 - [Create a Backstage plugin](create-a-plugin.md)
 - [Structure of a plugin](structure-of-a-plugin.md)
+- [Utility APIs](utility-apis.md)
 - Using Backstage components (TODO)

 [Back to Docs](../README.md)
@@ -0,0 +1,140 @@
+# Utility APIs
+
+## Introduction
+
+Backstage Plugins strive to be self-contained, with as much functionality
+as possible residing within the plugin itself and its backend APIs. There will however
+always be a need for plugins to communicate outside of its boundaries, both with other
+plugins and the app itself.
+
+Backstage provides two primary methods for plugins to communication across
+their boundaries in client-side code. The first one being the `createPlugin` API and the registration hooks
+passed to the `register` method, and the second one being Utility APIs.
+While the `createPlugin` API is focused on the initialization plugins and the app,
+the Utility APIs provide ways for plugins to communicate during their entire life cycle.
+
+## Usage
+
+Each Utility API is tied to an `ApiRef` instance, which is a global singleton object
+without any additional state or functionality, its only purpose is to reference Utility APIs.
+`ApiRef`s are create using `createApiRef`, which is exported by `@backstage/core`.
+There are many predefined Utility APIs defined in `@backstage/core`, and they're all exported with a name of the pattern `*ApiRef`, for example `errorApiRef`.
+
+To access one of the Utility APIs inside a React component, use the `useApi` hook exported by
+`@backstage/core`, or the `withApis` HOC if you prefer class components. For example, the `ErrorApi` can be accessed like this:
+
+```tsx
+import React, { FC } from 'react';
+import { useApi, errorApiRef } from '@backstage/core';
+
+export const MyComponent: FC<{}> = () => {
+  const errorApi = useApi(errorApiRef);
+
+  // Signal to the app that something went wrong, and display the error to the user.
+  const handleError = error => {
+    errorApi.post(error);
+  };
+
+  // the rest of the component ...
+};
+```
+
+Note that there is no explicit type given for `ErrorApi`. This is because the `errorApiRef` has the type embedded, and `useApi` is able to infer the type.
+
+Also note that consuming Utility APIs is not limited to plugins, it can be done from any component inside Backstage, including the ones in `@backstage/core`. The only requirement is that they are beneath the `AppProvider` in the react tree.
+
+## Registering Utility API Implementations
+
+The Backstage App is responsible for providing implementations for all Utility APIs required by plugins. The example app in this repo registers its APIs inside [src/apis.ts](/packages/app/src/apis.ts). Here's an example of how to wire up the `ErrorApi` inside an app:
+
+```ts
+import {
+  ApiRegistry,
+  createApp,
+  alertApiRef,
+  errorApiRef,
+  AlertApiForwarder,
+  ErrorApiForwarder,
+} from '@backstage/core';
+
+const builder = ApiRegistry.builder();
+
+// The alert API is a self-contained implementation that shows alerts to the user.
+builder.add(alertApiRef, new AlertApiForwarder());
+
+// The error API uses the alert API to send error notifications to the user.
+builder.add(errorApiRef, new ErrorApiForwarder(alertApiForwarder));
+
+const app = createApp({
+  apis: apiBuilder.build(),
+  // ... other config
+});
+```
+
+The `ApiRegistry` is used to register all Utility APIs in the app and associate them with `ApiRef`s. It implements the `ApiHolder` interface, which enables it to provide an API implementation given an `ApiRef`.
+
+Note that our `ErrorApi` implementation depends on another Utility API, the `AlertApi`. This is the method with which APIs can depend on other APIs, using manual dependency injection at the initialization of the app. In general, if you want
+to depend on another Utility API in an implementation of an API, you import the type for that API and make it a
+constructor parameter.
+
+## Custom implementations of Utility APIs
+
+Defining a custom implementation of a utility API is easy, you simply need to export a class
+that `implements` the target API, for example:
+
+```ts
+export class IgnoringErrorApi implements ErrorApi {
+  post(error: Error, context?: ErrorContext) {
+    // ignore error
+  }
+}
+```
+
+The `IgnoringErrorApi` would then be imported in the app, and wired up like this:
+
+```ts
+builder.add(errorApiRef, new IgnoringErrorApi());
+```
+
+Note that the above line will cause an error if `IgnoreErrorApi` does not fully
+implement the `ErrorApi`, as it is checked by the type embedded in the `errorApiRef` at compile time.
+
+## Defining custom Utility APIs
+
+The pattern for plugins defining their own Utility APIs is not fully established yet. The current way is for the plugin to export its own `ApiRef` and type for the API, along with one or more implementations. It is then up to the app to import, and register those APIs. See for example the [lighthouse](/plugins/lighthouse/src/api.ts) or [graphiql](/plugins/graphiql/src/lib/api/types.ts) plugins for examples of this.
+
+The goal is to make this process a bit smoother, but that requires work in other parts of Backstage, like configuration management. So it remains as a TODO. If you have more questions regarding this, or have an idea for an API that you want to share outside your plugin, hit us up in [GitHub issues](https://github.com/spotify/backstage/issues/new/choose) or the [Backstage Discord server](https://discord.gg/EBHEGzX).
+
+## Architecture
+
+The `ApiRef` instances mentioned above provide a point of indirection between consumers and producers of
+Utility APIs. It allows for plugins and components to depend on APIs in a type-safe way, without
+having a direct reference to a concrete implementation of the APIs. The Apps are also given a lot
+of flexibility in what implementations to provide. As long as they adhere to the contract established
+by an `ApiRef`, they are free to choose any implementation they want.
+
+The figure below shows the relationship between <span style="color: #82b366">different Apps</span>, that
+provide <span style="color: #6c8ebf">different implementations</span> of the
+<span style="color: #9673a6">FooApi</span>. <span style="color: #d6b656">Components</span> within Plugins
+then access the <span style="color: #9673a6">FooApi</span> via the <span style="color: #b85450">fooApiRef</span>.
+
+<div style="text-align:center">
+<img src="utility-apis-fig1.svg" alt="Figure showing the relationship between utility APIs, the apps that provide them, and the plugins that consume them">
+</div>
+
+The current method for connecting Utility API providers and consumers is via the React tree using
+an `ApiProvider`, which is added to the `AppProvider` of the `App`. In the future there may potentially
+be more ways to do this, in ways that are not tied to react. A design goal of the Utility APIs was to
+not have them directly tied to React.
+
+The indirection provided by Utility APIs also makes it straightforward to test components that depend
+on APIs, and to provide a standard common development environment for plugins. A proper test wrapper
+with mocked API implementations is not yet ready, but it will provided as a part of
+`@backstage/test-utils`. It will provide mocked variants of APIs, with additional methods for asserting
+a component's interaction with the API.
+
+The common development environment for plugins is included in `@backstage/dev-utils`, where the exported
+`createDevApp` function creates an application with implementations for all core APIs already present.
+Contrary to the method for wiring up Utility API implementations in an app created with `createApp`, `createDevApp`
+uses automatic dependency injection. This is to make it possible to replace any API implementation, and
+having that be reflected in dependents of that API.
@@ -15,3 +15,4 @@
 */

 export * from './rootLogger';
+export * from './voidLogger';
@@ -14,15 +14,14 @@
 * limitations under the License.
 */

-import { ReaderOutput } from '../types';
+import { PassThrough } from 'stream';
+import winston, { Logger } from 'winston';

-export type LocationSource = {
-  /**
-   * Reads the contents of a single location.
-   *
-   * @param target The location target to read
-   * @returns The parsed contents, as an array of unverified descriptors
-   * @throws An error if the location target could not be read
-   */
-  read(target: string): Promise<ReaderOutput[]>;
-};
+/**
+ * A logger that just throws away all messages.
+ */
+export function getVoidLogger(): Logger {
+  return winston.createLogger({
+    transports: [new winston.transports.Stream({ stream: new PassThrough() })],
+  });
+}
@@ -29,7 +29,7 @@ export default async function ({ logger, database }: PluginEnvironment) {
  const reader = LocationReaders.create();
  const parser = DescriptorParsers.create();

-  const db = await DatabaseManager.createDatabase(database);
+  const db = await DatabaseManager.createDatabase(database, logger);
  runPeriodically(
    () => DatabaseManager.refreshLocations(db, reader, parser, logger),
    10000,
@@ -0,0 +1,74 @@
+/*
+ * Copyright 2020 Spotify AB
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { Command } from 'commander';
+import {
+  diffTemplateFiles,
+  handlers,
+  handleAllFiles,
+  inquirerPromptFunc,
+  makeCheckPromptFunc,
+  yesPromptFunc,
+} from '../../lib/diff';
+import { version } from '../../lib/version';
+
+const fileHandlers = [
+  {
+    patterns: ['packages/app/package.json'],
+    handler: handlers.appPackageJson,
+  },
+  {
+    patterns: [/tsconfig\.json$/],
+    handler: handlers.exactMatch,
+  },
+  {
+    patterns: [
+      /README\.md$/,
+      /\.eslintrc\.js$/,
+      // make sure files in 1st level of src/ and dev/ exist
+      /^packages\/app\/(src|dev)\/[^/]+$/,
+    ],
+    handler: handlers.exists,
+  },
+  {
+    patterns: [
+      'lerna.json',
+      /^src\//,
+      /^patches\//,
+      /^packages\/app\/public\//,
+      /^packages\/app\/cypress/,
+      // Let plugin:diff take care of the plugins
+      /^plugins/,
+      /package\.json$/,
+    ],
+    handler: handlers.skip,
+  },
+];
+
+export default async (cmd: Command) => {
+  let promptFunc = inquirerPromptFunc;
+  let finalize = () => {};
+
+  if (cmd.check) {
+    [promptFunc, finalize] = makeCheckPromptFunc();
+  } else if (cmd.yes) {
+    promptFunc = yesPromptFunc;
+  }
+
+  const templateFiles = await diffTemplateFiles('default-app', { version });
+  await handleAllFiles(fileHandlers, templateFiles, promptFunc);
+  await finalize();
+};
@@ -0,0 +1,97 @@
+/*
+ * Copyright 2020 Spotify AB
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import fs from 'fs-extra';
+import { Command } from 'commander';
+import {
+  diffTemplateFiles,
+  handlers,
+  handleAllFiles,
+  inquirerPromptFunc,
+  makeCheckPromptFunc,
+  yesPromptFunc,
+} from '../../lib/diff';
+import { paths } from '../../lib/paths';
+import { version } from '../../lib/version';
+
+export type PluginData = {
+  id: string;
+  name: string;
+};
+
+const fileHandlers = [
+  {
+    patterns: ['package.json'],
+    handler: handlers.packageJson,
+  },
+  {
+    patterns: ['tsconfig.json'],
+    handler: handlers.exactMatch,
+  },
+  {
+    // make sure files in 1st level of src/ and dev/ exist
+    patterns: ['.eslintrc.js', /^(src|dev)\/[^/]+$/],
+    handler: handlers.exists,
+  },
+  {
+    patterns: ['README.md', /^src\//],
+    handler: handlers.skip,
+  },
+];
+
+export default async (cmd: Command) => {
+  let promptFunc = inquirerPromptFunc;
+  let finalize = () => {};
+
+  if (cmd.check) {
+    [promptFunc, finalize] = makeCheckPromptFunc();
+  } else if (cmd.yes) {
+    promptFunc = yesPromptFunc;
+  }
+
+  const data = await readPluginData();
+  const templateFiles = await diffTemplateFiles('default-plugin', {
+    version,
+    ...data,
+  });
+  await handleAllFiles(fileHandlers, templateFiles, promptFunc);
+  await finalize();
+};
+
+// Reads templating data from the existing plugin
+async function readPluginData(): Promise<PluginData> {
+  let name: string;
+  try {
+    const pkg = require(paths.resolveTarget('package.json'));
+    name = pkg.name;
+  } catch (error) {
+    throw new Error(`Failed to read target package, ${error}`);
+  }
+
+  const pluginTsContents = await fs.readFile(
+    paths.resolveTarget('src/plugin.ts'),
+    'utf8',
+  );
+  // TODO: replace with some proper parsing logic or plugin metadata file
+  const pluginIdMatch = pluginTsContents.match(/id: ['"`](.+?)['"`]/);
+  if (!pluginIdMatch) {
+    throw new Error(`Failed to parse plugin.ts, no plugin ID found`);
+  }
+
+  const id = pluginIdMatch[1];
+
+  return { id, name };
+}
@@ -1,112 +0,0 @@
-/*
- * Copyright 2020 Spotify AB
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import fs from 'fs-extra';
-import { relative as relativePath } from 'path';
-import handlebars from 'handlebars';
-import recursiveReadDir from 'recursive-readdir';
-import { paths } from '../../../lib/paths';
-import { version } from '../../../lib/version';
-import { PluginInfo, TemplateFile } from './types';
-
-// Reads info from the existing plugin
-export async function readPluginInfo(): Promise<PluginInfo> {
-  let name: string;
-  try {
-    const pkg = require(paths.resolveTarget('package.json'));
-    name = pkg.name;
-  } catch (error) {
-    throw new Error(`Failed to read target package, ${error}`);
-  }
-
-  const pluginTsContents = await fs.readFile(
-    paths.resolveTarget('src/plugin.ts'),
-    'utf8',
-  );
-  // TODO: replace with some proper parsing logic or plugin metadata file
-  const pluginIdMatch = pluginTsContents.match(/id: ['"`](.+?)['"`]/);
-  if (!pluginIdMatch) {
-    throw new Error(`Failed to parse plugin.ts, no plugin ID found`);
-  }
-
-  const id = pluginIdMatch[1];
-
-  return { id, name };
-}
-
-export async function readTemplateFile(
-  templateFile: string,
-  templateVars: any,
-): Promise<string> {
-  const contents = await fs.readFile(templateFile, 'utf8');
-
-  if (!templateFile.endsWith('.hbs')) {
-    return contents;
-  }
-
-  return handlebars.compile(contents)(templateVars);
-}
-
-export async function readTemplate(
-  templateDir: string,
-  templateVars: any,
-): Promise<TemplateFile[]> {
-  const templateFilePaths = await recursiveReadDir(templateDir).catch(
-    (error) => {
-      throw new Error(`Failed to read template directory: ${error.message}`);
-    },
-  );
-
-  const templateFiles = new Array<TemplateFile>();
-  for (const templateFile of templateFilePaths) {
-    // Target file inside the target dir without template extension
-    const targetFile = templateFile
-      .replace(templateDir, paths.targetDir)
-      .replace(/\.hbs$/, '');
-    const targetPath = relativePath(paths.targetDir, targetFile);
-
-    const templateContents = await readTemplateFile(templateFile, templateVars);
-
-    const targetExists = await fs.pathExists(targetFile);
-    if (targetExists) {
-      const targetContents = await fs.readFile(targetFile, 'utf8');
-      templateFiles.push({
-        targetPath,
-        targetExists,
-        targetContents,
-        templateContents,
-      });
-    } else {
-      templateFiles.push({
-        targetPath,
-        targetExists,
-        templateContents,
-      });
-    }
-  }
-
-  return templateFiles;
-}
-
-// Read all template files for a given template, along with all matching files in the target dir
-export async function readTemplateFiles(template: string) {
-  const pluginInfo = await readPluginInfo();
-  const templateVars = { version, ...pluginInfo };
-
-  const templateDir = paths.resolveOwn('templates', template);
-
-  return await readTemplate(templateDir, templateVars);
-}
@@ -39,6 +39,13 @@ const main = (argv: string[]) => {
    .option('--check', 'Enable type checking and linting')
    .action(actionHandler(() => require('./commands/app/serve')));

+  program
+    .command('app:diff')
+    .option('--check', 'Fail if changes are required')
+    .option('--yes', 'Apply all changes')
+    .description('Diff an existing app with the creation template')
+    .action(actionHandler(() => require('./commands/app/diff')));
+
  program
    .command('create-plugin')
    .description('Creates a new plugin in the current repository')
@@ -157,7 +164,7 @@ function actionHandler<T extends readonly any[]>(
  };
 }

-process.on('unhandledRejection', (rejection) => {
+process.on('unhandledRejection', rejection => {
  if (rejection instanceof Error) {
    exitWithError(rejection);
  } else {
@@ -14,43 +14,52 @@
 * limitations under the License.
 */

-import fs from 'fs-extra';
 import chalk from 'chalk';
-import { dirname } from 'path';
 import { diffLines } from 'diff';
-import { paths } from '../../../lib/paths';
-import { TemplateFile, PromptFunc, FileHandler } from './types';
-
-export async function writeTargetFile(targetPath: string, contents: string) {
-  const path = paths.resolveTarget(targetPath);
-  await fs.ensureDir(dirname(path));
-  await fs.writeFile(path, contents, 'utf8');
-}
+import { FileDiff, PromptFunc, FileHandler, WriteFileFunc } from './types';

 class PackageJsonHandler {
-  static async handler(file: TemplateFile, prompt: PromptFunc) {
+  static async handler(
+    { path, write, missing, targetContents, templateContents }: FileDiff,
+    prompt: PromptFunc,
+    variant?: string,
+  ) {
    console.log('Checking package.json');

-    if (!file.targetExists) {
-      throw new Error(`${file.targetPath} doesn't exist`);
+    if (missing) {
+      throw new Error(`${path} doesn't exist`);
    }

-    const pkg = JSON.parse(file.templateContents);
-    const targetPkg = JSON.parse(file.targetContents);
+    const pkg = JSON.parse(templateContents);
+    const targetPkg = JSON.parse(targetContents);

-    const handler = new PackageJsonHandler(file, prompt, pkg, targetPkg);
+    const handler = new PackageJsonHandler(
+      write,
+      prompt,
+      pkg,
+      targetPkg,
+      variant,
+    );
    await handler.handle();
  }

+  static async appHandler(file: FileDiff, prompt: PromptFunc) {
+    return PackageJsonHandler.handler(file, prompt, 'app');
+  }
+
  constructor(
-    private readonly file: TemplateFile,
+    private readonly writeFunc: WriteFileFunc,
    private readonly prompt: PromptFunc,
    private readonly pkg: any,
    private readonly targetPkg: any,
+    private readonly variant?: string,
  ) {}

  async handle() {
    await this.syncField('main');
+    if (this.variant !== 'app') {
+      await this.syncField('main:src');
+    }
    await this.syncField('types');
    await this.syncField('files');
    await this.syncScripts();
@@ -84,7 +93,7 @@ class PackageJsonHandler {
        targetObj[fieldName] = newValue;
        await this.write();
      }
-    } else {
+    } else if (fieldName in obj) {
      if (
        await this.prompt(
          `package.json is missing field ${fullFieldName}, set to ${coloredNewValue}?`,
@@ -101,6 +110,10 @@ class PackageJsonHandler {
    const targetScripts = (this.targetPkg.scripts =
      this.targetPkg.scripts || {});

+    if (!pkgScripts) {
+      return;
+    }
+
    for (const key of Object.keys(pkgScripts)) {
      await this.syncField(key, pkgScripts, targetScripts, 'scripts');
    }
@@ -138,37 +151,42 @@ class PackageJsonHandler {
    const targetDeps = (this.targetPkg[fieldName] =
      this.targetPkg[fieldName] || {});

+    if (!pkgDeps) {
+      return;
+    }
+
    for (const key of Object.keys(pkgDeps)) {
+      if (this.variant === 'app' && key.startsWith('plugin-')) {
+        continue;
+      }
      await this.syncField(key, pkgDeps, targetDeps, fieldName);
    }
  }

  private async write() {
-    await fs.writeFile(
-      paths.resolveTarget(this.file.targetPath),
-      `${JSON.stringify(this.targetPkg, null, 2)}\n`,
-    );
+    await this.writeFunc(`${JSON.stringify(this.targetPkg, null, 2)}\n`);
  }
 }

 // Make sure the file is an exact match of the template
-async function exactMatchHandler(file: TemplateFile, prompt: PromptFunc) {
-  console.log(`Checking ${file.targetPath}`);
+async function exactMatchHandler(
+  { path, write, missing, targetContents, templateContents }: FileDiff,
+  prompt: PromptFunc,
+) {
+  console.log(`Checking ${path}`);
+  const coloredPath = chalk.cyan(path);

-  const { targetPath, templateContents } = file;
-  const coloredPath = chalk.cyan(targetPath);
-
-  if (!file.targetExists) {
+  if (missing) {
    if (await prompt(`Missing ${coloredPath}, do you want to add it?`)) {
-      await writeTargetFile(targetPath, templateContents);
+      await write(templateContents);
    }
    return;
  }
-  if (file.targetContents === templateContents) {
+  if (targetContents === templateContents) {
    return;
  }

-  const diffs = diffLines(file.targetContents, templateContents);
+  const diffs = diffLines(targetContents, templateContents);
  for (const diff of diffs) {
    if (diff.added) {
      process.stdout.write(chalk.green(`+${diff.value}`));
@@ -184,27 +202,29 @@ async function exactMatchHandler(file: TemplateFile, prompt: PromptFunc) {
      `Outdated ${coloredPath}, do you want to apply the above patch?`,
    )
  ) {
-    await writeTargetFile(targetPath, templateContents);
+    await write(templateContents);
  }
 }

 // Adds the file if it is missing, but doesn't check existing files
-async function existsHandler(file: TemplateFile, prompt: PromptFunc) {
-  console.log(`Making sure ${file.targetPath} exists`);
+async function existsHandler(
+  { path, write, missing, templateContents }: FileDiff,
+  prompt: PromptFunc,
+) {
+  console.log(`Making sure ${path} exists`);

-  const { targetPath, templateContents } = file;
-  const coloredPath = chalk.cyan(targetPath);
+  const coloredPath = chalk.cyan(path);

-  if (!file.targetExists) {
+  if (missing) {
    if (await prompt(`Missing ${coloredPath}, do you want to add it?`)) {
-      await writeTargetFile(targetPath, templateContents);
+      await write(templateContents);
    }
    return;
  }
 }

-async function skipHandler(file: TemplateFile) {
-  console.log(`Skipping ${file.targetPath}`);
+async function skipHandler({ path }: FileDiff) {
+  console.log(`Skipping ${path}`);
 }

 export const handlers = {
@@ -212,26 +232,25 @@ export const handlers = {
  exists: existsHandler,
  exactMatch: exactMatchHandler,
  packageJson: PackageJsonHandler.handler,
+  appPackageJson: PackageJsonHandler.appHandler,
 };

 export async function handleAllFiles(
  fileHandlers: FileHandler[],
-  files: TemplateFile[],
+  files: FileDiff[],
  promptFunc: PromptFunc,
 ) {
  for (const file of files) {
-    const { targetPath } = file;
-    const fileHandler = fileHandlers.find((handler) =>
-      handler.patterns.some((pattern) =>
-        typeof pattern === 'string'
-          ? pattern === targetPath
-          : pattern.test(targetPath),
+    const { path } = file;
+    const fileHandler = fileHandlers.find(handler =>
+      handler.patterns.some(pattern =>
+        typeof pattern === 'string' ? pattern === path : pattern.test(path),
      ),
    );
    if (fileHandler) {
      await fileHandler.handler(file, promptFunc);
    } else {
-      throw new Error(`No template file handler found for ${targetPath}`);
+      throw new Error(`No template file handler found for ${path}`);
    }
  }
 }
@@ -0,0 +1,20 @@
+/*
+ * Copyright 2020 Spotify AB
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+export * from './handlers';
+export * from './prompts';
+export * from './read';
+export * from './types';
@@ -16,32 +16,9 @@

 import chalk from 'chalk';
 import inquirer from 'inquirer';
-import { Command } from 'commander';
-import { readTemplateFiles } from './read';
-import { handlers, handleAllFiles } from './handlers';
 import { PromptFunc } from './types';

-const fileHandlers = [
-  {
-    patterns: ['package.json'],
-    handler: handlers.packageJson,
-  },
-  {
-    patterns: ['tsconfig.json'],
-    handler: handlers.exactMatch,
-  },
-  {
-    // make sure files in 1st level of src/ and dev/ exist
-    patterns: ['.eslintrc.js', /^(src|dev)\/[^/]+$/],
-    handler: handlers.exists,
-  },
-  {
-    patterns: ['README.md', /^src\//],
-    handler: handlers.skip,
-  },
-];
-
-const inquirerPromptFunc: PromptFunc = async (msg) => {
+export const inquirerPromptFunc: PromptFunc = async msg => {
  const { result } = await inquirer.prompt({
    type: 'confirm',
    name: 'result',
@@ -50,10 +27,10 @@ const inquirerPromptFunc: PromptFunc = async (msg) => {
  return result;
 };

-const makeCheck = () => {
+export const makeCheckPromptFunc = () => {
  let failed = false;

-  const promptFunc: PromptFunc = async (msg) => {
+  const promptFunc: PromptFunc = async msg => {
    failed = true;
    console.log(chalk.red(`[Check Failed] ${msg}`));
    return false;
@@ -70,22 +47,7 @@ const makeCheck = () => {
  return [promptFunc, finalize] as const;
 };

-const yesPromptFunc: PromptFunc = async (msg) => {
+export const yesPromptFunc: PromptFunc = async msg => {
  console.log(`Accepting: "${msg}"`);
  return true;
 };
-
-export default async (cmd: Command) => {
-  let promptFunc = inquirerPromptFunc;
-  let finalize = () => {};
-
-  if (cmd.check) {
-    [promptFunc, finalize] = makeCheck();
-  } else if (cmd.yes) {
-    promptFunc = yesPromptFunc;
-  }
-
-  const templateFiles = await readTemplateFiles('default-plugin');
-  await handleAllFiles(fileHandlers, templateFiles, promptFunc);
-  await finalize();
-};
@@ -0,0 +1,109 @@
+/*
+ * Copyright 2020 Spotify AB
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import fs from 'fs-extra';
+import {
+  dirname,
+  resolve as resolvePath,
+  relative as relativePath,
+} from 'path';
+import handlebars from 'handlebars';
+import recursiveReadDir from 'recursive-readdir';
+import { paths } from '../paths';
+import { FileDiff } from './types';
+
+export type TemplatedFile = {
+  path: string;
+  contents: string;
+};
+
+async function readTemplateFile(
+  templateFile: string,
+  templateVars: any,
+): Promise<string> {
+  const contents = await fs.readFile(templateFile, 'utf8');
+
+  if (!templateFile.endsWith('.hbs')) {
+    return contents;
+  }
+
+  return handlebars.compile(contents)(templateVars);
+}
+
+async function readTemplate(
+  templateDir: string,
+  templateVars: any,
+): Promise<TemplatedFile[]> {
+  const templateFilePaths = await recursiveReadDir(templateDir).catch(error => {
+    throw new Error(`Failed to read template directory: ${error.message}`);
+  });
+
+  const templatedFiles = new Array<TemplatedFile>();
+  for (const templateFile of templateFilePaths) {
+    const path = relativePath(templateDir, templateFile).replace(/\.hbs$/, '');
+    const contents = await readTemplateFile(templateFile, templateVars);
+
+    templatedFiles.push({ path, contents });
+  }
+
+  return templatedFiles;
+}
+
+async function diffTemplatedFiles(
+  targetDir: string,
+  templatedFiles: TemplatedFile[],
+): Promise<FileDiff[]> {
+  const fileDiffs = new Array<FileDiff>();
+  for (const { path, contents: templateContents } of templatedFiles) {
+    const targetPath = resolvePath(targetDir, path);
+    const targetExists = await fs.pathExists(targetPath);
+
+    const write = async (contents: string) => {
+      await fs.ensureDir(dirname(targetPath));
+      await fs.writeFile(targetPath, contents, 'utf8');
+    };
+
+    if (targetExists) {
+      const targetContents = await fs.readFile(targetPath, 'utf8');
+      fileDiffs.push({
+        path,
+        write,
+        missing: false,
+        targetContents,
+        templateContents,
+      });
+    } else {
+      fileDiffs.push({
+        path,
+        write,
+        missing: true,
+        targetContents: '',
+        templateContents,
+      });
+    }
+  }
+
+  return fileDiffs;
+}
+
+// Read all template files for a given template, along with all matching files in the target dir
+export async function diffTemplateFiles(template: string, templateData: any) {
+  const templateDir = paths.resolveOwn('templates', template);
+
+  const templatedFiles = await readTemplate(templateDir, templateData);
+  const fileDiffs = await diffTemplatedFiles(paths.targetDir, templatedFiles);
+  return fileDiffs;
+}
@@ -14,35 +14,24 @@
 * limitations under the License.
 */

-export type PluginInfo = {
-  id: string;
-  name: string;
-};
+export type WriteFileFunc = (contents: string) => Promise<void>;

-export type TemplateFile = {
+export type FileDiff = {
  // Relative path within the target directory
-  targetPath: string;
+  path: string;
+  // Wether the target file exists in the target directory.
+  missing: boolean;
+  // Contents of the file in the target directory, or an empty string if the file is missing.
+  targetContents: string;
  // Contents of the compiled template file
  templateContents: string;
-} & (
-  | {
-      // Whether the template file exists in the target directory
-      targetExists: true;
-      // Contents of the file in the target directory, if it exists
-      targetContents: string;
-    }
-  | {
-      // Whether the template file exists in the target directory
-      targetExists: false;
-    }
-);
+  // Write new contents to the target file
+  write: WriteFileFunc;
+};

 export type PromptFunc = (msg: string) => Promise<boolean>;

-export type HandlerFunc = (
-  file: TemplateFile,
-  prompt: PromptFunc,
-) => Promise<void>;
+export type HandlerFunc = (file: FileDiff, prompt: PromptFunc) => Promise<void>;

 export type FileHandler = {
  patterns: Array<string | RegExp>;
@@ -27,7 +27,18 @@
  },
  "devDependencies": {
    "@backstage/cli": "^{{version}}",
+    "@spotify/prettier-config": "^7.0.0",
    "lerna": "^3.20.2",
    "prettier": "^1.19.1"
+  },
+  "prettier": "@spotify/prettier-config",
+  "lint-staged": {
+    "*.{js,jsx,ts,tsx}": [
+      "eslint --fix",
+      "prettier --write"
+    ],
+    "*.{json,md}": [
+      "prettier --write"
+    ]
  }
 }
@@ -1,20 +0,0 @@
-{
-  "extends": "@spotify/web-scripts/config/tsconfig.json",
-  "compilerOptions": {
-    "target": "es5",
-    "lib": ["dom", "dom.iterable", "esnext"],
-    "allowJs": true,
-    "skipLibCheck": true,
-    "esModuleInterop": true,
-    "allowSyntheticDefaultImports": true,
-    "strict": true,
-    "forceConsistentCasingInFileNames": true,
-    "module": "esnext",
-    "moduleResolution": "node",
-    "resolveJsonModule": true,
-    "isolatedModules": true,
-    "jsx": "react",
-    "incremental": false
-  },
-  "include": ["src"]
-}
@@ -1,5 +0,0 @@
-{
-  "extends": "../../tsconfig.json",
-  "include": ["src"],
-  "compilerOptions": {}
-}
@@ -1 +0,0 @@
-module.exports = require('@spotify/web-scripts/config/prettier.config.js');
@@ -11,4 +11,4 @@ kind: Component
 metadata:
  name: component2
 spec:
-  type: service
+  type: website
@@ -14,6 +14,7 @@
 * limitations under the License.
 */

+import { NotFoundError } from '@backstage/backend-common';
 import { Database } from '../database';
 import { DescriptorEnvelope } from '../ingestion/types';
 import { EntitiesCatalog } from './types';
@@ -22,12 +23,23 @@ export class DatabaseEntitiesCatalog implements EntitiesCatalog {
  constructor(private readonly database: Database) {}

  async entities(): Promise<DescriptorEnvelope[]> {
-    const items = await this.database.entities();
+    const items = await this.database.transaction(tx =>
+      this.database.entities(tx),
+    );
    return items.map(i => i.entity);
  }

-  async entity(name: string): Promise<DescriptorEnvelope> {
-    const item = await this.database.entity(name);
+  async entity(
+    kind: string,
+    name: string,
+    namespace: string | undefined,
+  ): Promise<DescriptorEnvelope | undefined> {
+    const item = await this.database.transaction(tx =>
+      this.database.entity(tx, kind, name, namespace),
+    );
+    if (!item) {
+      throw new NotFoundError('Entity cannot be found');
+    }
    return item.entity;
  }
 }
@@ -15,6 +15,7 @@
 */

 import { NotFoundError } from '@backstage/backend-common';
+import lodash from 'lodash';
 import { DescriptorEnvelope } from '../ingestion';
 import { EntitiesCatalog } from './types';

@@ -26,14 +27,23 @@ export class StaticEntitiesCatalog implements EntitiesCatalog {
  }

  async entities(): Promise<DescriptorEnvelope[]> {
-    return this._entities.slice();
+    return lodash.cloneDeep(this._entities);
  }

-  async entity(name: string): Promise<DescriptorEnvelope> {
-    const item = this._entities.find(e => e.metadata?.name === name);
+  async entity(
+    kind: string,
+    name: string,
+    namespace: string | undefined,
+  ): Promise<DescriptorEnvelope | undefined> {
+    const item = this._entities.find(
+      e =>
+        kind === e.kind &&
+        name === e.metadata?.name &&
+        namespace === e.metadata?.namespace,
+    );
    if (!item) {
-      throw new NotFoundError(`Found no entity with name ${name}`);
+      throw new NotFoundError('Entity cannot be found');
    }
-    return item;
+    return lodash.cloneDeep(item);
  }
 }
@@ -23,7 +23,11 @@ import { DescriptorEnvelope } from '../ingestion';

 export type EntitiesCatalog = {
  entities(): Promise<DescriptorEnvelope[]>;
-  entity(id: string): Promise<DescriptorEnvelope>;
+  entity(
+    kind: string,
+    name: string,
+    namespace: string | undefined,
+  ): Promise<DescriptorEnvelope | undefined>;
 };

 //
@@ -14,30 +14,74 @@
 * limitations under the License.
 */

-import knex from 'knex';
+import {
+  ConflictError,
+  getVoidLogger,
+  NotFoundError,
+} from '@backstage/backend-common';
+import Knex from 'knex';
 import path from 'path';
 import { Database } from './Database';
-import { AddDatabaseLocation, DbLocationsRow } from './types';
+import {
+  AddDatabaseLocation,
+  DbEntityRequest,
+  DbEntityResponse,
+  DbLocationsRow,
+} from './types';

 describe('Database', () => {
-  const database = knex({
-    client: 'sqlite3',
-    connection: ':memory:',
-    useNullAsDefault: true,
-  });
-  database.client.pool.on('createSuccess', (_eventId: any, resource: any) => {
-    resource.run('PRAGMA foreign_keys = ON', () => {});
-  });
+  let database: Knex;
+  let entityRequest: DbEntityRequest;
+  let entityResponse: DbEntityResponse;

  beforeEach(async () => {
+    database = Knex({
+      client: 'sqlite3',
+      connection: ':memory:',
+      useNullAsDefault: true,
+    });
+
+    await database.raw('PRAGMA foreign_keys = ON');
    await database.migrate.latest({
      directory: path.resolve(__dirname, 'migrations'),
      loadExtensions: ['.ts'],
    });
+
+    entityRequest = {
+      entity: {
+        apiVersion: 'a',
+        kind: 'b',
+        metadata: {
+          name: 'c',
+          namespace: 'd',
+          labels: { e: 'f' },
+          annotations: { g: 'h' },
+        },
+        spec: { i: 'j' },
+      },
+    };
+
+    entityResponse = {
+      locationId: undefined,
+      entity: {
+        apiVersion: 'a',
+        kind: 'b',
+        metadata: {
+          uid: expect.anything(),
+          etag: expect.anything(),
+          generation: expect.anything(),
+          name: 'c',
+          namespace: 'd',
+          labels: { e: 'f' },
+          annotations: { g: 'h' },
+        },
+        spec: { i: 'j' },
+      },
+    };
  });

  it('manages locations', async () => {
-    const db = new Database(database);
+    const db = new Database(database, getVoidLogger());
    const input: AddDatabaseLocation = { type: 'a', target: 'b' };
    const output: DbLocationsRow = {
      id: expect.anything(),
@@ -62,7 +106,7 @@ describe('Database', () => {

  it('instead of adding second location with the same target, returns existing one', async () => {
    // Prepare
-    const catalog = new Database(database);
+    const catalog = new Database(database, getVoidLogger());
    const input: AddDatabaseLocation = { type: 'a', target: 'b' };
    const output1: DbLocationsRow = await catalog.addLocation(input);

@@ -75,4 +119,127 @@ describe('Database', () => {
    // Locations contain only one record
    expect(locations).toEqual([output1]);
  });
+
+  describe('addEntity', () => {
+    it('happy path: adds entity to empty database', async () => {
+      const catalog = new Database(database, getVoidLogger());
+      const added = await catalog.transaction(tx =>
+        catalog.addEntity(tx, entityRequest),
+      );
+      expect(added).toStrictEqual(entityResponse);
+      expect(added.entity.metadata!.generation).toBe(1);
+    });
+
+    it('rejects adding the same-named entity twice', async () => {
+      const catalog = new Database(database, getVoidLogger());
+      await catalog.transaction(tx => catalog.addEntity(tx, entityRequest));
+      await expect(
+        catalog.transaction(tx => catalog.addEntity(tx, entityRequest)),
+      ).rejects.toThrow(ConflictError);
+    });
+
+    it('accepts adding the same-named entity twice if on different namespaces', async () => {
+      const catalog = new Database(database, getVoidLogger());
+      entityRequest.entity.metadata!.namespace = 'namespace1';
+      await catalog.transaction(tx => catalog.addEntity(tx, entityRequest));
+      entityRequest.entity.metadata!.namespace = 'namespace2';
+      await expect(
+        catalog.transaction(tx => catalog.addEntity(tx, entityRequest)),
+      ).resolves.toBeDefined();
+    });
+  });
+
+  describe('updateEntity', () => {
+    it('can read and no-op-update an entity', async () => {
+      const catalog = new Database(database, getVoidLogger());
+      const added = await catalog.transaction(tx =>
+        catalog.addEntity(tx, entityRequest),
+      );
+      const updated = await catalog.transaction(tx =>
+        catalog.updateEntity(tx, { entity: added.entity }),
+      );
+      expect(updated.entity.apiVersion).toEqual(added.entity.apiVersion);
+      expect(updated.entity.kind).toEqual(added.entity.kind);
+      expect(updated.entity.metadata!.etag).not.toEqual(
+        added.entity.metadata!.etag,
+      );
+      expect(updated.entity.metadata!.generation).toEqual(
+        added.entity.metadata!.generation,
+      );
+      expect(updated.entity.metadata!.name).toEqual(
+        added.entity.metadata!.name,
+      );
+      expect(updated.entity.metadata!.namespace).toEqual(
+        added.entity.metadata!.namespace,
+      );
+    });
+
+    it('can update name if uid matches', async () => {
+      const catalog = new Database(database, getVoidLogger());
+      const added = await catalog.transaction(tx =>
+        catalog.addEntity(tx, entityRequest),
+      );
+      added.entity.metadata!.name! = 'new!';
+      const updated = await catalog.transaction(tx =>
+        catalog.updateEntity(tx, { entity: added.entity }),
+      );
+      expect(updated.entity.metadata!.name).toEqual('new!');
+    });
+
+    it('can update fields if kind, name, and namespace match', async () => {
+      const catalog = new Database(database, getVoidLogger());
+      const added = await catalog.transaction(tx =>
+        catalog.addEntity(tx, entityRequest),
+      );
+      added.entity.apiVersion = 'something.new';
+      delete added.entity.metadata!.uid;
+      delete added.entity.metadata!.generation;
+      const updated = await catalog.transaction(tx =>
+        catalog.updateEntity(tx, { entity: added.entity }),
+      );
+      expect(updated.entity.apiVersion).toEqual('something.new');
+    });
+
+    it('rejects if kind, name, but not namespace match', async () => {
+      const catalog = new Database(database, getVoidLogger());
+      const added = await catalog.transaction(tx =>
+        catalog.addEntity(tx, entityRequest),
+      );
+      added.entity.apiVersion = 'something.new';
+      delete added.entity.metadata!.uid;
+      delete added.entity.metadata!.generation;
+      added.entity.metadata!.namespace = 'something.wrong';
+      await expect(
+        catalog.transaction(tx =>
+          catalog.updateEntity(tx, { entity: added.entity }),
+        ),
+      ).rejects.toThrow(NotFoundError);
+    });
+
+    it('fails to update an entity if etag does not match', async () => {
+      const catalog = new Database(database, getVoidLogger());
+      const added = await catalog.transaction(tx =>
+        catalog.addEntity(tx, entityRequest),
+      );
+      added.entity.metadata!.etag = 'garbage';
+      await expect(
+        catalog.transaction(tx =>
+          catalog.updateEntity(tx, { entity: added.entity }),
+        ),
+      ).rejects.toThrow(ConflictError);
+    });
+
+    it('fails to update an entity if generation does not match', async () => {
+      const catalog = new Database(database, getVoidLogger());
+      const added = await catalog.transaction(tx =>
+        catalog.addEntity(tx, entityRequest),
+      );
+      added.entity.metadata!.generation! += 100;
+      await expect(
+        catalog.transaction(tx =>
+          catalog.updateEntity(tx, { entity: added.entity }),
+        ),
+      ).rejects.toThrow(ConflictError);
+    });
+  });
 });
@@ -14,33 +14,43 @@
 * limitations under the License.
 */

-import { InputError, NotFoundError } from '@backstage/backend-common';
+import {
+  ConflictError,
+  InputError,
+  NotFoundError,
+} from '@backstage/backend-common';
 import Knex from 'knex';
+import lodash from 'lodash';
 import { v4 as uuidv4 } from 'uuid';
-import { DescriptorEnvelope } from '../ingestion';
+import { Logger } from 'winston';
+import { DescriptorEnvelope, EntityMeta } from '../ingestion';
+import { buildEntitySearch } from './search';
 import {
  AddDatabaseLocation,
  DatabaseLocationUpdateLogEvent,
  DatabaseLocationUpdateLogStatus,
  DbEntitiesRow,
+  DbEntitiesSearchRow,
  DbEntityRequest,
  DbEntityResponse,
  DbLocationsRow,
 } from './types';

-function serializeMetadata(
-  metadata: DescriptorEnvelope['metadata'],
-): DbEntitiesRow['metadata'] {
-  if (!metadata) {
-    return null;
-  }
-
-  const output = { ...metadata };
+function getStrippedMetadata(metadata: EntityMeta): EntityMeta {
+  const output = lodash.cloneDeep(metadata);
  delete output.uid;
  delete output.etag;
  delete output.generation;

-  return JSON.stringify(output);
+  return output;
+}
+
+function serializeMetadata(metadata: EntityMeta | undefined): string | null {
+  if (!metadata) {
+    return null;
+  }
+
+  return JSON.stringify(getStrippedMetadata(metadata));
 }

 function serializeSpec(
@@ -53,29 +63,32 @@ function serializeSpec(
  return JSON.stringify(spec);
 }

-function entityRequestToDb(request: DbEntityRequest): DbEntitiesRow {
+function toEntityRow(
+  locationId: string | undefined,
+  entity: DescriptorEnvelope,
+): DbEntitiesRow {
  return {
-    id: '',
-    location_id: request.locationId || null,
-    etag: new Buffer(uuidv4()).toString('base64').replace(/[^\w]/g, ''), // TODO(freben): Atomicity isn't checked using these yet
-    generation: 1, // TODO(freben): These aren't updated yet
-    api_version: request.entity.apiVersion,
-    kind: request.entity.kind,
-    name: request.entity.metadata?.name || null,
-    namespace: request.entity.metadata?.namespace || null,
-    metadata: serializeMetadata(request.entity.metadata),
-    spec: serializeSpec(request.entity.spec),
+    id: entity.metadata!.uid!,
+    location_id: locationId || null,
+    etag: entity.metadata!.etag!,
+    generation: entity.metadata!.generation!,
+    api_version: entity.apiVersion,
+    kind: entity.kind,
+    name: entity.metadata!.name || null,
+    namespace: entity.metadata!.namespace || null,
+    metadata: serializeMetadata(entity.metadata),
+    spec: serializeSpec(entity.spec),
  };
 }

-function entityDbToResponse(row: DbEntitiesRow): DbEntityResponse {
+function toEntityResponse(row: DbEntitiesRow): DbEntityResponse {
  const entity: DescriptorEnvelope = {
    apiVersion: row.api_version,
    kind: row.kind,
    metadata: {
      uid: row.id,
      etag: row.etag,
-      generation: row.generation,
+      generation: Number(row.generation), // cast because of sqlite
    },
  };

@@ -95,49 +108,229 @@ function entityDbToResponse(row: DbEntitiesRow): DbEntityResponse {
  };
 }

-export class Database {
-  constructor(private readonly database: Knex) {}
-
-  async addOrUpdateEntity(request: DbEntityRequest): Promise<void> {
-    if (!request.entity.metadata?.name) {
-      throw new InputError(`Entities without names are not yet supported`);
-    }
-
-    const newRow = entityRequestToDb(request);
-
-    await this.database.transaction(async tx => {
-      // TODO(freben): Currently, several locations can compete for the same entity
-      // TODO(freben): If locationId is unset in the input, it won't be overwritten - should we instead replace with null?
-      const count = await tx<DbEntitiesRow>('entities')
-        .where({ name: request.entity.metadata?.name })
-        .update({
-          ...newRow,
-          id: undefined,
-        });
-      if (!count) {
-        await tx<DbEntitiesRow>('entities').insert({
-          ...newRow,
-          id: uuidv4(),
-        });
-      }
-    });
+function specsAreEqual(
+  first: string | null,
+  second: object | undefined,
+): boolean {
+  if (!first && !second) {
+    return true;
+  } else if (!first || !second) {
+    return false;
  }

-  async entities(): Promise<DbEntityResponse[]> {
-    const items = await this.database<DbEntitiesRow>('entities')
+  return lodash.isEqual(JSON.parse(first), second);
+}
+
+function generateUid(): string {
+  return uuidv4();
+}
+
+function generateEtag(): string {
+  return Buffer.from(uuidv4(), 'utf8').toString('base64').replace(/[^\w]/g, '');
+}
+
+/**
+ * An abstraction on top of the underlying database, wrapping the basic CRUD
+ * needs.
+ */
+export class Database {
+  constructor(
+    private readonly database: Knex,
+    private readonly logger: Logger,
+  ) {}
+
+  /**
+   * Runs a transaction.
+   *
+   * The callback is expected to make calls back into this class. When it
+   * completes, the transaction is closed.
+   *
+   * @param fn The callback that implements the transaction
+   */
+  async transaction<T>(
+    fn: (tx: Knex.Transaction<any, any>) => Promise<T>,
+  ): Promise<T> {
+    try {
+      return await this.database.transaction<T>(fn);
+    } catch (e) {
+      this.logger.debug(`Error during transaction, ${e}`);
+
+      if (
+        /SQLITE_CONSTRAINT: UNIQUE/.test(e.message) ||
+        /unique constraint/.test(e.message)
+      ) {
+        throw new ConflictError(`Rejected due to a conflicting entity`, e);
+      }
+
+      throw e;
+    }
+  }
+
+  /**
+   * Adds a new entity to the catalog.
+   *
+   * @param tx An ongoing transaction
+   * @param request The entity being added
+   * @returns The added entity, with uid, etag and generation set
+   */
+  async addEntity(
+    tx: Knex.Transaction<any, any>,
+    request: DbEntityRequest,
+  ): Promise<DbEntityResponse> {
+    if (request.entity.metadata?.uid !== undefined) {
+      throw new InputError('May not specify uid for new entities');
+    } else if (request.entity.metadata?.etag !== undefined) {
+      throw new InputError('May not specify etag for new entities');
+    } else if (request.entity.metadata?.generation !== undefined) {
+      throw new InputError('May not specify generation for new entities');
+    }
+
+    const newEntity = lodash.cloneDeep(request.entity);
+    newEntity.metadata = Object.assign({}, newEntity.metadata, {
+      uid: generateUid(),
+      etag: generateEtag(),
+      generation: 1,
+    });
+
+    const newRow = toEntityRow(request.locationId, newEntity);
+    await tx<DbEntitiesRow>('entities').insert(newRow);
+    await this.updateEntitiesSearch(tx, newRow.id, newEntity);
+
+    return { locationId: request.locationId, entity: newEntity };
+  }
+
+  /**
+   * Updates an existing entity in the catalog.
+   *
+   * The given entity must contain enough information to identify an already
+   * stored entity in the catalog - either by uid, or by kind + namespace +
+   * name. If no matching entity is found, the operation fails.
+   *
+   * If etag or generation are given, they are taken into account. Attempts to
+   * update a matching entity, but where the etag and/or generation are not
+   * equal to the passed values, will fail.
+   *
+   * @param tx An ongoing transaction
+   * @param request The entity being updated
+   * @returns The updated entity
+   */
+  async updateEntity(
+    tx: Knex.Transaction<any, any>,
+    request: DbEntityRequest,
+  ): Promise<DbEntityResponse> {
+    const { kind } = request.entity;
+    const {
+      uid,
+      etag: expectedOldEtag,
+      generation: expectedOldGeneration,
+      name,
+      namespace,
+    } = request.entity.metadata ?? {};
+
+    // Find existing entities that match the given metadata
+    let entitySelector: Partial<DbEntitiesRow>;
+    if (uid) {
+      entitySelector = { id: uid };
+    } else if (kind && name) {
+      entitySelector = {
+        kind,
+        name: name,
+        namespace: namespace || null,
+      };
+    } else {
+      throw new InputError(
+        'Must specify either uid, or kind + name + namespace to be able to identify an entity',
+      );
+    }
+    const oldRows = await tx<DbEntitiesRow>('entities')
+      .where(entitySelector)
+      .select();
+    if (oldRows.length !== 1) {
+      throw new NotFoundError('No matching entity found');
+    }
+
+    // Validate the old entity
+    const oldRow = oldRows[0];
+    // The Number cast is here because sqlite reads it as a string, no matter
+    // what the table actually says
+    oldRow.generation = Number(oldRow.generation);
+    if (expectedOldEtag) {
+      if (expectedOldEtag !== oldRow.etag) {
+        throw new ConflictError(
+          `Etag mismatch, expected="${expectedOldEtag}" found="${oldRow.etag}"`,
+        );
+      }
+    }
+    if (expectedOldGeneration) {
+      if (expectedOldGeneration !== oldRow.generation) {
+        throw new ConflictError(
+          `Generation mismatch, expected="${expectedOldGeneration}" found="${oldRow.generation}"`,
+        );
+      }
+    }
+
+    // Build the new shape of the entity
+    const newEtag = generateEtag();
+    const newGeneration = specsAreEqual(oldRow.spec, request.entity.spec)
+      ? oldRow.generation
+      : oldRow.generation + 1;
+    const newEntity = lodash.cloneDeep(request.entity);
+    newEntity.metadata = Object.assign({}, request.entity.metadata, {
+      uid: oldRow.id,
+      etag: newEtag,
+      generation: newGeneration,
+    });
+
+    // Preserve annotations that were set on the old version of the entity,
+    // unless the new version overwrites them
+    if (oldRow.metadata) {
+      const oldMetadata = JSON.parse(oldRow.metadata) as EntityMeta;
+      if (oldMetadata.annotations) {
+        newEntity.metadata!.annotations = {
+          ...oldMetadata.annotations,
+          ...newEntity.metadata!.annotations,
+        };
+      }
+    }
+
+    // Store the updated entity; select on the old etag to ensure that we do
+    // not lose to another writer
+    const newRow = toEntityRow(request.locationId, newEntity);
+    const updatedRows = await tx<DbEntitiesRow>('entities')
+      .where({ id: oldRow.id, etag: oldRow.etag })
+      .update(newRow);
+
+    // If this happens, somebody else changed the entity just now
+    if (updatedRows !== 1) {
+      throw new ConflictError(`Failed to update entity`);
+    }
+
+    await this.updateEntitiesSearch(tx, oldRow.id, newEntity);
+    return { locationId: request.locationId, entity: newEntity };
+  }
+
+  async entities(tx: Knex.Transaction<any, any>): Promise<DbEntityResponse[]> {
+    const rows = await tx<DbEntitiesRow>('entities')
      .orderBy('namespace', 'name')
      .select();
-    return items.map(entityDbToResponse);
+    return rows.map(row => toEntityResponse(row));
  }

-  async entity(name: string): Promise<DbEntityResponse> {
-    const items = await this.database<DbEntitiesRow>('entities')
-      .where({ name })
+  async entity(
+    tx: Knex.Transaction<any, any>,
+    kind: string,
+    name: string,
+    namespace?: string,
+  ): Promise<DbEntityResponse | undefined> {
+    const rows = await tx<DbEntitiesRow>('entities')
+      .where({ kind, name, namespace: namespace || null })
      .select();
-    if (!items.length) {
-      throw new NotFoundError(`Found no entity with name ${name}`);
+
+    if (rows.length !== 1) {
+      return undefined;
    }
-    return entityDbToResponse(items[0]);
+
+    return toEntityResponse(rows[0]);
  }

  async addLocation(location: AddDatabaseLocation): Promise<DbLocationsRow> {
@@ -205,15 +398,20 @@ export class Database {
    });
  }

-  /*
  private async updateEntitiesSearch(
    tx: Knex.Transaction<any, any>,
    entityId: string,
    data: DescriptorEnvelope,
  ): Promise<void> {
-    const entries = buildEntitySearch(entityId, data);
-    await tx('entities_search').where({ entity_id: entityId }).del();
-    await tx('entities_search').insert(entries);
+    try {
+      const entries = buildEntitySearch(entityId, data);
+      await tx<DbEntitiesSearchRow>('entities_search')
+        .where({ entity_id: entityId })
+        .del();
+      await tx<DbEntitiesSearchRow>('entities_search').insert(entries);
+    } catch {
+      // ignore intentionally - if this happens, the entity was deleted before
+      // we got around to writing the entries
+    }
  }
-  */
 }
@@ -14,8 +14,7 @@
 * limitations under the License.
 */

-import { PassThrough } from 'stream';
-import winston from 'winston';
+import { getVoidLogger } from '@backstage/backend-common';
 import {
  ComponentDescriptor,
  DescriptorParser,
@@ -25,16 +24,12 @@ import {
 import { Database } from './Database';
 import { DatabaseManager } from './DatabaseManager';
 import { DatabaseLocationUpdateLogStatus, DbLocationsRow } from './types';
+import Knex from 'knex';

 describe('DatabaseManager', () => {
-  const logger = winston.createLogger({
-    transports: [new winston.transports.Stream({ stream: new PassThrough() })],
-  });
-
  describe('refreshLocations', () => {
    it('works with no locations added', async () => {
      const db = ({
-        addOrUpdateEntity: jest.fn(),
        locations: jest.fn().mockResolvedValue([]),
      } as unknown) as Database;
      const reader: LocationReader = {
@@ -45,33 +40,34 @@ describe('DatabaseManager', () => {
      };

      await expect(
-        DatabaseManager.refreshLocations(db, reader, parser, logger),
+        DatabaseManager.refreshLocations(db, reader, parser, getVoidLogger()),
      ).resolves.toBeUndefined();
      expect(reader.read).not.toHaveBeenCalled();
      expect(parser.parse).not.toHaveBeenCalled();
    });

    it('can update a single location', async () => {
-      const db = ({
-        addOrUpdateEntity: jest.fn(),
-        locations: jest.fn(() =>
-          Promise.resolve([
-            {
-              id: '123',
-              type: 'some',
-              target: 'thing',
-            } as DbLocationsRow,
-          ]),
-        ),
-        addLocationUpdateLogEvent: jest.fn(),
-      } as unknown) as Database;
-
+      const location: DbLocationsRow = {
+        id: '123',
+        type: 'some',
+        target: 'thing',
+      };
      const desc: ComponentDescriptor = {
        apiVersion: 'backstage.io/v1beta1',
        kind: 'Component',
        metadata: { name: 'c1' },
        spec: { type: 'service' },
      };
+      const tx = (undefined as unknown) as Knex.Transaction<any, any>;
+
+      const db = ({
+        transaction: jest.fn(f => f(tx)),
+        entity: jest.fn(() => Promise.resolve(undefined)),
+        addEntity: jest.fn(),
+        locations: jest.fn(() => Promise.resolve([location])),
+        addLocationUpdateLogEvent: jest.fn(),
+      } as Partial<Database>) as Database;
+
      const reader: LocationReader = {
        read: jest.fn(() => Promise.resolve([{ type: 'data', data: desc }])),
      };
@@ -80,12 +76,12 @@ describe('DatabaseManager', () => {
      };

      await expect(
-        DatabaseManager.refreshLocations(db, reader, parser, logger),
+        DatabaseManager.refreshLocations(db, reader, parser, getVoidLogger()),
      ).resolves.toBeUndefined();
      expect(reader.read).toHaveBeenCalledTimes(1);
      expect(reader.read).toHaveBeenNthCalledWith(1, 'some', 'thing');
-      expect(db.addOrUpdateEntity).toHaveBeenCalledTimes(1);
-      expect(db.addOrUpdateEntity).toHaveBeenNthCalledWith(1, {
+      expect(db.addEntity).toHaveBeenCalledTimes(1);
+      expect(db.addEntity).toHaveBeenNthCalledWith(1, undefined, {
        locationId: '123',
        entity: expect.objectContaining({
          metadata: expect.objectContaining({ name: 'c1' }),
@@ -94,8 +90,12 @@ describe('DatabaseManager', () => {
    });

    it('logs successful updates', async () => {
+      const tx = (undefined as unknown) as Knex.Transaction<any, any>;
+
      const db = ({
-        addOrUpdateEntity: jest.fn(),
+        transaction: jest.fn(f => f(tx)),
+        addEntity: jest.fn(),
+        entity: jest.fn(() => Promise.resolve(undefined)),
        locations: jest.fn(() =>
          Promise.resolve([
            {
@@ -122,7 +122,7 @@ describe('DatabaseManager', () => {
      };

      await expect(
-        DatabaseManager.refreshLocations(db, reader, parser, logger),
+        DatabaseManager.refreshLocations(db, reader, parser, getVoidLogger()),
      ).resolves.toBeUndefined();

      expect(db.addLocationUpdateLogEvent).toHaveBeenNthCalledWith(
@@ -141,8 +141,11 @@ describe('DatabaseManager', () => {
    });

    it('logs unsuccessful updates when parser fails', async () => {
+      const tx = (undefined as unknown) as Knex.Transaction<any, any>;
+
      const db = ({
-        addOrUpdateEntity: jest.fn(),
+        transaction: jest.fn(f => f(tx)),
+        addEntity: jest.fn(),
        locations: jest.fn(() =>
          Promise.resolve([
            {
@@ -171,7 +174,7 @@ describe('DatabaseManager', () => {
      };

      await expect(
-        DatabaseManager.refreshLocations(db, reader, parser, logger),
+        DatabaseManager.refreshLocations(db, reader, parser, getVoidLogger()),
      ).resolves.toBeUndefined();

      expect(db.addLocationUpdateLogEvent).toHaveBeenNthCalledWith(
@@ -191,8 +194,11 @@ describe('DatabaseManager', () => {
    });

    it('logs unsuccessful updates when reader fails', async () => {
+      const tx = (undefined as unknown) as Knex.Transaction<any, any>;
+
      const db = ({
-        addOrUpdateEntity: jest.fn(),
+        transaction: jest.fn(f => f(tx)),
+        addEntity: jest.fn(),
        locations: jest.fn(() =>
          Promise.resolve([
            {
@@ -217,7 +223,7 @@ describe('DatabaseManager', () => {
      };

      await expect(
-        DatabaseManager.refreshLocations(db, reader, parser, logger),
+        DatabaseManager.refreshLocations(db, reader, parser, getVoidLogger()),
      ).resolves.toBeUndefined();

      expect(db.addLocationUpdateLogEvent).toHaveBeenNthCalledWith(
@@ -15,19 +15,28 @@
 */

 import Knex from 'knex';
+import lodash from 'lodash';
 import path from 'path';
 import { Logger } from 'winston';
-import { DescriptorParser, LocationReader, ParserError } from '../ingestion';
+import {
+  DescriptorEnvelope,
+  DescriptorParser,
+  LocationReader,
+  ParserError,
+} from '../ingestion';
 import { Database } from './Database';
 import { DatabaseLocationUpdateLogStatus, DbEntityRequest } from './types';

 export class DatabaseManager {
-  public static async createDatabase(database: Knex): Promise<Database> {
+  public static async createDatabase(
+    database: Knex,
+    logger: Logger,
+  ): Promise<Database> {
    await database.migrate.latest({
      directory: path.resolve(__dirname, 'migrations'),
      loadExtensions: ['.js'],
    });
-    return new Database(database);
+    return new Database(database, logger);
  }

  private static async logUpdateSuccess(
@@ -66,20 +75,25 @@ export class DatabaseManager {
    for (const location of locations) {
      try {
        logger.debug(
-          `Refreshing location ${location.id} type "${location.type}" target "${location.target}"`,
+          `Refreshing location id="${location.id}" type="${location.type}" target="${location.target}"`,
        );

        const readerOutput = await reader.read(location.type, location.target);

        for (const readerItem of readerOutput) {
          if (readerItem.type === 'error') {
-            logger.debug(readerItem.error);
+            logger.info(readerItem.error);
            continue;
          }
+
          try {
            const entity = await parser.parse(readerItem.data);
-            const dbc: DbEntityRequest = { locationId: location.id, entity };
-            await database.addOrUpdateEntity(dbc);
+            await DatabaseManager.refreshSingleEntity(
+              database,
+              location.id,
+              entity,
+              logger,
+            );
            await DatabaseManager.logUpdateSuccess(
              database,
              location.id,
@@ -98,11 +112,76 @@ export class DatabaseManager {
            );
          }
        }
-        await DatabaseManager.logUpdateSuccess(database, location.id);
+        await DatabaseManager.logUpdateSuccess(
+          database,
+          location.id,
+          undefined,
+        );
      } catch (error) {
-        logger.debug(`Failed to refresh location ${location.id}, ${error}`);
+        logger.debug(
+          `Failed to refresh location id="${location.id}", ${error}`,
+        );
        await DatabaseManager.logUpdateFailure(database, location.id, error);
      }
    }
  }
+
+  private static async refreshSingleEntity(
+    database: Database,
+    locationId: string,
+    entity: DescriptorEnvelope,
+    logger: Logger,
+  ): Promise<void> {
+    const { kind } = entity;
+    const { name, namespace } = entity.metadata || {};
+    if (!name) {
+      throw new Error('Entities without names are not yet supported');
+    }
+
+    const request: DbEntityRequest = {
+      locationId: locationId,
+      entity: entity,
+    };
+
+    logger.debug(
+      `Read entity kind="${kind}" name="${name}" namespace="${namespace}"`,
+    );
+
+    await database.transaction(async tx => {
+      const previous = await database.entity(tx, kind, name, namespace);
+      if (!previous) {
+        logger.debug(`No such entity found, adding`);
+        await database.addEntity(tx, request);
+      } else if (
+        !DatabaseManager.entitiesAreEqual(previous.entity, request.entity)
+      ) {
+        logger.debug(`Different from existing entity, updating`);
+        await database.updateEntity(tx, request);
+      } else {
+        logger.debug(`Equal to existing entity, skipping update`);
+      }
+    });
+  }
+
+  private static entitiesAreEqual(
+    first: DescriptorEnvelope,
+    second: DescriptorEnvelope,
+  ) {
+    const firstClone = lodash.cloneDeep(first);
+    const secondClone = lodash.cloneDeep(second);
+
+    // Remove generated fields
+    if (firstClone.metadata) {
+      delete firstClone.metadata.uid;
+      delete firstClone.metadata.etag;
+      delete firstClone.metadata.generation;
+    }
+    if (secondClone.metadata) {
+      delete secondClone.metadata.uid;
+      delete secondClone.metadata.etag;
+      delete secondClone.metadata.generation;
+    }
+
+    return lodash.isEqual(firstClone, secondClone);
+  }
 }
@@ -27,7 +27,7 @@ export async function up(knex: Knex): Promise<any> {
      .inTable('locations')
      .onUpdate('CASCADE')
      .onDelete('CASCADE');
-    table.string('entity_name').notNullable();
+    table.string('entity_name').nullable();
  });
 }

@@ -14,9 +14,9 @@
 * limitations under the License.
 */

+import { DescriptorEnvelope } from '../ingestion';
 import { buildEntitySearch, visitEntityPart } from './search';
 import { DbEntitiesSearchRow } from './types';
-import { DescriptorEnvelope } from '../ingestion';

 describe('search', () => {
  describe('visitEntityPart', () => {
@@ -16,8 +16,7 @@

 import { FileLocationSource } from './sources/FileLocationSource';
 import { GitHubLocationSource } from './sources/GitHubLocationSource';
-import { LocationSource } from './sources/types';
-import { LocationReader, ReaderOutput } from './types';
+import { LocationReader, LocationSource, ReaderOutput } from './types';

 export class LocationReaders implements LocationReader {
  static create(): LocationReader {
@@ -15,8 +15,7 @@
 */

 import * as yup from 'yup';
-import { DescriptorEnvelope, ParserError } from '../types';
-import { KindParser } from './types';
+import { DescriptorEnvelope, KindParser, ParserError } from '../types';

 export interface ComponentDescriptorV1beta1 extends DescriptorEnvelope {
  spec: {
@@ -89,7 +89,7 @@ export class DescriptorEnvelopeParser {
      );

    const labelsSchema = yup
-      .object()
+      .object<Record<string, string>>()
      .notRequired()
      .test({
        name: 'metadata.labels.keys',
@@ -113,7 +113,7 @@ export class DescriptorEnvelopeParser {
      });

    const annotationsSchema = yup
-      .object()
+      .object<Record<string, string>>()
      .notRequired()
      .test({
        name: 'metadata.annotations.keys',
@@ -1,38 +0,0 @@
-/*
- * Copyright 2020 Spotify AB
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- *     http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-import { DescriptorEnvelope } from '../types';
-
-/**
- * Parses and validates a single envelope into its materialized kind.
- *
- * These parsers may assume that the envelope is already validated and well
- * formed.
- */
-export type KindParser = {
-  /**
-   * Try to parse an envelope into a materialized kind.
-   *
-   * @param envelope A valid descriptor envelope
-   * @returns A materialized type, or undefined if the given version/kind is
-   *          not meant to be handled by this parser
-   * @throws An Error if the type was handled and found to not be properly
-   *         formatted
-   */
-  tryParse(
-    envelope: DescriptorEnvelope,
-  ): Promise<DescriptorEnvelope | undefined>;
-};
@@ -15,8 +15,7 @@
 */

 import fs from 'fs-extra';
-import { ReaderOutput } from '../types';
-import { LocationSource } from './types';
+import { LocationSource, ReaderOutput } from '../types';
 import { readDescriptorYaml } from './util';

 export class FileLocationSource implements LocationSource {
@@ -15,10 +15,9 @@
 */

 import fetch from 'node-fetch';
-import { ReaderOutput } from '../types';
-import { LocationSource } from './types';
-import { readDescriptorYaml } from './util';
 import { URL } from 'url';
+import { LocationSource, ReaderOutput } from '../types';
+import { readDescriptorYaml } from './util';

 // Pointing to raw.githubusercontent.com for now
 // to be changed in the future, after auth and tokens are done
@@ -58,7 +57,7 @@ export class GitHubLocationSource implements LocationSource {

    let rawYaml;
    try {
-      rawYaml = await fetch(url.toString()).then((x) => {
+      rawYaml = await fetch(url.toString()).then(x => {
        return x.text();
      });
    } catch (e) {
@@ -18,6 +18,70 @@ import { ComponentDescriptorV1beta1 } from './descriptors/ComponentDescriptorV1b

 export type ComponentDescriptor = ComponentDescriptorV1beta1;

+/**
+ * Metadata fields common to all versions/kinds of entity.
+ *
+ * @see https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#objectmeta-v1-meta
+ */
+export type EntityMeta = {
+  /**
+   * A globally unique ID for the entity.
+   *
+   * This field can not be set by the user at creation time, and the server
+   * will reject an attempt to do so. The field will be populated in read
+   * operations. The field can (optionally) be specified when performing
+   * update or delete operations, but the server is free to reject requests
+   * that do so in such a way that it breaks semantics.
+   */
+  uid?: string;
+
+  /**
+   * An opaque string that changes for each update operation to any part of
+   * the entity, including metadata.
+   *
+   * This field can not be set by the user at creation time, and the server
+   * will reject an attempt to do so. The field will be populated in read
+   * operations. The field can (optionally) be specified when performing
+   * update or delete operations, and the server will then reject the
+   * operation if it does not match the current stored value.
+   */
+  etag?: string;
+
+  /**
+   * A positive nonzero number that indicates the current generation of data
+   * for this entity; the value is incremented each time the spec changes.
+   *
+   * This field can not be set by the user at creation time, and the server
+   * will reject an attempt to do so. The field will be populated in read
+   * operations.
+   */
+  generation?: number;
+
+  /**
+   * The name of the entity.
+   *
+   * Must be uniqe within the catalog at any given point in time, for any
+   * given namespace, for any given kind.
+   */
+  name?: string;
+
+  /**
+   * The namespace that the entity belongs to.
+   */
+  namespace?: string;
+
+  /**
+   * Key/value pairs of identifying information attached to the entity.
+   */
+  labels?: Record<string, string>;
+
+  /**
+   * Key/value pairs of non-identifying auxiliary information attached to the
+   * entity.
+   */
+  annotations?: Record<string, string>;
+};
+
 /**
 * The format envelope that's common to all versions/kinds.
 *
@@ -37,67 +101,8 @@ export type DescriptorEnvelope = {

  /**
   * Optional metadata related to the entity.
-   *
-   * @see https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#objectmeta-v1-meta
   */
-  metadata?: {
-    /**
-     * A globally unique ID for the entity.
-     *
-     * This field can not be set by the user at creation time, and the server
-     * will reject an attempt to do so. The field will be populated in read
-     * operations. The field can (optionally) be specified when performing
-     * update or delete operations, but the server is free to reject requests
-     * that do so in such a way that it breaks semantics.
-     */
-    uid?: string;
-
-    /**
-     * An opaque string that changes for each update operation to any part of
-     * the entity, including metadata.
-     *
-     * This field can not be set by the user at creation time, and the server
-     * will reject an attempt to do so. The field will be populated in read
-     * operations. The field can (optionally) be specified when performing
-     * update or delete operations, and the server will then reject the
-     * operation if it does not match the current stored value.
-     */
-    etag?: string;
-
-    /**
-     * A positive nonzero number that indicates the current generation of data
-     * for this entity; the value is incremented each time the spec changes.
-     *
-     * This field can not be set by the user at creation time, and the server
-     * will reject an attempt to do so. The field will be populated in read
-     * operations.
-     */
-    generation?: number;
-
-    /**
-     * The name of the entity.
-     *
-     * Must be uniqe within the catalog at any given point in time, for any
-     * given namespace, for any given kind.
-     */
-    name?: string;
-
-    /**
-     * The namespace that the entity belongs to.
-     */
-    namespace?: string;
-
-    /**
-     * Key/value pairs of identifying information attached to the entity.
-     */
-    labels?: object;
-
-    /**
-     * Key/value pairs of non-identifying auxiliary information attached to the
-     * entity.
-     */
-    annotations?: object;
-  };
+  metadata?: EntityMeta;

  /**
   * The specification data describing the entity itself.
@@ -38,16 +38,10 @@ export async function createRouter(

  if (entitiesCatalog) {
    // Entities
-    router
-      .get('/entities', async (_req, res) => {
-        const entities = await entitiesCatalog.entities();
-        res.status(200).send(entities);
-      })
-      .get('/entities/:id', async (req, res) => {
-        const { id } = req.params;
-        const entity = await entitiesCatalog.entity(id);
-        res.status(200).send(entity);
-      });
+    router.get('/entities', async (_req, res) => {
+      const entities = await entitiesCatalog.entities();
+      res.status(200).send(entities);
+    });
  }

  // Locations
				`@@ -1 +0,0 @@`
				`module.exports = require('@spotify/web-scripts/config/prettier.config.js');`