paulononato/backstage
1
0
Fork 0
Code Issues Pull Requests Actions 23 Packages Projects Releases Wiki Activity

ADR002: Default Software Catalog File Format

Browse Source
This commit is contained in:
Fredrik Adelöw
2020-05-17 20:21:17 +02:00
parent 55e45500fe
commit 86bb6cc6a8
2 changed files with 219 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture-decisions/adr001-add-adr-log.md
+1 -3
View File
@@ -1,4 +1,4 @@
## Date: 26.04.2020
## Date: 2020-04-26
## Title: Architecture Decision Record (ADR) log
@@ -7,5 +7,3 @@
## Discussion: There is a need to store big decisions made in a log as a reference point for the team, help with onboarding new members and give context to others interested in the project.
## Risks: People stop adding ADRs to the log and context gets lost
docs/architecture-decisions/adr002-default-catalog-file-format.md
+218
View File
@@ -0,0 +1,218 @@
# ADR002: Default Software Catalog File Format
| Created | Status |
| ---------- | ------ |
| 2020-05-17 | Open |
## Background
Backstage comes with a software catalog functionality, that you can use to
track all your software components and more. It can be powered by data from
various sources, and one of them that is included with the package, is a
custom database backed catalog. It has the ability to keep itself updated
automatically based on the contents of little descriptor files in your
version control system of choice. Developers create these files and maintain
them side by side with their code, and the catalog system reacts accordingly.
This ADR describes the default format of these descriptor files.
### Inspiration
Internally at Spotify, a home grown software catalog system is used pervasively
and forms a core part of Backstage and other important pieces of the
infrastructure. The user experience, learnings and certain pieces of metadata
from that catalog are being carried over to the open source effort.
The file format described herein, also draws heavy inspiration from the
[kubernetes object format](https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/).
### Core Concepts
There are a number of descriptor files, all of whose locations (e.g. within a
version control system) are registered with the software catalog. The method of
registration is not covered in this document; it could happen either manually
inside Backstage, or by push events from a CI/CD pipelines, or by webhook
triggers from the version control system, etc.
Each file describes one or more entities in accordance with the
[Backstage System Model](https://github.com/spotify/backstage/issues/390).
All of these entities have a common stucture and nomenclature, and they are
stored in the software catalog from which they then can be queried.
Entities have distinct names, and they may reference each other by those names.
## Format
Descriptor files use the [YAML](https://yaml.org/spec/1.2/spec.html) format.
They may be written by hand, or created using automated tools. Each file may
consist of several YAML documents (separated by `---`), where each document
describes a single entity.
This is an example entity definition with some mocked data.
```yaml
---
apiVersion: backstage.io/v1beta1
kind: Component
metadata:
name: frobs-awesome
description: |
Backend service that implements the Frobs API, as defined
in [the Frobs RFC](https://example.com/spec/frob.html).
labels:
system: frobs
lifecycle: production
example.com/service-discovery-name: frobsawesome
annotations:
circleci.com/project-slug: gh/example-org/frobs-awesome
spec:
type: service
```
The root fields `apiVersion`, `kind`, `metadata`, and `spec` are part of the
_envelope_, defining the overall structure of all kinds of entity. Likewise,
the `name`, `namespace`, `labels`, and `annotations` metadata fields are of
special significance and have reserved purposes and distinct shapes.
See below for details about these fields.
## Envelope
The root envelope object has the following structure.
### `apiVersion` and `kind`
The `kind` is the high level entity type being described, typically from the
[Backstage system model](https://github.com/spotify/backstage/issues/390). The
first versions of the catalog will focus on the `Component` kind.
The `apiVersion`is the version of specification format for that particular
entity that this file is written against. The version is used for being able to
evolve the format, and the tuple of `apiVersion` and `kind` should be enough
for a parser to know how to interpret the contents of the rest of the document.
Backstage specific entities have an `apiVersion` that is prefixed with
`backstage.io/`, to distinguish them from other types of object that share
the same type of structure. This may be relevant when co-hosting these
specifications with e.g. kubernetes object manifests.
Early versions of the catalog will be using beta versions, e.g. `backstage.io/v1beta1`,
to signal that the format may still change. After that, we will be using
`backstage.io/v1` and up.
### `metadata`
A structure that contains metadata about the entity, i.e. things that aren't directly
part of the entity specification itself. See below for more details about this structure.
### `spec`
The actual specification data that describes the entity.
The precise structure of the `spec` depends on the `apiVersion` and `kind` combination,
and some kinds may not even have a `spec` at all. See further down in this document for
the specification structure of specific kinds.
## Metadata
The `metadata` root field has the following nested structure.
### `name`
The name of the entity. This name is both meant for human eyes to recognize the entity,
and for machines and other components to reference the entity (e.g. in URLs or from
other entity specification files).
Names must be unique per kind, within a given namespace (if specified), at any point in
time. Names may be reused at a later time, after an entity is deleted from the registry.
Names are required to follow a certain format. Entities that do not follow those rules
will not be accepted for registration in the catalog. The ruleset is configurable to fit
your organization's needs, but the default behavior is as follows.
- Strings of length at least 1, and at most 253
- Consist only of URL safe characters, e.g. ones that `encodeURIComponent` or similar would
leave unchanged
- Must consist of sequences of `[a-z0-9A-Z]` possibly separated by other characters
Example: `visits-tracking-service`, `CircleciBuildsDs_avro_gcs`
In addition to this, names are passed through a normalization function and then compared
to the same normalized form of other entity names and made sure to not collide. This rule
of uniqueness exists to avoid situations where e.g. both `my-component` and `MyComponent`
are registered side by side, which leads to confusion and risk. The normalization function
is also configurable, but the default behavior is as follows.
- Strip out all characters outside of the set `[a-zA-Z0-9]`
- Convert to lowercase
Example: `CircleciBuildsDs_avro_gcs` -> `circlecibuildsdsavrogcs`
### `namespace`
The `name` of a namespace that the entity belongs to. This field is optional, and currently
has no special semantics apart from bounding the name uniqueness constraint if specified.
It is reserved for future use and may get broader semantic implication.
Namespaces may also be part of the catalog, and are `v1` / `Namespace` entities,
i.e. not Backstage specific but the same as in Kubernetes.
### `description`
A human readable description of the entity, to be shown in Backstage. Should be kept short
and informative, suitable to give an overview of the entity's purpose at a glance. More
detailed explanations and documentation should be placed elsewhere.
### `labels`
Labels are optional key/value pairs of that are attached to the entity, and their use is
identical to [kubernetes object labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).
Their main purpose is for references to other entities, and for information that is
in one way or another classifying for the current entity. They are often used as values
in queries or filters.
Both the key and the value are strings, subject to the following restrictions.
Keys have an optional prefix followed by a slash, and then the name part which is required.
The prefix must be a valid lowercase domain name, at most 253 characters in total. The name
part must be sequences of `[a-zA-Z0-9]` separated by any of `[-_.]`, at most 63 characters
in total.
The `backstage.io/` prefix is reserved for use by Backstage core components. Some keys such as
`system` also have predefined semantics.
Values are strings that follow the same restrictions as `name` above.
### `annotations`
An object with arbitrary non-identifying metadata attached to the entity,
identical in use to [kubernetes object annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/).
Their purpose is mainly, but not limited, to reference into external systems. This could
for example be a reference to the git ref the entity was ingested from, to monitoring
and logging systems, to pagerduty schedules, etc.
The keys are strings and the values are free-form, subject to the following restrictions.
Keys have an optional prefix followed by a slash, and then the name part which is required.
The prefix must be a valid lowercase domain name, at most 253 characters in total. The name
part must be sequences of `[a-zA-Z0-9]` separated by any of `[-_.]`, at most 63 characters
in total.
The `backstage.io/` prefix is reserved for use by Backstage core components.
Values can be of any JSON type.
## Component
| Field | Value |
| ------------ | ---------------------- |
| `apiVersion` | `backstage.io/v1beta1` |
| `kind` | `Component` |
The `spec` object for this kind is as follows:
| Field | Type | Required | Description |
| ------ | ------ | -------- | -------------------------------------- |
| `type` | String | Yes | The type of component, e.g. `service`. |