diff --git a/docs/features/software-catalog/bsc-edit.png b/docs/features/software-catalog/bsc-edit.png new file mode 100644 index 0000000000..d1ac1f52e5 Binary files /dev/null and b/docs/features/software-catalog/bsc-edit.png differ diff --git a/docs/features/software-catalog/bsc-extend.png b/docs/features/software-catalog/bsc-extend.png new file mode 100644 index 0000000000..54d660ea70 Binary files /dev/null and b/docs/features/software-catalog/bsc-extend.png differ diff --git a/docs/features/software-catalog/bsc-register-1.png b/docs/features/software-catalog/bsc-register-1.png new file mode 100644 index 0000000000..117b2ea8ea Binary files /dev/null and b/docs/features/software-catalog/bsc-register-1.png differ diff --git a/docs/features/software-catalog/bsc-register-2.png b/docs/features/software-catalog/bsc-register-2.png new file mode 100644 index 0000000000..fd1ea7b998 Binary files /dev/null and b/docs/features/software-catalog/bsc-register-2.png differ diff --git a/docs/features/software-catalog/bsc-search.png b/docs/features/software-catalog/bsc-search.png new file mode 100644 index 0000000000..8e417cb076 Binary files /dev/null and b/docs/features/software-catalog/bsc-search.png differ diff --git a/docs/features/software-catalog/bsc-starred.png b/docs/features/software-catalog/bsc-starred.png new file mode 100644 index 0000000000..27db19c842 Binary files /dev/null and b/docs/features/software-catalog/bsc-starred.png differ diff --git a/docs/features/software-catalog/extending-the-model.md b/docs/features/software-catalog/extending-the-model.md index 7b3bc826cf..ce5ea03b26 100644 --- a/docs/features/software-catalog/extending-the-model.md +++ b/docs/features/software-catalog/extending-the-model.md @@ -3,4 +3,39 @@ id: extending-the-model title: Extending the model --- -## TODO +Backstage natively supports tracking of the following component +[`type`](descriptor-format.md)'s: + +- Services +- Websites +- Libraries +- Documentation +- Other + +![](bsc-extend.png) + +Since these types are likely not the only kind of software you will want to +track in Backstage, it is possible to + +It is possible to add your own software types that fits your organization's data +model. Inside Spotify our model has grown significantly over the years, and now +includes ML models, Apps, data pipelines and many more. + +## Adding a new type + +TODO: Describe what changes are needed to add a new type that shows up in the +catalog. + +## The Other type + +It might be tempting to put software that doesn't fit into any of the existing +types into Other. There are a few reasons why we advice against this; firstly, +we have found that it is preferred to match the conceptual model that your +engineers have when describing your sofware. Secondly, Backstage helps your +engineers manage their software by integrating the infratrucure tooling through +plugins. Different plugins are used for managing different types of components. + +For example, the +[Lighthouse plugin](https://github.com/spotify/backstage/tree/master/plugins/lighthouse) +only makes sense for Websites. The more specific you can be in how you model +your software, the easier it is to provide plugins that are contextual. diff --git a/docs/features/software-catalog/external-integrations.md b/docs/features/software-catalog/external-integrations.md index fb8dc10c45..7ca869cdae 100644 --- a/docs/features/software-catalog/external-integrations.md +++ b/docs/features/software-catalog/external-integrations.md @@ -3,4 +3,9 @@ id: external-integrations title: External integrations --- -## TODO +Backstage natively supports storing software components in +[metadata YAML files](descriptor-format.md). However, companies that already +have an existing system for keeping track of software and its owners can +integrate such systems with Backstage. + +TODO: Describe the API contract. diff --git a/docs/features/software-catalog/index.md b/docs/features/software-catalog/index.md index ce27d821d7..3e4d3ddd18 100644 --- a/docs/features/software-catalog/index.md +++ b/docs/features/software-catalog/index.md @@ -9,25 +9,109 @@ The Backstage Service Catalog — actually, a software catalog, since it include more than just services — is a centralized system that keeps track of ownership and metadata for all the software in your ecosystem (services, websites, libraries, data pipelines, etc). The catalog is built around the concept of -[metadata yaml files](../../architecture-decisions/adr002-default-catalog-file-format.md#format) -stored together with the code, which are then harvested and visualized in -Backstage. +[metadata YAML files](descriptor-format.md) stored together with the code, which +are then harvested and visualized in Backstage. ![service-catalog](https://backstage.io/blog/assets/6/header.png) -We have also found that the service catalog is a great way to organise the -infrastructure tools you use to manage the software as well. This is how -Backstage creates one developer portal for all your tools. Rather than asking -teams to jump between different infrastructure UI’s (and incurring additional -cognitive overhead each time they make a context switch), most of these tools -can be organised around the entities in the catalog. +## How it works -## Using the Service Catalog +Backstage and the Backstage Service Catalog makes it easy for one team to manage +10 services — and makes it possible for your company to manage thousands of +them. -TODO +More specifically, the Service Catalog enables two main use-cases: + +1. Helping teams manage and maintain the software they own. Teams get a uniform + view of all their software; services, libraries, websites, ML models — you + name it, Backstage knows all about it. +2. Makes all the software in your company, and who owns it, discoverable. No + more orphan software hiding in the dark corners of your software ecosystem. ![](../../assets/software-catalog/service-catalog-home.png) +## Adding components to the catalog + +The source of truth for the components in your service catalog are +[metadata YAML files](descriptor-format.md) stored in source control (GitHub, +GitHub Enterprise, GitLab, ...). + +There are 3 ways to add components to the catalog: + +1. Manually register components +2. Creating new components through Backstage +3. Integrating with and [external source](external-integrations.md) + +### Manually register components + +Users can register new components by going to `/create` and clicking the +**REGSITER EXISTING COMPONENT** button: + +![](bsc-register-1.png) + +Backstage expects the full URL to the YAML in your source control. Example: + +``` +https://github.com/spotify/backstage/blob/master/packages/catalog-model/examples/artist-lookup-component.yaml +``` + +_More examples can be found +[here](https://github.com/spotify/backstage/tree/master/packages/catalog-model/examples)._ + +![](bsc-register-2.png) + +It is important to note that any kind of software can be registered in +Backstage. Even if the software is not maintained by your company (SaaS +offering, for example) it is still useful to create components for tracking +ownership. + +### Creating new components through Backstage + +All software created through the +[Backstage Software Templates](../software-templates/index.md) are automatically +registered in the catalog. + +### Updating component metadata + +Teams owning the components are responsible for maintaining the metadata about +them, and do so using their normal Git workflow. + +![](bsc-edit.png) + +Once the change has been merged, Backstage will automatically show the updated +metadata in the service catalog after a short while. + +## Finding software in the catlog + +By default the service catalog shows components owned by the team of the logged +in user. But you can also switch to _All_ to see all the components across your +companie's software ecosystem. Basic inline _search_ and _column filtering_ +makes it easy to browse a big set of components. + +![](bsc-search.png) + +## Starring components + +For easy and quick access to components you visit frequently, Backstage supports +_starring_ of components: + +![](bsc-starred.png) + +## Integrated tooling through plugins + +The service catalog is a great way to organise the infrastructure tools you use +to manage the software. This is how Backstage creates one developer portal for +all your tools. Rather than asking teams to jump between different +infrastructure UI’s (and incurring additional cognitive overhead each time they +make a context switch), most of these tools can be organised around the entities +in the catalog. + +![tools](https://backstage.io/blog/assets/20-05-20/tabs.png) + +The Backstage platform can be customized by incorporating +[existing open source plugins](https://github.com/spotify/backstage/tree/master/plugins), +or by [building your own](../../plugins/index.md). + ## Links - [[Blog post] Backstage Service Catalog released in alpha](https://backstage.io/blog/2020/06/22/backstage-service-catalog-alpha) diff --git a/docs/features/software-catalog/populating.md b/docs/features/software-catalog/populating.md deleted file mode 100644 index 68c61f2822..0000000000 --- a/docs/features/software-catalog/populating.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -id: populating-catalog -title: Populating the catalog ---- - -## TODO diff --git a/docs/features/software-catalog/system-model.md b/docs/features/software-catalog/system-model.md index b5a53080dc..0a30693455 100644 --- a/docs/features/software-catalog/system-model.md +++ b/docs/features/software-catalog/system-model.md @@ -3,4 +3,97 @@ id: system-model title: System Model --- -## TODO +We believe that a strong shared understanding and terminology around systems, +software and resources leads to a better Backstage experience. + +_This description originates from +[this RFC](https://github.com/spotify/backstage/issues/390). Note that some of +the concpets are not yet supported in Backstage._ + +## Concepts + +We model our technology using these five concepts (further explained below): + +- **Domains** are a high-level grouping of systems +- **Systems** encapsulate the implementation of APIs +- **APIs** are the boundaries between different components and systems +- **Components** are pieces of software +- **Resources** are physical or virtual infrastructure needed to operate a + system + +![Software Ecosystem Model_ Public Github version](https://user-images.githubusercontent.com/24575/77633084-39bcde80-6f4f-11ea-8251-f8df561a3652.png) + +### Domain + +While systems are the basic level of encapsulation for resources, components and +APIs, it is often useful to group a collection of systems that share +terminology, domain models, business purpose, or documentation, i.e. they form a +bounded context. + +For example, it would make sense if the different systems in the “Payments” +domain would come with some documentation on how to accept payments for a new +product or use-case, share the same entity types in their APIs, and integrate +well with each other. + +### System + +With increasing complexity in software, we believe that systems form an +important abstraction level to help us reason about software ecosystems. Systems +are a useful concept in that they allow us to ignore the implementation details +of a certain functionality for consumers, while allowing the owning team to make +changes as they see fit (leading to low coupling). + +A system, in this sense, is a collection of resources and components that +exposes one or several APIs. Components and resources in a system are typically +owned by the same team and are expected to co-evolve. As such, systems usually +consist of at most a handful of components. + +For example, a playlist management system might encapsulate a backend service to +update playlists, a backend service to query them, and a database to store them. +It could expose an RPC API, a daily snapshots dataset, and an event stream of +playlist updates. + +### Component + +A component is a piece of software, for example a mobile feature, web site, +backend service or data pipeline (list not exhaustive). A component can be +tracked in source control, or use some existing open source or commercial +software. + +A component can implement APIs for other components to consume. It might depend +on the resources of the system it belongs to, and APIs from other components or +other systems. All other aspects of the component, e.g. any code dependencies, +must be encapsulated. + +### API + +We believe APIs form an important (maybe the most important) abstraction that +allows large software ecosystems to scale. Thus, APIs are a first class citizen +in the Backstage model and the primary way to discover existing functionality in +the ecosystem. + +APIs are implemented by components and form boundaries between components and +systems. They might be defined using an RPC IDL (eg Protobuf, GraphQL, ...), a +data schema (eg Avro, TFRecord, ...), or as code interfaces. In any case, APIs +exposed by components need to be in a known machine-readable format so we can +build further tooling and analysis on top. + +Some APIs might be exposed by the system, making them available for any other +Spotify component to consume. Those public APIs must be documented and humanly +discoverable in Backstage. + +### Resource + +Resources are the infrastructure a system needs to operate, like BigTable +databases, Pub/Sub topics, S3 buckets or CDNs. Modelling them together with +components and systems will better allow us to visualize resource footprint, and +create tooling around them. + +## Current status + +Backstage currently supports Components and APIs. + +## Links + +- [Original RFC](https://github.com/spotify/backstage/issues/390) +- [YAML file format](../../architecture-decisions/adr002-default-catalog-file-format.md) diff --git a/docs/features/software-templates/index.md b/docs/features/software-templates/index.md index 72d12b10cc..9c1096b026 100644 --- a/docs/features/software-templates/index.md +++ b/docs/features/software-templates/index.md @@ -8,6 +8,10 @@ Components inside Backstage. It by default has the ability to load skeletons of code, template in some variables and then publish the template to some location like GitHub. + + ### Getting Started The Software Templates are available under `/create`, and if you've followed diff --git a/microsite/i18n/en.json b/microsite/i18n/en.json index c3faa0874a..8e333aab8a 100644 --- a/microsite/i18n/en.json +++ b/microsite/i18n/en.json @@ -86,8 +86,8 @@ "FAQ": { "title": "FAQ" }, - "features/software-catalog/api": { - "title": "features/software-catalog/api" + "features/software-catalog/software-catalog-api": { + "title": "API" }, "features/software-catalog/descriptor-format": { "title": "Descriptor Format of Catalog Entities", @@ -102,8 +102,8 @@ "features/software-catalog/software-catalog-overview": { "title": "Backstage Service Catalog (alpha)" }, - "features/software-catalog/populating": { - "title": "features/software-catalog/populating" + "features/software-catalog/populating-catalog": { + "title": "Populating the catalog" }, "features/software-catalog/system-model": { "title": "System Model" diff --git a/mkdocs.yml b/mkdocs.yml index 2380001203..adfdf6ae0a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -25,7 +25,6 @@ nav: - Overview: 'features/software-catalog/index.md' - System model: 'features/software-catalog/system-model.md' - YAML File Format: 'features/software-catalog/descriptor-format.md' - - Populating the catalog: 'features/software-catalog/populating.md' - Extending the model: 'features/software-catalog/extending-the-model.md' - External integrations: 'features/software-catalog/external-integrations.md' - API: 'features/software-catalog/api.md' diff --git a/plugins/register-component/README.md b/plugins/register-component/README.md index 729a38955c..d324b3941f 100644 --- a/plugins/register-component/README.md +++ b/plugins/register-component/README.md @@ -1,5 +1,92 @@ -# [WIP] register-component +# Register component plugin Welcome to the register-component plugin! This plugin allows you to submit your Backstage component using your software's YAML config. + +When installed it is accessible on [localhost:3000/register-component](localhost:3000/register-component). + + + + + + + +## Standalone setup + +0. Install plugin and its dependency `plugin-catalog` + +```bash +yarn add @backstage/plugin-register-component -W +yarn add @backstage/plugin-catalog -W +``` + +1. Add dependencies to the active plugins list + +```typescript +// packages/app/src/plugins.ts +export { plugin as RegisterComponent } from '@backstage/plugin-register-component'; +export { plugin as CatalogPlugin } from '@backstage/plugin-catalog'; +``` + +2. Create `packages/app/src/apis.ts` and register all the needed plugins + +```typescript +import { + alertApiRef, + AlertApiForwarder, + ApiRegistry, + ConfigApi, + errorApiRef, + ErrorApiForwarder, + ErrorAlerter, +} from '@backstage/core'; + +import { catalogApiRef, CatalogClient } from '@backstage/plugin-catalog'; + +export const apis = (config: ConfigApi) => { + const backendUrl = config.getString('backend.baseUrl'); + + const builder = ApiRegistry.builder(); + + const alertApi = builder.add(alertApiRef, new AlertApiForwarder()); + const errorApi = builder.add( + errorApiRef, + new ErrorAlerter(alertApi, new ErrorApiForwarder()), + ); + + builder.add( + catalogApiRef, + new CatalogClient({ + apiOrigin: backendUrl, + basePath: '/catalog', + }), + ); + + return builder.build(); +}; +``` + +3. Pass `apis` to createApp + +```typescript +// packages/app/src/App.tsx +import { apis } from './apis'; + +const app = createApp({ + apis, + plugins: Object.values(plugins), +}); +``` + +## Running + +Just run the backstage. + +``` +yarn start && yarn --cwd packages/backend start +``` + +## Usage + +Pretty straightforward, navigate to [localhost:3000/register-component](localhost:3000/register-component) and enter your component's YAML config URL. diff --git a/plugins/register-component/src/assets/screenshot-1.png b/plugins/register-component/src/assets/screenshot-1.png new file mode 100644 index 0000000000..ffaebbb2f5 Binary files /dev/null and b/plugins/register-component/src/assets/screenshot-1.png differ diff --git a/plugins/register-component/src/assets/screenshot-2.png b/plugins/register-component/src/assets/screenshot-2.png new file mode 100644 index 0000000000..e4e5dc579c Binary files /dev/null and b/plugins/register-component/src/assets/screenshot-2.png differ diff --git a/plugins/register-component/src/assets/screenshot-3.png b/plugins/register-component/src/assets/screenshot-3.png new file mode 100644 index 0000000000..4bbac74d82 Binary files /dev/null and b/plugins/register-component/src/assets/screenshot-3.png differ diff --git a/plugins/register-component/src/assets/screenshot-4.png b/plugins/register-component/src/assets/screenshot-4.png new file mode 100644 index 0000000000..f45f333135 Binary files /dev/null and b/plugins/register-component/src/assets/screenshot-4.png differ diff --git a/plugins/register-component/src/assets/screenshot-5.png b/plugins/register-component/src/assets/screenshot-5.png new file mode 100644 index 0000000000..a50decd748 Binary files /dev/null and b/plugins/register-component/src/assets/screenshot-5.png differ