From 7b59d3e395b1d2f665bff04a5bd9babc6c7174e7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fredrik=20Adel=C3=B6w?= Date: Mon, 6 Sep 2021 22:11:44 +0200 Subject: [PATCH] add some text about orphaning too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Fredrik Adelöw --- .../software-catalog/life-of-an-entity.md | 57 ++++++++++++++++++- 1 file changed, 55 insertions(+), 2 deletions(-) diff --git a/docs/features/software-catalog/life-of-an-entity.md b/docs/features/software-catalog/life-of-an-entity.md index 3b7b3e8117..2cdf8761be 100644 --- a/docs/features/software-catalog/life-of-an-entity.md +++ b/docs/features/software-catalog/life-of-an-entity.md @@ -183,6 +183,59 @@ The stitching is currently a fixed process, that cannot be modified or extended. This means that any modifications you want to make on the final result, has to happen during ingestion or processing. -## Deletion +## Orphaning -> TODO +As mentioned earlier, entities internally form a graph. The edges go from +processed parent entities, to child entities emitted while processing the +parent. + +The processing loop runs continuously, so these edges are reconsidered over +time. If processing a parent entity no longer emits a given child entity, then +that former edge is severed. If that child has no other edges pointing at it +either, it becomes _orphaned_. The end result is as follows: + +- The stitching process injects a `backstage.io/orphan: 'true'` annotation on + the child entity. +- The child entity is _not_ removed from the catalog, but stays around until + explicitly deleted via the catalog API, or "reclaimed" by the original parent + or another parent starting to reference it. +- The child entity stops being subjected to regular processing, frozen in time. +- The catalog page in Backstage for the child entity detects the new annotation + and informs users about the orphan status. + +Orphaning can occur in several different scenarios. One common cause is that the +end user edited a corresponding catalog catalog-info YAML file removing the +entity's entry, or in the case of a `Location` entity, removed the target row +pointing to the file containing the file. Another common cause is large batch +processors such as the ones that crawl through remote systems looking for +entities, no longer finding something that it used to find before. Maybe the +data was moved, or deleted, in the remote system. So for example when a person +leaves the company an LDAP org discovery processor might leave an orphaned +`User` entity behind. + +> Note that removing a file, or accidentally corrupting a file so that it cannot +> be read successfully, does _not_ lead to orphaning. Hard errors, including the +> inability to find or read a distinct remote, are marked as such on the entity +> to inform the owner that something is wrong. But processing and other +> behaviors continue as usual. + +The reason that the orphaning mechanism exists instead of having an eager +deletion triggered, is safety. Scenarios like these can happen purely by +accident, due to the asynchronous nature of the system and the fallible nature +of humans. In particular when external systems start consuming and relying on +the catalog, the consequences of suddenly dropping entities without your +explicit owner consent could be substantial. The catalog therefore takes the +stance that entities that often were added by direct user action should also be +deleted only by direct user action. + +It is possible to build automated "reaper" systems that finally delete entities +that are orphaned by using the catalog API. This is however not something that's +provided out of the box. + +## Implicit Deletion + +> TODO: Describe the process of entity providers eagerly deleting entities + +## Explicit Deletion + +> TODO: Describe direct deletion via the catalog API