add some text about orphaning too

Signed-off-by: Fredrik Adelöw <freben@gmail.com>
This commit is contained in:
Fredrik Adelöw
2021-09-06 22:11:44 +02:00
parent e33d346c12
commit 7b59d3e395
@@ -183,6 +183,59 @@ The stitching is currently a fixed process, that cannot be modified or extended.
This means that any modifications you want to make on the final result, has to
happen during ingestion or processing.
## Deletion
## Orphaning
> TODO
As mentioned earlier, entities internally form a graph. The edges go from
processed parent entities, to child entities emitted while processing the
parent.
The processing loop runs continuously, so these edges are reconsidered over
time. If processing a parent entity no longer emits a given child entity, then
that former edge is severed. If that child has no other edges pointing at it
either, it becomes _orphaned_. The end result is as follows:
- The stitching process injects a `backstage.io/orphan: 'true'` annotation on
the child entity.
- The child entity is _not_ removed from the catalog, but stays around until
explicitly deleted via the catalog API, or "reclaimed" by the original parent
or another parent starting to reference it.
- The child entity stops being subjected to regular processing, frozen in time.
- The catalog page in Backstage for the child entity detects the new annotation
and informs users about the orphan status.
Orphaning can occur in several different scenarios. One common cause is that the
end user edited a corresponding catalog catalog-info YAML file removing the
entity's entry, or in the case of a `Location` entity, removed the target row
pointing to the file containing the file. Another common cause is large batch
processors such as the ones that crawl through remote systems looking for
entities, no longer finding something that it used to find before. Maybe the
data was moved, or deleted, in the remote system. So for example when a person
leaves the company an LDAP org discovery processor might leave an orphaned
`User` entity behind.
> Note that removing a file, or accidentally corrupting a file so that it cannot
> be read successfully, does _not_ lead to orphaning. Hard errors, including the
> inability to find or read a distinct remote, are marked as such on the entity
> to inform the owner that something is wrong. But processing and other
> behaviors continue as usual.
The reason that the orphaning mechanism exists instead of having an eager
deletion triggered, is safety. Scenarios like these can happen purely by
accident, due to the asynchronous nature of the system and the fallible nature
of humans. In particular when external systems start consuming and relying on
the catalog, the consequences of suddenly dropping entities without your
explicit owner consent could be substantial. The catalog therefore takes the
stance that entities that often were added by direct user action should also be
deleted only by direct user action.
It is possible to build automated "reaper" systems that finally delete entities
that are orphaned by using the catalog API. This is however not something that's
provided out of the box.
## Implicit Deletion
> TODO: Describe the process of entity providers eagerly deleting entities
## Explicit Deletion
> TODO: Describe direct deletion via the catalog API