docs: add technical notes for catalog documentation

Signed-off-by: Hellgren Heikki <heikki.hellgren@op.fi>
This commit is contained in:
Hellgren Heikki
2025-08-14 11:24:44 +03:00
parent d2d6d325f9
commit e663767c31
@@ -125,6 +125,15 @@ all of the processors' contributions to one step are run in the order that the
processors were registered, then all of their contributions to the next step in
the same order, and so on.
:::note Technical note
Processors registered from the same catalog module always run
in the registration order, but this does not apply over multiple catalog modules as
the order of registration depends on the order in which the modules are loaded
by the framework.
:::
Each step has the opportunity to optionally modify the entity, and to optionally
emit other information. For example, the processor might look at information in
the `spec` field of the entity, and emit relations that correspond to those
@@ -182,6 +191,19 @@ The diagram shows how the stitcher reads from several sources:
The last part is noteworthy: This is how the stitcher is able to collect all of
the relation edges, both incoming and outgoing, no matter who produced them.
:::note Technical note
Whether the entity will be stitched depends on the entity hash value,
which is calculated based on the entity body, relations, errors, referred entities,
and entity parents after the processing. The hash value changes if any of these components
change. The hash value is then compared to hash value from the previous processing of the
entity and if they are different, the entity is stitched again.
In entity body, changes in array order, for example `metadata.tags`, changes the hash
value. It is a good idea to monitor the stitched entities count in the catalog as it
might cause performance issues if the number of entities gets too high.
:::
The stitching is currently a fixed process, that cannot be modified or extended.
This means that any modifications you want to make on the final result, has to
happen during ingestion or processing.