docs: add technical notes for catalog documentation
Signed-off-by: Hellgren Heikki <heikki.hellgren@op.fi>
This commit is contained in:
@@ -125,6 +125,15 @@ all of the processors' contributions to one step are run in the order that the
|
||||
processors were registered, then all of their contributions to the next step in
|
||||
the same order, and so on.
|
||||
|
||||
:::note Technical note
|
||||
|
||||
Processors registered from the same catalog module always run
|
||||
in the registration order, but this does not apply over multiple catalog modules as
|
||||
the order of registration depends on the order in which the modules are loaded
|
||||
by the framework.
|
||||
|
||||
:::
|
||||
|
||||
Each step has the opportunity to optionally modify the entity, and to optionally
|
||||
emit other information. For example, the processor might look at information in
|
||||
the `spec` field of the entity, and emit relations that correspond to those
|
||||
@@ -182,6 +191,19 @@ The diagram shows how the stitcher reads from several sources:
|
||||
The last part is noteworthy: This is how the stitcher is able to collect all of
|
||||
the relation edges, both incoming and outgoing, no matter who produced them.
|
||||
|
||||
:::note Technical note
|
||||
|
||||
Whether the entity will be stitched depends on the entity hash value,
|
||||
which is calculated based on the entity body, relations, errors, referred entities,
|
||||
and entity parents after the processing. The hash value changes if any of these components
|
||||
change. The hash value is then compared to hash value from the previous processing of the
|
||||
entity and if they are different, the entity is stitched again.
|
||||
In entity body, changes in array order, for example `metadata.tags`, changes the hash
|
||||
value. It is a good idea to monitor the stitched entities count in the catalog as it
|
||||
might cause performance issues if the number of entities gets too high.
|
||||
|
||||
:::
|
||||
|
||||
The stitching is currently a fixed process, that cannot be modified or extended.
|
||||
This means that any modifications you want to make on the final result, has to
|
||||
happen during ingestion or processing.
|
||||
|
||||
Reference in New Issue
Block a user