The whereIn('ref', refs) calls on ingestion_mark_entities generated
a unique prepared statement for every distinct array length, bloating
the Postgres query plan cache. On Postgres, use = ANY($1) with a
single array parameter instead. Falls back to regular whereIn on
SQLite/MySQL.
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
The relations table had indexes on originating_entity_id and
source_entity_ref but none on target_entity_ref. Several query paths
join or filter on this column:
- Orphan deletion (LEFT JOIN relations ON target_entity_ref)
- Entity ancestry (INNER JOIN relations ON target_entity_ref)
- Eager pruning (JOIN relations ON target_entity_ref)
Without an index these queries seq-scan the full table (~3.5M rows,
714 MB heap). On a production replica, a single point lookup takes
~122ms via seq scan. With the index it drops to <1ms.
The index is ~141 MB based on column width (~35 bytes avg) across
~3.5M rows. On PostgreSQL it's created with CONCURRENTLY to avoid
blocking reads/writes.
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
* feat(catalog-backend-module-msgraph): filter out disabled users by default
The Microsoft Graph provider now always applies an `accountEnabled eq true`
base filter when fetching users. Any custom `user.filter` is combined with
the base filter using `and`, so adopters no longer need to manually add
`accountEnabled eq true` to their configuration.
Also removes the legacy mutual exclusivity check between `userFilter` and
`userGroupMemberFilter` — these serve orthogonal purposes (user-level
filtering vs group selection) and the downstream code already handles
both being set.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: MT Lewis <mtlewis@users.noreply.github.com>
* chore: mark msgraph disabled-user filtering as breaking change
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: MT Lewis <mtlewis@users.noreply.github.com>
* docs(catalog-backend-module-msgraph): clarify automatic accountEnabled filter in docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: MT Lewis <mtlewis@users.noreply.github.com>
---------
Signed-off-by: MT Lewis <mtlewis@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(auth-backend): harden default allowed patterns for CIMD and DCR
Signed-off-by: benjdlambert <ben@blam.sh>
* address PR review feedback for OIDC defaults
- narrow CLI client ID pattern to exact cli.json path
- add BREAKING prefix to changeset
- add IPv6 [::1] to docs examples
- add loopback redirect URI tests for IPv6 and 127.0.0.1
Signed-off-by: benjdlambert <ben@blam.sh>
* remove dead ['*'] fallback when features are disabled
The restrictive defaults are now always used regardless of the enabled
flag, since the patterns are only consulted on code paths that require
the feature to be enabled.
Signed-off-by: benjdlambert <ben@blam.sh>
* add default pattern tests and fix docs cli example
Signed-off-by: benjdlambert <ben@blam.sh>
* use URL constructor for CLI client ID
Signed-off-by: benjdlambert <ben@blam.sh>
* use string templating for cliClientId to match OidcRouter
Signed-off-by: benjdlambert <ben@blam.sh>
* fix docs: remove misleading CLI client_id URL example
Signed-off-by: benjdlambert <ben@blam.sh>
---------
Signed-off-by: benjdlambert <ben@blam.sh>
* fix(catalog-backend): move generateStableHash out of shared util to fix Storybook build
The util.ts file mixed Node.js-only code (createHash from node:crypto)
with pure constants. Since InMemoryCatalogClient reaches into
buildEntitySearch via a relative import, and buildEntitySearch imports
from util.ts, the node:crypto dependency leaked into Vite's browser
bundle causing the Storybook build to fail.
Signed-off-by: benjdlambert <ben@blam.sh>
* add changeset
Signed-off-by: benjdlambert <ben@blam.sh>
---------
Signed-off-by: benjdlambert <ben@blam.sh>
Mark immediate mode stitching as deprecated in config.d.ts, the
StitchingStrategy type, and log a warning on startup when it is
detected. This is a precursor to removing immediate mode entirely
in the next release.
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
When using GithubMultiOrgEntityProvider with alwaysUseDefaultNamespace
and teams with identical slugs across orgs, the emitted entity order
was non-deterministic. Since the catalog's upsert path uses last-write-
wins for duplicate entity refs, this caused the winning org to flip
randomly on every refresh cycle, producing constant unnecessary
stitching and flickering entity data.
Sort the emitted entities by entity ref (primary) and location
annotation (tiebreaker) so that the same org consistently wins when
duplicate refs exist.
Fixes#34263
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
The mock credentials created by mockCredentials.none(), .user(), and
.service() were missing the internal version: 'v1' field. This caused
toInternalBackstageCredentials() to throw when used with mock
credentials in tests.
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Add container layout to Header sections so the title, actions, tags, metadata, and tabs align with surrounding page content.
Signed-off-by: Charles de Dreuille <charles.dedreuille@gmail.com>
* draft: restructure entities() ordering to drive from search-by-key
When a sort field is specified, build the query so the search table
filtered by that key is the driving relation, instead of left-joining
search onto final_entities and sorting after. The planner can then walk
the (key, value, entity_id) index in already-sorted order and short
circuit on LIMIT.
Measured against a production replica, the catalog UI's default
"first page of components ordered by metadata.name" query goes from
~940 ms to ~8 ms. See PR description for the full numbers.
Known caveats this draft does not yet address:
- Entities lacking the order field are excluded; the previous shape
put them at the end with NULLS LAST. A UNION ALL pattern can
preserve the old semantics.
- Multi-field order falls back to the OLD shape (only the first field
is taken when present today; subsequent fields acted as
tie-breakers). Restoring tie-breakers needs additional joins or a
CTE.
- queryEntities (/entities/by-query) is left untouched; the same
optimization applies but the CTE/cursor structure makes the rewrite
more involved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@spotify.com>
* two-phase entities() ordering: fast path + NULLs-last fallback
When an order field is specified, run a fast path first that drives
from the search-by-key index (excluding entities without the field).
If that path doesn't produce enough rows to cover offset+limit+1, run
a fallback that picks up the no-field entities in entity_id order.
This preserves NULLs-LAST semantics while keeping the fast plan for
the common case where every entity has the order field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@spotify.com>
* fix: multi-field order fallback + MySQL quoting in entities()
Fall back to the original LEFT JOIN shape when multiple order fields
are specified, since tie-breaking on secondary fields inherently
requires materialization. The fast INNER-JOIN-driven path is used only
for single-field order (the typical UI case).
Fix the phase 2 NOT EXISTS clause to use knex's ?? identifier escaping
instead of hardcoded double-quotes, which broke on MySQL.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@spotify.com>
* catalog-backend: fix phase 2 ordering and update changeset
Fix two issues raised in review:
- Phase 2 (entities lacking the sort field) now always sorts by
entity_id ASC regardless of the primary sort direction, matching the
original NULLS-LAST behaviour where the NULL group was always
entity_id ASC.
- Update changeset to describe the actual two-phase behaviour (NULLS
LAST preserved) instead of the stale description that said entities
without the field are excluded.
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* catalog-backend: apply offset when limit is undefined in runOrderedEntitiesQuery
Previously the fast-path returned the full combined array when no limit
was specified, silently ignoring any pagination offset. The old
implementation always pushed offset to SQL independently of limit.
Restore parity by slicing the combined array by the offset even when
limit is absent.
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* catalog-backend: add pagination and phase-boundary tests for entities() ordering
Add two new test cases that cover the parts of runOrderedEntitiesQuery not
exercised before:
- "paginates correctly through single-field ordering": exercises limit, offset,
and hasNextPage through the fast path (Phase 1 only) across all DB engines.
- "paginates across the Phase 1 / Phase 2 boundary": exercises the case where
the requested page straddles entities that have the sort field (Phase 1) and
those that do not (Phase 2), and verifies that Phase 2 entities are always
ordered ASC by entity_id regardless of the primary sort direction.
Also fixes a double-space caught by prettier in DefaultEntitiesCatalog.ts.
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* catalog-backend: treat null sort-field value the same as a missing sort field
buildEntitySearch stores value=NULL for entity fields that are explicitly
null or exceed MAX_VALUE_LENGTH. Previously, Phase 1 included those rows via
the INNER JOIN (key matches, value IS NULL), causing them to sort ahead of
entities that have no row for the key at all — changing semantics vs the old
LEFT JOIN shape where both cases landed in the same NULLS-LAST bucket.
Fix Phase 1 to require order_0.value IS NOT NULL, and fix Phase 2's NOT EXISTS
to check for no non-null value (value IS NOT NULL) so that both null-valued
and missing-key entities are collected in Phase 2 and ordered together by
entity_id ASC.
Adds a regression test covering an entity with spec.b=null alongside one with
no spec.b, asserting both appear after sorted entities regardless of direction.
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Signed-off-by: Fredrik Adelöw <freben@spotify.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Replace the LEFT OUTER JOIN + DISTINCT in the queryEntities CTE with
an INNER JOIN that drives from the search table for the sort field's
key. Entities lacking the sort field are excluded from both the result
and the count, aligning totalItems with navigable entities.
Removes DISTINCT (prerequisite: search table dedup migration).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@spotify.com>
The UNIQUE constraint on (entity_id, key, value) from the search
indices migration guarantees each entity appears at most once per
(key, original_value) group, making DISTINCT unnecessary. Removing
it lets the database use a simpler aggregation plan.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
* feat(ui): add DatePicker component
Add a single-date picker built on React Aria's DatePicker, mirroring the
existing DateRangePicker implementation. Includes the date field with
segmented input, calendar popover, BUI design tokens, bg consumer
pattern, and full keyboard/screen reader accessibility.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Erik Hughes <erikh@spotify.com>
* fix(ui): address DatePicker PR feedback
- Remove unused dataAttributes spread from DatePickerGroup
- Mark DatePickerGroupDefinition and DatePickerCalendarDefinition as
public so CSS class name changes appear in API reports
- Add Affected components line to changeset
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Erik Hughes <erikh@spotify.com>
* fix(ui): restore dataAttributes spread in DatePickerGroup
The bg: 'consumer' config on DatePickerGroupDefinition emits
data-on-bg attributes via useDefinition. Without spreading
dataAttributes onto <Group>, the CSS [data-on-bg] selectors
never match and background auto-increment doesn't work.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Erik Hughes <erikh@spotify.com>
---------
Signed-off-by: Erik Hughes <erikh@spotify.com>
Co-authored-by: Erik Hughes <erikh@spotify.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(scaffolder): add BUI theme for scaffolder forms
Add a Backstage UI (BUI) form theme as an alternative to the Material
UI theme. Toggled via formProps.theme or enableBackstageUi page config.
Includes BUI widgets, templates, field extension variants, and a ported
React Aria Autocomplete component.
Signed-off-by: benjdlambert <ben@blam.sh>
* refactor(scaffolder): use BUI Combobox and CheckboxGroup for form widgets
Signed-off-by: benjdlambert <ben@blam.sh>
* chore(scaffolder): enable BUI form flag and add kitchen sink demo template
Signed-off-by: benjdlambert <ben@blam.sh>
* fix(scaffolder): use outlined input style for BUI form widgets
Signed-off-by: benjdlambert <ben@blam.sh>
* fix(scaffolder): address BUI form PR feedback
Signed-off-by: benjdlambert <ben@blam.sh>
* fix(scaffolder): format CSS and regen API reports
Signed-off-by: benjdlambert <ben@blam.sh>
---------
Signed-off-by: benjdlambert <ben@blam.sh>
Andre Wanlin