docs: fix page-size docs and note groupIncludeSubGroups unsupported
Update all references to "up to 999 items" to accurately reflect that users are fetched in pages of 999 while groups use a smaller page of 100. Also document groupIncludeSubGroups as unsupported in the JSDoc @remarks, README, and comparison table. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: pillaris <pillaris@adobe.com>
This commit is contained in:
@@ -4,4 +4,4 @@
|
||||
|
||||
New package: `@backstage/plugin-catalog-backend-module-msgraph-incremental`
|
||||
|
||||
Introduces a cursor-based incremental ingestion provider for Microsoft Graph that processes users and groups one page at a time. Unlike `MicrosoftGraphOrgEntityProvider`, this module never holds the full dataset in memory — each burst processes a single page of up to 999 items. The `@odata.nextLink` cursor is persisted so a pod restart resumes from the last completed page rather than starting over.
|
||||
Introduces a cursor-based incremental ingestion provider for Microsoft Graph that processes users and groups one page at a time. Unlike `MicrosoftGraphOrgEntityProvider`, this module never holds the full dataset in memory — each burst processes a single page (up to 999 users or 100 groups). The `@odata.nextLink` cursor is persisted so a pod restart resumes from the last completed page rather than starting over.
|
||||
|
||||
@@ -9,8 +9,8 @@ AD tenants where holding the full dataset in memory at once is not practical.
|
||||
- **Cursor-based resumption** — the `@odata.nextLink` URL is persisted as the
|
||||
cursor, so a pod restart during ingestion resumes from the last completed page
|
||||
rather than starting over.
|
||||
- **Memory-efficient** — each burst processes a single page (up to 999 items),
|
||||
keeping memory usage flat regardless of tenant size.
|
||||
- **Memory-efficient** — each burst processes a single page (up to 999 users
|
||||
or 100 groups), keeping memory usage flat regardless of tenant size.
|
||||
- **Photo support** — user profile photos are fetched with a gated pre-check to
|
||||
avoid unnecessary API calls for users without photos.
|
||||
- **Transformer extension point** — user, group, organization, and provider
|
||||
@@ -70,4 +70,5 @@ catalog:
|
||||
| Memory usage | Full dataset in RAM | One page at a time |
|
||||
| Resume on restart | Starts from scratch | Resumes from cursor |
|
||||
| `userGroupMember*` options | Supported | Not supported |
|
||||
| `groupIncludeSubGroups` | Supported | Not supported |
|
||||
| Suitable for large tenants | No | Yes |
|
||||
|
||||
+7
-5
@@ -153,8 +153,8 @@ export interface MicrosoftGraphIncrementalEntityProviderOptions {
|
||||
* at a time, and provides them as User and Group entities for the catalog.
|
||||
*
|
||||
* Unlike `MicrosoftGraphOrgEntityProvider`, this provider never holds the full
|
||||
* dataset in memory at once. Each burst processes a single page of users or
|
||||
* groups (up to 999 items). This makes it suitable for very large tenants and
|
||||
* dataset in memory at once. Each burst processes a single page (up to 999
|
||||
* users or 100 groups). This makes it suitable for very large tenants and
|
||||
* avoids the memory pressure and long-running task issues of the full-scan
|
||||
* provider.
|
||||
*
|
||||
@@ -166,9 +166,11 @@ export interface MicrosoftGraphIncrementalEntityProviderOptions {
|
||||
* stitching derives `spec.memberOf` on users from these group membership lists.
|
||||
*
|
||||
* @remarks
|
||||
* `userGroupMemberFilter`, `userGroupMemberSearch`, and `userGroupMemberPath`
|
||||
* are not supported. Use `userFilter` / `userPath` to restrict which users are
|
||||
* ingested, and `groupFilter` / `groupSearch` to restrict which groups.
|
||||
* `userGroupMemberFilter`, `userGroupMemberSearch`, `userGroupMemberPath`, and
|
||||
* `groupIncludeSubGroups` are not supported. Use `userFilter` / `userPath` to
|
||||
* restrict which users are ingested, and `groupFilter` / `groupSearch` to
|
||||
* restrict which groups. Switch to `MicrosoftGraphOrgEntityProvider` if you
|
||||
* require any of these options.
|
||||
*
|
||||
* @public
|
||||
*/
|
||||
|
||||
Reference in New Issue
Block a user