paulononato/backstage
1
0
Fork 0
Code Issues Pull Requests Actions 17 Packages Projects Releases Wiki Activity

Update docs and code based on review comments

Browse Source 
Signed-off-by: abhishekbvs <bvsabhishek@gmail.com>
This commit is contained in:
abhishekbvs
2025-10-24 00:32:24 +05:30
parent 637a3de8d8
commit 999d1c1747
12 changed files with 32 additions and 154 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/integrations/github/discovery.md
+4 -18
View File
@@ -210,6 +210,8 @@ catalog:
filters: # optional filters
branch: 'develop' # optional string
repository: '.*' # optional Regex
pageSizes:
repositories: 25
wildcardProviderId:
organization: 'new-org' # string
catalogPath: '/groups/**/*.yaml' # this will search all folders for files that end in .yaml
@@ -309,25 +311,9 @@ If you do so, `default` will be used as provider ID.
- **`scope`** _(optional)_:
`'global'` or `'local'`. Sets the scope of concurrency control.
- **`pageSizes`** _(optional)_:
Configure page sizes for GitHub GraphQL API queries. This can help prevent `RESOURCE_LIMITS_EXCEEDED` errors with large organizations.
Configure page sizes for GitHub GraphQL API queries. This can help prevent `RESOURCE_LIMITS_EXCEEDED` errors.
- **`repositories`** _(optional)_:
Number of repositories to fetch per page. Defaults to `25`.
Example with page sizes configuration:
```yaml
catalog:
providers:
github:
myOrganization:
organization: 'my-large-org'
catalogPath: '/catalog-info.yaml'
schedule:
frequency: { minutes: 30 }
timeout: { minutes: 3 }
pageSizes:
repositories: 15 # Reduce if hitting API limits
```
Number of repositories to fetch per page. Defaults to `25`. Reduce this value if hitting API resource limits.
## GitHub API Rate Limits
docs/integrations/github/org.md
+9 -29
View File
@@ -79,6 +79,10 @@ catalog:
initialDelay: { seconds: 30 }
frequency: { hours: 1 }
timeout: { minutes: 50 }
pageSizes:
teams: 25
teamMembers: 50
organizationMembers: 50
- id: ghe
githubUrl: https://ghe.mycompany.com
orgs: ['internal-1', 'internal-2', 'internal-3']
@@ -94,37 +98,13 @@ Directly under the `githubOrg` is a list of configurations, each entry is a stru
- `githubUrl`: The target that this provider should consume
- `orgs` (optional): The list of the GitHub orgs to consume. If you only list a single org the generated group entities will use the `default` namespace, otherwise they will use the org name as the namespace. By default the provider will consume all accessible orgs on the given GitHub instance (support for GitHub App integration only).
- `schedule`: The refresh schedule to use, matches the structure of [`SchedulerServiceTaskScheduleDefinitionConfig`](https://backstage.io/docs/reference/backend-plugin-api.schedulerservicetaskscheduledefinitionconfig/)
- `pageSizes` (optional): Configure page sizes for GitHub GraphQL API queries to prevent `RESOURCE_LIMITS_EXCEEDED` errors with large organizations. See [Page Sizes Configuration](#page-sizes-configuration) below for details.
- `pageSizes` (optional): Configure page sizes for GitHub GraphQL API queries to prevent `RESOURCE_LIMITS_EXCEEDED` errors. You can configure the following page sizes:
### Page Sizes Configuration
- `teams`: Number of teams to fetch per page when querying organization teams (default: 25)
- `teamMembers`: Number of team members to fetch per page when querying team members (default: 50)
- `organizationMembers`: Number of organization members to fetch per page (default: 50)
For large GitHub organizations (200+ teams), you may encounter `RESOURCE_LIMITS_EXCEEDED` errors due to GitHub's GraphQL API resource limits. You can configure page sizes to reduce the number of records fetched per API request:
```yaml title="app-config.yaml"
catalog:
providers:
githubOrg:
- id: production
githubUrl: https://github.com
orgs: ['large-org']
schedule:
frequency: { hours: 1 }
timeout: { minutes: 50 }
pageSizes:
teams: 25 # Default: 25
teamMembers: 50 # Default: 50
organizationMembers: 50 # Default: 50
repositories: 25 # Default: 25
```
**Configuration Options:**
- `teams`: Number of teams to fetch per page when querying organization teams (default: 25)
- `teamMembers`: Number of team members to fetch per page when querying team members (default: 50)
- `organizationMembers`: Number of organization members to fetch per page (default: 50)
- `repositories`: Number of repositories to fetch per page (default: 25)
**Note:** Reducing page sizes will result in more API calls and slightly longer sync times, but will prevent resource limit errors for large organizations.
Reducing page sizes will result in more API calls and slightly longer sync times, but will prevent API resource limits for organizations with large number of teams and members.
### Events Support