Commit Graph

99 Commits

Author SHA1 Message Date
Esteban Barrios 8f53351a32 Rebase from master (#19)
* Fix client reference

* Fix CatalogClient reference in create-app too

* Move catalog-client dep to create-app

* Add catalog-client dep again..

* Use RequestOptions

* tsc

* Backwards compatible scaffolder

* Fix test

* Avoid importing core

* Use request options

* Increase code coverage

* Type those variables

* use the second mock

* Trying to make tests work on Github too

* Get clean yarn from master

* Only add authorization header if token exists

* Use request options

* Forward authorization header token

* Refactor to use context argument

* Allow the registration of optional locations via the catalog-client

* Make EntityRefLink a React.forwardRef in order to use it as root component in other components like ListItem

* Rework the user flow of the catalog-import plugin

* Update the create-app template

* Add luxon and remove moment

* Port moment to luxon

* Add change set

* Split the step label and content to be able to refactor the stepper from vertical to horizontal in the future

* Fix tsc, prettier and import issues

* Update .changeset/strange-olives-unite.md

Co-authored-by: Himanshu Mishra <himanshu@orkohunter.net>

* Update plugins/auth-backend/src/identity/TokenFactory.ts

Co-authored-by: Himanshu Mishra <himanshu@orkohunter.net>

* Change from minor to patch

* Update DatabaseKeyStore.ts

* feat: added a test for running a simple cypress server

* Add luxon and remove moment

* Port from moment to Luxon

* Add changeset

* Fixes following CR

* Rebased and reinstalled yarn.lock

* Changed to fromSQL

* Fix prettier formatting

* chore: reworking the lock file for the cypress deps

* chore: adding cypress.config

* chore: reworking some more examples

* chore: added some more simple stuff for e2e tests with cypress

* chore: reworking again

* chore: making cypress nice

* chore: added a sample test that works

* chore: reworking tests to use he cypress github-action instead

* chore: reconfigure Cypress run

* chore: don't install - we have deps

* chore: dump video recordings too

* chore: slimming down install script a little

* chore: make pretty

* chore: fixing syntax

* chore: fixing syntax again

* Align card elements with list

* Align card elements with List

* Add changeset

* Removed unused import

* chore: archive video footage with the correct path

* chore: fix lint warnings

* chore: fix prettier again

* chore: move files around for ease

* Added pattern for component name

* Add changeset

* stash: push to stash

* Return AboutCard title to group name

* Refactor children to use shared function

* TechDocs: Add visbility to migrate away from basic setup

* bugfix: [3310] Favoriting a component resets interface to your owned components

update changeset

remove unused import

fix changeset message

fix vale errors?

* Create utc DateTime object directly

* integration: update the gitlab config mandatory fields to match reality

* Resolve review comments

* Remove shadowed variables

* generateEntityDefinitions should return a PartialEntity

* Update roadmap with Backstage Community Sessions

* Make sure that SidebarItems are also active when on sub route

* Add whitespace around variable

* Add changeset

* Add word to dictionary

* Smater duration display

* Add changeset

* chore(deps-dev): bump @graphql-codegen/typescript from 1.18.1 to 1.20.2

Bumps [@graphql-codegen/typescript](https://github.com/dotansimha/graphql-code-generator/tree/HEAD/packages/plugins/typescript/typescript) from 1.18.1 to 1.20.2.
- [Release notes](https://github.com/dotansimha/graphql-code-generator/releases)
- [Changelog](https://github.com/dotansimha/graphql-code-generator/blob/master/packages/plugins/typescript/typescript/CHANGELOG.md)
- [Commits](https://github.com/dotansimha/graphql-code-generator/commits/@graphql-codegen/typescript@1.20.2/packages/plugins/typescript/typescript)

Signed-off-by: dependabot[bot] <support@github.com>

* chore: reworking folder structure to move cypress outside of the project

* chore: updating workflow to point at the right folder now

* chore: fixing cypress build

* microsite: Fix color contrast for pre tags

* Add package name to lockfile.ts error

* Create new-mangos-tap.md

* Changed changeset bump to be patch

* Pass registered Logger in ServiceBuilderImpl to requestLoggingHandler

`requestLoggingHandler` takes an optional `logger` parameter that it can use
to log incoming requests. The `ServiceBuilderImpl` was not passing  on this logger, if
set, to `requestLoggingHandler` middleware

* Add indices on columns referring location(id)

* Add changeset

* Catch catalog errors

* Add changeset

* Fixed parseUrl to output catalogPaths beginning with '/'

* Add configurable OAuth 2.0 scopes

- Add oauth2 config for optional scopes
- Document oauth2 config keys
- Add OAuth2 to demo app list of identity providers

* incorrectly added callbackUrl

* chore: added a simple readme to run some simple tests against a backstage instance

* chore(deps): bump @svgr/plugin-jsx from 5.4.0 to 5.5.0

Bumps [@svgr/plugin-jsx](https://github.com/gregberge/svgr) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/gregberge/svgr/releases)
- [Changelog](https://github.com/gregberge/svgr/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gregberge/svgr/compare/v5.4.0...v5.5.0)

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump @types/cors from 2.8.6 to 2.8.9

Bumps [@types/cors](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/cors) from 2.8.6 to 2.8.9.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/cors)

Signed-off-by: dependabot[bot] <support@github.com>

* fix: removing test e2e-test tsconfig.json file

* TechDocs: Preparers will take and return an etag for cache invalidation

* TechDocs: Implement etag based caching for url preparer

Signed-off-by: Himanshu Mishra <himanshu@orkohunter.net>

* TechDocs: Implement etag based caching for git preparer

Signed-off-by: Himanshu Mishra <himanshu@orkohunter.net>

* TechDocs: Warn when using legacy git preparer and dir preparer in backstage.io/techdocs-ref

Context: https://github.com/backstage/backstage/issues/4409

* TechDocs: Update tests for proper url preparer caching

* TechDocs: Deduplicate git clone for cache check

Signed-off-by: Himanshu Mishra <himanshu@orkohunter.net>

* TechDocs: Add changesets for deprecation and proper caching

* TechDocs: Implement caching for dir preparer

* Update the changesets

* scaffolder-backend: remove auth-backend dependency

* catalog-client: rename ApiContext to CatalogRequestOptions + avoid export

* refactor existing usage of ApiContext

* techdocs: don't swallow errors other than NotModifiedError

* catalog-import: removed bonus code

* techdocs: meaningful logs when readTree starts

* Flatten the options of the CatalogImportPage

* inline optional id token auth headers

* Define relationship to software catalog and loose coupling by convention.

* Clarify intentions around bulk vs. incremental index management.

* Break apart backend plugins & clarify indexer/plugin relationship.

* update changeset bump levels

* [Search] documentation update (#4459)

* delete link to issue as it is closed

* replace usage of easy as its very subjective

* Update docs/features/search/architecture.md

Co-authored-by: Adam Harvey <adam.harvey@dxc.com>

* prettier....:

Co-authored-by: Adam Harvey <adam.harvey@dxc.com>

* Own it.

* scaffolder: include backstage identity token in requests

* review feedback tweaks

* docs/apis: update to use named plugin var

* sentry: update plugin instance export name

* clear other field when toggling reason

* changeset

* fix(pagerduty): use the luxon date library

* Don't pass default as a scope to OIDC providers

* docs(TechDocs): Add GitHub Actions CI example with AWS S3

* tech-radar: migrated to new composability API

* cost-insights: migrate to new composability API

* Changeset

* Fix line endings

* Prettier

* Remove unnecessary scopes from OIDC defaultApi

* Bump plugin-auth-backend to a minor change,
add documentation for fixing it

* Fix md syntax error

* Fix Vale spelling error

* Remove defaultScopes from OIDC api

* [ImgBot] Optimize images

/plugins/catalog-import/docs/catalog-import-screenshot.png -- 613.39kb -> 356.48kb (41.88%)

Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com>

* github/workflows: use lax config checks

* Use commented-out example value for scope

* feat(pagerduty): add changeset

* Apply suggestions from code review

Co-authored-by: Rémi Doreau <32459935+ayshiff@users.noreply.github.com>

* feat(catalog): add entity links card component

* update entity links changeset

* Fix prettier

* chore(deps-dev): bump @types/http-proxy from 1.17.4 to 1.17.5

Bumps [@types/http-proxy](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/http-proxy) from 1.17.4 to 1.17.5.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/http-proxy)

Signed-off-by: dependabot[bot] <support@github.com>

* Revert "Merge pull request #3953 from ayshiff/feature/techdocs-aws-sdkv3"

This reverts commit 1c7771871e, reversing
changes made to 65d5c8e6fe.

* Revert "Merge pull request #4035 from ayshiff/feature/catalog-backend-aws-sdk-v3"

This reverts commit 1df9134648, reversing
changes made to d45a510069.

* Add changeset

* feat: support custom app icons

* backend-common: implement UrlReader.search for the other providers too

* Remove changeset

* docs(TechDocs): Improvements to the CI example

Signed-off-by: Himanshu Mishra <himanshu@orkohunter.net>

* fix(catalog): entity links incorrect wrapping

* Remove unused functions and luxon

* Removed old imports

* Fix broken links in documentation (#4418)

* Update IdentityApi.md

* Update docs/reference/utility-apis/IdentityApi.md

* Update docs/reference/utility-apis/IdentityApi.md

* ran yarn docgen

* Use JS Date instead of Datetime

* Add line

* Prettier fix

* Move parseDate function

* Version Packages

* chore: fix the create-app version

* Limit the props that are forwarded to the Link component in the EntityRefLink

* Use routed tabs to link to every settings page

* fix up yarn after release

* chore: add lockfile to words

* catalog-info: add links

* Export Select component from core

I saw the Select component on storybook and went
to use it but it seems it's not exported. Any chance
it could be exported?

* bug: filepath can be returned as undefined from `git-url-parse` let's default to empty

* chore: might as well do this for all parsing

* chore: changeset

* Add changeset for fixing requestLoggingHandler

* chore(deps-dev): bump @storybook/addon-actions from 6.1.11 to 6.1.17

Bumps [@storybook/addon-actions](https://github.com/storybookjs/storybook/tree/HEAD/addons/actions) from 6.1.11 to 6.1.17.
- [Release notes](https://github.com/storybookjs/storybook/releases)
- [Changelog](https://github.com/storybookjs/storybook/blob/next/CHANGELOG.md)
- [Commits](https://github.com/storybookjs/storybook/commits/v6.1.17/addons/actions)

Signed-off-by: dependabot[bot] <support@github.com>

* Port AboutCard

* Fix tests

* Revert EntityPage changes

* docs: Merge auth glossary with main glossary

* Remove the "Move repository" menu entry from the catalog page, as it's just a placeholder

It will be easy to bring it back later, but for now it just confuses users that it's not doing anything. It's also hard to remove for integrators.

* make the template cards conform to mui standard

Co-authored-by: Erik Larsson <erik.larsson@schibsted.com>
Co-authored-by: Dominik Henneke <dominik.henneke@sda-se.com>
Co-authored-by: Nils Streijffert <nstreijffert@spotify.com>
Co-authored-by: Nils Streijffert <nils.streijffert@gmail.com>
Co-authored-by: Himanshu Mishra <himanshu@orkohunter.net>
Co-authored-by: blam <ben@blam.sh>
Co-authored-by: Nir Gazit <nir.gzt@gmail.com>
Co-authored-by: Adam Harvey <adam.harvey@dxc.com>
Co-authored-by: NHI TRAN <nhid@ntran.io>
Co-authored-by: Fredrik Adelöw <freben@gmail.com>
Co-authored-by: Oliver Sand <oliver.sand@sda-se.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Debajyoti Halder <rondebajyoti@gmail.com>
Co-authored-by: Patrik Oldsberg <poldsberg@gmail.com>
Co-authored-by: Gowind <petrovgovind@gmail.com>
Co-authored-by: Alan Crosswell <alan@columbia.edu>
Co-authored-by: Eric Peterson <ericpeterson@spotify.com>
Co-authored-by: Emma Indal <emma.indahl@gmail.com>
Co-authored-by: Ryan Vazquez <ryanv@spotify.com>
Co-authored-by: Ryan Vazquez <ryanmvazquez@gmail.com>
Co-authored-by: Remi <remi.d45@gmail.com>
Co-authored-by: Ryan Manny <rmanny@apptio.com>
Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com>
Co-authored-by: Rémi Doreau <32459935+ayshiff@users.noreply.github.com>
Co-authored-by: Andrew Thauer <6507159+andrewthauer@users.noreply.github.com>
Co-authored-by: Joel Low <joel@joelsplace.sg>
Co-authored-by: Eric Peterson <iamEAP@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Iain Billett <iain@roadie.io>
2021-02-12 16:08:46 +01:00
Fredrik Adelöw 5a51635197 backend-common: implement UrlReader.search that does glob matching 2021-02-09 12:16:12 +01:00
Himanshu Mishra 38469b66bf techdocs: fix tests by reusing the instantiated discovery 2021-02-02 13:51:05 +01:00
Parth Shandilya e44925723e TechDocs: Make requestUrl and storageUrl optional configs by using discovery APIs
closes #3715

Co-authored-by: Himanshu Mishra <himanshu@orkohunter.net>
2021-02-02 13:51:04 +01:00
Himanshu Mishra 8eb862c58e Merge pull request #4017 from viavarejo/feature/techdocs-azure-storage 2021-01-31 16:03:24 +01:00
vitorgrenzel 59b8d5a0d9 feat(techdocs-common): add Azure Blob Storage 2021-01-28 16:16:22 -03:00
vitorgrenzel c777df180a feat(techdocs-common): add Azure Storage 2021-01-28 16:16:21 -03:00
Himanshu Mishra 03a2d95c71 TechDocs: Update tests with mock url reader 2021-01-28 15:32:09 +01:00
Matei David a5e27d5c1d Introduce techdocs metadata in techdocs-common
* Add TechdocsMetadata type in backend plugin endpoint
 * Introduce TechDocsMetadata type in frontend
 * Add changeset
 * Remove old thennable metadata resp
 * Address PR feedback
  - Remove explicit type annotation on TechDocsMetadata
  - Reintroduce res.send instead of throwing an error
  - Change logger info to logger error
 * Add TechDocsMetadata type in frontend plugin
 * Commit yarn.lock
 * Introduce JSON5 and remove parsing in local pub
Signed-off-by: Matei David <matei.david.35@gmail.com>
2021-01-16 18:33:53 +00:00
Himanshu Mishra cb7af51e73 techdocs: cache docs site when built using urlReader for 30 minutes
This caching makes it usable experience, so that docs are not built on every load.
In future readTree will support a method to fetch the timestamp of the latest HEAD. And
it should be used to invalidate the cache.
2021-01-13 21:09:44 +01:00
Himanshu Mishra 2165901c00 TechDocs: Generator now requires input and output directory and does not create them
The creation of temporary directories and their clean up should be handled on its own. techdocs-cli already has a fixed input output dir requirement. It makes that generator accepts these values as requirements.
2021-01-11 22:33:10 +01:00
Remi b9e6d305e2 fix(techdocs): typos + add tests + requested changes 2020-12-30 10:25:57 +01:00
Remi c4fcd8b180 feat(techdocs-backend): add awsS3 2020-12-30 10:22:12 +01:00
Himanshu Mishra c91e4d6b04 techdocs: Use @backstage/integration for scm tokens and request options 2020-12-28 21:08:54 +01:00
Himanshu Mishra 2ac77aeec2 techdocs-backend: Remove individual preparers and generators from app backend, to use factory methods
When a new Backstage app is setup, the techdocs backend file contains all the preparers (dir, github, gitlab, etc.) and same for generators. While this providers added customizibility, it comes with a cost of complexity in setting up the app backend. New preparers' updates would need users to update their app's techdocs backend file.

Scaffolder backend has already moved to this proposed pattern.
2020-12-17 11:33:56 +01:00
Patrik Oldsberg 1da7d2c657 plugins,packages: replace usages of ConfigReader.from 2020-12-15 19:04:50 +01:00
Himanshu Mishra 24bcdd0829 techdocs: Use googleGcs for publisher type instead of google_gcs 2020-12-09 18:58:12 +01:00
Himanshu Mishra 9bd130a73a techdocs: Use 'local' and 'external' for alternatives for techdocs.builder config
Co-authored-by: freben <freben@gmail.com>
2020-12-09 12:31:40 +01:00
Himanshu Mishra 372160ede5 Update README of all three techdocs packages/plugins 2020-12-05 18:07:50 +01:00
Himanshu Mishra ff3dd9c02b create-app: Add new techdocs default configs 2020-12-05 11:13:27 +01:00
Himanshu Mishra a212e43142 techdocs: Add JSON Schema config
Show better error if techdocs.builder is set to 'ci' and if no docs are found.
Return 404 from googleStorage client when a file is not found.
2020-12-05 10:48:54 +01:00
Himanshu Mishra c91fd25a53 techdocs-backend: Fix logic to determine if docs have been published 2020-12-04 22:37:14 +01:00
Himanshu Mishra 07e7c201e8 techdocs-backend: Wait for some time before published files show up in storage 2020-12-04 22:01:45 +01:00
Himanshu Mishra 084670b37a Google Cloud Storage working with TechDocs (publish and fetch)
Plus
1. Introduce ncessary configs to connect with storage
2. Introduce config to prefer CI build of docs, or techdocs-backend to build and publish them.
3. Write guide how to use Google Cloud Storage with TechDocs
4. Add a TechDocs Configuration reference page in docucmentation
2020-12-03 22:40:15 +01:00
Himanshu Mishra e09cff1682 Abstract Publisher APIs from the router
1. Move fetchMetadata and serve static files to local publisher
2. Create factory class to create publishers based on app cofig
2020-12-02 13:05:44 +01:00
Himanshu Mishra 64737f447a App/Backend should import techdocs plugin only, not common package
If a backstage App wants to enable techdocs plugin, it should only have to care about installing @backstage/plugin-techdocs and @backstage/plugin-techdocs-backend.
@backstage/plugin-techdocs-backend should re-export necessary APIs from techdocs-common
2020-12-01 20:48:58 +01:00
Himanshu Mishra 7481601883 techdocs-backend: Add non-zero tests for the CI to pass 2020-12-01 20:48:58 +01:00
Himanshu Mishra ff348df9fe techdocs-common: Initialize package and move common code from techdocs-backend 2020-12-01 20:48:58 +01:00
Himanshu Mishra 9ebc897351 TechDocs: Refactor Publisher to pave way for external publishers 2020-12-01 20:48:58 +01:00
Himanshu Mishra 21f86aa478 Merge pull request #3337 from adrianke77/master 2020-12-01 09:48:02 +01:00
Chongyang Adrian, Ke ae95c7ff3e techdocs-backend: update Gitlab clone auth 2020-12-01 10:18:33 +08:00
Himanshu Mishra dbca620ff9 Use fs-extra and async file read method 2020-11-26 20:41:40 +01:00
Himanshu Mishra 4b53294a6c TechDocs: Update mkdocs.yml with repo_url when possible before building docs 2020-11-26 10:55:38 +01:00
Himanshu Mishra 266e93b478 TechDocs: Refactor metadata retrieval
1. Don't use mkdocs in name of APIs or variables. It is an implementation detail and liable to change in future.
2. techDocsApi.getMetadata('mkdocs') and techDocsAPI.getMetadata('entity') should be two separate functions just because their responses differ in structure. They should also be type checked.
3. Use either 'techdocs' or 'TechDocs' consistently. 'techDocs' seems like an unnecessary third way to write TechDocs, which can be avoided.
4. Remove unused /plugins/techdocs-backend/src/service/metadata.ts file
2020-11-25 19:39:48 +01:00
Fredrik Adelöw fae3fafcfa silence some new ts squigglies in the editor (#3435) 2020-11-25 09:57:16 +01:00
Himanshu Mishra 8fd5e64351 TechDocs: Tell users when index.md is missing (better error message) (#3429)
* TechDocs: Logger already prints the name of plugin

* TechDocs: Display a custom error message if provided

* TechDocs: throw custom error message if index.md is not present

* Language improvements. Thanks @freben

Co-authored-by: Fredrik Adelöw <freben@gmail.com>

Co-authored-by: Fredrik Adelöw <freben@gmail.com>
2020-11-25 09:52:19 +01:00
Himanshu Mishra f8877545c5 TechDocs: Reject promise if call to getBaseUrl fails 2020-11-18 20:35:22 +01:00
Himanshu Mishra 66daa47c6a 1. Remove DiscoveryApi (frontend) and use PluginEndpointDiscovery(SingleHostDiscovery) (backend)
2. Mock class for testing
2020-11-18 20:29:07 +01:00
Himanshu Mishra 3b314807c6 TechDocs: Use discovery instead of config to read base url 2020-11-17 21:03:56 +01:00
Himanshu Mishra c01bf361a4 Modify tests to include new config param in LocalPublish 2020-11-17 20:19:30 +01:00
Himanshu Mishra 65edcdee22 TechDocs: Remove hardcoded backend URL from techdocs-backend 2020-11-17 20:03:44 +01:00
Patrik Oldsberg bd1cef1847 backend-common: review comments for readTree + switch archive return type to a stream 2020-11-13 00:43:20 +01:00
Patrik Oldsberg b672492e21 backend-common: make .readTree required 2020-11-12 13:59:10 +01:00
Patrik Oldsberg 6417d966bb backend-common,techdocs-backend: refactor .readTree to accept options and implement GitHub reader with response utils 2020-11-12 13:53:22 +01:00
Patrik Oldsberg e74422b4d8 backend-common: make ReadTreeResponse.files async 2020-11-12 12:27:33 +01:00
Sebastian Qvarfordt f059eaf755 readTree method on UrlReader (#2925)
* Basic implementation of readTree on github and use it in techdocs

* Added a UrlPreparer and cleaned up some minor things

* Download entire archive and filter out the wanted files in readTree

* Added test for dir output of tree reader

* Fixed formatting

* Fixed formatting

* Some cleanups and fixes

* Fix typing issues

* Fix prettier

* Fix prettier

* Removed unused dependency

* Fixed comments on PR

* Prettier

* Moved @types/fs-extra to devDeps

* Fixed another PR comment

* Added a test for getDocFilesFromRepository

* Prettier
2020-11-09 16:28:51 +01:00
Patrik Oldsberg af141c0766 rewrite repo location urls 2020-11-03 11:03:49 +01:00
Iain Billett 65792dd070 Use discovery api to build URLs in techdocs
Use the discovery api to build catalog API URLs and URLs to locally published
metadata.
2020-11-02 10:59:51 +00:00
Patrik Oldsberg 24e5730200 techdocs-backend: remove duplicate log tagging 2020-10-27 10:14:47 +01:00
Patrik Oldsberg bc9f536545 Merge pull request #2994 from spotify/rugvip/conflagrate
config-loader: switch to using --config options to load in config
2020-10-22 18:01:21 +02:00