From 85709f15de88cbc5bcc0724614cf4ee6e5ecc1c8 Mon Sep 17 00:00:00 2001 From: Heikki Hellgren Date: Thu, 1 Feb 2024 16:17:11 +0200 Subject: [PATCH 1/3] beps: add information that was agreed on the meeting Signed-off-by: Heikki Hellgren --- beps/0001-notifications-system/README.md | 275 ++++++++++++++++------- 1 file changed, 199 insertions(+), 76 deletions(-) diff --git a/beps/0001-notifications-system/README.md b/beps/0001-notifications-system/README.md index 2257e07261..f3e98f107d 100644 --- a/beps/0001-notifications-system/README.md +++ b/beps/0001-notifications-system/README.md @@ -2,7 +2,9 @@ title: Backstage Notifications System status: provisional authors: - - Rugvip + - '@Rugvip' + - '@drodil' + - '@mareklibra' project-areas: - core creation-date: 2023-01-12 @@ -76,7 +78,7 @@ In the backend the signal plugin implements a general purpose message bus for se In the frontend the signal plugin has a persistent connection to the signal backend. This is initially implemented as a WebSocket connection, but could in the future also receive fallback mechanisms such as Server Sent Events or long polling. It is important that this connection is authenticated as we will be routing signals to specific users. The exact implementation of the authentication is not part of this proposal, but it should use whatever the outcome of the discussion in issue [#19581](https://github.com/backstage/backstage/issues/19581) is. -In order to route signals from the sender to the intended receiver the signal plugin uses the concept of signaling channels. They are just like the topics on the message bus, but we use separate terminology in order to avoid confusion. They also sit one layer beneath the even topic in the protocol stack, where the signal channel is communicated as part of the event payload. In the frontend, the signal plugin exposes an API to subscribe to a signal channel. Each time the user receives a signal on the specified channel, a listener is called with the signal payload. +In order to route signals from the sender to the intended recipient the signal plugin uses the concept of signaling channels. They are just like the topics on the message bus, but we use separate terminology in order to avoid confusion. They also sit one layer beneath the even topic in the protocol stack, where the signal channel is communicated as part of the event payload. In the frontend, the signal plugin exposes an API to subscribe to a signal channel. Each time the user receives a signal on the specified channel, a listener is called with the signal payload. ### Notifications Plugin @@ -85,38 +87,44 @@ The role of the notifications plugin is to manage the lifecycle of notifications The notification backend stores notification using the [database service](https://backstage.io/docs/backend-system/core-services/index#database). In particular it needs to store the following information for each notification: - ID -- Title -- Priority -- Origin -- Topic (optional) -- Creation timestamp (generated) -- Receivers -- Read status by user -- Payload - - icon (optional) - - message (optional) - - links (list, optional) +- Read date +- Done date +- Saved status +- Creation date +- Updated date (optional, for topic notifications) +- Payload: + - Title + - Description + - Priority + - Origin + - Link + - Recipients + - Topic (optional) + - Icon (optional) + - Image URL (optional) -The receivers is **not** a list of users, but rather a filter that describes who should receive the notification. It must be possible to evaluate this filter in a database query, so that we can efficiently fetch all notifications for a given user. The same filter will also be used by the signal backend to determine which users should receive a signal. +The recipients is **not** a list of users, but rather a filter that describes who should receive the notification. It must be possible to evaluate this filter in a database query, so that we can efficiently fetch all notifications for a given user. The same filter will also be used by the signal backend to determine which users should receive a signal. -The title is mandatory human readable text shortly describing the notifications. +The title is mandatory human-readable text shortly describing the notifications. -The priority is one of the predefined values to help with presentation of the notifications or their filtering. Values: Critical, High, Normal, Low. +The description is an optional human-readable text providing more details to the user. + +The priority is one of the predefined values to help with presentation of the notifications or their filtering. Allowed values are: `critical`, `high`, `normal` (default), and `low`. The origin is a string identifying the creator of the notification, i.e. the BE plugin or external system. -The topic is an optional string helping to group notifications of particular context. Its use cases include: +If a topic is provided for a notification, it will either: -- several notifications emitted by an asynchronous external task can be grouped by a single topic -- a BE plugin can group several related messages to a particular processing, i.e. asynchronous progress monitoring +- Create a new notification if user doesn't have existing notification in that topic +- Update existing notification in that topic so that it's unread considered as a new notification -The timestamp of message creation is auto-generated by the notifications backend at the time of receiving a request to create the notification. +The idea of topic is to be able to reuse notifications about same information for example specific failing CI build. -The icon is optional and is meant to improve UX. A string referencing an icon name from MUI icon library of a defined version. If missing, it will be determined from the priority +The timestamp of notification creation is auto-generated by the notifications backend at the time of receiving a request to create the notification. -The message is an optional human readable text providing more details to the user. +The icon and image URL are optional and is meant to improve UX. A string referencing an icon name from MUI icon library of a defined version. If missing, it will be determined from the priority. -The links are an array of title-URL pairs. As an example, they can be used +The link is a relative or absolute URL. As an example, it can be used: - by an external system to provide more details related to the notification - by an external system to request an action within an asynchronous task @@ -160,89 +168,202 @@ The following backend service interfaces are added as part of this proposal. ```ts // TODO - We may want to add an additional wrapping here with interfaces for Notification and NotificationParameters -interface SendNotificationRequest { - receivers: SignalReceivers; - - /* TODO - please contribute :) */ -} - -interface SendNotificationResponse { - id: string; - receivers: SignalReceivers; - - /* TODO - please contribute :) */ -} - -interface NotificationService { - sendNotification( - request: SendNotificationRequest, - ): Promise; -} -``` - -#### `SignalService` - -```ts -type SignalReceivers = +export type NotificationRecipients = | { type: 'entity'; - entityRef: string; // typically a user or group entity + entityRef: string | string[]; // typically a user or group entity, otherwise `spec.owner` of the entity/entities } | { type: 'broadcast'; // all users }; -interface Signal { - channel: string; - receivers: SignalReceivers; - payload: JsonObject; +export type NotificationType = 'undone' | 'done' | 'saved'; + +export type Notification = { + id: string; + userRef: string; + title: string; + description: string; + origin: string; + link: string; + created: Date; + saved: boolean; + read?: Date; + topic?: string; + updated?: Date; + icon?: string; + image?: string; +}; + +interface SendNotificationRequest { + recipients: NotificationRecipients; + title: string; + description: string; + link: string; + origin: string; + topic?: string; + icon?: string; + image?: string; } -interface SignalService { - signal(signal: Signal): Promise; +interface NotificationService { + sendNotification(request: SendNotificationRequest): Promise; } ``` +Notification is always targeted to a single user unless it's a broadcast. The `notification-backend` will figure out which users will receive notification based on the `recipients` parameter. + +Each notification contains a `title`, a `description` and a `link` for further information. The `created`, `id`, `read` and `saved` properties are handled by the backend based and cannot be changed during the notification creation. + +Calling `sendNotification` should never throw an error so that it doesn't block the current processing. Notifications should be considered as second-level citizens that are not critical if not delivered. + +The `notification-backend` shall provide necessary parameters for paging and filtering notifications from the frontend. + +#### `SignalService` + +Upon creating a new notification, a signal will be emitted by the `notification-backend`. + +The channel for notifications will be named `notifications` and the message will contain necessary information about the notification for rendering as well as action telling to refresh the notifications using the REST API. + +The notification payload must be in the signal to be able to show notification in the UI immediately after receiving the signal. + +If the notification status is updated, the signal service shall emit a signal without the notification payload and only with the action. + +```ts +export type SignalPayload = { + recipients: string[] | string | null; + channel: string; + message: SignalType; +}; + +interface SignalService { + publish( + signal: SignalPayload, + ): Promise; +} +``` + +Example signal payload for a new notification: + +```json +{ + "action": "new_notification", + "notification": { + "title": "New notification", + "description": "This is the longer description of the notification", + "link": "/catalog" + } +} +``` + +#### Future considerations + +- Add icon for the notification request (for UX purposes) +- Add image url for the notification request (for Web Notification API purposes) +- Broadcast messages are to be saved to a separate table for performance reasons +- OpenAPI tooling is taken into use for the notification router and client + ### Frontend API +Notification frontend shows users their own notifications in its own page and the number of unread notifications in the main menu item. + +By default, notifications that are `undone` will be shown in the notifications view. The notification `read` status is indicated by the UI. + +Notifications are set to `read` when the notification link is opened or the notification is set as `done` by the user. + +Notifications can be set to `done` by the user, and they are filtered out of the main view. + +Notifications can be saved for later use by the user. + +Notifications can be filtered by their `read`, `done` and `saved` statuses. + +Notifications are displayed in pages. + +User can search for notifications based on their title and description. + The following frontend API is added as part of this proposal. +#### `NotificationsApi` + +```ts +export type GetNotificationsOptions = { + type?: NotificationType; + offset?: number; + limit?: number; + search?: string; +}; + +export type NotificationStatus = { + unread: number; + read: number; +}; + +export type NotificationIds = { + ids: string[]; +}; + +interface NotificationsApi { + getNotifications(options?: GetNotificationsOptions): Promise; + + getStatus(): Promise; + + markDone(ids: string[]): Promise; + + markUndone(ids: string[]): Promise; + + markRead(ids: string[]): Promise; + + markUnread(ids: string[]): Promise; + + markSaved(ids: string[]): Promise; + + markUnsaved(ids: string[]): Promise; +} +``` + #### `SignalApi` > TODO - we likely need a slightly different approach here, since APIs are lazy loaded. +> TODO - signal typing in https://github.com/backstage/backstage/pull/22656 + +```ts +export type NotificationSignal = { + action: string; + notification?: { + title: string; + description: string; + link: string; + icon?: string; + image?: string; + }; +}; + +const { lastSignal, isSignalsAvailable } = + useSignal('notifications'); +// isSignalsAvailable can be used to fallback to long-polling if wanted +``` + ```ts interface SignalSubscriber { unsubscribe(): void; } interface SignalApi { - subscribe( + subscribe( channel: string, - listener: (payload: JsonObject) => void, + onMessage: (message: SignalType) => void, ): SignalSubscriber; } ``` -> TODO: Add payload formats and API details +#### Future considerations - +- Notification frontend utilizes [Web Notification API](https://developer.mozilla.org/en-US/docs/Web/API/Notification) to notify user for new notifications +- Unread notifications count is displayed in the title of the page +- Configuration can be used to enable or disable features in the notification system (Web Notification API, title change, etc.) +- Replace absent signal service with long polling. This requires changes to the `signals` plugin as well. +- Notifications can have priority that is used to determine how notifications are shown to the user ## Release Plan @@ -251,12 +372,12 @@ The notification and signal plugins are released as two new plugins in the Backs For the notification plugin to reach a stable release we much reach the following: - A stable notifications payload format. -- A stable notifications receiver filter format. +- A stable notifications recipient filter format. - The event broker must have at least one implementation that supports scaled deployments. For the signal plugin to reach a stable release we much reach the following: -- A stable signal receiver filter format. +- A stable signal recipient filter format. - A stable signal channel API in the frontend. If any changes are required to the frontend framework to facilitate the implementation of notifications or signals, these will be released as experimental alpha features. They will stay in alpha until they are deemed stable enough, which must happen before a stable release of the notifications system. @@ -265,6 +386,8 @@ If any changes are required to the frontend framework to facilitate the implemen Since the signal plugin relies on the event broker for communication, it is a dependency for the notifications system as a whole. The event broker does not currently implement any transport for scaled deployments, which is a requirement for scaled deployments of the notification system. +Alternatively the notifications can work without the signals, but in this case the notifications are updated only during page refresh. + ## Alternatives ### Signal Plugin Separation From 3af10366c3ce9944b74aa512a91999b1cf6e2cb7 Mon Sep 17 00:00:00 2001 From: Heikki Hellgren Date: Fri, 2 Feb 2024 12:16:58 +0200 Subject: [PATCH 2/3] beps: fix review comments Signed-off-by: Heikki Hellgren --- beps/0001-notifications-system/README.md | 65 ++++++++++++------------ 1 file changed, 32 insertions(+), 33 deletions(-) diff --git a/beps/0001-notifications-system/README.md b/beps/0001-notifications-system/README.md index f3e98f107d..726a9ff9a6 100644 --- a/beps/0001-notifications-system/README.md +++ b/beps/0001-notifications-system/README.md @@ -80,6 +80,14 @@ In the frontend the signal plugin has a persistent connection to the signal back In order to route signals from the sender to the intended recipient the signal plugin uses the concept of signaling channels. They are just like the topics on the message bus, but we use separate terminology in order to avoid confusion. They also sit one layer beneath the even topic in the protocol stack, where the signal channel is communicated as part of the event payload. In the frontend, the signal plugin exposes an API to subscribe to a signal channel. Each time the user receives a signal on the specified channel, a listener is called with the signal payload. +Upon creating a new notification, a signal will be emitted by the `notification-backend`. + +The channel for notifications will be named `notifications` and the message will contain necessary information about the notification for rendering as well as action telling to refresh the notifications using the REST API. + +The notification payload must be in the signal to be able to show notification in the UI immediately after receiving the signal. + +If the notification status is updated, the signal service shall emit a signal without the notification payload and only with the action. + ### Notifications Plugin The role of the notifications plugin is to manage the lifecycle of notifications. The backend plugin provides an API for other backends to send notifications, as well as an accompanying [backend service](https://backstage.io/docs/backend-system/architecture/services). It also provides a separate API for the frontend plugin to read notifications for an individual user and manage the read status of notifications. @@ -99,7 +107,7 @@ The notification backend stores notification using the [database service](https: - Origin - Link - Recipients - - Topic (optional) + - Scope (optional) - Icon (optional) - Image URL (optional) @@ -113,12 +121,12 @@ The priority is one of the predefined values to help with presentation of the no The origin is a string identifying the creator of the notification, i.e. the BE plugin or external system. -If a topic is provided for a notification, it will either: +If a scope is provided for a notification, it will either: -- Create a new notification if user doesn't have existing notification in that topic -- Update existing notification in that topic so that it's unread considered as a new notification +- Create a new notification if user doesn't have existing notification in that scope and origin +- Update existing notification in that scope and origin so that it's unread considered as a new notification -The idea of topic is to be able to reuse notifications about same information for example specific failing CI build. +The idea of scope is to be able to reuse notifications about same information for example specific failing CI build. The timestamp of notification creation is auto-generated by the notifications backend at the time of receiving a request to create the notification. @@ -130,7 +138,9 @@ The link is a relative or absolute URL. As an example, it can be used: - by an external system to request an action within an asynchronous task - by a BE plugin to provide link to other part of the Backstage UI (i.e. to the Catalog) -The notification backend does not provide any new permissions, since creating notifications can only be done by other backend plugins, while reading notifications can only be done by the authenticated user. It is possible that we want to add a permissions for reading notifications, in particular for admin and impersonation use cases, but that is not part of this proposal or the initial implementation. +The `notification-backend` does not provide any new permissions, since creating notifications can only be done by other backend plugins, while reading notifications can only be done by the authenticated user. It is possible that we want to add a permissions for reading notifications, in particular for admin and impersonation use cases, but that is not part of this proposal or the initial implementation. + +The `notification-backend` shall provide necessary parameters for paging and filtering notifications from the frontend. The notification frontend plugin provides a UI for viewing notifications, which in the initial implementation can be as simple as needed. The only requirement is that a user is able to view recent notifications and distinguish between read and unread notifications. A notification as marked as read once it has been interacted with. The frontend plugin also subscribes to the notifications signal channel and alerts the user when a new notification is received. @@ -177,8 +187,6 @@ export type NotificationRecipients = type: 'broadcast'; // all users }; -export type NotificationType = 'undone' | 'done' | 'saved'; - export type Notification = { id: string; userRef: string; @@ -189,7 +197,7 @@ export type Notification = { created: Date; saved: boolean; read?: Date; - topic?: string; + scope?: string; updated?: Date; icon?: string; image?: string; @@ -201,34 +209,24 @@ interface SendNotificationRequest { description: string; link: string; origin: string; - topic?: string; + scope?: string; icon?: string; image?: string; } interface NotificationService { - sendNotification(request: SendNotificationRequest): Promise; + sendNotification(request: SendNotificationRequest): Promise; } ``` -Notification is always targeted to a single user unless it's a broadcast. The `notification-backend` will figure out which users will receive notification based on the `recipients` parameter. +Each notification is always routed to individual users unless it's a broadcast. The `notification-backend` will figure out which users will receive a notification based on the `recipients` parameter, resolving it to the concrete list of users at the time of sending the notification. Each notification contains a `title`, a `description` and a `link` for further information. The `created`, `id`, `read` and `saved` properties are handled by the backend based and cannot be changed during the notification creation. Calling `sendNotification` should never throw an error so that it doesn't block the current processing. Notifications should be considered as second-level citizens that are not critical if not delivered. -The `notification-backend` shall provide necessary parameters for paging and filtering notifications from the frontend. - #### `SignalService` -Upon creating a new notification, a signal will be emitted by the `notification-backend`. - -The channel for notifications will be named `notifications` and the message will contain necessary information about the notification for rendering as well as action telling to refresh the notifications using the REST API. - -The notification payload must be in the signal to be able to show notification in the UI immediately after receiving the signal. - -If the notification status is updated, the signal service shall emit a signal without the notification payload and only with the action. - ```ts export type SignalPayload = { recipients: string[] | string | null; @@ -286,6 +284,8 @@ The following frontend API is added as part of this proposal. #### `NotificationsApi` ```ts +export type NotificationType = 'undone' | 'done' | 'saved'; + export type GetNotificationsOptions = { type?: NotificationType; offset?: number; @@ -293,6 +293,13 @@ export type GetNotificationsOptions = { search?: string; }; +export type NotificationUpdateOptions = { + ids: string[]; + done?: boolean; + read?: boolean; + saved?: boolean; +}; + export type NotificationStatus = { unread: number; read: number; @@ -307,17 +314,9 @@ interface NotificationsApi { getStatus(): Promise; - markDone(ids: string[]): Promise; - - markUndone(ids: string[]): Promise; - - markRead(ids: string[]): Promise; - - markUnread(ids: string[]): Promise; - - markSaved(ids: string[]): Promise; - - markUnsaved(ids: string[]): Promise; + updateNotifications( + options: UpdateNotificationsOptions, + ): Promise; } ``` From bc6641d87d701d57f3da365286230fc5d95af92b Mon Sep 17 00:00:00 2001 From: Heikki Hellgren Date: Fri, 2 Feb 2024 12:24:17 +0200 Subject: [PATCH 3/3] beps: restore topic to the document Signed-off-by: Heikki Hellgren --- beps/0001-notifications-system/README.md | 64 ++++++++++++------------ 1 file changed, 31 insertions(+), 33 deletions(-) diff --git a/beps/0001-notifications-system/README.md b/beps/0001-notifications-system/README.md index 726a9ff9a6..c25d78b5df 100644 --- a/beps/0001-notifications-system/README.md +++ b/beps/0001-notifications-system/README.md @@ -99,14 +99,15 @@ The notification backend stores notification using the [database service](https: - Done date - Saved status - Creation date -- Updated date (optional, for topic notifications) +- Updated date (optional, for scoped notifications) - Payload: - Title - - Description - - Priority + - Description (optional) - Origin - Link - Recipients + - Severity (optional, default normal) + - Topic (optional) - Scope (optional) - Icon (optional) - Image URL (optional) @@ -117,10 +118,15 @@ The title is mandatory human-readable text shortly describing the notifications. The description is an optional human-readable text providing more details to the user. -The priority is one of the predefined values to help with presentation of the notifications or their filtering. Allowed values are: `critical`, `high`, `normal` (default), and `low`. +The severity is one of the predefined values to help with presentation of the notifications or their filtering. Allowed values are: `critical`, `high`, `normal` (default), and `low`. The origin is a string identifying the creator of the notification, i.e. the BE plugin or external system. +The topic is an optional string helping to group notifications of particular context. Its use cases include: + +- Several notifications emitted by an asynchronous external task can be grouped by a single topic +- A backend plugin can group several related messages to a particular processing, i.e. asynchronous progress monitoring + If a scope is provided for a notification, it will either: - Create a new notification if user doesn't have existing notification in that scope and origin @@ -130,7 +136,7 @@ The idea of scope is to be able to reuse notifications about same information fo The timestamp of notification creation is auto-generated by the notifications backend at the time of receiving a request to create the notification. -The icon and image URL are optional and is meant to improve UX. A string referencing an icon name from MUI icon library of a defined version. If missing, it will be determined from the priority. +The icon and image URL are optional and is meant to improve UX. A string referencing an icon name from MUI icon library of a defined version. If missing, it will be determined from the severity. The link is a relative or absolute URL. As an example, it can be used: @@ -187,31 +193,33 @@ export type NotificationRecipients = type: 'broadcast'; // all users }; +export type NotificationSeverity = 'critical' | 'high' | 'normal' | 'low'; + +export type NotificationPayload = { + title: string; + description?: string; + link: string; + additionalLinks?: string[]; + severity: NotificationSeverity; + topic?: string; + scope?: string; + icon?: string; +}; + export type Notification = { id: string; userRef: string; - title: string; - description: string; - origin: string; - link: string; created: Date; - saved: boolean; + saved?: Date; read?: Date; - scope?: string; updated?: Date; - icon?: string; - image?: string; + origin: string; + payload: NotificationPayload; }; interface SendNotificationRequest { recipients: NotificationRecipients; - title: string; - description: string; - link: string; - origin: string; - scope?: string; - icon?: string; - image?: string; + payload: NotificationPayload; } interface NotificationService { @@ -221,7 +229,7 @@ interface NotificationService { Each notification is always routed to individual users unless it's a broadcast. The `notification-backend` will figure out which users will receive a notification based on the `recipients` parameter, resolving it to the concrete list of users at the time of sending the notification. -Each notification contains a `title`, a `description` and a `link` for further information. The `created`, `id`, `read` and `saved` properties are handled by the backend based and cannot be changed during the notification creation. +Each notification contains a `title` and a `link` for user to see further information. The `created`, `id`, `read` and `saved` properties are handled by the backend based and cannot be changed during the notification creation. Calling `sendNotification` should never throw an error so that it doesn't block the current processing. Notifications should be considered as second-level citizens that are not critical if not delivered. @@ -305,10 +313,6 @@ export type NotificationStatus = { read: number; }; -export type NotificationIds = { - ids: string[]; -}; - interface NotificationsApi { getNotifications(options?: GetNotificationsOptions): Promise; @@ -329,13 +333,7 @@ interface NotificationsApi { ```ts export type NotificationSignal = { action: string; - notification?: { - title: string; - description: string; - link: string; - icon?: string; - image?: string; - }; + notification?: NotificationPayload; }; const { lastSignal, isSignalsAvailable } = @@ -362,7 +360,7 @@ interface SignalApi { - Unread notifications count is displayed in the title of the page - Configuration can be used to enable or disable features in the notification system (Web Notification API, title change, etc.) - Replace absent signal service with long polling. This requires changes to the `signals` plugin as well. -- Notifications can have priority that is used to determine how notifications are shown to the user +- Notifications can have severity that is used to determine how notifications are shown to the user ## Release Plan