From acea14b422f1acec216c16f0f62b13ec72a4ad2b Mon Sep 17 00:00:00 2001 From: Paul Schultz Date: Tue, 4 Jun 2024 16:08:51 -0500 Subject: [PATCH 1/8] docs: add BEP for audit log Signed-off-by: Paul Schultz --- beps/0009-audit-log/README.md | 298 ++++++++++++++++++++++++++++++++++ 1 file changed, 298 insertions(+) create mode 100644 beps/0009-audit-log/README.md diff --git a/beps/0009-audit-log/README.md b/beps/0009-audit-log/README.md new file mode 100644 index 0000000000..2693dd30eb --- /dev/null +++ b/beps/0009-audit-log/README.md @@ -0,0 +1,298 @@ +--- +title: Audit Log +status: provisional +authors: + - '@schultzp2020' +owners: +project-areas: + - core +creation-date: 2024-06-04 +--- + +# BEP: Audit Log + +[**Discussion Issue**](https://github.com/backstage/backstage/issues/23950) + +- [BEP: Audit Log](#bep-audit-log) + - [Summary](#summary) + - [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) + - [Proposal](#proposal) + - [Design Details](#design-details) + - [Winston Configuration Changes](#winston-configuration-changes) + - [Data Model for Audit Logs](#data-model-for-audit-logs) + - [Actor Details Interface](#actor-details-interface) + - [Audit Request/Response Interfaces](#audit-requestresponse-interfaces) + - [Audit Log Status Interfaces](#audit-log-status-interfaces) + - [AuditLogger Interface](#auditlogger-interface) + - [Release Plan](#release-plan) + - [Dependencies](#dependencies) + - [Alternatives](#alternatives) + +## Summary + +This feature introduces a dedicated system for recording critical security-related actions and events within Backstage. By maintaining a distinct audit log stream, organizations benefit from enhanced security, improved regulatory compliance, efficient incident response, and ensured data integrity. + +## Motivation + +- Strengthen security by tracking user authentication, authorization, data access, and configuration changes. +- Facilitate adherence to regulatory requirements through logging security-sensitive operations. +- Enable efficient forensic analysis and investigation of security incidents. +- Uphold the integrity of critical audit data by implementing robust access controls and tamper-proof measures. + +### Goals + +- Implement a separate audit log event stream for recording security-critical events. +- Defining a specific data format for audit logs. +- Ensure adherence to regulatory compliance requirements through comprehensive audit logging. + +### Non-Goals + +- Implementing mechanisms for log storage, analysis, or visualization. +- Addressing security aspects of log storage and access control beyond initial separation from regular logs. + +Both of these non-goals can be implemented as separate plugins. + +## Proposal + +The proposal introduces two key changes to implement a separate audit log event stream in Backstage. First, we recommend modifying Backstage's Winston configuration to create a distinct channel specifically for audit logs. This can be achieved by extending the [existing configuration file](https://github.com/backstage/backstage/blob/master/packages/backend-app-api/src/logging/WinstonLogger.ts). A reference implementation for such a configuration can be found [here](https://github.com/janus-idp/backstage-showcase/blob/main/packages/backend/src/logger/customLogger.ts). Separating the configuration ensures clear distinction between regular application logs and critical security events. + +Secondly, the proposal suggests creating a new shared package within the Backstage ecosystem. This package would define a standardized data format for audit logs. The format would include mandatory fields relevant for regulatory compliance and security investigations, such as actor, IP address, timestamp, and event details. The package would also provide helper functions to simplify logging audit events throughout the Backstage application. A reference implementation for such a package can be found [here](https://github.com/janus-idp/backstage-plugins/tree/main/plugins/audit-log-node). This standardized format would streamline analysis and investigation of security-related events. + +Overall, this approach offers several benefits. By separating the configuration, security-critical events are clearly distinguished for improved monitoring and analysis. The standardized data format within the shared package would ensure consistency and facilitate compliance with regulations. Finally, the helper functions within the package would simplify the process of logging audit events. + +## Design Details + +### Winston Configuration Changes + +The proposal involves modifying Backstage's logging configuration using the Winston library. This modification creates a separate channel specifically for audit logs. + +```ts +/** + * A default formatting function is defined. This function adds timestamps, error details, and other relevant information to all logs. + */ +const defaultFormat = winston.format.combine( + winston.format.timestamp({ + format: 'YYYY-MM-DD HH:mm:ss', + }), + winston.format.errors({ stack: true }), + winston.format.splat(), +); + +const auditLogFormat = winston.format((info, opts) => { + const { isAuditLog, ...newInfo } = info; + + if (isAuditLog) { + /** + * If the flag `isAuditLog` is present and set to true, the entire message is included in the audit log format. + */ + return opts.isAuditLog ? info : false; + } + + /** + * If the flag `isAuditLog` is absent or set to false, the message is excluded from the audit log but included in the regular application log. + */ + return !opts.isAuditLog ? newInfo : false; +}); + +/** + * Two separate transport configurations are created: one for regular logs and one for audit logs. JSON formatting was chosen for easier parsing by logging tools. + */ +const transports = { + log: [ + new winston.transports.Console({ + format: winston.format.combine( + auditLogFormat({ isAuditLog: false }), + defaultFormat, + winston.format.json(), + ), + }), + ], + auditLog: [ + new winston.transports.Console({ + format: winston.format.combine( + auditLogFormat({ isAuditLog: true }), + defaultFormat, + winston.format.json(), + ), + }), + ], +}; + +const logger = WinstonLogger.create({ + meta: { + service: 'backstage', + }, + level: process.env.LOG_LEVEL ?? 'info', + format: winston.format.combine(defaultFormat, winston.format.json()), + transports: [...transports.log, ...transports.auditLog], +}); +``` + +### Data Model for Audit Logs + +To ensure consistency and facilitate regulatory compliance, the proposal suggests creating a shared package that defines a data model for audit logs. This model consists of several key components. + +#### Actor Details Interface + +This interface defines the information related to the actor who triggered the logged event. It includes fields like actor ID, IP address, hostname, and user agent. + +```ts +export type ActorDetails = { + actorId: string | null; + ip?: string; + hostname?: string; + userAgent?: string; +}; +``` + +#### Audit Request/Response Interfaces + +These interfaces define the structure of request and response data that might be included in the audit log. It's important to note that these interfaces exclude sensitive information like tokens from headers or other irrelevant details to avoid security risks. + +```ts +export type AuditRequest = { + body: any; + url: string; + method: string; + params?: any; + query?: any; +}; + +export type AuditResponse = { + status: number; + body?: any; +}; +``` + +#### Audit Log Status Interfaces + +These interfaces define the possible statuses for an audit log entry. There are three options: + +```ts +/** + * Indicates the event was successful. + */ +export type AuditLogSuccessStatus = { status: 'succeeded' }; + +/** + * Indicates the event failed and includes details about the encountered errors. + */ +export type AuditLogFailureStatus = { + status: 'failed'; + errors: ErrorLike[]; +}; + +/** + * Indicates the event failed with errors that don't perfectly fit the expected structure. + */ +export type AuditLogUnknownFailureStatus = { + status: 'failed'; + errors: unknown[]; +}; + +export type AuditLogStatus = AuditLogSuccessStatus | AuditLogFailureStatus; +``` + +#### AuditLogger Interface + +This interface defines the functionalities of an `AuditLogger` class. This class provides methods for: +- Extracting the actor ID from an Express request (if available). +- Creating detailed audit log information based on provided options. +- Logging an audit event with a specific level (info, debug, warn, or error). + +```ts +/** + * Common fields of an audit log. Note: timestamp and pluginId are automatically added at log creation. + * + * @public + */ +export type AuditLogDetails = { + actor: ActorDetails; + eventName: string; + stage: string; + request?: AuditRequest; + response?: AuditResponse; + meta: JsonValue; + isAuditLog: true; +} & AuditLogStatus; + +export type AuditLogDetailsOptions = { + eventName: string; + stage: string; + metadata?: JsonValue; + response?: AuditResponse; + actorId?: string; + request?: Request; +} & (AuditLogSuccessStatus | AuditLogUnknownFailureStatus); + +export type AuditLogOptions = { + eventName: string; + message: string; + stage: string; + level?: 'info' | 'debug' | 'warn' | 'error'; + actorId?: string; + metadata?: JsonValue; + response?: AuditResponse; + request?: Request; +} & (AuditLogSuccessStatus | AuditLogUnknownFailureStatus); + +export type AuditLoggerOptions = { + logger: LoggerService; + authService: AuthService; + httpAuthService: HttpAuthService; +}; + +export interface AuditLogger { + /** + * Processes an express request and obtains the actorId from it. Returns undefined if actorId is not obtainable. + * + * @public + */ + getActorId(request?: Request): Promise; + + /** + * Generates the audit log details to place in the metadata argument of the logger + * + * Secrets in the metadata field and request body, params and query field should be redacted by the user before passing in the request object + * @public + */ + createAuditLogDetails( + options: AuditLogDetailsOptions, + ): Promise; + + /** + * Generates an Audit Log and logs it at the level passed by the user. + * Supports `info`, `debug`, `warn` or `error` level. Defaults to `info` if no level is passed. + * + * Secrets in the metadata field and request body, params and query field should be redacted by the user before passing in the request object + * @public + */ + auditLog(options: AuditLogOptions): Promise; +} +``` + +## Release Plan + +WIP + + + +## Dependencies + +- `@backstage/backend-app-api` + +## Alternatives + +WIP + + From 62abf939d80dc531ae14c6e62217889413be1ad4 Mon Sep 17 00:00:00 2001 From: Paul Schultz Date: Mon, 10 Jun 2024 12:32:32 -0500 Subject: [PATCH 2/8] add release plan and alternatives Signed-off-by: Paul Schultz --- beps/0009-audit-log/README.md | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/beps/0009-audit-log/README.md b/beps/0009-audit-log/README.md index 2693dd30eb..8f1955f72f 100644 --- a/beps/0009-audit-log/README.md +++ b/beps/0009-audit-log/README.md @@ -29,6 +29,7 @@ creation-date: 2024-06-04 - [Release Plan](#release-plan) - [Dependencies](#dependencies) - [Alternatives](#alternatives) + - [Create a Separate Winston Logger Instance](#create-a-separate-winston-logger-instance) ## Summary @@ -275,13 +276,7 @@ export interface AuditLogger { ## Release Plan -WIP - - +The release plan involves initially creating a shared audit log package. Following this, the audit log will be implemented in core packages and other plugins. The first targets should be high-priority areas, such as the scaffolder and catalog systems. Since adding the audit log will not disrupt existing functionality, the release plan is simplified. ## Dependencies @@ -289,10 +284,6 @@ If there is any particular feedback to be gathered during the rollout, this shou ## Alternatives -WIP +### Create a Separate Winston Logger Instance - +We could create a separate instance of the Winston logger. However, this approach would necessitate core packages and plugins to include an additional argument in their constructor. From c27c109921425b241d4ffbd827c7b3c795929bbf Mon Sep 17 00:00:00 2001 From: Paul Schultz Date: Mon, 10 Jun 2024 12:53:34 -0500 Subject: [PATCH 3/8] add goal Signed-off-by: Paul Schultz --- beps/{0009-audit-log => 00010-audit-log}/README.md | 1 + 1 file changed, 1 insertion(+) rename beps/{0009-audit-log => 00010-audit-log}/README.md (99%) diff --git a/beps/0009-audit-log/README.md b/beps/00010-audit-log/README.md similarity index 99% rename from beps/0009-audit-log/README.md rename to beps/00010-audit-log/README.md index 8f1955f72f..4c08daa2ac 100644 --- a/beps/0009-audit-log/README.md +++ b/beps/00010-audit-log/README.md @@ -47,6 +47,7 @@ This feature introduces a dedicated system for recording critical security-relat - Implement a separate audit log event stream for recording security-critical events. - Defining a specific data format for audit logs. - Ensure adherence to regulatory compliance requirements through comprehensive audit logging. +- Provide access to the `WinstonLogger` transport layer to enable exporting logs to a file. ### Non-Goals From 287b0039d93960629c712208b32b70c750d6b770 Mon Sep 17 00:00:00 2001 From: Paul Schultz Date: Tue, 11 Jun 2024 10:13:49 -0500 Subject: [PATCH 4/8] change from audit log to audit event Signed-off-by: Paul Schultz --- beps/00010-audit-log/README.md | 290 ----------------------------- beps/00010-event-auditor/README.md | 220 ++++++++++++++++++++++ 2 files changed, 220 insertions(+), 290 deletions(-) delete mode 100644 beps/00010-audit-log/README.md create mode 100644 beps/00010-event-auditor/README.md diff --git a/beps/00010-audit-log/README.md b/beps/00010-audit-log/README.md deleted file mode 100644 index 4c08daa2ac..0000000000 --- a/beps/00010-audit-log/README.md +++ /dev/null @@ -1,290 +0,0 @@ ---- -title: Audit Log -status: provisional -authors: - - '@schultzp2020' -owners: -project-areas: - - core -creation-date: 2024-06-04 ---- - -# BEP: Audit Log - -[**Discussion Issue**](https://github.com/backstage/backstage/issues/23950) - -- [BEP: Audit Log](#bep-audit-log) - - [Summary](#summary) - - [Motivation](#motivation) - - [Goals](#goals) - - [Non-Goals](#non-goals) - - [Proposal](#proposal) - - [Design Details](#design-details) - - [Winston Configuration Changes](#winston-configuration-changes) - - [Data Model for Audit Logs](#data-model-for-audit-logs) - - [Actor Details Interface](#actor-details-interface) - - [Audit Request/Response Interfaces](#audit-requestresponse-interfaces) - - [Audit Log Status Interfaces](#audit-log-status-interfaces) - - [AuditLogger Interface](#auditlogger-interface) - - [Release Plan](#release-plan) - - [Dependencies](#dependencies) - - [Alternatives](#alternatives) - - [Create a Separate Winston Logger Instance](#create-a-separate-winston-logger-instance) - -## Summary - -This feature introduces a dedicated system for recording critical security-related actions and events within Backstage. By maintaining a distinct audit log stream, organizations benefit from enhanced security, improved regulatory compliance, efficient incident response, and ensured data integrity. - -## Motivation - -- Strengthen security by tracking user authentication, authorization, data access, and configuration changes. -- Facilitate adherence to regulatory requirements through logging security-sensitive operations. -- Enable efficient forensic analysis and investigation of security incidents. -- Uphold the integrity of critical audit data by implementing robust access controls and tamper-proof measures. - -### Goals - -- Implement a separate audit log event stream for recording security-critical events. -- Defining a specific data format for audit logs. -- Ensure adherence to regulatory compliance requirements through comprehensive audit logging. -- Provide access to the `WinstonLogger` transport layer to enable exporting logs to a file. - -### Non-Goals - -- Implementing mechanisms for log storage, analysis, or visualization. -- Addressing security aspects of log storage and access control beyond initial separation from regular logs. - -Both of these non-goals can be implemented as separate plugins. - -## Proposal - -The proposal introduces two key changes to implement a separate audit log event stream in Backstage. First, we recommend modifying Backstage's Winston configuration to create a distinct channel specifically for audit logs. This can be achieved by extending the [existing configuration file](https://github.com/backstage/backstage/blob/master/packages/backend-app-api/src/logging/WinstonLogger.ts). A reference implementation for such a configuration can be found [here](https://github.com/janus-idp/backstage-showcase/blob/main/packages/backend/src/logger/customLogger.ts). Separating the configuration ensures clear distinction between regular application logs and critical security events. - -Secondly, the proposal suggests creating a new shared package within the Backstage ecosystem. This package would define a standardized data format for audit logs. The format would include mandatory fields relevant for regulatory compliance and security investigations, such as actor, IP address, timestamp, and event details. The package would also provide helper functions to simplify logging audit events throughout the Backstage application. A reference implementation for such a package can be found [here](https://github.com/janus-idp/backstage-plugins/tree/main/plugins/audit-log-node). This standardized format would streamline analysis and investigation of security-related events. - -Overall, this approach offers several benefits. By separating the configuration, security-critical events are clearly distinguished for improved monitoring and analysis. The standardized data format within the shared package would ensure consistency and facilitate compliance with regulations. Finally, the helper functions within the package would simplify the process of logging audit events. - -## Design Details - -### Winston Configuration Changes - -The proposal involves modifying Backstage's logging configuration using the Winston library. This modification creates a separate channel specifically for audit logs. - -```ts -/** - * A default formatting function is defined. This function adds timestamps, error details, and other relevant information to all logs. - */ -const defaultFormat = winston.format.combine( - winston.format.timestamp({ - format: 'YYYY-MM-DD HH:mm:ss', - }), - winston.format.errors({ stack: true }), - winston.format.splat(), -); - -const auditLogFormat = winston.format((info, opts) => { - const { isAuditLog, ...newInfo } = info; - - if (isAuditLog) { - /** - * If the flag `isAuditLog` is present and set to true, the entire message is included in the audit log format. - */ - return opts.isAuditLog ? info : false; - } - - /** - * If the flag `isAuditLog` is absent or set to false, the message is excluded from the audit log but included in the regular application log. - */ - return !opts.isAuditLog ? newInfo : false; -}); - -/** - * Two separate transport configurations are created: one for regular logs and one for audit logs. JSON formatting was chosen for easier parsing by logging tools. - */ -const transports = { - log: [ - new winston.transports.Console({ - format: winston.format.combine( - auditLogFormat({ isAuditLog: false }), - defaultFormat, - winston.format.json(), - ), - }), - ], - auditLog: [ - new winston.transports.Console({ - format: winston.format.combine( - auditLogFormat({ isAuditLog: true }), - defaultFormat, - winston.format.json(), - ), - }), - ], -}; - -const logger = WinstonLogger.create({ - meta: { - service: 'backstage', - }, - level: process.env.LOG_LEVEL ?? 'info', - format: winston.format.combine(defaultFormat, winston.format.json()), - transports: [...transports.log, ...transports.auditLog], -}); -``` - -### Data Model for Audit Logs - -To ensure consistency and facilitate regulatory compliance, the proposal suggests creating a shared package that defines a data model for audit logs. This model consists of several key components. - -#### Actor Details Interface - -This interface defines the information related to the actor who triggered the logged event. It includes fields like actor ID, IP address, hostname, and user agent. - -```ts -export type ActorDetails = { - actorId: string | null; - ip?: string; - hostname?: string; - userAgent?: string; -}; -``` - -#### Audit Request/Response Interfaces - -These interfaces define the structure of request and response data that might be included in the audit log. It's important to note that these interfaces exclude sensitive information like tokens from headers or other irrelevant details to avoid security risks. - -```ts -export type AuditRequest = { - body: any; - url: string; - method: string; - params?: any; - query?: any; -}; - -export type AuditResponse = { - status: number; - body?: any; -}; -``` - -#### Audit Log Status Interfaces - -These interfaces define the possible statuses for an audit log entry. There are three options: - -```ts -/** - * Indicates the event was successful. - */ -export type AuditLogSuccessStatus = { status: 'succeeded' }; - -/** - * Indicates the event failed and includes details about the encountered errors. - */ -export type AuditLogFailureStatus = { - status: 'failed'; - errors: ErrorLike[]; -}; - -/** - * Indicates the event failed with errors that don't perfectly fit the expected structure. - */ -export type AuditLogUnknownFailureStatus = { - status: 'failed'; - errors: unknown[]; -}; - -export type AuditLogStatus = AuditLogSuccessStatus | AuditLogFailureStatus; -``` - -#### AuditLogger Interface - -This interface defines the functionalities of an `AuditLogger` class. This class provides methods for: -- Extracting the actor ID from an Express request (if available). -- Creating detailed audit log information based on provided options. -- Logging an audit event with a specific level (info, debug, warn, or error). - -```ts -/** - * Common fields of an audit log. Note: timestamp and pluginId are automatically added at log creation. - * - * @public - */ -export type AuditLogDetails = { - actor: ActorDetails; - eventName: string; - stage: string; - request?: AuditRequest; - response?: AuditResponse; - meta: JsonValue; - isAuditLog: true; -} & AuditLogStatus; - -export type AuditLogDetailsOptions = { - eventName: string; - stage: string; - metadata?: JsonValue; - response?: AuditResponse; - actorId?: string; - request?: Request; -} & (AuditLogSuccessStatus | AuditLogUnknownFailureStatus); - -export type AuditLogOptions = { - eventName: string; - message: string; - stage: string; - level?: 'info' | 'debug' | 'warn' | 'error'; - actorId?: string; - metadata?: JsonValue; - response?: AuditResponse; - request?: Request; -} & (AuditLogSuccessStatus | AuditLogUnknownFailureStatus); - -export type AuditLoggerOptions = { - logger: LoggerService; - authService: AuthService; - httpAuthService: HttpAuthService; -}; - -export interface AuditLogger { - /** - * Processes an express request and obtains the actorId from it. Returns undefined if actorId is not obtainable. - * - * @public - */ - getActorId(request?: Request): Promise; - - /** - * Generates the audit log details to place in the metadata argument of the logger - * - * Secrets in the metadata field and request body, params and query field should be redacted by the user before passing in the request object - * @public - */ - createAuditLogDetails( - options: AuditLogDetailsOptions, - ): Promise; - - /** - * Generates an Audit Log and logs it at the level passed by the user. - * Supports `info`, `debug`, `warn` or `error` level. Defaults to `info` if no level is passed. - * - * Secrets in the metadata field and request body, params and query field should be redacted by the user before passing in the request object - * @public - */ - auditLog(options: AuditLogOptions): Promise; -} -``` - -## Release Plan - -The release plan involves initially creating a shared audit log package. Following this, the audit log will be implemented in core packages and other plugins. The first targets should be high-priority areas, such as the scaffolder and catalog systems. Since adding the audit log will not disrupt existing functionality, the release plan is simplified. - -## Dependencies - -- `@backstage/backend-app-api` - -## Alternatives - -### Create a Separate Winston Logger Instance - -We could create a separate instance of the Winston logger. However, this approach would necessitate core packages and plugins to include an additional argument in their constructor. diff --git a/beps/00010-event-auditor/README.md b/beps/00010-event-auditor/README.md new file mode 100644 index 0000000000..d94e421433 --- /dev/null +++ b/beps/00010-event-auditor/README.md @@ -0,0 +1,220 @@ +--- +title: Event Auditor +status: provisional +authors: + - '@schultzp2020' +owners: +project-areas: + - core +creation-date: 2024-06-04 +--- + +# BEP: Event Auditor + +[**Discussion Issue**](https://github.com/backstage/backstage/issues/23950) + +- [BEP: Event Auditor](#bep-event-auditor) + - [Summary](#summary) + - [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) + - [Proposal](#proposal) + - [Design Details](#design-details) + - [Data Model for Audit Events](#data-model-for-audit-events) + - [Actor Details Interface](#actor-details-interface) + - [Audit Request/Response Interfaces](#audit-requestresponse-interfaces) + - [Audit Event Status Interfaces](#audit-event-status-interfaces) + - [EventAuditor Interface](#eventauditor-interface) + - [Release Plan](#release-plan) + - [Dependencies](#dependencies) + - [Alternatives](#alternatives) + +## Summary + +This feature introduces a dedicated system for recording critical security-related actions and events within Backstage. By maintaining a distinct audit event stream, organizations benefit from enhanced security, improved regulatory compliance, efficient incident response, and ensured data integrity. + +## Motivation + +- Strengthen security by tracking user authentication, authorization, data access, and configuration changes. +- Facilitate adherence to regulatory requirements through logging security-sensitive operations. +- Enable efficient forensic analysis and investigation of security incidents. +- Uphold the integrity of critical audit data by implementing robust access controls and tamper-proof measures. + +### Goals + +- Develop a backend service for the audit event stream to record security-critical events. +- Ensure compliance with regulatory requirements through detailed audit event logging. + - Refer to [NIST: Audit and Accountability](https://csrc.nist.gov/projects/cprt/catalog#/cprt/framework/version/SP_800_53_5_1_1/home?element=AU) +- Establish a standardized data format for audit events. + - Refer to [NIST: Content of Audit Records](https://csrc.nist.gov/projects/cprt/catalog#/cprt/framework/version/SP_800_53_5_1_1/home?element=AU-03) +- Provide access to the transport layer for customizable output options. + +### Non-Goals + +- Implementing mechanisms for event storage, analysis, or visualization. +- Addressing security aspects of event storage and access control beyond initial separation from regular events. + +Both of these non-goals can be implemented as separate plugins. + +## Proposal + +The proposal introduces a crucial change to implement a dedicated audit event stream in Backstage. We recommend creating a new backend service using Winston to establish a distinct channel specifically for audit events. This service would act as a wrapper around Winston, providing methods with strict interfaces to ensure uniformity across audit events throughout the Backstage application. By separating the configuration, we can clearly distinguish between regular application events and critical security events. The event format would include mandatory fields relevant to regulatory compliance and security investigations, such as actor, IP address, timestamp, and event details. This standardized format would streamline the analysis and investigation of security-related events. + +Overall, this approach offers several benefits. Separating the configuration allows for the clear distinction of security-critical events, enhancing monitoring and analysis. The standardized data format within the service ensures consistency and facilitates compliance with regulations. Finally, the service methods simplify the process of logging audit events. + +## Design Details + +### Data Model for Audit Events + +To ensure consistency and facilitate regulatory compliance, the proposal suggests creating a shared package that defines a data model for audit events. This model consists of several key components. + +#### Actor Details Interface + +This interface defines the information related to the actor who triggered the logged event. It includes fields like actor ID, IP address, hostname, and user agent. + +```ts +export type ActorDetails = { + actorId: string | null; + ip?: string; + hostname?: string; + userAgent?: string; +}; +``` + +#### Audit Request/Response Interfaces + +These interfaces define the structure of request and response data that might be included in the audit event. It's important to note that these interfaces exclude sensitive information like tokens from headers or other irrelevant details to avoid security risks. + +```ts +export type AuditRequest = { + body: any; + url: string; + method: string; + params?: any; + query?: any; +}; + +export type AuditResponse = { + status: number; + body?: any; +}; +``` + +#### Audit Event Status Interfaces + +These interfaces define the possible statuses for an audit event entry. There are three options: + +```ts +/** + * Indicates the event was successful. + */ +export type AuditEventSuccessStatus = { status: 'succeeded' }; + +/** + * Indicates the event failed and includes details about the encountered errors. + */ +export type AuditEventFailureStatus = { + status: 'failed'; + errors: ErrorLike[]; +}; + +/** + * Indicates the event failed with errors that don't perfectly fit the expected structure. + */ +export type AuditEventUnknownFailureStatus = { + status: 'failed'; + errors: unknown[]; +}; + +export type AuditEventStatus = AuditEventSuccessStatus | AuditEventFailureStatus; +``` + +#### EventAuditor Interface + +This interface defines the functionalities of an `EventAuditor` class. This class provides methods for: +- Extracting the actor ID from an Express request (if available). +- Creating detailed audit event information based on provided options. +- Logging an audit event with a specific level (info, debug, warn, or error). + +```ts +/** + * Common fields of an audit event. Note: timestamp and pluginId are automatically added at event creation. + * + * @public + */ +export type AuditEventDetails = { + actor: ActorDetails; + eventName: string; + stage: string; + request?: AuditRequest; + response?: AuditResponse; + meta: JsonValue; + isAuditEvent: true; +} & AuditEventStatus; + +export type AuditEventDetailsOptions = { + eventName: string; + stage: string; + metadata?: JsonValue; + response?: AuditResponse; + actorId?: string; + request?: Request; +} & (AuditEventSuccessStatus | AuditEventUnknownFailureStatus); + +export type AuditEventOptions = { + eventName: string; + message: string; + stage: string; + level?: 'info' | 'debug' | 'warn' | 'error'; + actorId?: string; + metadata?: JsonValue; + response?: AuditResponse; + request?: Request; +} & (AuditEventSuccessStatus | AuditEventUnknownFailureStatus); + +export type EventAuditorOptions = { + logger: LoggerService; + authService: AuthService; + httpAuthService: HttpAuthService; +}; + +export interface EventAuditor { + /** + * Processes an express request and obtains the actorId from it. Returns undefined if actorId is not obtainable. + * + * @public + */ + getActorId(request?: Request): Promise; + + /** + * Generates the audit event details to place in the metadata argument of the logger + * + * Secrets in the metadata field and request body, params and query field should be redacted by the user before passing in the request object + * @public + */ + createAuditEventDetails( + options: AuditEventDetailsOptions, + ): Promise; + + /** + * Generates an Audit Event and logs it at the level passed by the user. + * Supports `info`, `debug`, `warn` or `error` level. Defaults to `info` if no level is passed. + * + * Secrets in the metadata field and request body, params and query field should be redacted by the user before passing in the request object + * @public + */ + auditEvent(options: AuditEventOptions): Promise; +} +``` + +## Release Plan + +The release plan involves initially creating a shared audit event package. Following this, the audit event will be implemented in core packages and other plugins. The first targets should be high-priority areas, such as the scaffolder and catalog systems. Since adding the audit event will not disrupt existing functionality, the release plan is simplified. + +## Dependencies + +- `@backstage/types` + +## Alternatives + +N/A From bbacf63e67f5ce044eae395ae535551c55015d73 Mon Sep 17 00:00:00 2001 From: Paul Schultz Date: Tue, 25 Jun 2024 09:09:49 -0500 Subject: [PATCH 5/8] add non-repudiation Signed-off-by: Paul Schultz --- beps/00010-event-auditor/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/beps/00010-event-auditor/README.md b/beps/00010-event-auditor/README.md index d94e421433..5f4e767806 100644 --- a/beps/00010-event-auditor/README.md +++ b/beps/00010-event-auditor/README.md @@ -45,6 +45,7 @@ This feature introduces a dedicated system for recording critical security-relat - Develop a backend service for the audit event stream to record security-critical events. - Ensure compliance with regulatory requirements through detailed audit event logging. - Refer to [NIST: Audit and Accountability](https://csrc.nist.gov/projects/cprt/catalog#/cprt/framework/version/SP_800_53_5_1_1/home?element=AU) + - Refer to [NIST: Non-Repudiation](https://csrc.nist.gov/projects/cprt/catalog#/cprt/framework/version/SP_800_53_5_1_1/home?element=AU-10) - Establish a standardized data format for audit events. - Refer to [NIST: Content of Audit Records](https://csrc.nist.gov/projects/cprt/catalog#/cprt/framework/version/SP_800_53_5_1_1/home?element=AU-03) - Provide access to the transport layer for customizable output options. From 405916c10a5f38ed3bc332dcbcde35812422c486 Mon Sep 17 00:00:00 2001 From: Paul Schultz Date: Thu, 1 Aug 2024 14:54:17 -0500 Subject: [PATCH 6/8] apply requested changes Signed-off-by: Paul Schultz --- beps/00010-event-auditor/README.md | 69 +++++++----------------------- 1 file changed, 15 insertions(+), 54 deletions(-) diff --git a/beps/00010-event-auditor/README.md b/beps/00010-event-auditor/README.md index 5f4e767806..b1af69cf95 100644 --- a/beps/00010-event-auditor/README.md +++ b/beps/00010-event-auditor/README.md @@ -75,7 +75,7 @@ This interface defines the information related to the actor who triggered the lo ```ts export type ActorDetails = { - actorId: string | null; + actorId?: string; ip?: string; hostname?: string; userAgent?: string; @@ -88,16 +88,12 @@ These interfaces define the structure of request and response data that might be ```ts export type AuditRequest = { - body: any; url: string; method: string; - params?: any; - query?: any; }; export type AuditResponse = { status: number; - body?: any; }; ``` @@ -114,20 +110,12 @@ export type AuditEventSuccessStatus = { status: 'succeeded' }; /** * Indicates the event failed and includes details about the encountered errors. */ -export type AuditEventFailureStatus = { +export type AuditEventFailureStatus = { status: 'failed'; - errors: ErrorLike[]; + errors: E; }; -/** - * Indicates the event failed with errors that don't perfectly fit the expected structure. - */ -export type AuditEventUnknownFailureStatus = { - status: 'failed'; - errors: unknown[]; -}; - -export type AuditEventStatus = AuditEventSuccessStatus | AuditEventFailureStatus; +export type AuditEventStatus = AuditEventSuccessStatus | AuditEventFailureStatus | undefined; ``` #### EventAuditor Interface @@ -139,45 +127,28 @@ This interface defines the functionalities of an `EventAuditor` class. This clas ```ts /** - * Common fields of an audit event. Note: timestamp and pluginId are automatically added at event creation. + * Common fields of an audit event. * * @public */ -export type AuditEventDetails = { - actor: ActorDetails; - eventName: string; - stage: string; - request?: AuditRequest; - response?: AuditResponse; - meta: JsonValue; - isAuditEvent: true; -} & AuditEventStatus; - -export type AuditEventDetailsOptions = { - eventName: string; - stage: string; - metadata?: JsonValue; - response?: AuditResponse; - actorId?: string; - request?: Request; -} & (AuditEventSuccessStatus | AuditEventUnknownFailureStatus); - -export type AuditEventOptions = { +export type AuditEventOptions = AuditEventStatus & { eventName: string; message: string; stage: string; level?: 'info' | 'debug' | 'warn' | 'error'; - actorId?: string; metadata?: JsonValue; response?: AuditResponse; request?: Request; -} & (AuditEventSuccessStatus | AuditEventUnknownFailureStatus); +} & ({ actorId: string; } | { credentials: BackstageCredentials } | undefined); -export type EventAuditorOptions = { - logger: LoggerService; - authService: AuthService; - httpAuthService: HttpAuthService; -}; +export type AuditEvent = { + actor: ActorDetails; + eventName: string; + stage: string; + isAuditLog: true; + request?: AuditRequest; + response?: AuditResponse; +} & AuditLogStatus; export interface EventAuditor { /** @@ -187,16 +158,6 @@ export interface EventAuditor { */ getActorId(request?: Request): Promise; - /** - * Generates the audit event details to place in the metadata argument of the logger - * - * Secrets in the metadata field and request body, params and query field should be redacted by the user before passing in the request object - * @public - */ - createAuditEventDetails( - options: AuditEventDetailsOptions, - ): Promise; - /** * Generates an Audit Event and logs it at the level passed by the user. * Supports `info`, `debug`, `warn` or `error` level. Defaults to `info` if no level is passed. From 716c9fbaa9e75f74e96412b5bb09c1bc5b21a180 Mon Sep 17 00:00:00 2001 From: Paul Schultz Date: Thu, 1 Aug 2024 15:41:59 -0500 Subject: [PATCH 7/8] Update beps/00010-event-auditor/README.md Co-authored-by: Frank Kong <50030060+Zaperex@users.noreply.github.com> Signed-off-by: Paul Schultz --- beps/00010-event-auditor/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/beps/00010-event-auditor/README.md b/beps/00010-event-auditor/README.md index b1af69cf95..96cafb701e 100644 --- a/beps/00010-event-auditor/README.md +++ b/beps/00010-event-auditor/README.md @@ -112,7 +112,7 @@ export type AuditEventSuccessStatus = { status: 'succeeded' }; */ export type AuditEventFailureStatus = { status: 'failed'; - errors: E; + errors: E[]; }; export type AuditEventStatus = AuditEventSuccessStatus | AuditEventFailureStatus | undefined; From 1284ac0915c66e6f8742dbaefecf1658cf19aadf Mon Sep 17 00:00:00 2001 From: Paul Schultz Date: Tue, 20 Aug 2024 09:27:19 -0500 Subject: [PATCH 8/8] add dependency Signed-off-by: Paul Schultz --- beps/00010-event-auditor/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/beps/00010-event-auditor/README.md b/beps/00010-event-auditor/README.md index b1af69cf95..8a0091c972 100644 --- a/beps/00010-event-auditor/README.md +++ b/beps/00010-event-auditor/README.md @@ -176,6 +176,7 @@ The release plan involves initially creating a shared audit event package. Follo ## Dependencies - `@backstage/types` +- ## Alternatives