databuild/design/build-event-log.md

Build Event Log (BEL)

Purpose: Store build events and define views summarizing databuild application state, like partition catalog, build status summary, job run statistics, etc.

Architecture

• Uses event sourcing / CQRS philosophy.
• BELs are only ever written to by graph processes (e.g. CLI or service), not the jobs themselves.
• BEL uses only two types of tables:
  • The root event table, with event ID, timestamp, message, event type, and ID fields for related event types.
  • Type-specific event tables (e.g. task even, partition event, build request event, etc).
• This makes it easy to support multiple backends (SQLite, postgres, and delta tables are supported initially).
• Exposes an access layer that mediates writes, and which exposes entity-specific repositories for reads.

Correctness Strategy

• Access layer will evaluate events requested to be written, returning an error if the event is not a correct next. state based on the involved component's governing state diagram.
• Events are versioned, with each versions' schemas stored in databuild.proto.

Write Interface

See trait definition.

Read Repositories

There are repositories for the following entities:

• Builds
• Jobs
• Partitions
• Tasks

Generally the following verbs are available for each:

• Show
• List
• Cancel