databuild/plans/build-event-log.md

Build Event Log Design

The foundation of persistence for DataBuild is the build event log, a fact table recording events related to build requests, partitions, and jobs. Each graph has exactly one build event log, upon which other views (potentially materialized) rely and aggregate, e.g. powering the partition liveness catalog and enabling delegation to in-progress partition builds.

1. Schema

// Partition lifecycle states
enum PartitionStatus {
  PARTITION_UNKNOWN = 0;
  PARTITION_REQUESTED = 1;     // Partition requested but not yet scheduled
  PARTITION_SCHEDULED = 2;     // Job scheduled to produce this partition
  PARTITION_BUILDING = 3;      // Job actively building this partition
  PARTITION_AVAILABLE = 4;     // Partition successfully built and available
  PARTITION_FAILED = 5;        // Partition build failed
  PARTITION_STALE = 6;         // Partition exists but upstream dependencies changed
  PARTITION_DELEGATED = 7;     // Request delegated to existing build
}

// Job lifecycle
enum JobStatus {
  // TODO implement me
}

// Individual partition activity event
message BuildEvent {
  // TODO implement me
}

TODO narrative

2. Persistence

TODO narrative + design, with requirements:

• Should target postgres, sqlite, and delta tables

3. Access Layer

TODO narrative + design