CLAUDE.md

@@ -12,7 +12,7 @@ DataBuild is a bazel-based data build system. Key files:
   - [Graph specification](./design/graph-specification.md) - Describes the different libraries that enable more succinct declaration of databuild applications than the core bazel-based interface.
   - [Observability](./design/observability.md) - How observability is systematically achieved throughout databuild applications.
   - [Deploy strategies](./design/deploy-strategies.md) - Different strategies for deploying databuild applications.
-  - [Wants](./design/wants.md) - How triggering works in databuild applications.
+  - [Triggers](./design/triggers.md) - How triggering works in databuild applications.
   - [Why databuild?](./design/why-databuild.md) - Why to choose databuild instead of other better established orchestration solutions.
 
 Please reference these for any related work, as they indicate key technical bias/direction of the project.

DESIGN.md

@@ -58,7 +58,7 @@ The BEL encodes all relevant build actions that occur, enabling concurrent build
 The BEL is similar to [event-sourced](https://martinfowler.com/eaaDev/EventSourcing.html) systems, as all application state is rendered from aggregations over the BEL. This enables the BEL to stay simple while also powering concurrent builds, the data catalog, and the DataBuild service. 
 
 ### Triggers and Wants (Coming Soon)
-["Wants"](./design/wants.md) are the main mechanism for continually building partitions over time. In real world scenarios, it is standard for data to arrive late, or not at all. Wants cause the databuild graph to continually attempt to build the wanted partitions until a) the partitions are live or b) the want expires, at which another script can be run. Wants are the mechanism that implements SLA checking. 
+["Wants"](./design/triggers.md) are the main mechanism for continually building partitions over time. In real world scenarios, it is standard for data to arrive late, or not at all. Wants cause the databuild graph to continually attempt to build the wanted partitions until a) the partitions are live or b) the want expires, at which another script can be run. Wants are the mechanism that implements SLA checking. 
 
 You can also use cron-based triggers, which return partition refs that they want built.
 

README.md

@@ -17,22 +17,7 @@ For important context, check out [DESIGN.md](./DESIGN.md), along with designs in
 
 ## Usage
 
-### Graph Description Methods
+See the [podcast example BUILD file](examples/podcast_reviews/BUILD.bazel).
-
-- **Bazel targets**: The foundational 
-- **Python DSL**: A more succinct method with partition patterns and decorator-based auto graph wiring. [Example usage.](databuild/test/app/dsl/graph.py)
-
-### Examples
-- Test app: [color votes](databuild/test/app/README.md)
-  - [Bazel graph description example](databuild/test/app/bazel/BUILD.bazel)
-  - [Python DSL description example](databuild/test/app/dsl/graph.py)
-- See the [podcast example BUILD file](examples/podcast_reviews/BUILD.bazel).
-
-### Ways to Use DataBuild in Production
-
-- **As a CLI build tool**: You can run DataBuild builds from the command line or in a remote environment - no build event log required!
-- **As a standalone service**: Similar to Dagster or Airflow, you can run a persistent service that you send build requests to, and which serves an API and web dashboard.
-- **As a cloud-native containerized build tool**: Build containers from your graphs and launch scheduled builds using a container service like ECS, or even your own kubernetes cluster.
 
 ## Development
 

databuild/BUILD.bazel

@@ -20,11 +20,12 @@ rust_binary(
 rust_library(
     name = "databuild",
     srcs = [
+        "event_log/delta.rs",
         "event_log/mock.rs",
         "event_log/mod.rs",
-        "event_log/query_engine.rs",
+        "event_log/postgres.rs",
-        "event_log/sqlite_storage.rs",
+        "event_log/sqlite.rs",
-        "event_log/storage.rs",
+        "event_log/stdout.rs",
         "event_log/writer.rs",
         "format_consistency_test.rs",
         "lib.rs",
@@ -56,7 +57,9 @@ rust_library(
         "@crates//:axum",
         "@crates//:axum-jsonschema",
         "@crates//:chrono",
+        "@crates//:deltalake",
         "@crates//:log",
+        "@crates//:parquet",
         "@crates//:prost",
         "@crates//:prost-types",
         "@crates//:rusqlite",

databuild/cli/main.rs

@@ -1,5 +1,5 @@
 use databuild::*;
-use databuild::event_log::create_bel_query_engine;
+use databuild::event_log::create_build_event_log;
 use databuild::orchestration::{BuildOrchestrator, BuildResult};
 use databuild::repositories::{
     partitions::PartitionsRepository,
@@ -12,6 +12,7 @@ use log::{info, error};
 use simple_logger::SimpleLogger;
 use std::env;
 use std::process::{Command, Stdio};
+use std::sync::Arc;
 use uuid::Uuid;
 
 mod error;
@@ -139,14 +140,14 @@ async fn handle_build_command(matches: &ArgMatches) -> Result<()> {
     info!("Event log URI: {}", event_log_uri);
     
     // Create event log and orchestrator
-    let query_engine = create_bel_query_engine(&event_log_uri).await?;
+    let event_log = create_build_event_log(&event_log_uri).await?;
     
     let requested_partitions: Vec<PartitionRef> = partitions.iter()
         .map(|p| PartitionRef { str: p.clone() })
         .collect();
     
     let orchestrator = BuildOrchestrator::new(
-        query_engine.clone(),
+        std::sync::Arc::from(event_log),
         build_request_id,
         requested_partitions,
     );
@@ -385,10 +386,10 @@ async fn main() -> Result<()> {
 }
 
 async fn handle_partitions_command(matches: &ArgMatches, event_log_uri: &str) -> Result<()> {
-    let query_engine = create_bel_query_engine(event_log_uri).await
+    let event_log = create_build_event_log(event_log_uri).await
         .map_err(|e| CliError::Database(format!("Failed to connect to event log: {}", e)))?;
     
-    let repository = PartitionsRepository::new(query_engine);
+    let repository = PartitionsRepository::new(Arc::from(event_log));
     
     match matches.subcommand() {
         Some(("list", sub_matches)) => {
@@ -511,10 +512,10 @@ async fn handle_partitions_command(matches: &ArgMatches, event_log_uri: &str) ->
 }
 
 async fn handle_jobs_command(matches: &ArgMatches, event_log_uri: &str) -> Result<()> {
-    let query_engine = create_bel_query_engine(event_log_uri).await
+    let event_log = create_build_event_log(event_log_uri).await
         .map_err(|e| CliError::Database(format!("Failed to connect to event log: {}", e)))?;
     
-    let repository = JobsRepository::new(query_engine);
+    let repository = JobsRepository::new(Arc::from(event_log));
     
     match matches.subcommand() {
         Some(("list", sub_matches)) => {
@@ -647,10 +648,10 @@ async fn handle_jobs_command(matches: &ArgMatches, event_log_uri: &str) -> Resul
 }
 
 async fn handle_tasks_command(matches: &ArgMatches, event_log_uri: &str) -> Result<()> {
-    let query_engine = create_bel_query_engine(event_log_uri).await
+    let event_log = create_build_event_log(event_log_uri).await
         .map_err(|e| CliError::Database(format!("Failed to connect to event log: {}", e)))?;
     
-    let repository = TasksRepository::new(query_engine);
+    let repository = TasksRepository::new(Arc::from(event_log));
     
     match matches.subcommand() {
         Some(("list", sub_matches)) => {
@@ -814,10 +815,10 @@ async fn handle_tasks_command(matches: &ArgMatches, event_log_uri: &str) -> Resu
 }
 
 async fn handle_builds_command(matches: &ArgMatches, event_log_uri: &str) -> Result<()> {
-    let query_engine = create_bel_query_engine(event_log_uri).await
+    let event_log = create_build_event_log(event_log_uri).await
         .map_err(|e| CliError::Database(format!("Failed to connect to event log: {}", e)))?;
     
-    let repository = BuildsRepository::new(query_engine);
+    let repository = BuildsRepository::new(Arc::from(event_log));
     
     match matches.subcommand() {
         Some(("list", sub_matches)) => {

databuild/client/BUILD.bazel

@@ -48,6 +48,7 @@ genrule(
         "typescript_generated/src/models/BuildsListApiResponse.ts",
         "typescript_generated/src/models/BuildsListResponse.ts",
         "typescript_generated/src/models/CancelBuildRepositoryRequest.ts",
+        "typescript_generated/src/models/CancelTaskRequest.ts",
         "typescript_generated/src/models/InvalidatePartitionRequest.ts",
         "typescript_generated/src/models/JobDailyStats.ts",
         "typescript_generated/src/models/JobDetailRequest.ts",
@@ -55,6 +56,7 @@ genrule(
         "typescript_generated/src/models/JobMetricsRequest.ts",
         "typescript_generated/src/models/JobMetricsResponse.ts",
         "typescript_generated/src/models/JobRunDetail.ts",
+        "typescript_generated/src/models/JobRunSummary.ts",
         "typescript_generated/src/models/JobSummary.ts",
         "typescript_generated/src/models/JobsListApiResponse.ts",
         "typescript_generated/src/models/JobsListResponse.ts",
@@ -72,16 +74,14 @@ genrule(
         "typescript_generated/src/models/PartitionTimelineEvent.ts",
         "typescript_generated/src/models/PartitionsListApiResponse.ts",
         "typescript_generated/src/models/PartitionsListResponse.ts",
-        "typescript_generated/src/models/CancelTaskRequest.ts",
-        "typescript_generated/src/models/JobRunDetailResponse.ts",
-        "typescript_generated/src/models/JobRunSummary.ts",
-        "typescript_generated/src/models/JobRunSummary2.ts",
-        "typescript_generated/src/models/JobRunTimelineEvent.ts",
-        "typescript_generated/src/models/JobRunsListApiResponse.ts",
-        "typescript_generated/src/models/JobRunsListResponse.ts",
         "typescript_generated/src/models/TaskCancelPathRequest.ts",
         "typescript_generated/src/models/TaskCancelResponse.ts",
         "typescript_generated/src/models/TaskDetailRequest.ts",
+        "typescript_generated/src/models/TaskDetailResponse.ts",
+        "typescript_generated/src/models/TaskSummary.ts",
+        "typescript_generated/src/models/TaskTimelineEvent.ts",
+        "typescript_generated/src/models/TasksListApiResponse.ts",
+        "typescript_generated/src/models/TasksListResponse.ts",
         "typescript_generated/src/runtime.ts",
         "typescript_generated/src/index.ts",
     ],
@@ -122,6 +122,7 @@ genrule(
     cp $$TEMP_DIR/src/models/BuildsListApiResponse.ts $(location typescript_generated/src/models/BuildsListApiResponse.ts)
     cp $$TEMP_DIR/src/models/BuildsListResponse.ts $(location typescript_generated/src/models/BuildsListResponse.ts)
     cp $$TEMP_DIR/src/models/CancelBuildRepositoryRequest.ts $(location typescript_generated/src/models/CancelBuildRepositoryRequest.ts)
+    cp $$TEMP_DIR/src/models/CancelTaskRequest.ts $(location typescript_generated/src/models/CancelTaskRequest.ts)
     cp $$TEMP_DIR/src/models/InvalidatePartitionRequest.ts $(location typescript_generated/src/models/InvalidatePartitionRequest.ts)
     cp $$TEMP_DIR/src/models/JobDailyStats.ts $(location typescript_generated/src/models/JobDailyStats.ts)
     cp $$TEMP_DIR/src/models/JobDetailRequest.ts $(location typescript_generated/src/models/JobDetailRequest.ts)
@@ -147,16 +148,14 @@ genrule(
     cp $$TEMP_DIR/src/models/PartitionTimelineEvent.ts $(location typescript_generated/src/models/PartitionTimelineEvent.ts)
     cp $$TEMP_DIR/src/models/PartitionsListApiResponse.ts $(location typescript_generated/src/models/PartitionsListApiResponse.ts)
     cp $$TEMP_DIR/src/models/PartitionsListResponse.ts $(location typescript_generated/src/models/PartitionsListResponse.ts)
-    cp $$TEMP_DIR/src/models/JobRunSummary.ts $(location typescript_generated/src/models/JobRunSummary.ts)
-    cp $$TEMP_DIR/src/models/JobRunTimelineEvent.ts $(location typescript_generated/src/models/JobRunTimelineEvent.ts)
-    cp $$TEMP_DIR/src/models/JobRunsListApiResponse.ts $(location typescript_generated/src/models/JobRunsListApiResponse.ts)
-    cp $$TEMP_DIR/src/models/JobRunsListResponse.ts $(location typescript_generated/src/models/JobRunsListResponse.ts)
-    cp $$TEMP_DIR/src/models/CancelTaskRequest.ts $(location typescript_generated/src/models/CancelTaskRequest.ts)
-    cp $$TEMP_DIR/src/models/JobRunDetailResponse.ts $(location typescript_generated/src/models/JobRunDetailResponse.ts)
-    cp $$TEMP_DIR/src/models/JobRunSummary2.ts $(location typescript_generated/src/models/JobRunSummary2.ts)
     cp $$TEMP_DIR/src/models/TaskCancelPathRequest.ts $(location typescript_generated/src/models/TaskCancelPathRequest.ts)
     cp $$TEMP_DIR/src/models/TaskCancelResponse.ts $(location typescript_generated/src/models/TaskCancelResponse.ts)
     cp $$TEMP_DIR/src/models/TaskDetailRequest.ts $(location typescript_generated/src/models/TaskDetailRequest.ts)
+    cp $$TEMP_DIR/src/models/TaskDetailResponse.ts $(location typescript_generated/src/models/TaskDetailResponse.ts)
+    cp $$TEMP_DIR/src/models/TaskSummary.ts $(location typescript_generated/src/models/TaskSummary.ts)
+    cp $$TEMP_DIR/src/models/TaskTimelineEvent.ts $(location typescript_generated/src/models/TaskTimelineEvent.ts)
+    cp $$TEMP_DIR/src/models/TasksListApiResponse.ts $(location typescript_generated/src/models/TasksListApiResponse.ts)
+    cp $$TEMP_DIR/src/models/TasksListResponse.ts $(location typescript_generated/src/models/TasksListResponse.ts)
     cp $$TEMP_DIR/src/runtime.ts $(location typescript_generated/src/runtime.ts)
     cp $$TEMP_DIR/src/index.ts $(location typescript_generated/src/index.ts)
     """,

databuild/dashboard/pages.ts

@@ -385,7 +385,7 @@ export const BuildStatus: TypedComponent<BuildStatusAttrs> = {
           if (typeof window !== 'undefined' && (window as any).mermaid) {
             (window as any).mermaid.init();
           }
-        }, 200);
+        }, 100);
       } else {
         this.mermaidError = 'No job graph available for this build';
       }
@@ -462,8 +462,6 @@ export const BuildStatus: TypedComponent<BuildStatusAttrs> = {
       ...(build.completed_at ? [{stage: 'Build Completed', time: build.completed_at, icon: '✅'}] : []),
     ];
 
-    let startedAt = build.started_at || build.requested_at;
-
     return m('div.container.mx-auto.p-4', [
       // Build Header
       m('.build-header.mb-6', [
@@ -487,8 +485,8 @@ export const BuildStatus: TypedComponent<BuildStatusAttrs> = {
           ]),
           m('.stat.bg-base-100.shadow.rounded-lg.p-4', [
             m('.stat-title', 'Duration'),
-            m('.stat-value.text-2xl', (build.completed_at - startedAt) ? formatDuration((build.completed_at - startedAt)) : '—'),
+            m('.stat-value.text-2xl', (build.completed_at - build.started_at) ? formatDuration((build.completed_at - build.started_at)) : '—'),
-            m('.stat-desc', startedAt ? formatDateTime(startedAt) : 'Not started')
+            m('.stat-desc', build.started_at ? formatDateTime(build.started_at) : 'Not started')
           ])
         ])
       ]),

databuild/dashboard/services.ts

@@ -462,7 +462,6 @@ export function formatDateTime(epochNanos: number): string {
 
 export function formatDuration(durationNanos?: number | null): string {
   let durationMs = durationNanos ? durationNanos / 1000000 : null;
-  console.warn('Formatting duration:', durationMs);
   if (!durationMs || durationMs <= 0) {
     return '—';
   }

databuild/databuild.proto

@@ -163,22 +163,6 @@ message GraphBuildResponse { repeated PartitionManifest manifests = 1; }
 // Build Event Log
 ///////////////////////////////////////////////////////////////////////////////////////////////
 
-// Filter for querying build events
-message EventFilter {
-  repeated string partition_refs = 1;
-  repeated string partition_patterns = 2;  
-  repeated string job_labels = 3;
-  repeated string job_run_ids = 4;
-  repeated string build_request_ids = 5;
-}
-
-// Paginated response for build events
-message EventPage {
-  repeated BuildEvent events = 1;
-  int64 next_idx = 2;
-  bool has_more = 3;
-}
-
 // Partition lifecycle states
 enum PartitionStatus {
   PARTITION_UNKNOWN = 0;
@@ -261,8 +245,8 @@ message PartitionInvalidationEvent {
   string reason = 2;                      // Reason for invalidation
 }
 
-// Job run cancellation event
+// Task cancellation event
-message JobRunCancelEvent {
+message TaskCancelEvent {
   string job_run_id = 1;                  // UUID of the job run being cancelled
   string reason = 2;                      // Reason for cancellation
 }
@@ -272,22 +256,6 @@ message BuildCancelEvent {
   string reason = 1;                      // Reason for cancellation
 }
 
-// Partition Want
-message WantSource {
-  // TODO
-}
-
-message PartitionWant {
-  PartitionRef partition_ref = 1;         // Partition being requested
-  uint64 created_at = 2;                  // Server time when want registered
-  optional uint64 data_timestamp = 3;     // Business time this partition represents
-  optional uint64 ttl_seconds = 4;        // Give up after this long (from created_at)
-  optional uint64 sla_seconds = 5;        // SLA violation after this long (from data_timestamp)
-  repeated string external_dependencies = 6; // Cross-graph dependencies
-  string want_id = 7;                     // Unique identifier
-  WantSource source = 8;                  // How this want was created
-}
-
 // Individual build event
 message BuildEvent {
   // Event metadata
@@ -303,7 +271,7 @@ message BuildEvent {
     DelegationEvent delegation_event = 13;
     JobGraphEvent job_graph_event = 14;
     PartitionInvalidationEvent partition_invalidation_event = 15;
-    JobRunCancelEvent job_run_cancel_event = 16;
+    TaskCancelEvent task_cancel_event = 16;
     BuildCancelEvent build_cancel_event = 17;
   }
 }
@@ -415,19 +383,19 @@ message JobSummary {
 }
 
 //
-// Job Runs List
+// Tasks List
 //
 
-message JobRunsListRequest {
+message TasksListRequest {
   optional uint32 limit = 1;
 }
 
-message JobRunsListResponse {
+message TasksListResponse {
-  repeated JobRunSummary tasks = 1;
+  repeated TaskSummary tasks = 1;
   uint32 total_count = 2;
 }
 
-message JobRunSummary {
+message TaskSummary {
   string job_run_id = 1;
   string job_label = 2;
   string build_request_id = 3;
@@ -589,14 +557,14 @@ message JobRunDetail {
 }
 
 //
-// Job Run Detail
+// Task Detail
 //
 
-message JobRunDetailRequest {
+message TaskDetailRequest {
   string job_run_id = 1;
 }
 
-message JobRunDetailResponse {
+message TaskDetailResponse {
   string job_run_id = 1;
   string job_label = 2;
   string build_request_id = 3;
@@ -610,10 +578,10 @@ message JobRunDetailResponse {
   bool cancelled = 11;
   optional string cancel_reason = 12;
   string message = 13;
-  repeated JobRunTimelineEvent timeline = 14;
+  repeated TaskTimelineEvent timeline = 14;
 }
 
-message JobRunTimelineEvent {
+message TaskTimelineEvent {
   int64 timestamp = 1;
   optional JobStatus status_code = 2;     // Enum for programmatic use
   optional string status_name = 3;        // Human-readable string

databuild/dsl/python/dsl.py

@@ -120,7 +120,7 @@ class DataBuildGraph:
         import os
         
         # Get job classes from the lookup table
-        job_classes = sorted(set(self.lookup.values()), key=lambda cls: cls.__name__)
+        job_classes = list(set(self.lookup.values()))
         
         # Format deps for BUILD.bazel
         if deps:
@@ -172,15 +172,6 @@ databuild_graph(
     lookup = ":{name}_job_lookup",
     visibility = ["//visibility:public"],
 )
-
-# Create tar archive of generated files for testing
-genrule(
-    name = "existing_generated",
-    srcs = glob(["*.py", "BUILD.bazel"]),
-    outs = ["existing_generated.tar"],
-    cmd = "mkdir -p temp && cp $(SRCS) temp/ && find temp -exec touch -t 197001010000 {{}} + && tar -cf $@ -C temp .",
-    visibility = ["//visibility:public"],
-)
 '''
         
         with open(os.path.join(output_dir, "BUILD.bazel"), "w") as f:

databuild/event_log/mock.rs

@@ -1,10 +1,8 @@
 use crate::*;
-use crate::event_log::{BuildEventLogError, Result};
+use crate::event_log::{BuildEventLog, BuildEventLogError, Result, QueryResult, BuildRequestSummary, PartitionSummary, ActivitySummary};
-use crate::event_log::storage::BELStorage;
-use crate::event_log::query_engine::BELQueryEngine;
 use async_trait::async_trait;
 use std::sync::{Arc, Mutex};
-use rusqlite::Connection;
+use rusqlite::{Connection, params};
 
 /// MockBuildEventLog provides an in-memory SQLite database for testing
 /// 
@@ -23,7 +21,7 @@ pub struct MockBuildEventLog {
 impl MockBuildEventLog {
     /// Create a new MockBuildEventLog with an in-memory SQLite database
     pub async fn new() -> Result<Self> {
-        let conn = Connection::open(":memory:")
+        let mut conn = Connection::open(":memory:")
             .map_err(|e| BuildEventLogError::ConnectionError(e.to_string()))?;
         
         // Disable foreign key constraints for simplicity in testing
@@ -106,9 +104,342 @@ impl MockBuildEventLog {
         
         Ok(())
     }
+}
 
-    /// Initialize the database schema for testing
+#[async_trait]
-    pub async fn initialize(&self) -> Result<()> {
+impl BuildEventLog for MockBuildEventLog {
+    async fn append_event(&self, event: BuildEvent) -> Result<()> {
+        let conn = self.connection.lock().unwrap();
+        
+        // Serialize the entire event for storage
+        let event_data = serde_json::to_string(&event)
+            .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+        
+        // Insert into main events table
+        conn.execute(
+            "INSERT INTO build_events (event_id, timestamp, build_request_id, event_type, event_data) VALUES (?1, ?2, ?3, ?4, ?5)",
+            params![
+                event.event_id,
+                event.timestamp,
+                event.build_request_id,
+                match &event.event_type {
+                    Some(crate::build_event::EventType::BuildRequestEvent(_)) => "build_request",
+                    Some(crate::build_event::EventType::PartitionEvent(_)) => "partition",
+                    Some(crate::build_event::EventType::JobEvent(_)) => "job",
+                    Some(crate::build_event::EventType::DelegationEvent(_)) => "delegation",
+                    Some(crate::build_event::EventType::JobGraphEvent(_)) => "job_graph",
+                    Some(crate::build_event::EventType::PartitionInvalidationEvent(_)) => "partition_invalidation",
+                    Some(crate::build_event::EventType::TaskCancelEvent(_)) => "task_cancel",
+                    Some(crate::build_event::EventType::BuildCancelEvent(_)) => "build_cancel",
+                    None => "unknown",
+                },
+                event_data
+            ],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        // Insert into specific event type table for better querying
+        match &event.event_type {
+            Some(crate::build_event::EventType::BuildRequestEvent(br_event)) => {
+                let partitions_json = serde_json::to_string(&br_event.requested_partitions)
+                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+                
+                conn.execute(
+                    "INSERT INTO build_request_events (event_id, status, requested_partitions, message) VALUES (?1, ?2, ?3, ?4)",
+                    params![
+                        event.event_id,
+                        br_event.status_code.to_string(),
+                        partitions_json,
+                        br_event.message
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::PartitionEvent(p_event)) => {
+                conn.execute(
+                    "INSERT INTO partition_events (event_id, partition_ref, status, message, job_run_id) VALUES (?1, ?2, ?3, ?4, ?5)",
+                    params![
+                        event.event_id,
+                        p_event.partition_ref.as_ref().map(|r| &r.str).unwrap_or(&String::new()),
+                        p_event.status_code.to_string(),
+                        p_event.message,
+                        if p_event.job_run_id.is_empty() { None } else { Some(&p_event.job_run_id) }
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::JobEvent(j_event)) => {
+                let partitions_json = serde_json::to_string(&j_event.target_partitions)
+                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+                let config_json = j_event.config.as_ref()
+                    .map(|c| serde_json::to_string(c))
+                    .transpose()
+                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+                let manifests_json = serde_json::to_string(&j_event.manifests)
+                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+                
+                conn.execute(
+                    "INSERT INTO job_events (event_id, job_run_id, job_label, target_partitions, status, message, config_json, manifests_json) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)",
+                    params![
+                        event.event_id,
+                        j_event.job_run_id,
+                        j_event.job_label.as_ref().map(|l| &l.label).unwrap_or(&String::new()),
+                        partitions_json,
+                        j_event.status_code.to_string(),
+                        j_event.message,
+                        config_json,
+                        manifests_json
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::DelegationEvent(d_event)) => {
+                conn.execute(
+                    "INSERT INTO delegation_events (event_id, partition_ref, delegated_to_build_request_id, message) VALUES (?1, ?2, ?3, ?4)",
+                    params![
+                        event.event_id,
+                        d_event.partition_ref.as_ref().map(|r| &r.str).unwrap_or(&String::new()),
+                        d_event.delegated_to_build_request_id,
+                        d_event.message
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::JobGraphEvent(jg_event)) => {
+                let job_graph_json = match serde_json::to_string(&jg_event.job_graph) {
+                    Ok(json) => json,
+                    Err(e) => {
+                        return Err(BuildEventLogError::DatabaseError(format!("Failed to serialize job graph: {}", e)));
+                    }
+                };
+                conn.execute(
+                    "INSERT INTO job_graph_events (event_id, job_graph_json, message) VALUES (?1, ?2, ?3)",
+                    params![
+                        event.event_id,
+                        job_graph_json,
+                        jg_event.message
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::PartitionInvalidationEvent(_pi_event)) => {
+                // For now, just store in main events table
+            }
+            Some(crate::build_event::EventType::TaskCancelEvent(_tc_event)) => {
+                // For now, just store in main events table
+            }
+            Some(crate::build_event::EventType::BuildCancelEvent(_bc_event)) => {
+                // For now, just store in main events table
+            }
+            None => {}
+        }
+        
+        Ok(())
+    }
+    
+    async fn get_build_request_events(
+        &self, 
+        build_request_id: &str,
+        since: Option<i64>
+    ) -> Result<Vec<BuildEvent>> {
+        let conn = self.connection.lock().unwrap();
+        let (query, params): (String, Vec<_>) = match since {
+            Some(timestamp) => (
+                "SELECT event_data FROM build_events WHERE build_request_id = ?1 AND timestamp > ?2 ORDER BY timestamp ASC".to_string(),
+                vec![build_request_id.to_string(), timestamp.to_string()]
+            ),
+            None => (
+                "SELECT event_data FROM build_events WHERE build_request_id = ?1 ORDER BY timestamp ASC".to_string(),
+                vec![build_request_id.to_string()]
+            )
+        };
+        
+        let mut stmt = conn.prepare(&query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let rows = stmt.query_map(rusqlite::params_from_iter(params.iter()), |row| {
+            let event_data: String = row.get(0)?;
+            Ok(event_data)
+        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut events = Vec::new();
+        for row in rows {
+            let event_data = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            let event: BuildEvent = serde_json::from_str(&event_data)
+                .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+            events.push(event);
+        }
+        
+        Ok(events)
+    }
+    
+    async fn get_partition_events(
+        &self,
+        partition_ref: &str,
+        since: Option<i64>
+    ) -> Result<Vec<BuildEvent>> {
+        let conn = self.connection.lock().unwrap();
+        let (query, params): (String, Vec<_>) = match since {
+            Some(timestamp) => (
+                "SELECT be.event_data FROM build_events be JOIN partition_events pe ON be.event_id = pe.event_id WHERE pe.partition_ref = ?1 AND be.timestamp > ?2 ORDER BY be.timestamp ASC".to_string(),
+                vec![partition_ref.to_string(), timestamp.to_string()]
+            ),
+            None => (
+                "SELECT be.event_data FROM build_events be JOIN partition_events pe ON be.event_id = pe.event_id WHERE pe.partition_ref = ?1 ORDER BY be.timestamp ASC".to_string(),
+                vec![partition_ref.to_string()]
+            )
+        };
+        
+        let mut stmt = conn.prepare(&query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let rows = stmt.query_map(rusqlite::params_from_iter(params.iter()), |row| {
+            let event_data: String = row.get(0)?;
+            Ok(event_data)
+        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut events = Vec::new();
+        for row in rows {
+            let event_data = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            let event: BuildEvent = serde_json::from_str(&event_data)
+                .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+            events.push(event);
+        }
+        
+        Ok(events)
+    }
+    
+    async fn get_job_run_events(
+        &self,
+        job_run_id: &str
+    ) -> Result<Vec<BuildEvent>> {
+        let conn = self.connection.lock().unwrap();
+        let mut stmt = conn.prepare(
+            "SELECT be.event_data FROM build_events be JOIN job_events je ON be.event_id = je.event_id WHERE je.job_run_id = ?1 ORDER BY be.timestamp ASC"
+        ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let rows = stmt.query_map([job_run_id], |row| {
+            let event_data: String = row.get(0)?;
+            Ok(event_data)
+        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut events = Vec::new();
+        for row in rows {
+            let event_data = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            let event: BuildEvent = serde_json::from_str(&event_data)
+                .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+            events.push(event);
+        }
+        
+        Ok(events)
+    }
+    
+    async fn get_events_in_range(
+        &self,
+        start_time: i64,
+        end_time: i64
+    ) -> Result<Vec<BuildEvent>> {
+        let conn = self.connection.lock().unwrap();
+        let mut stmt = conn.prepare(
+            "SELECT event_data FROM build_events WHERE timestamp >= ?1 AND timestamp <= ?2 ORDER BY timestamp ASC"
+        ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let rows = stmt.query_map([start_time, end_time], |row| {
+            let event_data: String = row.get(0)?;
+            Ok(event_data)
+        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut events = Vec::new();
+        for row in rows {
+            let event_data = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            let event: BuildEvent = serde_json::from_str(&event_data)
+                .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+            events.push(event);
+        }
+        
+        Ok(events)
+    }
+    
+    async fn execute_query(&self, query: &str) -> Result<QueryResult> {
+        let conn = self.connection.lock().unwrap();
+        let mut stmt = conn.prepare(query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let column_names: Vec<String> = stmt.column_names().iter().map(|s| s.to_string()).collect();
+        
+        let rows = stmt.query_map([], |row| {
+            let mut values = Vec::new();
+            for i in 0..column_names.len() {
+                let value: String = row.get::<_, Option<String>>(i)?.unwrap_or_default();
+                values.push(value);
+            }
+            Ok(values)
+        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut result_rows = Vec::new();
+        for row in rows {
+            let values = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            result_rows.push(values);
+        }
+        
+        Ok(QueryResult {
+            columns: column_names,
+            rows: result_rows,
+        })
+    }
+    
+    async fn get_latest_partition_status(
+        &self,
+        partition_ref: &str
+    ) -> Result<Option<(PartitionStatus, i64)>> {
+        let conn = self.connection.lock().unwrap();
+        let mut stmt = conn.prepare(
+            "SELECT pe.status, be.timestamp FROM build_events be JOIN partition_events pe ON be.event_id = pe.event_id WHERE pe.partition_ref = ?1 ORDER BY be.timestamp DESC LIMIT 1"
+        ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let result = stmt.query_row([partition_ref], |row| {
+            let status_str: String = row.get(0)?;
+            let timestamp: i64 = row.get(1)?;
+            let status: i32 = status_str.parse().unwrap_or(0);
+            Ok((status, timestamp))
+        });
+        
+        match result {
+            Ok((status, timestamp)) => {
+                let partition_status = match status {
+                    1 => PartitionStatus::PartitionRequested,
+                    2 => PartitionStatus::PartitionAnalyzed,
+                    3 => PartitionStatus::PartitionBuilding,
+                    4 => PartitionStatus::PartitionAvailable,
+                    5 => PartitionStatus::PartitionFailed,
+                    6 => PartitionStatus::PartitionDelegated,
+                    _ => PartitionStatus::PartitionUnknown,
+                };
+                Ok(Some((partition_status, timestamp)))
+            }
+            Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
+            Err(e) => Err(BuildEventLogError::QueryError(e.to_string())),
+        }
+    }
+    
+    async fn get_active_builds_for_partition(
+        &self,
+        partition_ref: &str
+    ) -> Result<Vec<String>> {
+        let conn = self.connection.lock().unwrap();
+        let mut stmt = conn.prepare(
+            "SELECT DISTINCT be.build_request_id FROM build_events be JOIN partition_events pe ON be.event_id = pe.event_id WHERE pe.partition_ref = ?1 AND pe.status IN ('1', '2', '3') ORDER BY be.timestamp DESC"
+        ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let rows = stmt.query_map([partition_ref], |row| {
+            let build_request_id: String = row.get(0)?;
+            Ok(build_request_id)
+        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut build_ids = Vec::new();
+        for row in rows {
+            let build_id = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            build_ids.push(build_id);
+        }
+        
+        Ok(build_ids)
+    }
+    
+    async fn initialize(&self) -> Result<()> {
         let conn = self.connection.lock().unwrap();
         
         // Create main events table
@@ -123,7 +454,7 @@ impl MockBuildEventLog {
             [],
         ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
         
-        // Create supporting tables for easier queries
+        // Create specific event type tables
         conn.execute(
             "CREATE TABLE IF NOT EXISTS build_request_events (
                 event_id TEXT PRIMARY KEY,
@@ -172,199 +503,90 @@ impl MockBuildEventLog {
         conn.execute(
             "CREATE TABLE IF NOT EXISTS job_graph_events (
                 event_id TEXT PRIMARY KEY,
-                job_graph_json TEXT NOT NULL
+                job_graph_json TEXT NOT NULL,
+                message TEXT NOT NULL
             )",
             [],
         ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
         
+        // Create indexes for common queries
+        conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_build_events_build_request_id ON build_events (build_request_id)",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_build_events_timestamp ON build_events (timestamp)",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_partition_events_partition_ref ON partition_events (partition_ref)",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_job_events_job_run_id ON job_events (job_run_id)",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
         Ok(())
     }
     
-    /// Append an event to the mock event log
+    async fn list_build_requests(
-    pub async fn append_event(&self, event: BuildEvent) -> Result<()> {
+        &self,
-        let conn = self.connection.lock().unwrap();
+        limit: u32,
-        
+        offset: u32,
-        // Serialize the entire event for storage
+        status_filter: Option<BuildRequestStatus>,
-        let event_data = serde_json::to_string(&event)
+    ) -> Result<(Vec<BuildRequestSummary>, u32)> {
-            .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+        // For simplicity in the mock, return empty results
-        
+        // Real implementation would query the database
-        // Insert into main events table
+        Ok((vec![], 0))
-        conn.execute(
-            "INSERT INTO build_events (event_id, timestamp, build_request_id, event_type, event_data) VALUES (?1, ?2, ?3, ?4, ?5)",
-            rusqlite::params![
-                event.event_id,
-                event.timestamp,
-                event.build_request_id,
-                match &event.event_type {
-                    Some(crate::build_event::EventType::BuildRequestEvent(_)) => "build_request",
-                    Some(crate::build_event::EventType::PartitionEvent(_)) => "partition",
-                    Some(crate::build_event::EventType::JobEvent(_)) => "job",
-                    Some(crate::build_event::EventType::DelegationEvent(_)) => "delegation",
-                    Some(crate::build_event::EventType::JobGraphEvent(_)) => "job_graph",
-                    Some(crate::build_event::EventType::PartitionInvalidationEvent(_)) => "partition_invalidation",
-                    Some(crate::build_event::EventType::JobRunCancelEvent(_)) => "job_run_cancel",
-                    Some(crate::build_event::EventType::BuildCancelEvent(_)) => "build_cancel",
-                    None => "unknown",
-                },
-                event_data
-            ],
-        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
-        
-        // Insert into specific event type table for better querying
-        match &event.event_type {
-            Some(crate::build_event::EventType::BuildRequestEvent(br_event)) => {
-                let partitions_json = serde_json::to_string(&br_event.requested_partitions)
-                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-                
-                conn.execute(
-                    "INSERT INTO build_request_events (event_id, status, requested_partitions, message) VALUES (?1, ?2, ?3, ?4)",
-                    rusqlite::params![
-                        event.event_id,
-                        br_event.status_code.to_string(),
-                        partitions_json,
-                        br_event.message
-                    ],
-                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
-            }
-            Some(crate::build_event::EventType::PartitionEvent(p_event)) => {
-                conn.execute(
-                    "INSERT INTO partition_events (event_id, partition_ref, status, message, job_run_id) VALUES (?1, ?2, ?3, ?4, ?5)",
-                    rusqlite::params![
-                        event.event_id,
-                        p_event.partition_ref.as_ref().map(|r| &r.str).unwrap_or(&String::new()),
-                        p_event.status_code.to_string(),
-                        p_event.message,
-                        if p_event.job_run_id.is_empty() { None } else { Some(&p_event.job_run_id) }
-                    ],
-                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
-            }
-            Some(crate::build_event::EventType::JobEvent(j_event)) => {
-                let partitions_json = serde_json::to_string(&j_event.target_partitions)
-                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-                let config_json = j_event.config.as_ref()
-                    .map(|c| serde_json::to_string(c))
-                    .transpose()
-                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-                let manifests_json = serde_json::to_string(&j_event.manifests)
-                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-                
-                conn.execute(
-                    "INSERT INTO job_events (event_id, job_run_id, job_label, target_partitions, status, message, config_json, manifests_json) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)",
-                    rusqlite::params![
-                        event.event_id,
-                        j_event.job_run_id,
-                        j_event.job_label.as_ref().map(|l| &l.label).unwrap_or(&String::new()),
-                        partitions_json,
-                        j_event.status_code.to_string(),
-                        j_event.message,
-                        config_json,
-                        manifests_json
-                    ],
-                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
-            }
-            _ => {} // Other event types don't need special handling for testing
     }
 
-        Ok(())
+    async fn list_recent_partitions(
+        &self,
+        limit: u32, 
+        offset: u32,
+        status_filter: Option<PartitionStatus>,
+    ) -> Result<(Vec<PartitionSummary>, u32)> {
+        // For simplicity in the mock, return empty results
+        // Real implementation would query the database
+        Ok((vec![], 0))
     }
 
-    /// Get all events for a specific build request
+    async fn get_activity_summary(&self) -> Result<ActivitySummary> {
-    pub async fn get_build_request_events(&self, build_request_id: &str, _limit: Option<u32>) -> Result<Vec<BuildEvent>> {
+        // For simplicity in the mock, return empty activity
+        Ok(ActivitySummary {
+            active_builds_count: 0,
+            recent_builds: vec![],
+            recent_partitions: vec![],
+            total_partitions_count: 0,
+        })
+    }
+    
+    async fn get_build_request_for_available_partition(
+        &self,
+        partition_ref: &str
+    ) -> Result<Option<String>> {
         let conn = self.connection.lock().unwrap();
         let mut stmt = conn.prepare(
-            "SELECT event_data FROM build_events WHERE build_request_id = ? ORDER BY timestamp ASC"
+            "SELECT be.build_request_id FROM build_events be JOIN partition_events pe ON be.event_id = pe.event_id WHERE pe.partition_ref = ?1 AND pe.status = '4' ORDER BY be.timestamp DESC LIMIT 1"
-        ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-
-        let rows = stmt.query_map([build_request_id], |row| {
-            let event_data: String = row.get(0)?;
-            Ok(event_data)
-        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-
-        let mut events = Vec::new();
-        for row in rows {
-            let event_data = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-            let event: BuildEvent = serde_json::from_str(&event_data)
-                .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-            events.push(event);
-        }
-
-        Ok(events)
-    }
-
-    /// Get all events for a specific partition
-    pub async fn get_partition_events(&self, partition_ref: &str, _limit: Option<u32>) -> Result<Vec<BuildEvent>> {
-        let conn = self.connection.lock().unwrap();
-        let mut stmt = conn.prepare(
-            "SELECT e.event_data FROM build_events e 
-             JOIN partition_events p ON e.event_id = p.event_id 
-             WHERE p.partition_ref = ? ORDER BY e.timestamp ASC"
-        ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-
-        let rows = stmt.query_map([partition_ref], |row| {
-            let event_data: String = row.get(0)?;
-            Ok(event_data)
-        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-
-        let mut events = Vec::new();
-        for row in rows {
-            let event_data = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-            let event: BuildEvent = serde_json::from_str(&event_data)
-                .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-            events.push(event);
-        }
-
-        Ok(events)
-    }
-
-    /// Get the latest status for a partition
-    pub async fn get_latest_partition_status(&self, partition_ref: &str) -> Result<Option<(PartitionStatus, i64)>> {
-        let conn = self.connection.lock().unwrap();
-        let mut stmt = conn.prepare(
-            "SELECT p.status, e.timestamp FROM build_events e 
-             JOIN partition_events p ON e.event_id = p.event_id 
-             WHERE p.partition_ref = ? ORDER BY e.timestamp DESC LIMIT 1"
         ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
         
         let result = stmt.query_row([partition_ref], |row| {
-            let status_str: String = row.get(0)?;
+            let build_request_id: String = row.get(0)?;
-            let timestamp: i64 = row.get(1)?;
+            Ok(build_request_id)
-            let status_code = status_str.parse::<i32>().unwrap_or(0);
-            let status = PartitionStatus::try_from(status_code).unwrap_or(PartitionStatus::PartitionUnknown);
-            Ok((status, timestamp))
         });
         
         match result {
-            Ok(status_and_timestamp) => Ok(Some(status_and_timestamp)),
+            Ok(build_request_id) => Ok(Some(build_request_id)),
             Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
             Err(e) => Err(BuildEventLogError::QueryError(e.to_string())),
         }
     }
-
-    /// Get events in a timestamp range (used by BELStorage)
-    pub async fn get_events_in_range(&self, start: i64, end: i64) -> Result<Vec<BuildEvent>> {
-        let conn = self.connection.lock().unwrap();
-        let mut stmt = conn.prepare(
-            "SELECT event_data FROM build_events WHERE timestamp >= ? AND timestamp <= ? ORDER BY timestamp ASC"
-        ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-
-        let rows = stmt.query_map([start, end], |row| {
-            let event_data: String = row.get(0)?;
-            Ok(event_data)
-        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-
-        let mut events = Vec::new();
-        for row in rows {
-            let event_data = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-            let event: BuildEvent = serde_json::from_str(&event_data)
-                .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-            events.push(event);
-        }
-
-        Ok(events)
-    }
 }
 
-
 /// Utility functions for creating test events with sensible defaults
 pub mod test_events {
     use super::*;
@@ -531,130 +753,3 @@ mod tests {
         assert!(matches!(j_event.event_type, Some(build_event::EventType::JobEvent(_))));
     }
 }
-
-/// MockBELStorage is a BELStorage implementation that wraps MockBuildEventLog
-/// This allows us to use the real BELQueryEngine in tests while having control over the data
-pub struct MockBELStorage {
-    mock_log: Arc<MockBuildEventLog>,
-}
-
-impl MockBELStorage {
-    pub async fn new() -> Result<Self> {
-        let mock_log = Arc::new(MockBuildEventLog::new().await?);
-        Ok(Self { mock_log })
-    }
-    
-    pub async fn with_events(events: Vec<BuildEvent>) -> Result<Self> {
-        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await?);
-        Ok(Self { mock_log })
-    }
-}
-
-#[async_trait]
-impl BELStorage for MockBELStorage {
-    async fn append_event(&self, event: BuildEvent) -> Result<i64> {
-        self.mock_log.append_event(event).await?;
-        Ok(0) // Return dummy index for mock storage
-    }
-
-    async fn list_events(&self, since_idx: i64, filter: EventFilter) -> Result<EventPage> {
-        // Get all events first (MockBELEventLog uses timestamps, so we get all events)
-        let mut events = self.mock_log.get_events_in_range(0, i64::MAX).await?;
-        
-        // Apply filtering based on EventFilter
-        events.retain(|event| {
-            // Filter by build request IDs if specified
-            if !filter.build_request_ids.is_empty() {
-                if !filter.build_request_ids.contains(&event.build_request_id) {
-                    return false;
-                }
-            }
-            
-            // Filter by partition refs if specified
-            if !filter.partition_refs.is_empty() {
-                let has_matching_partition = match &event.event_type {
-                    Some(build_event::EventType::PartitionEvent(pe)) => {
-                        pe.partition_ref.as_ref()
-                            .map(|pr| filter.partition_refs.contains(&pr.str))
-                            .unwrap_or(false)
-                    }
-                    Some(build_event::EventType::BuildRequestEvent(bre)) => {
-                        bre.requested_partitions.iter()
-                            .any(|pr| filter.partition_refs.contains(&pr.str))
-                    }
-                    Some(build_event::EventType::JobEvent(je)) => {
-                        je.target_partitions.iter()
-                            .any(|pr| filter.partition_refs.contains(&pr.str))
-                    }
-                    _ => false,
-                };
-                if !has_matching_partition {
-                    return false;
-                }
-            }
-            
-            // Filter by job labels if specified
-            if !filter.job_labels.is_empty() {
-                let has_matching_job = match &event.event_type {
-                    Some(build_event::EventType::JobEvent(je)) => {
-                        je.job_label.as_ref()
-                            .map(|jl| filter.job_labels.contains(&jl.label))
-                            .unwrap_or(false)
-                    }
-                    _ => false,
-                };
-                if !has_matching_job {
-                    return false;
-                }
-            }
-            
-            // Filter by job run IDs if specified  
-            if !filter.job_run_ids.is_empty() {
-                let has_matching_job_run = match &event.event_type {
-                    Some(build_event::EventType::JobEvent(je)) => {
-                        filter.job_run_ids.contains(&je.job_run_id)
-                    }
-                    Some(build_event::EventType::JobRunCancelEvent(jrce)) => {
-                        filter.job_run_ids.contains(&jrce.job_run_id)
-                    }
-                    Some(build_event::EventType::PartitionEvent(pe)) => {
-                        if pe.job_run_id.is_empty() {
-                            false
-                        } else {
-                            filter.job_run_ids.contains(&pe.job_run_id)
-                        }
-                    }
-                    // Add other job-run-related events here if they exist
-                    _ => false,
-                };
-                if !has_matching_job_run {
-                    return false;
-                }
-            }
-            
-            true
-        });
-
-        Ok(EventPage {
-            events,
-            next_idx: since_idx + 1, // Simple increment for testing
-            has_more: false, // Simplify for testing
-        })
-    }
-
-    async fn initialize(&self) -> Result<()> {
-        self.mock_log.initialize().await
-    }
-}
-
-/// Helper function to create a BELQueryEngine for testing with mock data
-pub async fn create_mock_bel_query_engine() -> Result<Arc<BELQueryEngine>> {
-    let storage: Arc<dyn BELStorage> = Arc::new(MockBELStorage::new().await?);
-    Ok(Arc::new(BELQueryEngine::new(storage)))
-}
-
-/// Helper function to create a BELQueryEngine for testing with predefined events
-pub async fn create_mock_bel_query_engine_with_events(events: Vec<BuildEvent>) -> Result<Arc<BELQueryEngine>> {
-    let storage: Arc<dyn BELStorage> = Arc::new(MockBELStorage::with_events(events).await?);
-    Ok(Arc::new(BELQueryEngine::new(storage)))
-}

databuild/event_log/mod.rs

@@ -1,12 +1,14 @@
 use crate::*;
+use async_trait::async_trait;
 use std::error::Error as StdError;
 use uuid::Uuid;
 
+pub mod stdout;
+pub mod sqlite;
+pub mod postgres;
+pub mod delta;
 pub mod writer;
 pub mod mock;
-pub mod storage;
-pub mod sqlite_storage;
-pub mod query_engine;
 
 #[derive(Debug)]
 pub enum BuildEventLogError {
@@ -63,6 +65,82 @@ pub struct ActivitySummary {
     pub total_partitions_count: u32,
 }
 
+#[async_trait]
+pub trait BuildEventLog: Send + Sync {
+    // Append new event to the log
+    async fn append_event(&self, event: BuildEvent) -> Result<()>;
+    
+    // Query events by build request
+    async fn get_build_request_events(
+        &self, 
+        build_request_id: &str,
+        since: Option<i64>
+    ) -> Result<Vec<BuildEvent>>;
+    
+    // Query events by partition
+    async fn get_partition_events(
+        &self,
+        partition_ref: &str,
+        since: Option<i64>
+    ) -> Result<Vec<BuildEvent>>;
+    
+    // Query events by job run
+    async fn get_job_run_events(
+        &self,
+        job_run_id: &str
+    ) -> Result<Vec<BuildEvent>>;
+    
+    // Query events in time range
+    async fn get_events_in_range(
+        &self,
+        start_time: i64,
+        end_time: i64
+    ) -> Result<Vec<BuildEvent>>;
+    
+    // Execute raw SQL queries (for dashboard and debugging)
+    // Note: Non-SQL backends should return QueryError for unsupported queries
+    async fn execute_query(&self, query: &str) -> Result<QueryResult>;
+    
+    // Get latest partition availability status
+    async fn get_latest_partition_status(
+        &self,
+        partition_ref: &str
+    ) -> Result<Option<(PartitionStatus, i64)>>; // status and timestamp
+    
+    // Check if partition is being built by another request
+    async fn get_active_builds_for_partition(
+        &self,
+        partition_ref: &str
+    ) -> Result<Vec<String>>; // build request IDs
+    
+    // Initialize/setup the storage backend
+    async fn initialize(&self) -> Result<()>;
+    
+    // List recent build requests with pagination and filtering
+    async fn list_build_requests(
+        &self,
+        limit: u32,
+        offset: u32,
+        status_filter: Option<BuildRequestStatus>,
+    ) -> Result<(Vec<BuildRequestSummary>, u32)>;
+
+    // List recent partitions with pagination and filtering  
+    async fn list_recent_partitions(
+        &self,
+        limit: u32, 
+        offset: u32,
+        status_filter: Option<PartitionStatus>,
+    ) -> Result<(Vec<PartitionSummary>, u32)>;
+
+    // Get aggregated activity summary for dashboard
+    async fn get_activity_summary(&self) -> Result<ActivitySummary>;
+    
+    // Get the build request ID that created an available partition
+    async fn get_build_request_for_available_partition(
+        &self,
+        partition_ref: &str
+    ) -> Result<Option<String>>; // build request ID that made partition available
+}
 
 // Helper function to generate event ID
 pub fn generate_event_id() -> String {
@@ -90,24 +168,27 @@ pub fn create_build_event(
     }
 }
 
-
+// Parse build event log URI and create appropriate implementation
-// Parse build event log URI and create BEL query engine with appropriate storage backend
+pub async fn create_build_event_log(uri: &str) -> Result<Box<dyn BuildEventLog>> {
-pub async fn create_bel_query_engine(uri: &str) -> Result<std::sync::Arc<query_engine::BELQueryEngine>> {
-    use std::sync::Arc;
-    use storage::BELStorage;
-    
     if uri == "stdout" {
-        let storage: Arc<dyn BELStorage> = Arc::new(storage::StdoutBELStorage::new());
+        Ok(Box::new(stdout::StdoutBuildEventLog::new()))
-        storage.initialize().await?;
-        Ok(Arc::new(query_engine::BELQueryEngine::new(storage)))
     } else if uri.starts_with("sqlite://") {
         let path = &uri[9..]; // Remove "sqlite://" prefix
-        let storage: Arc<dyn BELStorage> = Arc::new(sqlite_storage::SqliteBELStorage::new(path)?);
+        let log = sqlite::SqliteBuildEventLog::new(path).await?;
-        storage.initialize().await?;
+        log.initialize().await?;
-        Ok(Arc::new(query_engine::BELQueryEngine::new(storage)))
+        Ok(Box::new(log))
+    } else if uri.starts_with("postgres://") {
+        let log = postgres::PostgresBuildEventLog::new(uri).await?;
+        log.initialize().await?;
+        Ok(Box::new(log))
+    } else if uri.starts_with("delta://") {
+        let path = &uri[8..]; // Remove "delta://" prefix
+        let log = delta::DeltaBuildEventLog::new(path).await?;
+        log.initialize().await?;
+        Ok(Box::new(log))
     } else {
         Err(BuildEventLogError::ConnectionError(
-            format!("Unsupported build event log URI for BEL query engine: {}", uri)
+            format!("Unsupported build event log URI: {}", uri)
         ))
     }
 }

databuild/event_log/postgres.rs

@@ -0,0 +1,132 @@
+use super::*;
+use async_trait::async_trait;
+
+pub struct PostgresBuildEventLog {
+    _connection_string: String,
+}
+
+impl PostgresBuildEventLog {
+    pub async fn new(connection_string: &str) -> Result<Self> {
+        // For now, just store the connection string
+        // In a real implementation, we'd establish a connection pool here
+        Ok(Self {
+            _connection_string: connection_string.to_string(),
+        })
+    }
+}
+
+#[async_trait]
+impl BuildEventLog for PostgresBuildEventLog {
+    async fn append_event(&self, _event: BuildEvent) -> Result<()> {
+        // TODO: Implement PostgreSQL event storage
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn get_build_request_events(
+        &self, 
+        _build_request_id: &str,
+        _since: Option<i64>
+    ) -> Result<Vec<BuildEvent>> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn get_partition_events(
+        &self,
+        _partition_ref: &str,
+        _since: Option<i64>
+    ) -> Result<Vec<BuildEvent>> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn get_job_run_events(
+        &self,
+        _job_run_id: &str
+    ) -> Result<Vec<BuildEvent>> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn get_events_in_range(
+        &self,
+        _start_time: i64,
+        _end_time: i64
+    ) -> Result<Vec<BuildEvent>> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn execute_query(&self, _query: &str) -> Result<QueryResult> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn get_latest_partition_status(
+        &self,
+        _partition_ref: &str
+    ) -> Result<Option<(PartitionStatus, i64)>> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn get_active_builds_for_partition(
+        &self,
+        _partition_ref: &str
+    ) -> Result<Vec<String>> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn initialize(&self) -> Result<()> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn list_build_requests(
+        &self,
+        _limit: u32,
+        _offset: u32,
+        _status_filter: Option<BuildRequestStatus>,
+    ) -> Result<(Vec<BuildRequestSummary>, u32)> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+
+    async fn list_recent_partitions(
+        &self,
+        _limit: u32, 
+        _offset: u32,
+        _status_filter: Option<PartitionStatus>,
+    ) -> Result<(Vec<PartitionSummary>, u32)> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+
+    async fn get_activity_summary(&self) -> Result<ActivitySummary> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+    
+    async fn get_build_request_for_available_partition(
+        &self,
+        _partition_ref: &str
+    ) -> Result<Option<String>> {
+        Err(BuildEventLogError::DatabaseError(
+            "PostgreSQL implementation not yet available".to_string()
+        ))
+    }
+}

databuild/event_log/query_engine.rs

@@ -1,388 +0,0 @@
-use super::*;
-use super::storage::BELStorage;
-use std::sync::Arc;
-use std::collections::HashMap;
-
-/// App-layer aggregation that scans storage events
-pub struct BELQueryEngine {
-    storage: Arc<dyn BELStorage>,
-}
-
-impl BELQueryEngine {
-    pub fn new(storage: Arc<dyn BELStorage>) -> Self {
-        Self { storage }
-    }
-    
-    /// Get latest status for a partition by scanning recent events
-    pub async fn get_latest_partition_status(&self, partition_ref: &str) -> Result<Option<(PartitionStatus, i64)>> {
-        let filter = EventFilter {
-            partition_refs: vec![partition_ref.to_string()],
-            partition_patterns: vec![],
-            job_labels: vec![],
-            job_run_ids: vec![],
-            build_request_ids: vec![],
-        };
-        
-        let events = self.storage.list_events(0, filter).await?;
-        self.aggregate_partition_status(&events.events)
-    }
-    
-    /// Get all build requests that are currently building a partition
-    pub async fn get_active_builds_for_partition(&self, partition_ref: &str) -> Result<Vec<String>> {
-        let filter = EventFilter {
-            partition_refs: vec![partition_ref.to_string()],
-            partition_patterns: vec![],
-            job_labels: vec![],
-            job_run_ids: vec![],
-            build_request_ids: vec![],
-        };
-        
-        let events = self.storage.list_events(0, filter).await?;
-        let mut active_builds = Vec::new();
-        let mut build_states: HashMap<String, BuildRequestStatus> = HashMap::new();
-        
-        // Process events chronologically to track build states
-        for event in events.events {
-            match &event.event_type {
-                Some(crate::build_event::EventType::BuildRequestEvent(br_event)) => {
-                    if let Ok(status) = BuildRequestStatus::try_from(br_event.status_code) {
-                        build_states.insert(event.build_request_id.clone(), status);
-                    }
-                }
-                Some(crate::build_event::EventType::PartitionEvent(p_event)) => {
-                    if let Some(partition_event_ref) = &p_event.partition_ref {
-                        if partition_event_ref.str == partition_ref {
-                            // Check if this partition is actively being built
-                            if let Ok(status) = PartitionStatus::try_from(p_event.status_code) {
-                                if matches!(status, PartitionStatus::PartitionBuilding | PartitionStatus::PartitionAnalyzed) {
-                                    // Check if the build request is still active
-                                    if let Some(build_status) = build_states.get(&event.build_request_id) {
-                                        if matches!(build_status, 
-                                            BuildRequestStatus::BuildRequestReceived |
-                                            BuildRequestStatus::BuildRequestPlanning |
-                                            BuildRequestStatus::BuildRequestExecuting |
-                                            BuildRequestStatus::BuildRequestAnalysisCompleted
-                                        ) {
-                                            if !active_builds.contains(&event.build_request_id) {
-                                                active_builds.push(event.build_request_id.clone());
-                                            }
-                                        }
-                                    }
-                                }
-                            }
-                        }
-                    }
-                }
-                _ => {}
-            }
-        }
-        
-        Ok(active_builds)
-    }
-    
-    /// Get summary of a build request by aggregating its events
-    pub async fn get_build_request_summary(&self, build_id: &str) -> Result<BuildRequestSummary> {
-        let filter = EventFilter {
-            partition_refs: vec![],
-            partition_patterns: vec![],
-            job_labels: vec![],
-            job_run_ids: vec![],
-            build_request_ids: vec![build_id.to_string()],
-        };
-        
-        let events = self.storage.list_events(0, filter).await?;
-        
-        // If no events found, build doesn't exist
-        if events.events.is_empty() {
-            return Err(BuildEventLogError::QueryError(format!("Build request '{}' not found", build_id)));
-        }
-        
-        let mut status = BuildRequestStatus::BuildRequestUnknown;
-        let mut requested_partitions = Vec::new();
-        let mut created_at = 0i64;
-        let mut updated_at = 0i64;
-        
-        for event in events.events {
-            if event.timestamp > 0 {
-                if created_at == 0 || event.timestamp < created_at {
-                    created_at = event.timestamp;
-                }
-                if event.timestamp > updated_at {
-                    updated_at = event.timestamp;
-                }
-            }
-            
-            if let Some(crate::build_event::EventType::BuildRequestEvent(br_event)) = &event.event_type {
-                if let Ok(event_status) = BuildRequestStatus::try_from(br_event.status_code) {
-                    status = event_status;
-                }
-                if !br_event.requested_partitions.is_empty() {
-                    requested_partitions = br_event.requested_partitions.iter()
-                        .map(|p| p.str.clone())
-                        .collect();
-                }
-            }
-        }
-        
-        Ok(BuildRequestSummary {
-            build_request_id: build_id.to_string(),
-            status,
-            requested_partitions,
-            created_at,
-            updated_at,
-        })
-    }
-    
-    /// List build requests with pagination and filtering
-    pub async fn list_build_requests(&self, request: BuildsListRequest) -> Result<BuildsListResponse> {
-        // For now, scan all events and aggregate
-        let filter = EventFilter {
-            partition_refs: vec![],
-            partition_patterns: vec![],
-            job_labels: vec![],
-            job_run_ids: vec![],
-            build_request_ids: vec![],
-        };
-        
-        let events = self.storage.list_events(0, filter).await?;
-        let mut build_summaries: HashMap<String, BuildRequestSummary> = HashMap::new();
-        
-        // Aggregate by build request ID
-        for event in events.events {
-            if let Some(crate::build_event::EventType::BuildRequestEvent(br_event)) = &event.event_type {
-                let build_id = &event.build_request_id;
-                let entry = build_summaries.entry(build_id.clone()).or_insert_with(|| {
-                    BuildRequestSummary {
-                        build_request_id: build_id.clone(),
-                        status: BuildRequestStatus::BuildRequestUnknown,
-                        requested_partitions: Vec::new(),
-                        created_at: event.timestamp,
-                        updated_at: event.timestamp,
-                    }
-                });
-                
-                if let Ok(status) = BuildRequestStatus::try_from(br_event.status_code) {
-                    entry.status = status;
-                }
-                entry.updated_at = event.timestamp.max(entry.updated_at);
-                if !br_event.requested_partitions.is_empty() {
-                    entry.requested_partitions = br_event.requested_partitions.iter()
-                        .map(|p| p.str.clone())
-                        .collect();
-                }
-            }
-        }
-        
-        let mut builds: Vec<_> = build_summaries.into_values().collect();
-        builds.sort_by(|a, b| b.created_at.cmp(&a.created_at)); // Most recent first
-        
-        // Apply status filter if provided
-        if let Some(status_filter) = &request.status_filter {
-            if let Ok(filter_status) = status_filter.parse::<i32>() {
-                if let Ok(status) = BuildRequestStatus::try_from(filter_status) {
-                    builds.retain(|b| b.status == status);
-                }
-            }
-        }
-        
-        let total_count = builds.len() as u32;
-        let offset = request.offset.unwrap_or(0) as usize;
-        let limit = request.limit.unwrap_or(50) as usize;
-        
-        let paginated_builds = builds.into_iter()
-            .skip(offset)
-            .take(limit)
-            .map(|summary| BuildSummary {
-                build_request_id: summary.build_request_id,
-                status_code: summary.status as i32,
-                status_name: summary.status.to_display_string(),
-                requested_partitions: summary.requested_partitions.into_iter()
-                    .map(|s| PartitionRef { str: s })
-                    .collect(),
-                total_jobs: 0, // TODO: Implement
-                completed_jobs: 0, // TODO: Implement
-                failed_jobs: 0, // TODO: Implement
-                cancelled_jobs: 0, // TODO: Implement
-                requested_at: summary.created_at,
-                started_at: None, // TODO: Implement
-                completed_at: None, // TODO: Implement
-                duration_ms: None, // TODO: Implement
-                cancelled: false, // TODO: Implement
-            })
-            .collect();
-        
-        Ok(BuildsListResponse {
-            builds: paginated_builds,
-            total_count,
-            has_more: (offset + limit) < total_count as usize,
-        })
-    }
-    
-    /// Get activity summary for dashboard
-    pub async fn get_activity_summary(&self) -> Result<ActivitySummary> {
-        let builds_response = self.list_build_requests(BuildsListRequest {
-            limit: Some(5),
-            offset: Some(0),
-            status_filter: None,
-        }).await?;
-        
-        let active_builds_count = builds_response.builds.iter()
-            .filter(|b| matches!(
-                BuildRequestStatus::try_from(b.status_code).unwrap_or(BuildRequestStatus::BuildRequestUnknown),
-                BuildRequestStatus::BuildRequestReceived |
-                BuildRequestStatus::BuildRequestPlanning |
-                BuildRequestStatus::BuildRequestExecuting |
-                BuildRequestStatus::BuildRequestAnalysisCompleted
-            ))
-            .count() as u32;
-        
-        let recent_builds = builds_response.builds.into_iter()
-            .map(|b| BuildRequestSummary {
-                build_request_id: b.build_request_id,
-                status: BuildRequestStatus::try_from(b.status_code).unwrap_or(BuildRequestStatus::BuildRequestUnknown),
-                requested_partitions: b.requested_partitions.into_iter().map(|p| p.str).collect(),
-                created_at: b.requested_at,
-                updated_at: b.completed_at.unwrap_or(b.requested_at),
-            })
-            .collect();
-        
-        // For partitions, we'd need a separate implementation
-        let recent_partitions = Vec::new(); // TODO: Implement partition listing
-        
-        Ok(ActivitySummary {
-            active_builds_count,
-            recent_builds,
-            recent_partitions,
-            total_partitions_count: 0, // TODO: Implement
-        })
-    }
-    
-    /// Helper to aggregate partition status from events
-    fn aggregate_partition_status(&self, events: &[BuildEvent]) -> Result<Option<(PartitionStatus, i64)>> {
-        let mut latest_status = None;
-        let mut latest_timestamp = 0i64;
-        
-        // Look for the most recent partition event for this partition
-        for event in events {
-            if let Some(crate::build_event::EventType::PartitionEvent(p_event)) = &event.event_type {
-                if event.timestamp >= latest_timestamp {
-                    if let Ok(status) = PartitionStatus::try_from(p_event.status_code) {
-                        latest_status = Some(status);
-                        latest_timestamp = event.timestamp;
-                    }
-                }
-            }
-        }
-        
-        Ok(latest_status.map(|status| (status, latest_timestamp)))
-    }
-    
-    /// Get build request ID that created an available partition
-    pub async fn get_build_request_for_available_partition(&self, partition_ref: &str) -> Result<Option<String>> {
-        let filter = EventFilter {
-            partition_refs: vec![partition_ref.to_string()],
-            partition_patterns: vec![],
-            job_labels: vec![],
-            job_run_ids: vec![],
-            build_request_ids: vec![],
-        };
-        
-        let events = self.storage.list_events(0, filter).await?;
-        
-        // Find the most recent PARTITION_AVAILABLE event
-        let mut latest_available_build_id = None;
-        let mut latest_timestamp = 0i64;
-        
-        for event in events.events {
-            if let Some(crate::build_event::EventType::PartitionEvent(p_event)) = &event.event_type {
-                if let Some(partition_event_ref) = &p_event.partition_ref {
-                    if partition_event_ref.str == partition_ref {
-                        if let Ok(status) = PartitionStatus::try_from(p_event.status_code) {
-                            if status == PartitionStatus::PartitionAvailable && event.timestamp >= latest_timestamp {
-                                latest_available_build_id = Some(event.build_request_id.clone());
-                                latest_timestamp = event.timestamp;
-                            }
-                        }
-                    }
-                }
-            }
-        }
-        
-        Ok(latest_available_build_id)
-    }
-    
-    /// Append an event to storage
-    pub async fn append_event(&self, event: BuildEvent) -> Result<i64> {
-        self.storage.append_event(event).await
-    }
-    
-    /// Get all events for a specific partition
-    pub async fn get_partition_events(&self, partition_ref: &str, _limit: Option<u32>) -> Result<Vec<BuildEvent>> {
-        let filter = EventFilter {
-            partition_refs: vec![partition_ref.to_string()],
-            partition_patterns: vec![],
-            job_labels: vec![],
-            job_run_ids: vec![],
-            build_request_ids: vec![],
-        };
-        
-        let events = self.storage.list_events(0, filter).await?;
-        Ok(events.events)
-    }
-    
-    /// Execute a raw SQL query (for backwards compatibility)
-    pub async fn execute_query(&self, _query: &str) -> Result<QueryResult> {
-        // TODO: Implement SQL query execution if needed
-        // For now, return empty result to avoid compilation errors
-        Ok(QueryResult {
-            columns: vec![],
-            rows: vec![],
-        })
-    }
-
-    /// Get all events in a timestamp range
-    pub async fn get_events_in_range(&self, _start: i64, _end: i64) -> Result<Vec<BuildEvent>> {
-        // TODO: Implement range filtering
-        // For now, get all events
-        let filter = EventFilter {
-            partition_refs: vec![],
-            partition_patterns: vec![],
-            job_labels: vec![],
-            job_run_ids: vec![],
-            build_request_ids: vec![],
-        };
-        
-        let events = self.storage.list_events(0, filter).await?;
-        Ok(events.events)
-    }
-
-    /// Get all events for a specific job run
-    pub async fn get_job_run_events(&self, job_run_id: &str) -> Result<Vec<BuildEvent>> {
-        let filter = EventFilter {
-            partition_refs: vec![],
-            partition_patterns: vec![],
-            job_labels: vec![],
-            job_run_ids: vec![job_run_id.to_string()],
-            build_request_ids: vec![],
-        };
-        
-        let events = self.storage.list_events(0, filter).await?;
-        Ok(events.events)
-    }
-
-    /// Get all events for a specific build request
-    pub async fn get_build_request_events(&self, build_request_id: &str, _limit: Option<u32>) -> Result<Vec<BuildEvent>> {
-        let filter = EventFilter {
-            partition_refs: vec![],
-            partition_patterns: vec![],
-            job_labels: vec![],
-            job_run_ids: vec![],
-            build_request_ids: vec![build_request_id.to_string()],
-        };
-        
-        let events = self.storage.list_events(0, filter).await?;
-        Ok(events.events)
-    }
-}
-
-

databuild/event_log/sqlite.rs

@@ -0,0 +1,961 @@
+use super::*;
+use async_trait::async_trait;
+use rusqlite::{params, Connection, Row};
+use serde_json;
+use std::sync::{Arc, Mutex};
+
+// Helper functions to convert integer values back to enum values
+fn int_to_build_request_status(i: i32) -> BuildRequestStatus {
+    match i {
+        0 => BuildRequestStatus::BuildRequestUnknown,
+        1 => BuildRequestStatus::BuildRequestReceived,
+        2 => BuildRequestStatus::BuildRequestPlanning,
+        3 => BuildRequestStatus::BuildRequestExecuting,
+        4 => BuildRequestStatus::BuildRequestCompleted,
+        5 => BuildRequestStatus::BuildRequestFailed,
+        6 => BuildRequestStatus::BuildRequestCancelled,
+        _ => BuildRequestStatus::BuildRequestUnknown,
+    }
+}
+
+fn int_to_partition_status(i: i32) -> PartitionStatus {
+    match i {
+        0 => PartitionStatus::PartitionUnknown,
+        1 => PartitionStatus::PartitionRequested,
+        2 => PartitionStatus::PartitionAnalyzed,
+        3 => PartitionStatus::PartitionBuilding,
+        4 => PartitionStatus::PartitionAvailable,
+        5 => PartitionStatus::PartitionFailed,
+        6 => PartitionStatus::PartitionDelegated,
+        _ => PartitionStatus::PartitionUnknown,
+    }
+}
+
+pub struct SqliteBuildEventLog {
+    connection: Arc<Mutex<Connection>>,
+}
+
+impl SqliteBuildEventLog {
+    pub async fn new(path: &str) -> Result<Self> {
+        // Create parent directory if it doesn't exist
+        if let Some(parent) = std::path::Path::new(path).parent() {
+            std::fs::create_dir_all(parent)
+                .map_err(|e| BuildEventLogError::ConnectionError(
+                    format!("Failed to create directory {}: {}", parent.display(), e)
+                ))?;
+        }
+        
+        let conn = Connection::open(path)
+            .map_err(|e| BuildEventLogError::ConnectionError(e.to_string()))?;
+        
+        Ok(Self {
+            connection: Arc::new(Mutex::new(conn)),
+        })
+    }
+
+    // Proper event reconstruction from joined query results
+    fn row_to_build_event_from_join(row: &Row) -> rusqlite::Result<BuildEvent> {
+        let event_id: String = row.get(0)?;
+        let timestamp: i64 = row.get(1)?;
+        let build_request_id: String = row.get(2)?;
+        let event_type_name: String = row.get(3)?;
+        
+        // Read the actual event data from the joined columns
+        let event_type = match event_type_name.as_str() {
+            "build_request" => {
+                // Read from build_request_events columns (indices 4, 5, 6)
+                let status_str: String = row.get(4)?;
+                let requested_partitions_json: String = row.get(5)?;
+                let message: String = row.get(6)?;
+                
+                let status = status_str.parse::<i32>().unwrap_or(0);
+                let requested_partitions: Vec<PartitionRef> = serde_json::from_str(&requested_partitions_json)
+                    .unwrap_or_default();
+                
+                Some(crate::build_event::EventType::BuildRequestEvent(BuildRequestEvent {
+                    status_code: status,
+                    status_name: match status {
+                        1 => BuildRequestStatus::BuildRequestReceived.to_display_string(),
+                        2 => BuildRequestStatus::BuildRequestPlanning.to_display_string(),
+                        3 => BuildRequestStatus::BuildRequestExecuting.to_display_string(),
+                        4 => BuildRequestStatus::BuildRequestCompleted.to_display_string(),
+                        5 => BuildRequestStatus::BuildRequestFailed.to_display_string(),
+                        6 => BuildRequestStatus::BuildRequestCancelled.to_display_string(),
+                        7 => BuildRequestStatus::BuildRequestAnalysisCompleted.to_display_string(),
+                        _ => BuildRequestStatus::BuildRequestUnknown.to_display_string(),
+                    },
+                    requested_partitions,
+                    message,
+                }))
+            }
+            "partition" => {
+                // Read from partition_events columns (indices 4, 5, 6, 7)
+                let partition_ref: String = row.get(4)?;
+                let status_str: String = row.get(5)?;
+                let message: String = row.get(6)?;
+                let job_run_id: String = row.get(7).unwrap_or_default();
+                
+                let status = status_str.parse::<i32>().unwrap_or(0);
+                
+                Some(crate::build_event::EventType::PartitionEvent(PartitionEvent {
+                    partition_ref: Some(PartitionRef { str: partition_ref }),
+                    status_code: status,
+                    status_name: match status {
+                        1 => PartitionStatus::PartitionRequested.to_display_string(),
+                        2 => PartitionStatus::PartitionAnalyzed.to_display_string(),
+                        3 => PartitionStatus::PartitionBuilding.to_display_string(),
+                        4 => PartitionStatus::PartitionAvailable.to_display_string(),
+                        5 => PartitionStatus::PartitionFailed.to_display_string(),
+                        6 => PartitionStatus::PartitionDelegated.to_display_string(),
+                        _ => PartitionStatus::PartitionUnknown.to_display_string(),
+                    },
+                    message,
+                    job_run_id,
+                }))
+            }
+            "job" => {
+                // Read from job_events columns (indices 4-10)
+                let job_run_id: String = row.get(4)?;
+                let job_label: String = row.get(5)?;
+                let target_partitions_json: String = row.get(6)?;
+                let status_str: String = row.get(7)?;
+                let message: String = row.get(8)?;
+                let config_json: Option<String> = row.get(9).ok();
+                let manifests_json: String = row.get(10)?;
+                
+                let status = status_str.parse::<i32>().unwrap_or(0);
+                let target_partitions: Vec<PartitionRef> = serde_json::from_str(&target_partitions_json)
+                    .unwrap_or_default();
+                let config: Option<JobConfig> = config_json
+                    .and_then(|json| serde_json::from_str(&json).ok());
+                let manifests: Vec<PartitionManifest> = serde_json::from_str(&manifests_json)
+                    .unwrap_or_default();
+                
+                Some(crate::build_event::EventType::JobEvent(JobEvent {
+                    job_run_id,
+                    job_label: Some(JobLabel { label: job_label }),
+                    target_partitions,
+                    status_code: status,
+                    status_name: match status {
+                        1 => JobStatus::JobScheduled.to_display_string(),
+                        2 => JobStatus::JobRunning.to_display_string(),
+                        3 => JobStatus::JobCompleted.to_display_string(),
+                        4 => JobStatus::JobFailed.to_display_string(),
+                        5 => JobStatus::JobCancelled.to_display_string(),
+                        6 => JobStatus::JobSkipped.to_display_string(),
+                        _ => JobStatus::JobUnknown.to_display_string(),
+                    },
+                    message,
+                    config,
+                    manifests,
+                }))
+            }
+            "delegation" => {
+                // Read from delegation_events columns (indices 4, 5, 6)
+                let partition_ref: String = row.get(4)?;
+                let delegated_to_build_request_id: String = row.get(5)?;
+                let message: String = row.get(6)?;
+                
+                Some(crate::build_event::EventType::DelegationEvent(DelegationEvent {
+                    partition_ref: Some(PartitionRef { str: partition_ref }),
+                    delegated_to_build_request_id,
+                    message,
+                }))
+            }
+            "job_graph" => {
+                // Read from job_graph_events columns (indices 4, 5)
+                let job_graph_json: String = row.get(4)?;
+                let message: String = row.get(5)?;
+                
+                let job_graph: Option<JobGraph> = serde_json::from_str(&job_graph_json).ok();
+                
+                Some(crate::build_event::EventType::JobGraphEvent(JobGraphEvent {
+                    job_graph,
+                    message,
+                }))
+            }
+            _ => None,
+        };
+
+        Ok(BuildEvent {
+            event_id,
+            timestamp,
+            build_request_id,
+            event_type,
+        })
+    }
+}
+
+#[async_trait]
+impl BuildEventLog for SqliteBuildEventLog {
+    async fn append_event(&self, event: BuildEvent) -> Result<()> {
+        let conn = self.connection.lock().unwrap();
+        
+        // First insert into build_events table
+        conn.execute(
+            "INSERT INTO build_events (event_id, timestamp, build_request_id, event_type) VALUES (?1, ?2, ?3, ?4)",
+            params![
+                event.event_id,
+                event.timestamp,
+                event.build_request_id,
+                match &event.event_type {
+                    Some(crate::build_event::EventType::BuildRequestEvent(_)) => "build_request",
+                    Some(crate::build_event::EventType::PartitionEvent(_)) => "partition",
+                    Some(crate::build_event::EventType::JobEvent(_)) => "job",
+                    Some(crate::build_event::EventType::DelegationEvent(_)) => "delegation",
+                    Some(crate::build_event::EventType::JobGraphEvent(_)) => "job_graph",
+                    Some(crate::build_event::EventType::PartitionInvalidationEvent(_)) => "partition_invalidation",
+                    Some(crate::build_event::EventType::TaskCancelEvent(_)) => "task_cancel",
+                    Some(crate::build_event::EventType::BuildCancelEvent(_)) => "build_cancel",
+                    None => "unknown",
+                }
+            ],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+
+        // Insert into specific event type table
+        match &event.event_type {
+            Some(crate::build_event::EventType::BuildRequestEvent(br_event)) => {
+                let partitions_json = serde_json::to_string(&br_event.requested_partitions)
+                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+                
+                conn.execute(
+                    "INSERT INTO build_request_events (event_id, status, requested_partitions, message) VALUES (?1, ?2, ?3, ?4)",
+                    params![
+                        event.event_id,
+                        br_event.status_code.to_string(),
+                        partitions_json,
+                        br_event.message
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::PartitionEvent(p_event)) => {
+                conn.execute(
+                    "INSERT INTO partition_events (event_id, partition_ref, status, message, job_run_id) VALUES (?1, ?2, ?3, ?4, ?5)",
+                    params![
+                        event.event_id,
+                        p_event.partition_ref.as_ref().map(|r| &r.str).unwrap_or(&String::new()),
+                        p_event.status_code.to_string(),
+                        p_event.message,
+                        if p_event.job_run_id.is_empty() { None } else { Some(&p_event.job_run_id) }
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::JobEvent(j_event)) => {
+                let partitions_json = serde_json::to_string(&j_event.target_partitions)
+                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+                let config_json = j_event.config.as_ref()
+                    .map(|c| serde_json::to_string(c))
+                    .transpose()
+                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+                let manifests_json = serde_json::to_string(&j_event.manifests)
+                    .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+                
+                conn.execute(
+                    "INSERT INTO job_events (event_id, job_run_id, job_label, target_partitions, status, message, config_json, manifests_json) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)",
+                    params![
+                        event.event_id,
+                        j_event.job_run_id,
+                        j_event.job_label.as_ref().map(|l| &l.label).unwrap_or(&String::new()),
+                        partitions_json,
+                        j_event.status_code.to_string(),
+                        j_event.message,
+                        config_json,
+                        manifests_json
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::DelegationEvent(d_event)) => {
+                conn.execute(
+                    "INSERT INTO delegation_events (event_id, partition_ref, delegated_to_build_request_id, message) VALUES (?1, ?2, ?3, ?4)",
+                    params![
+                        event.event_id,
+                        d_event.partition_ref.as_ref().map(|r| &r.str).unwrap_or(&String::new()),
+                        d_event.delegated_to_build_request_id,
+                        d_event.message
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::JobGraphEvent(jg_event)) => {
+                let job_graph_json = match serde_json::to_string(&jg_event.job_graph) {
+                    Ok(json) => json,
+                    Err(e) => {
+                        return Err(BuildEventLogError::DatabaseError(format!("Failed to serialize job graph: {}", e)));
+                    }
+                };
+                conn.execute(
+                    "INSERT INTO job_graph_events (event_id, job_graph_json, message) VALUES (?1, ?2, ?3)",
+                    params![
+                        event.event_id,
+                        job_graph_json,
+                        jg_event.message
+                    ],
+                ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+            }
+            Some(crate::build_event::EventType::PartitionInvalidationEvent(_pi_event)) => {
+                // For now, we'll just store these in the main events table
+                // In a later phase, we could add a specific table for invalidation events
+            }
+            Some(crate::build_event::EventType::TaskCancelEvent(_tc_event)) => {
+                // For now, we'll just store these in the main events table
+                // In a later phase, we could add a specific table for task cancel events
+            }
+            Some(crate::build_event::EventType::BuildCancelEvent(_bc_event)) => {
+                // For now, we'll just store these in the main events table
+                // In a later phase, we could add a specific table for build cancel events
+            }
+            None => {}
+        }
+        
+        Ok(())
+    }
+    
+    async fn get_build_request_events(
+        &self, 
+        build_request_id: &str,
+        since: Option<i64>
+    ) -> Result<Vec<BuildEvent>> {
+        let conn = self.connection.lock().unwrap();
+        
+        // Use a UNION query to get all event types with their specific data
+        let base_query = "
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   bre.status, bre.requested_partitions, bre.message, NULL, NULL, NULL, NULL
+            FROM build_events be 
+            LEFT JOIN build_request_events bre ON be.event_id = bre.event_id
+            WHERE be.build_request_id = ? AND be.event_type = 'build_request'
+            UNION ALL
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   pe.partition_ref, pe.status, pe.message, pe.job_run_id, NULL, NULL, NULL
+            FROM build_events be 
+            LEFT JOIN partition_events pe ON be.event_id = pe.event_id
+            WHERE be.build_request_id = ? AND be.event_type = 'partition'
+            UNION ALL
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   je.job_run_id, je.job_label, je.target_partitions, je.status, je.message, je.config_json, je.manifests_json
+            FROM build_events be 
+            LEFT JOIN job_events je ON be.event_id = je.event_id
+            WHERE be.build_request_id = ? AND be.event_type = 'job'
+            UNION ALL
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   de.partition_ref, de.delegated_to_build_request_id, de.message, NULL, NULL, NULL, NULL
+            FROM build_events be 
+            LEFT JOIN delegation_events de ON be.event_id = de.event_id
+            WHERE be.build_request_id = ? AND be.event_type = 'delegation'
+            UNION ALL
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   jge.job_graph_json, jge.message, NULL, NULL, NULL, NULL, NULL
+            FROM build_events be 
+            LEFT JOIN job_graph_events jge ON be.event_id = jge.event_id
+            WHERE be.build_request_id = ? AND be.event_type = 'job_graph'
+        ";
+        
+        let query = if since.is_some() {
+            format!("{} AND be.timestamp > ? ORDER BY be.timestamp", base_query)
+        } else {
+            format!("{} ORDER BY be.timestamp", base_query)
+        };
+        
+        let mut stmt = conn.prepare(&query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let rows = if let Some(since_timestamp) = since {
+            // We need 6 parameters: build_request_id for each UNION + since_timestamp
+            stmt.query_map(params![build_request_id, build_request_id, build_request_id, build_request_id, build_request_id, since_timestamp], Self::row_to_build_event_from_join)
+        } else {
+            // We need 5 parameters: build_request_id for each UNION
+            stmt.query_map(params![build_request_id, build_request_id, build_request_id, build_request_id, build_request_id], Self::row_to_build_event_from_join)
+        }.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut events = Vec::new();
+        for row in rows {
+            events.push(row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?);
+        }
+        
+        Ok(events)
+    }
+    
+    async fn get_partition_events(
+        &self,
+        partition_ref: &str,
+        since: Option<i64>
+    ) -> Result<Vec<BuildEvent>> {
+        // First get the build request IDs (release the connection lock quickly)
+        let build_ids: Vec<String> = {
+            let conn = self.connection.lock().unwrap();
+            
+            // Get all events for builds that included this partition
+            // First find all build request IDs that have events for this partition
+            let build_ids_query = if since.is_some() {
+                "SELECT DISTINCT be.build_request_id 
+                 FROM build_events be 
+                 JOIN partition_events pe ON be.event_id = pe.event_id 
+                 WHERE pe.partition_ref = ? AND be.timestamp > ?"
+            } else {
+                "SELECT DISTINCT be.build_request_id 
+                 FROM build_events be 
+                 JOIN partition_events pe ON be.event_id = pe.event_id 
+                 WHERE pe.partition_ref = ?"
+            };
+            
+            let mut stmt = conn.prepare(build_ids_query)
+                .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            
+            let row_mapper = |row: &Row| -> rusqlite::Result<String> {
+                Ok(row.get::<_, String>(0)?)
+            };
+            
+            let build_ids_result: Vec<String> = if let Some(since_timestamp) = since {
+                stmt.query_map(params![partition_ref, since_timestamp], row_mapper)
+            } else {
+                stmt.query_map(params![partition_ref], row_mapper)
+            }.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?
+            .collect::<std::result::Result<Vec<_>, _>>()
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            
+            build_ids_result
+        }; // Connection lock is released here
+        
+        // Now get all events for those build requests (this gives us complete event reconstruction)
+        let mut all_events = Vec::new();
+        for build_id in build_ids {
+            let events = self.get_build_request_events(&build_id, since).await?;
+            all_events.extend(events);
+        }
+        
+        // Sort events by timestamp
+        all_events.sort_by_key(|e| e.timestamp);
+        
+        Ok(all_events)
+    }
+    
+    async fn get_job_run_events(
+        &self,
+        _job_run_id: &str
+    ) -> Result<Vec<BuildEvent>> {
+        // This method is not implemented because it would require complex joins
+        // to reconstruct complete event data. Use get_build_request_events instead
+        // which properly reconstructs all event types for a build request.
+        Err(BuildEventLogError::QueryError(
+            "get_job_run_events is not implemented - use get_build_request_events to get complete event data".to_string()
+        ))
+    }
+    
+    async fn get_events_in_range(
+        &self,
+        start_time: i64,
+        end_time: i64
+    ) -> Result<Vec<BuildEvent>> {
+        let conn = self.connection.lock().unwrap();
+        
+        // Use a UNION query to get all event types with their specific data in the time range
+        let query = "
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   bre.status, bre.requested_partitions, bre.message, NULL, NULL, NULL, NULL
+            FROM build_events be 
+            LEFT JOIN build_request_events bre ON be.event_id = bre.event_id
+            WHERE be.timestamp >= ?1 AND be.timestamp <= ?2 AND be.event_type = 'build_request'
+            UNION ALL
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   pe.partition_ref, pe.status, pe.message, pe.job_run_id, NULL, NULL, NULL
+            FROM build_events be 
+            LEFT JOIN partition_events pe ON be.event_id = pe.event_id
+            WHERE be.timestamp >= ?3 AND be.timestamp <= ?4 AND be.event_type = 'partition'
+            UNION ALL
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   je.job_run_id, je.job_label, je.target_partitions, je.status, je.message, je.config_json, je.manifests_json
+            FROM build_events be 
+            LEFT JOIN job_events je ON be.event_id = je.event_id
+            WHERE be.timestamp >= ?5 AND be.timestamp <= ?6 AND be.event_type = 'job'
+            UNION ALL
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   de.partition_ref, de.delegated_to_build_request_id, de.message, NULL, NULL, NULL, NULL
+            FROM build_events be 
+            LEFT JOIN delegation_events de ON be.event_id = de.event_id
+            WHERE be.timestamp >= ?7 AND be.timestamp <= ?8 AND be.event_type = 'delegation'
+            UNION ALL
+            SELECT be.event_id, be.timestamp, be.build_request_id, be.event_type,
+                   jge.job_graph_json, jge.message, NULL, NULL, NULL, NULL, NULL
+            FROM build_events be 
+            LEFT JOIN job_graph_events jge ON be.event_id = jge.event_id
+            WHERE be.timestamp >= ?9 AND be.timestamp <= ?10 AND be.event_type = 'job_graph'
+            ORDER BY timestamp ASC
+        ";
+        
+        let mut stmt = conn.prepare(query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        // We need 10 parameters: start_time and end_time for each of the 5 UNION queries
+        let rows = stmt.query_map(
+            params![start_time, end_time, start_time, end_time, start_time, end_time, start_time, end_time, start_time, end_time], 
+            Self::row_to_build_event_from_join
+        ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut events = Vec::new();
+        for row in rows {
+            events.push(row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?);
+        }
+        
+        Ok(events)
+    }
+    
+    async fn execute_query(&self, query: &str) -> Result<QueryResult> {
+        let conn = self.connection.lock().unwrap();
+        
+        let mut stmt = conn.prepare(query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let column_count = stmt.column_count();
+        let columns: Vec<String> = (0..column_count)
+            .map(|i| stmt.column_name(i).unwrap_or("unknown").to_string())
+            .collect();
+        
+        let rows = stmt.query_map([], |row| {
+            let mut row_data = Vec::new();
+            for i in 0..column_count {
+                // Try to get as different types and convert to string
+                let value: String = if let Ok(int_val) = row.get::<_, i64>(i) {
+                    int_val.to_string()
+                } else if let Ok(float_val) = row.get::<_, f64>(i) {
+                    float_val.to_string()
+                } else if let Ok(str_val) = row.get::<_, String>(i) {
+                    str_val
+                } else if let Ok(str_val) = row.get::<_, Option<String>>(i) {
+                    str_val.unwrap_or_default()
+                } else {
+                    String::new()
+                };
+                row_data.push(value);
+            }
+            Ok(row_data)
+        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut result_rows = Vec::new();
+        for row in rows {
+            result_rows.push(row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?);
+        }
+        
+        Ok(QueryResult {
+            columns,
+            rows: result_rows,
+        })
+    }
+    async fn get_latest_partition_status(
+        &self,
+        partition_ref: &str
+    ) -> Result<Option<(PartitionStatus, i64)>> {
+        match self.get_meaningful_partition_status(partition_ref).await? {
+            Some((status, timestamp, _build_request_id)) => Ok(Some((status, timestamp))),
+            None => Ok(None),
+        }
+    }
+    
+    async fn get_active_builds_for_partition(
+        &self,
+        partition_ref: &str
+    ) -> Result<Vec<String>> {
+        let conn = self.connection.lock().unwrap();
+        
+        // Look for build requests that are actively building this partition
+        // A build is considered active if:
+        // 1. It has scheduled/building events for this partition, AND
+        // 2. The build request itself has not completed (status 4=COMPLETED or 5=FAILED)
+        let query = "SELECT DISTINCT be.build_request_id 
+                     FROM partition_events pe 
+                     JOIN build_events be ON pe.event_id = be.event_id 
+                     WHERE pe.partition_ref = ?1 
+                     AND pe.status IN ('2', '3')  -- PARTITION_ANALYZED or PARTITION_BUILDING
+                     AND be.build_request_id NOT IN (
+                         SELECT DISTINCT be3.build_request_id
+                         FROM build_request_events bre
+                         JOIN build_events be3 ON bre.event_id = be3.event_id
+                         WHERE bre.status IN ('4', '5')  -- BUILD_REQUEST_COMPLETED or BUILD_REQUEST_FAILED
+                     )";
+        
+        let mut stmt = conn.prepare(query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let rows = stmt.query_map([partition_ref], |row| {
+            let build_request_id: String = row.get(0)?;
+            Ok(build_request_id)
+        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut build_request_ids = Vec::new();
+        for row in rows {
+            build_request_ids.push(row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?);
+        }
+        
+        Ok(build_request_ids)
+    }
+    
+    async fn list_build_requests(
+        &self,
+        limit: u32,
+        offset: u32,
+        status_filter: Option<BuildRequestStatus>,
+    ) -> Result<(Vec<BuildRequestSummary>, u32)> {
+        let conn = self.connection.lock().unwrap();
+        
+        // Build query based on status filter
+        let (where_clause, count_where_clause) = match status_filter {
+            Some(_) => (" WHERE bre.status = ?1", " WHERE bre.status = ?1"),
+            None => ("", ""),
+        };
+        
+        let query = format!(
+            "SELECT DISTINCT be.build_request_id, bre.status, bre.requested_partitions, 
+                     MIN(be.timestamp) as created_at, MAX(be.timestamp) as updated_at
+             FROM build_events be 
+             JOIN build_request_events bre ON be.event_id = bre.event_id{}
+             GROUP BY be.build_request_id 
+             ORDER BY created_at DESC 
+             LIMIT {} OFFSET {}",
+            where_clause, limit, offset
+        );
+        
+        let count_query = format!(
+            "SELECT COUNT(DISTINCT be.build_request_id) 
+             FROM build_events be 
+             JOIN build_request_events bre ON be.event_id = bre.event_id{}",
+            count_where_clause
+        );
+        
+        // Execute count query first
+        let total_count: u32 = if let Some(status) = status_filter {
+            let status_str = format!("{:?}", status);
+            conn.query_row(&count_query, params![status_str], |row| row.get(0))
+                .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?
+        } else {
+            conn.query_row(&count_query, [], |row| row.get(0))
+                .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?
+        };
+        
+        // Execute main query
+        let mut stmt = conn.prepare(&query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let build_row_mapper = |row: &Row| -> rusqlite::Result<BuildRequestSummary> {
+            let status_str: String = row.get(1)?;
+            let status = status_str.parse::<i32>()
+                .map(int_to_build_request_status)
+                .unwrap_or(BuildRequestStatus::BuildRequestUnknown);
+            
+            Ok(BuildRequestSummary {
+                build_request_id: row.get(0)?,
+                status,
+                requested_partitions: serde_json::from_str(&row.get::<_, String>(2)?).unwrap_or_default(),
+                created_at: row.get(3)?,
+                updated_at: row.get(4)?,
+            })
+        };
+        
+        let rows = if let Some(status) = status_filter {
+            let status_str = format!("{:?}", status);
+            stmt.query_map(params![status_str], build_row_mapper)
+        } else {
+            stmt.query_map([], build_row_mapper)
+        }.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let mut summaries = Vec::new();
+        for row in rows {
+            summaries.push(row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?);
+        }
+        
+        Ok((summaries, total_count))
+    }
+
+    async fn list_recent_partitions(
+        &self,
+        limit: u32, 
+        offset: u32,
+        status_filter: Option<PartitionStatus>,
+    ) -> Result<(Vec<PartitionSummary>, u32)> {
+        // Get all unique partition refs first, ordered by most recent activity
+        let (total_count, partition_refs) = {
+            let conn = self.connection.lock().unwrap();
+            
+            let count_query = "SELECT COUNT(DISTINCT pe.partition_ref) 
+                              FROM partition_events pe";
+            let total_count: u32 = conn.query_row(count_query, [], |row| row.get(0))
+                .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            
+            let refs_query = "SELECT DISTINCT pe.partition_ref 
+                             FROM partition_events pe 
+                             JOIN build_events be ON pe.event_id = be.event_id 
+                             GROUP BY pe.partition_ref
+                             ORDER BY MAX(be.timestamp) DESC 
+                             LIMIT ? OFFSET ?";
+            
+            let mut stmt = conn.prepare(refs_query)
+                .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            
+            let rows = stmt.query_map([limit, offset], |row| {
+                let partition_ref: String = row.get(0)?;
+                Ok(partition_ref)
+            }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            
+            let mut partition_refs = Vec::new();
+            for row in rows {
+                partition_refs.push(row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?);
+            }
+            
+            (total_count, partition_refs)
+        };
+        
+        // Get meaningful status for each partition using shared helper
+        let mut summaries = Vec::new();
+        for partition_ref in partition_refs {
+            if let Some((status, updated_at, build_request_id)) = self.get_meaningful_partition_status(&partition_ref).await? {
+                // Apply status filter if specified
+                if let Some(filter_status) = status_filter {
+                    if status != filter_status {
+                        continue;
+                    }
+                }
+                
+                summaries.push(PartitionSummary {
+                    partition_ref,
+                    status,
+                    updated_at,
+                    build_request_id: Some(build_request_id),
+                });
+            }
+        }
+        
+        // Sort by updated_at descending (most recent first)
+        summaries.sort_by(|a, b| b.updated_at.cmp(&a.updated_at));
+        
+        Ok((summaries, total_count))
+    }
+
+    async fn get_activity_summary(&self) -> Result<ActivitySummary> {
+        // First get the simple counts without holding the lock across awaits
+        let (active_builds_count, total_partitions_count) = {
+            let conn = self.connection.lock().unwrap();
+            
+            // Get active builds count (builds that are not completed, failed, or cancelled)
+            let active_builds_count: u32 = conn.query_row(
+                "SELECT COUNT(DISTINCT be.build_request_id) 
+                 FROM build_events be 
+                 JOIN build_request_events bre ON be.event_id = bre.event_id 
+                 WHERE bre.status IN ('BuildRequestReceived', 'BuildRequestPlanning', 'BuildRequestExecuting')",
+                [],
+                |row| row.get(0)
+            ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            
+            // Get total partitions count
+            let total_partitions_count: u32 = conn.query_row(
+                "SELECT COUNT(DISTINCT pe.partition_ref) 
+                 FROM partition_events pe 
+                 JOIN build_events be ON pe.event_id = be.event_id",
+                [],
+                |row| row.get(0)
+            ).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+            
+            (active_builds_count, total_partitions_count)
+        };
+        
+        // Get recent builds (limit to 5 for summary)
+        let (recent_builds, _) = self.list_build_requests(5, 0, None).await?;
+        
+        // Get recent partitions (limit to 5 for summary)  
+        let (recent_partitions, _) = self.list_recent_partitions(5, 0, None).await?;
+        
+        Ok(ActivitySummary {
+            active_builds_count,
+            recent_builds,
+            recent_partitions,
+            total_partitions_count,
+        })
+    }
+    
+    async fn initialize(&self) -> Result<()> {
+        let conn = self.connection.lock().unwrap();
+        
+        // Create tables
+        conn.execute(
+            "CREATE TABLE IF NOT EXISTS build_events (
+                event_id TEXT PRIMARY KEY,
+                timestamp INTEGER NOT NULL,
+                build_request_id TEXT NOT NULL,
+                event_type TEXT NOT NULL
+            )",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE TABLE IF NOT EXISTS build_request_events (
+                event_id TEXT PRIMARY KEY REFERENCES build_events(event_id),
+                status TEXT NOT NULL,
+                requested_partitions TEXT NOT NULL,
+                message TEXT
+            )",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE TABLE IF NOT EXISTS partition_events (
+                event_id TEXT PRIMARY KEY REFERENCES build_events(event_id),
+                partition_ref TEXT NOT NULL,
+                status TEXT NOT NULL,
+                message TEXT,
+                job_run_id TEXT
+            )",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE TABLE IF NOT EXISTS job_events (
+                event_id TEXT PRIMARY KEY REFERENCES build_events(event_id),
+                job_run_id TEXT NOT NULL,
+                job_label TEXT NOT NULL,
+                target_partitions TEXT NOT NULL,
+                status TEXT NOT NULL,
+                message TEXT,
+                config_json TEXT,
+                manifests_json TEXT
+            )",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE TABLE IF NOT EXISTS delegation_events (
+                event_id TEXT PRIMARY KEY REFERENCES build_events(event_id),
+                partition_ref TEXT NOT NULL,
+                delegated_to_build_request_id TEXT NOT NULL,
+                message TEXT
+            )",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE TABLE IF NOT EXISTS job_graph_events (
+                event_id TEXT PRIMARY KEY REFERENCES build_events(event_id),
+                job_graph_json TEXT NOT NULL,
+                message TEXT
+            )",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        // Create indexes
+        conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_build_events_build_request ON build_events(build_request_id, timestamp)",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_build_events_timestamp ON build_events(timestamp)",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_partition_events_partition ON partition_events(partition_ref)",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_job_events_job_run ON job_events(job_run_id)",
+            [],
+        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
+        
+        Ok(())
+    }
+    
+    async fn get_build_request_for_available_partition(
+        &self,
+        partition_ref: &str
+    ) -> Result<Option<String>> {
+        let conn = self.connection.lock().unwrap();
+        
+        // Find the most recent PARTITION_AVAILABLE event for this partition
+        let query = "SELECT be.build_request_id 
+                     FROM partition_events pe 
+                     JOIN build_events be ON pe.event_id = be.event_id 
+                     WHERE pe.partition_ref = ?1 AND pe.status = '4'
+                     ORDER BY be.timestamp DESC 
+                     LIMIT 1";
+        
+        let result = conn.query_row(query, [partition_ref], |row| {
+            let build_request_id: String = row.get(0)?;
+            Ok(build_request_id)
+        });
+        
+        match result {
+            Ok(build_request_id) => Ok(Some(build_request_id)),
+            Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
+            Err(e) => Err(BuildEventLogError::QueryError(e.to_string())),
+        }
+    }
+}
+
+impl SqliteBuildEventLog {
+    // Shared helper method to get the meaningful partition status for build coordination and display
+    // This implements the "delegation-friendly" logic: if a partition was ever available, it remains available
+    async fn get_meaningful_partition_status(
+        &self,
+        partition_ref: &str
+    ) -> Result<Option<(PartitionStatus, i64, String)>> { // (status, timestamp, build_request_id)
+        let conn = self.connection.lock().unwrap();
+        
+        // Check for ANY historical completion first - this is resilient to later events being added
+        let available_query = "SELECT pe.status, be.timestamp, be.build_request_id
+                              FROM partition_events pe 
+                              JOIN build_events be ON pe.event_id = be.event_id 
+                              WHERE pe.partition_ref = ?1 AND pe.status = '4'
+                              ORDER BY be.timestamp DESC 
+                              LIMIT 1";
+        
+        let mut available_stmt = conn.prepare(available_query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let available_result = available_stmt.query_row([partition_ref], |row| {
+            let status_str: String = row.get(0)?;
+            let timestamp: i64 = row.get(1)?;
+            let build_request_id: String = row.get(2)?;
+            let status = status_str.parse::<i32>()
+                .map_err(|_e| rusqlite::Error::InvalidColumnType(0, status_str.clone(), rusqlite::types::Type::Integer))?;
+            Ok((status, timestamp, build_request_id))
+        });
+        
+        match available_result {
+            Ok((status, timestamp, build_request_id)) => {
+                let partition_status = PartitionStatus::try_from(status)
+                    .map_err(|_| BuildEventLogError::QueryError(format!("Invalid partition status: {}", status)))?;
+                return Ok(Some((partition_status, timestamp, build_request_id)));
+            }
+            Err(rusqlite::Error::QueryReturnedNoRows) => {
+                // No available partition found, fall back to latest status
+            }
+            Err(e) => return Err(BuildEventLogError::QueryError(e.to_string())),
+        }
+        
+        // Fall back to latest status if no available partition found
+        let latest_query = "SELECT pe.status, be.timestamp, be.build_request_id
+                          FROM partition_events pe 
+                          JOIN build_events be ON pe.event_id = be.event_id 
+                          WHERE pe.partition_ref = ?1 
+                          ORDER BY be.timestamp DESC 
+                          LIMIT 1";
+        
+        let mut latest_stmt = conn.prepare(latest_query)
+            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
+        
+        let result = latest_stmt.query_row([partition_ref], |row| {
+            let status_str: String = row.get(0)?;
+            let timestamp: i64 = row.get(1)?;
+            let build_request_id: String = row.get(2)?;
+            let status = status_str.parse::<i32>()
+                .map_err(|_e| rusqlite::Error::InvalidColumnType(0, status_str.clone(), rusqlite::types::Type::Integer))?;
+            Ok((status, timestamp, build_request_id))
+        });
+        
+        match result {
+            Ok((status, timestamp, build_request_id)) => {
+                let partition_status = PartitionStatus::try_from(status)
+                    .map_err(|_| BuildEventLogError::QueryError(format!("Invalid partition status: {}", status)))?;
+                Ok(Some((partition_status, timestamp, build_request_id)))
+            }
+            Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
+            Err(e) => Err(BuildEventLogError::QueryError(e.to_string())),
+        }
+    }
+}

databuild/event_log/sqlite_storage.rs

@@ -1,154 +0,0 @@
-use super::*;
-use super::storage::BELStorage;
-use async_trait::async_trait;
-use rusqlite::{params, Connection};
-use std::path::Path;
-use std::sync::{Arc, Mutex};
-
-pub struct SqliteBELStorage {
-    connection: Arc<Mutex<Connection>>,
-}
-
-impl SqliteBELStorage {
-    pub fn new(path: &str) -> Result<Self> {
-        // Create parent directory if it doesn't exist
-        if let Some(parent) = Path::new(path).parent() {
-            std::fs::create_dir_all(parent)
-                .map_err(|e| BuildEventLogError::ConnectionError(
-                    format!("Failed to create directory {}: {}", parent.display(), e)
-                ))?;
-        }
-        
-        let conn = Connection::open(path)
-            .map_err(|e| BuildEventLogError::ConnectionError(e.to_string()))?;
-        
-        Ok(Self {
-            connection: Arc::new(Mutex::new(conn)),
-        })
-    }
-}
-
-#[async_trait]
-impl BELStorage for SqliteBELStorage {
-    async fn append_event(&self, event: BuildEvent) -> Result<i64> {
-        let serialized = serde_json::to_string(&event)
-            .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-        
-        let conn = self.connection.lock().unwrap();
-        let _row_id = conn.execute(
-            "INSERT INTO build_events (event_data) VALUES (?)",
-            params![serialized],
-        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
-        
-        Ok(conn.last_insert_rowid())
-    }
-    
-    async fn list_events(&self, since_idx: i64, filter: EventFilter) -> Result<EventPage> {
-        let conn = self.connection.lock().unwrap();
-        
-        // For simplicity in the initial implementation, we'll do basic filtering
-        // More sophisticated JSON path filtering can be added later if needed
-        let mut query = "SELECT rowid, event_data FROM build_events WHERE rowid > ?".to_string();
-        let mut params_vec = vec![since_idx.to_string()];
-        
-        // Add build request ID filter if provided
-        if !filter.build_request_ids.is_empty() {
-            query.push_str(" AND (");
-            for (i, build_id) in filter.build_request_ids.iter().enumerate() {
-                if i > 0 { query.push_str(" OR "); }
-                query.push_str("JSON_EXTRACT(event_data, '$.build_request_id') = ?");
-                params_vec.push(build_id.clone());
-            }
-            query.push_str(")");
-        }
-        
-        // Add ordering and pagination
-        query.push_str(" ORDER BY rowid ASC LIMIT 1000");
-        
-        let mut stmt = conn.prepare(&query)
-            .map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-        
-        // Convert params to rusqlite params
-        let param_refs: Vec<&dyn rusqlite::ToSql> = params_vec.iter()
-            .map(|p| p as &dyn rusqlite::ToSql)
-            .collect();
-        
-        let rows = stmt.query_map(&param_refs[..], |row| {
-            let rowid: i64 = row.get(0)?;
-            let event_data: String = row.get(1)?;
-            Ok((rowid, event_data))
-        }).map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-        
-        let mut events = Vec::new();
-        let mut max_idx = since_idx;
-        
-        for row in rows {
-            let (rowid, event_data) = row.map_err(|e| BuildEventLogError::QueryError(e.to_string()))?;
-            
-            let event: BuildEvent = serde_json::from_str(&event_data)
-                .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-            
-            // Apply additional filtering in memory for now
-            let mut include_event = true;
-            
-            if !filter.partition_refs.is_empty() {
-                include_event = false;
-                if let Some(event_type) = &event.event_type {
-                    if let crate::build_event::EventType::PartitionEvent(pe) = event_type {
-                        if let Some(partition_ref) = &pe.partition_ref {
-                            if filter.partition_refs.contains(&partition_ref.str) {
-                                include_event = true;
-                            }
-                        }
-                    }
-                }
-            }
-            
-            if !filter.job_run_ids.is_empty() && include_event {
-                include_event = false;
-                if let Some(event_type) = &event.event_type {
-                    if let crate::build_event::EventType::JobEvent(je) = event_type {
-                        if filter.job_run_ids.contains(&je.job_run_id) {
-                            include_event = true;
-                        }
-                    }
-                }
-            }
-            
-            if include_event {
-                events.push(event);
-                max_idx = rowid;
-            }
-        }
-        
-        let has_more = events.len() >= 1000; // If we got the max limit, there might be more
-        
-        Ok(EventPage {
-            events,
-            next_idx: max_idx,
-            has_more,
-        })
-    }
-    
-    async fn initialize(&self) -> Result<()> {
-        let conn = self.connection.lock().unwrap();
-        
-        conn.execute(
-            "CREATE TABLE IF NOT EXISTS build_events (
-                rowid INTEGER PRIMARY KEY AUTOINCREMENT,
-                event_data TEXT NOT NULL
-            )",
-            [],
-        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
-        
-        // Create index for efficient JSON queries
-        conn.execute(
-            "CREATE INDEX IF NOT EXISTS idx_build_request_id ON build_events(
-                JSON_EXTRACT(event_data, '$.build_request_id')
-            )",
-            [],
-        ).map_err(|e| BuildEventLogError::DatabaseError(e.to_string()))?;
-        
-        Ok(())
-    }
-}

databuild/event_log/stdout.rs

@@ -0,0 +1,139 @@
+use super::*;
+use async_trait::async_trait;
+use serde_json;
+
+pub struct StdoutBuildEventLog;
+
+impl StdoutBuildEventLog {
+    pub fn new() -> Self {
+        Self
+    }
+}
+
+#[async_trait]
+impl BuildEventLog for StdoutBuildEventLog {
+    async fn append_event(&self, event: BuildEvent) -> Result<()> {
+        // Serialize the event to JSON and print to stdout
+        let json = serde_json::to_string(&event)
+            .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
+        
+        println!("BUILD_EVENT: {}", json);
+        Ok(())
+    }
+    
+    async fn get_build_request_events(
+        &self, 
+        _build_request_id: &str,
+        _since: Option<i64>
+    ) -> Result<Vec<BuildEvent>> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+    
+    async fn get_partition_events(
+        &self,
+        _partition_ref: &str,
+        _since: Option<i64>
+    ) -> Result<Vec<BuildEvent>> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+    
+    async fn get_job_run_events(
+        &self,
+        _job_run_id: &str
+    ) -> Result<Vec<BuildEvent>> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+    
+    async fn get_events_in_range(
+        &self,
+        _start_time: i64,
+        _end_time: i64
+    ) -> Result<Vec<BuildEvent>> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+    
+    async fn execute_query(&self, _query: &str) -> Result<QueryResult> {
+        // Stdout implementation doesn't support raw queries
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support raw queries".to_string()
+        ))
+    }
+    
+    async fn get_latest_partition_status(
+        &self,
+        _partition_ref: &str
+    ) -> Result<Option<(PartitionStatus, i64)>> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+    
+    async fn get_active_builds_for_partition(
+        &self,
+        _partition_ref: &str
+    ) -> Result<Vec<String>> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+    
+    async fn initialize(&self) -> Result<()> {
+        // No initialization needed for stdout
+        Ok(())
+    }
+    
+    async fn list_build_requests(
+        &self,
+        _limit: u32,
+        _offset: u32,
+        _status_filter: Option<BuildRequestStatus>,
+    ) -> Result<(Vec<BuildRequestSummary>, u32)> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+
+    async fn list_recent_partitions(
+        &self,
+        _limit: u32, 
+        _offset: u32,
+        _status_filter: Option<PartitionStatus>,
+    ) -> Result<(Vec<PartitionSummary>, u32)> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+
+    async fn get_activity_summary(&self) -> Result<ActivitySummary> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+    
+    async fn get_build_request_for_available_partition(
+        &self,
+        _partition_ref: &str
+    ) -> Result<Option<String>> {
+        // Stdout implementation doesn't support querying
+        Err(BuildEventLogError::QueryError(
+            "Stdout build event log does not support querying".to_string()
+        ))
+    }
+}

databuild/event_log/storage.rs

@@ -1,75 +0,0 @@
-use crate::*;
-use async_trait::async_trait;
-use super::Result;
-
-/// Simple stdout storage backend for debugging
-pub struct StdoutBELStorage;
-
-impl StdoutBELStorage {
-    pub fn new() -> Self {
-        Self
-    }
-}
-
-#[async_trait]
-impl BELStorage for StdoutBELStorage {
-    async fn append_event(&self, event: BuildEvent) -> Result<i64> {
-        let json = serde_json::to_string(&event)
-            .map_err(|e| BuildEventLogError::SerializationError(e.to_string()))?;
-        
-        println!("BUILD_EVENT: {}", json);
-        Ok(0) // Return dummy index for stdout
-    }
-    
-    async fn list_events(&self, _since_idx: i64, _filter: EventFilter) -> Result<EventPage> {
-        // Stdout implementation doesn't support querying
-        Err(BuildEventLogError::QueryError(
-            "Stdout storage backend doesn't support querying".to_string()
-        ))
-    }
-    
-    async fn initialize(&self) -> Result<()> {
-        Ok(()) // Nothing to initialize for stdout
-    }
-}
-
-/// Minimal append-only interface optimized for sequential scanning
-#[async_trait]
-pub trait BELStorage: Send + Sync {
-    /// Append a single event, returns the sequential index
-    async fn append_event(&self, event: BuildEvent) -> Result<i64>;
-    
-    /// List events with filtering, starting from a given index
-    async fn list_events(&self, since_idx: i64, filter: EventFilter) -> Result<EventPage>;
-    
-    /// Initialize storage backend (create tables, etc.)
-    async fn initialize(&self) -> Result<()>;
-}
-
-/// Factory function to create storage backends from URI
-pub async fn create_bel_storage(uri: &str) -> Result<Box<dyn BELStorage>> {
-    if uri == "stdout" {
-        Ok(Box::new(StdoutBELStorage::new()))
-    } else if uri.starts_with("sqlite://") {
-        let path = &uri[9..]; // Remove "sqlite://" prefix
-        let storage = crate::event_log::sqlite_storage::SqliteBELStorage::new(path)?;
-        storage.initialize().await?;
-        Ok(Box::new(storage))
-    } else if uri.starts_with("postgres://") {
-        // TODO: Implement PostgresBELStorage
-        Err(BuildEventLogError::ConnectionError(
-            "PostgreSQL storage backend not yet implemented".to_string()
-        ))
-    } else {
-        Err(BuildEventLogError::ConnectionError(
-            format!("Unsupported build event log URI: {}", uri)
-        ))
-    }
-}
-
-/// Factory function to create query engine from URI
-pub async fn create_bel_query_engine(uri: &str) -> Result<std::sync::Arc<crate::event_log::query_engine::BELQueryEngine>> {
-    let storage = create_bel_storage(uri).await?;
-    let storage_arc = std::sync::Arc::from(storage);
-    Ok(std::sync::Arc::new(crate::event_log::query_engine::BELQueryEngine::new(storage_arc)))
-}

databuild/event_log/writer.rs

@@ -1,27 +1,22 @@
 use crate::*;
-use crate::event_log::{BuildEventLogError, Result, create_build_event, current_timestamp_nanos, generate_event_id, query_engine::BELQueryEngine};
+use crate::event_log::{BuildEventLog, BuildEventLogError, Result, create_build_event, current_timestamp_nanos, generate_event_id};
 use std::sync::Arc;
 use log::debug;
 
 /// Common interface for writing events to the build event log with validation
 pub struct EventWriter {
-    query_engine: Arc<BELQueryEngine>,
+    event_log: Arc<dyn BuildEventLog>,
 }
 
 impl EventWriter {
-    /// Create a new EventWriter with the specified query engine
+    /// Create a new EventWriter with the specified event log backend
-    pub fn new(query_engine: Arc<BELQueryEngine>) -> Self {
+    pub fn new(event_log: Arc<dyn BuildEventLog>) -> Self {
-        Self { query_engine }
+        Self { event_log }
     }
 
-    /// Append an event directly to the event log
+    /// Get access to the underlying event log for direct operations
-    pub async fn append_event(&self, event: BuildEvent) -> Result<()> {
+    pub fn event_log(&self) -> &dyn BuildEventLog {
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.as_ref()
-    }
-
-    /// Get access to the underlying query engine for direct operations
-    pub fn query_engine(&self) -> &BELQueryEngine {
-        self.query_engine.as_ref()
     }
 
     /// Request a new build for the specified partitions
@@ -42,7 +37,7 @@ impl EventWriter {
             }),
         );
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 
     /// Update build request status
@@ -64,7 +59,7 @@ impl EventWriter {
             }),
         );
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 
     /// Update build request status with partition list
@@ -87,7 +82,7 @@ impl EventWriter {
             }),
         );
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 
     /// Update partition status
@@ -114,7 +109,7 @@ impl EventWriter {
             })),
         };
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 
     /// Invalidate a partition with a reason
@@ -125,7 +120,7 @@ impl EventWriter {
         reason: String,
     ) -> Result<()> {
         // First validate that the partition exists by checking its current status
-        let current_status = self.query_engine.get_latest_partition_status(&partition_ref.str).await?;
+        let current_status = self.event_log.get_latest_partition_status(&partition_ref.str).await?;
         
         if current_status.is_none() {
             return Err(BuildEventLogError::QueryError(
@@ -145,7 +140,7 @@ impl EventWriter {
             )),
         };
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 
     /// Schedule a job for execution
@@ -175,7 +170,7 @@ impl EventWriter {
             })),
         };
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 
     /// Update job status
@@ -207,7 +202,7 @@ impl EventWriter {
             })),
         };
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 
     /// Cancel a task (job run) with a reason
@@ -218,7 +213,7 @@ impl EventWriter {
         reason: String,
     ) -> Result<()> {
         // Validate that the job run exists and is in a cancellable state
-        let job_events = self.query_engine.get_job_run_events(&job_run_id).await?;
+        let job_events = self.event_log.get_job_run_events(&job_run_id).await?;
         
         if job_events.is_empty() {
             return Err(BuildEventLogError::QueryError(
@@ -257,13 +252,13 @@ impl EventWriter {
             event_id: generate_event_id(),
             timestamp: current_timestamp_nanos(),
             build_request_id,
-            event_type: Some(build_event::EventType::JobRunCancelEvent(JobRunCancelEvent {
+            event_type: Some(build_event::EventType::TaskCancelEvent(TaskCancelEvent {
                 job_run_id,
                 reason,
             })),
         };
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 
     /// Cancel a build request with a reason
@@ -273,7 +268,7 @@ impl EventWriter {
         reason: String,
     ) -> Result<()> {
         // Validate that the build exists and is in a cancellable state
-        let build_events = self.query_engine.get_build_request_events(&build_request_id, None).await?;
+        let build_events = self.event_log.get_build_request_events(&build_request_id, None).await?;
         
         if build_events.is_empty() {
             return Err(BuildEventLogError::QueryError(
@@ -317,7 +312,7 @@ impl EventWriter {
             })),
         };
         
-        self.query_engine.append_event(event).await.map(|_| ())?;
+        self.event_log.append_event(event).await?;
 
         // Also emit a build request status update
         self.update_build_status(
@@ -346,7 +341,7 @@ impl EventWriter {
             }),
         );
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 
     /// Record the analyzed job graph
@@ -368,19 +363,19 @@ impl EventWriter {
             })),
         };
         
-        self.query_engine.append_event(event).await.map(|_| ())
+        self.event_log.append_event(event).await
     }
 }
 
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::event_log::mock::create_mock_bel_query_engine;
+    use crate::event_log::stdout::StdoutBuildEventLog;
 
     #[tokio::test]
     async fn test_event_writer_build_lifecycle() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let event_log = Arc::new(StdoutBuildEventLog::new());
-        let writer = EventWriter::new(query_engine);
+        let writer = EventWriter::new(event_log);
 
         let build_id = "test-build-123".to_string();
         let partitions = vec![PartitionRef { str: "test/partition".to_string() }];
@@ -410,8 +405,8 @@ mod tests {
 
     #[tokio::test]
     async fn test_event_writer_partition_and_job() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let event_log = Arc::new(StdoutBuildEventLog::new());
-        let writer = EventWriter::new(query_engine);
+        let writer = EventWriter::new(event_log);
 
         let build_id = "test-build-456".to_string();
         let partition = PartitionRef { str: "data/users".to_string() };

databuild/format_consistency_test.rs

@@ -3,7 +3,7 @@ mod format_consistency_tests {
     use super::*;
     use crate::*;
     use crate::repositories::partitions::PartitionsRepository;
-    use crate::event_log::mock::{create_mock_bel_query_engine_with_events, test_events};
+    use crate::event_log::mock::{MockBuildEventLog, test_events};
     use std::sync::Arc;
 
     #[tokio::test]
@@ -21,8 +21,8 @@ mod format_consistency_tests {
             test_events::partition_status(Some(build_id.clone()), partition2.clone(), PartitionStatus::PartitionFailed, None),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repository = PartitionsRepository::new(query_engine);
+        let repository = PartitionsRepository::new(mock_log);
         
         // Test the new unified protobuf format
         let request = PartitionsListRequest {

databuild/graph/analyze.rs

@@ -8,7 +8,7 @@ use simple_logger::SimpleLogger;
 use clap::{Arg, Command as ClapCommand};
 use uuid::Uuid;
 use databuild::*;
-use databuild::event_log::{create_bel_query_engine, create_build_event};
+use databuild::event_log::{BuildEventLog, create_build_event_log, create_build_event};
 use databuild::mermaid_utils::generate_mermaid_diagram;
 
 // Configure a job to produce the desired outputs
@@ -179,7 +179,7 @@ fn configure_parallel(job_refs: HashMap<String, Vec<String>>, num_workers: usize
 // Delegation optimization happens in execution phase
 async fn check_partition_staleness(
     partition_refs: &[String],
-    _query_engine: &std::sync::Arc<databuild::event_log::query_engine::BELQueryEngine>,
+    _event_log: &Box<dyn BuildEventLog>,
     _build_request_id: &str
 ) -> Result<(Vec<String>, Vec<String>), String> {
     // Analysis phase creates jobs for all requested partitions
@@ -193,13 +193,13 @@ async fn check_partition_staleness(
 // Plan creates a job graph for given output references
 async fn plan(
     output_refs: &[String], 
-    query_engine: Option<std::sync::Arc<databuild::event_log::query_engine::BELQueryEngine>>,
+    build_event_log: Option<Box<dyn BuildEventLog>>,
     build_request_id: &str
 ) -> Result<JobGraph, String> {
     info!("Starting planning for {} output refs: {:?}", output_refs.len(), output_refs);
     
     // Log build request received event
-    if let Some(ref query_engine_ref) = query_engine {
+    if let Some(ref event_log) = build_event_log {
         let event = create_build_event(
             build_request_id.to_string(),
             crate::build_event::EventType::BuildRequestEvent(BuildRequestEvent {
@@ -209,14 +209,14 @@ async fn plan(
                 message: "Analysis started".to_string(),
             })
         );
-        if let Err(e) = query_engine_ref.append_event(event).await {
+        if let Err(e) = event_log.append_event(event).await {
             error!("Failed to log build request event: {}", e);
         }
     }
     
     // Check for partition staleness and delegation opportunities
-    let (stale_refs, _delegated_refs) = if let Some(ref query_engine_ref) = query_engine {
+    let (stale_refs, _delegated_refs) = if let Some(ref event_log) = build_event_log {
-        match check_partition_staleness(output_refs, query_engine_ref, build_request_id).await {
+        match check_partition_staleness(output_refs, event_log, build_request_id).await {
             Ok((stale, delegated)) => {
                 info!("Staleness check: {} stale, {} delegated partitions", stale.len(), delegated.len());
                 (stale, delegated)
@@ -260,7 +260,7 @@ async fn plan(
     info!("Using {} workers for parallel execution", num_workers);
     
     // Log planning phase start
-    if let Some(ref query_engine_ref) = query_engine {
+    if let Some(ref event_log) = build_event_log {
         let event = create_build_event(
             build_request_id.to_string(),
             crate::build_event::EventType::BuildRequestEvent(BuildRequestEvent {
@@ -270,7 +270,7 @@ async fn plan(
                 message: "Graph analysis in progress".to_string(),
             })
         );
-        if let Err(e) = query_engine_ref.append_event(event).await {
+        if let Err(e) = event_log.append_event(event).await {
             error!("Failed to log planning event: {}", e);
         }
     }
@@ -330,7 +330,7 @@ async fn plan(
         info!("Planning complete: created graph with {} nodes for {} output refs", nodes.len(), output_refs.len());
         
         // Log analysis completion event
-        if let Some(ref query_engine) = query_engine {
+        if let Some(ref event_log) = build_event_log {
             let event = create_build_event(
                 build_request_id.to_string(),
                 crate::build_event::EventType::BuildRequestEvent(BuildRequestEvent {
@@ -340,7 +340,7 @@ async fn plan(
                     message: format!("Analysis completed successfully, {} tasks planned", nodes.len()),
                 })
             );
-            if let Err(e) = query_engine.append_event(event).await {
+            if let Err(e) = event_log.append_event(event).await {
                 error!("Failed to log analysis completion event: {}", e);
             }
             
@@ -358,7 +358,7 @@ async fn plan(
                     message: format!("Job graph analysis completed with {} tasks", nodes.len()),
                 }),
             );
-            if let Err(e) = query_engine.append_event(job_graph_event).await {
+            if let Err(e) = event_log.append_event(job_graph_event).await {
                 error!("Failed to log job graph event: {}", e);
             }
         }
@@ -372,7 +372,7 @@ async fn plan(
         error!("Planning failed: no nodes created for output refs {:?}", output_refs);
         
         // Log planning failure
-        if let Some(ref query_engine) = query_engine {
+        if let Some(ref event_log) = build_event_log {
             let event = create_build_event(
                 build_request_id.to_string(),
                 crate::build_event::EventType::BuildRequestEvent(BuildRequestEvent {
@@ -382,7 +382,7 @@ async fn plan(
                     message: "No jobs found for requested partitions".to_string(),
                 })
             );
-            if let Err(e) = query_engine.append_event(event).await {
+            if let Err(e) = event_log.append_event(event).await {
                 error!("Failed to log failure event: {}", e);
             }
         }
@@ -556,11 +556,11 @@ async fn main() {
         .unwrap_or_else(|_| Uuid::new_v4().to_string());
     
     // Initialize build event log if provided
-    let query_engine = if let Some(uri) = build_event_log_uri {
+    let build_event_log = if let Some(uri) = build_event_log_uri {
-        match create_bel_query_engine(&uri).await {
+        match create_build_event_log(&uri).await {
-            Ok(engine) => {
+            Ok(log) => {
                 info!("Initialized build event log: {}", uri);
-                Some(engine)
+                Some(log)
             }
             Err(e) => {
                 error!("Failed to initialize build event log {}: {}", uri, e);
@@ -575,7 +575,7 @@ async fn main() {
     match mode.as_str() {
         "plan" => {
             // Get output refs from command line arguments
-            match plan(&args, query_engine, &build_request_id).await {
+            match plan(&args, build_event_log, &build_request_id).await {
                 Ok(graph) => {
                     // Output the job graph as JSON
                     match serde_json::to_string(&graph) {

databuild/graph/execute.rs

@@ -1,5 +1,5 @@
 use databuild::{JobGraph, Task, JobStatus, BuildRequestStatus, PartitionStatus, BuildRequestEvent, JobEvent, PartitionEvent, PartitionRef};
-use databuild::event_log::{create_bel_query_engine, create_build_event};
+use databuild::event_log::{create_build_event_log, create_build_event};
 use databuild::build_event::EventType;
 use databuild::log_collector::{LogCollector, LogCollectorError};
 use crossbeam_channel::{Receiver, Sender};
@@ -296,7 +296,7 @@ fn is_task_ready(task: &Task, completed_outputs: &HashSet<String>) -> bool {
 // Check if partitions are already available or being built by other build requests
 async fn check_build_coordination(
     task: &Task,
-    query_engine: &Arc<databuild::event_log::query_engine::BELQueryEngine>,
+    event_log: &Box<dyn databuild::event_log::BuildEventLog>,
     build_request_id: &str
 ) -> Result<(bool, bool, Vec<(PartitionRef, String)>), String> {
     let outputs = &task.config.as_ref().unwrap().outputs;
@@ -307,12 +307,12 @@ async fn check_build_coordination(
         debug!("Checking build coordination for partition: {}", output_ref.str);
         
         // First check if this partition is already available
-        match query_engine.get_latest_partition_status(&output_ref.str).await {
+        match event_log.get_latest_partition_status(&output_ref.str).await {
             Ok(Some((status, _timestamp))) => {
                 debug!("Partition {} has status: {:?}", output_ref.str, status);
                 if status == databuild::PartitionStatus::PartitionAvailable {
                     // Get which build request created this partition
-                    match query_engine.get_build_request_for_available_partition(&output_ref.str).await {
+                    match event_log.get_build_request_for_available_partition(&output_ref.str).await {
                         Ok(Some(source_build_id)) => {
                             info!("Partition {} already available from build {}", output_ref.str, source_build_id);
                             available_partitions.push((output_ref.clone(), source_build_id));
@@ -343,7 +343,7 @@ async fn check_build_coordination(
         }
         
         // Check if this partition is being built by another request
-        match query_engine.get_active_builds_for_partition(&output_ref.str).await {
+        match event_log.get_active_builds_for_partition(&output_ref.str).await {
             Ok(active_builds) => {
                 let other_builds: Vec<String> = active_builds.into_iter()
                     .filter(|id| id != build_request_id)
@@ -363,7 +363,7 @@ async fn check_build_coordination(
                                 message: "Delegated to active build during execution".to_string(),
                             })
                         );
-                        if let Err(e) = query_engine.append_event(event).await {
+                        if let Err(e) = event_log.append_event(event).await {
                             error!("Failed to log delegation event: {}", e);
                         }
                     }
@@ -434,7 +434,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
 
     // Initialize build event log if provided
     let build_event_log = if let Some(uri) = build_event_log_uri {
-        match create_bel_query_engine(&uri).await {
+        match create_build_event_log(&uri).await {
             Ok(log) => {
                 info!("Initialized build event log: {}", uri);
                 Some(log)
@@ -456,7 +456,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
 
 
     // Log build request execution start (existing detailed event)
-    if let Some(ref query_engine) = build_event_log {
+    if let Some(ref event_log) = build_event_log {
         let event = create_build_event(
             build_request_id.clone(),
             EventType::BuildRequestEvent(BuildRequestEvent {
@@ -466,7 +466,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                 message: format!("Starting execution of {} jobs", graph.nodes.len()),
             })
         );
-        if let Err(e) = query_engine.append_event(event).await {
+        if let Err(e) = event_log.append_event(event).await {
             error!("Failed to log execution start event: {}", e);
         }
     }
@@ -522,7 +522,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
             task_states.insert(result.task_key.clone(), current_state);
 
             // Log job completion events
-            if let Some(ref query_engine) = build_event_log {
+            if let Some(ref event_log) = build_event_log {
                 if let Some(original_task) = original_tasks_by_key.get(&result.task_key) {
                     let job_run_id = Uuid::new_v4().to_string();
                     
@@ -540,7 +540,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                             manifests: vec![], // Would be populated from actual job output
                         })
                     );
-                    if let Err(e) = query_engine.append_event(job_event).await {
+                    if let Err(e) = event_log.append_event(job_event).await {
                         error!("Failed to log job completion event: {}", e);
                     }
                     
@@ -556,7 +556,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                                 job_run_id: job_run_id.clone(),
                             })
                         );
-                        if let Err(e) = query_engine.append_event(partition_event).await {
+                        if let Err(e) = event_log.append_event(partition_event).await {
                             error!("Failed to log partition status event: {}", e);
                         }
                     }
@@ -592,8 +592,8 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                 if task_states.get(&task_key) == Some(&TaskState::Pending) {
                     if is_task_ready(task_node, &completed_outputs) {
                         // Check build coordination if event log is available
-                        let (should_build, is_skipped, available_partitions) = if let Some(ref query_engine) = build_event_log {
+                        let (should_build, is_skipped, available_partitions) = if let Some(ref event_log) = build_event_log {
-                            match check_build_coordination(task_node, query_engine, &build_request_id).await {
+                            match check_build_coordination(task_node, event_log, &build_request_id).await {
                                 Ok((should_build, is_skipped, available_partitions)) => (should_build, is_skipped, available_partitions),
                                 Err(e) => {
                                     error!("Error checking build coordination for {}: {}", 
@@ -611,7 +611,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                                 info!("Task {} skipped - all target partitions already available", task_node.job.as_ref().unwrap().label);
                                 
                                 // Log delegation events for each available partition
-                                if let Some(ref query_engine) = build_event_log {
+                                if let Some(ref event_log) = build_event_log {
                                     for (partition_ref, source_build_id) in &available_partitions {
                                         let delegation_event = create_build_event(
                                             build_request_id.clone(),
@@ -621,7 +621,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                                                 message: "Delegated to historical build - partition already available".to_string(),
                                             })
                                         );
-                                        if let Err(e) = query_engine.append_event(delegation_event).await {
+                                        if let Err(e) = event_log.append_event(delegation_event).await {
                                             error!("Failed to log historical delegation event: {}", e);
                                         }
                                     }
@@ -641,7 +641,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                                             manifests: vec![],
                                         })
                                     );
-                                    if let Err(e) = query_engine.append_event(job_event).await {
+                                    if let Err(e) = event_log.append_event(job_event).await {
                                         error!("Failed to log job skipped event: {}", e);
                                     }
                                 }
@@ -662,7 +662,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                         info!("Dispatching task: {}", task_node.job.as_ref().unwrap().label);
                         
                         // Log job scheduling events
-                        if let Some(ref query_engine) = build_event_log {
+                        if let Some(ref event_log) = build_event_log {
                             let job_run_id = Uuid::new_v4().to_string();
                             
                             // Log job scheduled
@@ -679,7 +679,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                                     manifests: vec![],
                                 })
                             );
-                            if let Err(e) = query_engine.append_event(job_event).await {
+                            if let Err(e) = event_log.append_event(job_event).await {
                                 error!("Failed to log job scheduled event: {}", e);
                             }
                             
@@ -695,7 +695,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                                         job_run_id: job_run_id.clone(),
                                     })
                                 );
-                                if let Err(e) = query_engine.append_event(partition_event).await {
+                                if let Err(e) = event_log.append_event(partition_event).await {
                                     error!("Failed to log partition building event: {}", e);
                                 }
                             }
@@ -785,7 +785,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
 
 
     // Log final build request status (existing detailed event)
-    if let Some(ref query_engine) = build_event_log {
+    if let Some(ref event_log) = build_event_log {
         let final_status = if failure_count > 0 || fail_fast_triggered {
             BuildRequestStatus::BuildRequestFailed
         } else {
@@ -801,7 +801,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
                 message: format!("Execution completed: {} succeeded, {} failed", success_count, failure_count),
             })
         );
-        if let Err(e) = query_engine.append_event(event).await {
+        if let Err(e) = event_log.append_event(event).await {
             error!("Failed to log final build request event: {}", e);
         }
     }

databuild/lib.rs

@@ -35,7 +35,7 @@ pub mod metrics_aggregator;
 mod format_consistency_test;
 
 // Re-export commonly used types from event_log
-pub use event_log::{BuildEventLogError, create_bel_query_engine};
+pub use event_log::{BuildEventLog, BuildEventLogError, create_build_event_log};
 
 // Re-export orchestration types
 pub use orchestration::{BuildOrchestrator, BuildResult, OrchestrationError};

databuild/orchestration/mod.rs

@@ -1,5 +1,5 @@
 use crate::*;
-use crate::event_log::{writer::EventWriter, query_engine::BELQueryEngine};
+use crate::event_log::{BuildEventLog, writer::EventWriter};
 use log::info;
 use std::sync::Arc;
 
@@ -26,12 +26,12 @@ pub struct BuildOrchestrator {
 impl BuildOrchestrator {
     /// Create a new build orchestrator
     pub fn new(
-        query_engine: Arc<BELQueryEngine>,
+        event_log: Arc<dyn BuildEventLog>,
         build_request_id: String,
         requested_partitions: Vec<PartitionRef>,
     ) -> Self {
         Self {
-            event_writer: EventWriter::new(query_engine),
+            event_writer: EventWriter::new(event_log),
             build_request_id,
             requested_partitions,
         }
@@ -138,7 +138,7 @@ impl BuildOrchestrator {
             job,
         );
         
-        self.event_writer.append_event(event).await
+        self.event_writer.event_log().append_event(event).await
             .map_err(OrchestrationError::EventLog)?;
             
         Ok(())
@@ -151,7 +151,7 @@ impl BuildOrchestrator {
             job,
         );
         
-        self.event_writer.append_event(event).await
+        self.event_writer.event_log().append_event(event).await
             .map_err(OrchestrationError::EventLog)?;
             
         Ok(())
@@ -164,7 +164,7 @@ impl BuildOrchestrator {
             partition,
         );
         
-        self.event_writer.append_event(event).await
+        self.event_writer.event_log().append_event(event).await
             .map_err(OrchestrationError::EventLog)?;
             
         Ok(())
@@ -190,22 +190,138 @@ impl BuildOrchestrator {
         Ok(())
     }
 
+    /// Get reference to the event log for direct access if needed
+    pub fn event_log(&self) -> &dyn BuildEventLog {
+        self.event_writer.event_log()
+    }
 }
 
 #[cfg(test)]
 mod tests {
     use super::*;
+    use async_trait::async_trait;
+    use std::sync::{Arc, Mutex};
 
+    /// Mock event log for testing that captures events
+    struct MockEventLog {
+        events: Arc<Mutex<Vec<BuildEvent>>>,
+    }
 
+    impl MockEventLog {
+        fn new() -> (Self, Arc<Mutex<Vec<BuildEvent>>>) {
+            let events = Arc::new(Mutex::new(Vec::new()));
+            let log = Self {
+                events: events.clone(),
+            };
+            (log, events)
+        }
+    }
+
+    #[async_trait]
+    impl BuildEventLog for MockEventLog {
+        async fn append_event(&self, event: BuildEvent) -> crate::event_log::Result<()> {
+            self.events.lock().unwrap().push(event);
+            Ok(())
+        }
+
+        async fn get_build_request_events(
+            &self,
+            _build_request_id: &str,
+            _since: Option<i64>,
+        ) -> crate::event_log::Result<Vec<BuildEvent>> {
+            Ok(self.events.lock().unwrap().clone())
+        }
+
+        async fn get_partition_events(
+            &self,
+            _partition_ref: &str,
+            _since: Option<i64>,
+        ) -> crate::event_log::Result<Vec<BuildEvent>> {
+            Ok(vec![])
+        }
+
+        async fn get_job_run_events(
+            &self,
+            _job_run_id: &str,
+        ) -> crate::event_log::Result<Vec<BuildEvent>> {
+            Ok(vec![])
+        }
+
+        async fn get_events_in_range(
+            &self,
+            _start_time: i64,
+            _end_time: i64,
+        ) -> crate::event_log::Result<Vec<BuildEvent>> {
+            Ok(vec![])
+        }
+
+        async fn execute_query(&self, _query: &str) -> crate::event_log::Result<crate::event_log::QueryResult> {
+            Ok(crate::event_log::QueryResult {
+                columns: vec![],
+                rows: vec![],
+            })
+        }
+
+        async fn get_latest_partition_status(
+            &self,
+            _partition_ref: &str,
+        ) -> crate::event_log::Result<Option<(PartitionStatus, i64)>> {
+            Ok(None)
+        }
+
+        async fn get_active_builds_for_partition(
+            &self,
+            _partition_ref: &str,
+        ) -> crate::event_log::Result<Vec<String>> {
+            Ok(vec![])
+        }
+
+        async fn initialize(&self) -> crate::event_log::Result<()> {
+            Ok(())
+        }
+
+        async fn list_build_requests(
+            &self,
+            _limit: u32,
+            _offset: u32,
+            _status_filter: Option<BuildRequestStatus>,
+        ) -> crate::event_log::Result<(Vec<crate::event_log::BuildRequestSummary>, u32)> {
+            Ok((vec![], 0))
+        }
+
+        async fn list_recent_partitions(
+            &self,
+            _limit: u32,
+            _offset: u32,
+            _status_filter: Option<PartitionStatus>,
+        ) -> crate::event_log::Result<(Vec<crate::event_log::PartitionSummary>, u32)> {
+            Ok((vec![], 0))
+        }
+
+        async fn get_activity_summary(&self) -> crate::event_log::Result<crate::event_log::ActivitySummary> {
+            Ok(crate::event_log::ActivitySummary {
+                active_builds_count: 0,
+                recent_builds: vec![],
+                recent_partitions: vec![],
+                total_partitions_count: 0,
+            })
+        }
+
+        async fn get_build_request_for_available_partition(
+            &self,
+            _partition_ref: &str,
+        ) -> crate::event_log::Result<Option<String>> {
+            Ok(None)
+        }
+    }
 
     #[tokio::test]
     async fn test_build_lifecycle_events() {
-        // Use mock BEL query engine for testing
+        let (mock_log, events) = MockEventLog::new();
-        let query_engine = crate::event_log::mock::create_mock_bel_query_engine().await.unwrap();
         let partitions = vec![PartitionRef { str: "test/partition".to_string() }];
         
         let orchestrator = BuildOrchestrator::new(
-            query_engine,
+            Arc::new(mock_log),
             "test-build-123".to_string(),
             partitions.clone(),
         );
@@ -216,24 +332,29 @@ mod tests {
         orchestrator.start_execution().await.unwrap();
         orchestrator.complete_build(BuildResult::Success { jobs_completed: 5 }).await.unwrap();
 
-        // Note: Since we're using the real BELQueryEngine with mock storage,
+        let emitted_events = events.lock().unwrap();
-        // we can't easily inspect emitted events in this test without significant refactoring.
+        assert_eq!(emitted_events.len(), 4);
-        // The test verifies that the orchestration methods complete without errors,
-        // which exercises the event emission code paths.
 
-        // TODO: If we need to verify specific events, we could:
+        // Verify event types and build request IDs
-        // 1. Query the mock storage through the query engine
+        for event in emitted_events.iter() {
-        // 2. Create a specialized test storage that captures events
+            assert_eq!(event.build_request_id, "test-build-123");
-        // 3. Use the existing MockBuildEventLog test pattern with dependency injection
+        }
+
+        // Verify first event is build request received
+        if let Some(build_event::EventType::BuildRequestEvent(br_event)) = &emitted_events[0].event_type {
+            assert_eq!(br_event.status_code, BuildRequestStatus::BuildRequestReceived as i32);
+            assert_eq!(br_event.requested_partitions, partitions);
+        } else {
+            panic!("First event should be BuildRequestEvent");
+        }
     }
 
     #[tokio::test]
     async fn test_partition_and_job_events() {
-        // Use mock BEL query engine for testing
+        let (mock_log, events) = MockEventLog::new();
-        let query_engine = crate::event_log::mock::create_mock_bel_query_engine().await.unwrap();
         
         let orchestrator = BuildOrchestrator::new(
-            query_engine,
+            Arc::new(mock_log),
             "test-build-456".to_string(),
             vec![],
         );
@@ -255,7 +376,12 @@ mod tests {
         };
         orchestrator.emit_job_scheduled(&job_event).await.unwrap();
 
-        // Note: Same testing limitation as above.
+        let emitted_events = events.lock().unwrap();
-        // We verify that the methods complete successfully without panicking.
+        assert_eq!(emitted_events.len(), 2);
+
+        // All events should have the correct build request ID
+        for event in emitted_events.iter() {
+            assert_eq!(event.build_request_id, "test-build-456");
+        }
     }
 }

databuild/repositories/builds/mod.rs

@@ -1,14 +1,13 @@
 use crate::*;
-use crate::event_log::{BuildEventLogError, Result};
+use crate::event_log::{BuildEventLog, BuildEventLogError, Result};
-use crate::event_log::query_engine::BELQueryEngine;
 use crate::{BuildDetailResponse, BuildTimelineEvent as ServiceBuildTimelineEvent};
 use std::sync::Arc;
-// use std::collections::HashMap; // Commented out since not used with new query engine
+use std::collections::HashMap;
 use serde::Serialize;
 
 /// Repository for querying build data from the build event log
 pub struct BuildsRepository {
-    query_engine: Arc<BELQueryEngine>,
+    event_log: Arc<dyn BuildEventLog>,
 }
 
 /// Summary of a build request and its current status
@@ -41,8 +40,8 @@ pub struct BuildEvent {
 
 impl BuildsRepository {
     /// Create a new BuildsRepository
-    pub fn new(query_engine: Arc<BELQueryEngine>) -> Self {
+    pub fn new(event_log: Arc<dyn BuildEventLog>) -> Self {
-        Self { query_engine }
+        Self { event_log }
     }
 
     /// List all builds with their current status
@@ -50,32 +49,108 @@ impl BuildsRepository {
     /// Returns a list of all build requests that have been made,
     /// including their current status and execution details.
     pub async fn list(&self, limit: Option<usize>) -> Result<Vec<BuildInfo>> {
-        // Use query engine to list builds with the protobuf request format
+        // Get all events from the event log
-        let request = BuildsListRequest {
+        let events = self.event_log.get_events_in_range(0, i64::MAX).await?;
-            limit: limit.map(|l| l as u32),
-            offset: Some(0),
-            status_filter: None,
-        };
-        let response = self.query_engine.list_build_requests(request).await?;
         
-        // Convert from protobuf BuildSummary to repository BuildInfo
+        let mut build_data: HashMap<String, BuildInfo> = HashMap::new();
-        let builds = response.builds.into_iter().map(|build| {
+        let mut build_cancellations: HashMap<String, String> = HashMap::new();
-            BuildInfo {
+        let mut job_counts: HashMap<String, (usize, usize, usize, usize)> = HashMap::new(); // total, completed, failed, cancelled
-                build_request_id: build.build_request_id,
+        
-                status: BuildRequestStatus::try_from(build.status_code).unwrap_or(BuildRequestStatus::BuildRequestUnknown),
+        // First pass: collect all build cancel events
-                requested_partitions: build.requested_partitions,
+        for event in &events {
-                requested_at: build.requested_at,
+            if let Some(build_event::EventType::BuildCancelEvent(bc_event)) = &event.event_type {
-                started_at: build.started_at,
+                build_cancellations.insert(event.build_request_id.clone(), bc_event.reason.clone());
-                completed_at: build.completed_at,
+            }
-                duration_ms: build.duration_ms,
+        }
-                total_jobs: build.total_jobs as usize,
+        
-                completed_jobs: build.completed_jobs as usize,
+        // Second pass: collect job statistics for each build
-                failed_jobs: build.failed_jobs as usize,
+        for event in &events {
-                cancelled_jobs: build.cancelled_jobs as usize,
+            if let Some(build_event::EventType::JobEvent(j_event)) = &event.event_type {
-                cancelled: build.cancelled,
+                let build_id = &event.build_request_id;
-                cancel_reason: None, // TODO: Add cancel reason to BuildSummary if needed
+                let (total, completed, failed, cancelled) = job_counts.entry(build_id.clone()).or_insert((0, 0, 0, 0));
+                
+                match j_event.status_code {
+                    1 => *total = (*total).max(1), // JobScheduled - count unique jobs
+                    3 => *completed += 1,          // JobCompleted
+                    4 => *failed += 1,             // JobFailed  
+                    5 => *cancelled += 1,          // JobCancelled
+                    _ => {}
+                }
+            }
+        }
+        
+        // Third pass: collect all build request events and build information
+        for event in events {
+            if let Some(build_event::EventType::BuildRequestEvent(br_event)) = &event.event_type {
+                let status = match br_event.status_code {
+                    1 => BuildRequestStatus::BuildRequestReceived,
+                    2 => BuildRequestStatus::BuildRequestPlanning,
+                    3 => BuildRequestStatus::BuildRequestExecuting,
+                    4 => BuildRequestStatus::BuildRequestCompleted,
+                    5 => BuildRequestStatus::BuildRequestFailed,
+                    6 => BuildRequestStatus::BuildRequestCancelled,
+                    _ => BuildRequestStatus::BuildRequestUnknown,
+                };
+                
+                // Create or update build info
+                let build = build_data.entry(event.build_request_id.clone()).or_insert_with(|| {
+                    let (total_jobs, completed_jobs, failed_jobs, cancelled_jobs) = 
+                        job_counts.get(&event.build_request_id).unwrap_or(&(0, 0, 0, 0));
+                    
+                    BuildInfo {
+                        build_request_id: event.build_request_id.clone(),
+                        status: BuildRequestStatus::BuildRequestUnknown,
+                        requested_partitions: br_event.requested_partitions.clone(),
+                        requested_at: event.timestamp,
+                        started_at: None,
+                        completed_at: None,
+                        duration_ms: None,
+                        total_jobs: *total_jobs,
+                        completed_jobs: *completed_jobs,
+                        failed_jobs: *failed_jobs,
+                        cancelled_jobs: *cancelled_jobs,
+                        cancelled: false,
+                        cancel_reason: None,
+                    }
+                });
+                
+                // Update build with new information
+                build.status = status;
+                
+                match status {
+                    BuildRequestStatus::BuildRequestReceived => {
+                        build.requested_at = event.timestamp;
+                    }
+                    BuildRequestStatus::BuildRequestExecuting => {
+                        build.started_at = Some(event.timestamp);
+                    }
+                    BuildRequestStatus::BuildRequestCompleted | 
+                    BuildRequestStatus::BuildRequestFailed | 
+                    BuildRequestStatus::BuildRequestCancelled => {
+                        build.completed_at = Some(event.timestamp);
+                        if let Some(started) = build.started_at {
+                            build.duration_ms = Some((event.timestamp - started) / 1_000_000); // Convert to ms
+                        }
+                    }
+                    _ => {}
+                }
+                
+                // Check if this build was cancelled
+                if let Some(cancel_reason) = build_cancellations.get(&event.build_request_id) {
+                    build.cancelled = true;
+                    build.cancel_reason = Some(cancel_reason.clone());
+                }
+            }
+        }
+        
+        // Convert to vector and sort by requested time (most recent first)
+        let mut builds: Vec<BuildInfo> = build_data.into_values().collect();
+        builds.sort_by(|a, b| b.requested_at.cmp(&a.requested_at));
+        
+        // Apply limit if specified
+        if let Some(limit) = limit {
+            builds.truncate(limit);
         }
-        }).collect();
         
         Ok(builds)
     }
@@ -85,59 +160,121 @@ impl BuildsRepository {
     /// Returns the complete timeline of events for the specified build,
     /// including all status changes and any cancellation events.
     pub async fn show(&self, build_request_id: &str) -> Result<Option<(BuildInfo, Vec<BuildEvent>)>> {
-        // Use query engine to get build summary
+        // Get all events for this specific build
-        let summary_result = self.query_engine.get_build_request_summary(build_request_id).await;
+        let build_events = self.event_log.get_build_request_events(build_request_id, None).await?;
         
-        match summary_result {
+        if build_events.is_empty() {
-            Ok(summary) => {
+            return Ok(None);
-                // Convert BuildRequestSummary to BuildInfo
+        }
-                let build_info = BuildInfo {
+        
-                    build_request_id: summary.build_request_id,
+        let mut build_info: Option<BuildInfo> = None;
-                    status: summary.status,
+        let mut timeline: Vec<BuildEvent> = Vec::new();
-                    requested_partitions: summary.requested_partitions.into_iter()
+        let mut job_counts = (0, 0, 0, 0); // total, completed, failed, cancelled
-                        .map(|s| PartitionRef { str: s })
+        
-                        .collect(),
+        // Process all events to get job statistics
-                    requested_at: summary.created_at,
+        let all_events = self.event_log.get_events_in_range(0, i64::MAX).await?;
-                    started_at: None, // TODO: Track started_at in query engine
+        for event in &all_events {
-                    completed_at: Some(summary.updated_at),
+            if event.build_request_id == build_request_id {
-                    duration_ms: None, // TODO: Calculate duration in query engine
+                if let Some(build_event::EventType::JobEvent(j_event)) = &event.event_type {
-                    total_jobs: 0, // TODO: Implement job counting in query engine
+                    match j_event.status_code {
-                    completed_jobs: 0,
+                        1 => job_counts.0 = job_counts.0.max(1), // JobScheduled - count unique jobs
-                    failed_jobs: 0,
+                        3 => job_counts.1 += 1,                 // JobCompleted
-                    cancelled_jobs: 0,
+                        4 => job_counts.2 += 1,                 // JobFailed  
-                    cancelled: false, // TODO: Track cancellation in query engine
+                        5 => job_counts.3 += 1,                 // JobCancelled
-                    cancel_reason: None,
+                        _ => {}
+                    }
+                }
+            }
+        }
+        
+        // Process build request events to build timeline
+        for event in &build_events {
+            if let Some(build_event::EventType::BuildRequestEvent(br_event)) = &event.event_type {
+                let status = match br_event.status_code {
+                    1 => BuildRequestStatus::BuildRequestReceived,
+                    2 => BuildRequestStatus::BuildRequestPlanning,
+                    3 => BuildRequestStatus::BuildRequestExecuting,
+                    4 => BuildRequestStatus::BuildRequestCompleted,
+                    5 => BuildRequestStatus::BuildRequestFailed,
+                    6 => BuildRequestStatus::BuildRequestCancelled,
+                    _ => BuildRequestStatus::BuildRequestUnknown,
                 };
                 
-                // Get all events for this build to create a proper timeline
+                // Create or update build info
-                let all_events = self.query_engine.get_build_request_events(build_request_id, None).await?;
+                if build_info.is_none() {
+                    build_info = Some(BuildInfo {
+                        build_request_id: event.build_request_id.clone(),
+                        status: BuildRequestStatus::BuildRequestUnknown,
+                        requested_partitions: br_event.requested_partitions.clone(),
+                        requested_at: event.timestamp,
+                        started_at: None,
+                        completed_at: None,
+                        duration_ms: None,
+                        total_jobs: job_counts.0,
+                        completed_jobs: job_counts.1,
+                        failed_jobs: job_counts.2,
+                        cancelled_jobs: job_counts.3,
+                        cancelled: false,
+                        cancel_reason: None,
+                    });
+                }
                 
-                // Create timeline from build request events
+                let build = build_info.as_mut().unwrap();
-                let mut timeline = Vec::new();
+                build.status = status;
-                for event in all_events {
+                
-                    if let Some(crate::build_event::EventType::BuildRequestEvent(br_event)) = &event.event_type {
+                match status {
-                        if let Ok(status) = BuildRequestStatus::try_from(br_event.status_code) {
+                    BuildRequestStatus::BuildRequestReceived => {
+                        build.requested_at = event.timestamp;
+                    }
+                    BuildRequestStatus::BuildRequestExecuting => {
+                        build.started_at = Some(event.timestamp);
+                    }
+                    BuildRequestStatus::BuildRequestCompleted | 
+                    BuildRequestStatus::BuildRequestFailed | 
+                    BuildRequestStatus::BuildRequestCancelled => {
+                        build.completed_at = Some(event.timestamp);
+                        if let Some(started) = build.started_at {
+                            build.duration_ms = Some((event.timestamp - started) / 1_000_000); // Convert to ms
+                        }
+                    }
+                    _ => {}
+                }
+                
+                // Add to timeline
                 timeline.push(BuildEvent {
                     timestamp: event.timestamp,
-                                event_type: "build_status".to_string(),
+                    event_type: "build_status_change".to_string(),
                     status: Some(status),
-                                message: br_event.message.clone(),
+                    message: format!("Build status: {:?}", status),
                     cancel_reason: None,
                 });
             }
         }
+        
+        // Also check for build cancel events in all events
+        for event in all_events {
+            if event.build_request_id == build_request_id {
+                if let Some(build_event::EventType::BuildCancelEvent(bc_event)) = &event.event_type {
+                    if let Some(build) = build_info.as_mut() {
+                        build.cancelled = true;
+                        build.cancel_reason = Some(bc_event.reason.clone());
+                    }
+                    
+                    timeline.push(BuildEvent {
+                        timestamp: event.timestamp,
+                        event_type: "build_cancel".to_string(),
+                        status: None,
+                        message: "Build cancelled".to_string(),
+                        cancel_reason: Some(bc_event.reason.clone()),
+                    });
+                }
+            }
         }
         
         // Sort timeline by timestamp
         timeline.sort_by_key(|e| e.timestamp);
         
-                Ok(Some((build_info, timeline)))
+        Ok(build_info.map(|info| (info, timeline)))
-            }
-            Err(_) => {
-                // Build not found
-                Ok(None)
-            }
-        }
     }
 
     /// Show detailed information about a specific build using protobuf response format
@@ -187,7 +324,7 @@ impl BuildsRepository {
     /// 
     /// This method uses the EventWriter to write a build cancellation event.
     /// It validates that the build exists and is in a cancellable state.
-    pub async fn cancel(&self, build_request_id: &str, _reason: String) -> Result<()> {
+    pub async fn cancel(&self, build_request_id: &str, reason: String) -> Result<()> {
         // First check if the build exists and get its current status
         let build_info = self.show(build_request_id).await?;
         
@@ -219,23 +356,9 @@ impl BuildsRepository {
             _ => {}
         }
         
-        // Create a build cancellation event
+        // Use EventWriter to write the cancellation event
-        use crate::event_log::{create_build_event, current_timestamp_nanos, generate_event_id};
+        let event_writer = crate::event_log::writer::EventWriter::new(self.event_log.clone());
-        
+        event_writer.cancel_build(build_request_id.to_string(), reason).await
-        let cancel_event = create_build_event(
-            build_request_id.to_string(),
-            crate::build_event::EventType::BuildRequestEvent(crate::BuildRequestEvent {
-                status_code: BuildRequestStatus::BuildRequestCancelled as i32,
-                status_name: BuildRequestStatus::BuildRequestCancelled.to_display_string(),
-                requested_partitions: build.requested_partitions,
-                message: format!("Build cancelled"),
-            })
-        );
-        
-        // Append the cancellation event
-        self.query_engine.append_event(cancel_event).await?;
-        
-        Ok(())
     }
 
     /// List builds using protobuf response format with dual status fields
@@ -272,12 +395,12 @@ impl BuildsRepository {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::event_log::mock::{create_mock_bel_query_engine, create_mock_bel_query_engine_with_events, test_events};
+    use crate::event_log::mock::{MockBuildEventLog, test_events};
 
     #[tokio::test]
     async fn test_builds_repository_list_empty() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::new().await.unwrap());
-        let repo = BuildsRepository::new(query_engine);
+        let repo = BuildsRepository::new(mock_log);
         
         let builds = repo.list(None).await.unwrap();
         assert!(builds.is_empty());
@@ -298,8 +421,8 @@ mod tests {
             test_events::build_request_event(Some(build_id2.clone()), vec![partition2.clone()], BuildRequestStatus::BuildRequestFailed),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = BuildsRepository::new(query_engine);
+        let repo = BuildsRepository::new(mock_log);
         
         let builds = repo.list(None).await.unwrap();
         assert_eq!(builds.len(), 2);
@@ -329,8 +452,8 @@ mod tests {
             test_events::build_request_event(Some(build_id.clone()), vec![partition.clone()], BuildRequestStatus::BuildRequestCompleted),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = BuildsRepository::new(query_engine);
+        let repo = BuildsRepository::new(mock_log);
         
         let result = repo.show(&build_id).await.unwrap();
         assert!(result.is_some());
@@ -349,8 +472,8 @@ mod tests {
 
     #[tokio::test]
     async fn test_builds_repository_show_nonexistent() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::new().await.unwrap());
-        let repo = BuildsRepository::new(query_engine);
+        let repo = BuildsRepository::new(mock_log);
         
         let result = repo.show("nonexistent-build").await.unwrap();
         assert!(result.is_none());
@@ -367,14 +490,14 @@ mod tests {
             test_events::build_request_event(Some(build_id.clone()), vec![partition.clone()], BuildRequestStatus::BuildRequestExecuting),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = BuildsRepository::new(query_engine.clone());
+        let repo = BuildsRepository::new(mock_log.clone());
         
         // Cancel the build
         repo.cancel(&build_id, "User requested cancellation".to_string()).await.unwrap();
         
         // Verify the cancellation was recorded
-        // Note: This test demonstrates the pattern, but the MockBELStorage would need
+        // Note: This test demonstrates the pattern, but the MockBuildEventLog would need
         // to be enhanced to properly store build cancel events for full verification
         
         // Try to cancel a non-existent build
@@ -393,8 +516,8 @@ mod tests {
             test_events::build_request_event(Some(build_id.clone()), vec![partition.clone()], BuildRequestStatus::BuildRequestCompleted),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = BuildsRepository::new(query_engine);
+        let repo = BuildsRepository::new(mock_log);
         
         // Try to cancel the completed build - should fail
         let result = repo.cancel(&build_id, "Should fail".to_string()).await;

databuild/repositories/jobs/mod.rs

@@ -1,6 +1,5 @@
 use crate::*;
-use crate::event_log::{BuildEventLogError, Result};
+use crate::event_log::{BuildEventLog, Result};
-use crate::event_log::query_engine::BELQueryEngine;
 use crate::{JobDetailResponse, JobRunDetail as ServiceJobRunDetail};
 use std::sync::Arc;
 use std::collections::HashMap;
@@ -8,7 +7,7 @@ use serde::Serialize;
 
 /// Repository for querying job data from the build event log
 pub struct JobsRepository {
-    query_engine: Arc<BELQueryEngine>,
+    event_log: Arc<dyn BuildEventLog>,
 }
 
 /// Summary of a job's execution history and statistics
@@ -44,8 +43,8 @@ pub struct JobRunDetail {
 
 impl JobsRepository {
     /// Create a new JobsRepository
-    pub fn new(query_engine: Arc<BELQueryEngine>) -> Self {
+    pub fn new(event_log: Arc<dyn BuildEventLog>) -> Self {
-        Self { query_engine }
+        Self { event_log }
     }
 
     /// List all jobs with their execution statistics
@@ -54,7 +53,7 @@ impl JobsRepository {
     /// success/failure statistics and recent activity.
     pub async fn list(&self, limit: Option<usize>) -> Result<Vec<JobInfo>> {
         // Get all job events from the event log
-        let events = self.query_engine.get_events_in_range(0, i64::MAX).await?;
+        let events = self.event_log.get_events_in_range(0, i64::MAX).await?;
         
         let mut job_data: HashMap<String, Vec<JobRunDetail>> = HashMap::new();
         
@@ -180,7 +179,7 @@ impl JobsRepository {
     /// detailed timing, status, and output information.
     pub async fn show(&self, job_label: &str) -> Result<Option<(JobInfo, Vec<JobRunDetail>)>> {
         // Get all job events for this specific job
-        let events = self.query_engine.get_events_in_range(0, i64::MAX).await?;
+        let events = self.event_log.get_events_in_range(0, i64::MAX).await?;
         
         let mut job_runs: Vec<JobRunDetail> = Vec::new();
         
@@ -375,12 +374,12 @@ impl JobsRepository {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::event_log::mock::{create_mock_bel_query_engine, create_mock_bel_query_engine_with_events, test_events};
+    use crate::event_log::mock::{MockBuildEventLog, test_events};
 
     #[tokio::test]
     async fn test_jobs_repository_list_empty() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::new().await.unwrap());
-        let repo = JobsRepository::new(query_engine);
+        let repo = JobsRepository::new(mock_log);
         
         let jobs = repo.list(None).await.unwrap();
         assert!(jobs.is_empty());
@@ -402,8 +401,8 @@ mod tests {
             test_events::job_event(Some(build_id.clone()), Some("job-run-2".to_string()), job_label2.clone(), vec![partition2.clone()], JobStatus::JobFailed),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = JobsRepository::new(query_engine);
+        let repo = JobsRepository::new(mock_log);
         
         let jobs = repo.list(None).await.unwrap();
         assert_eq!(jobs.len(), 2);
@@ -435,8 +434,8 @@ mod tests {
             test_events::job_event(Some(build_id.clone()), Some("job-run-123".to_string()), job_label.clone(), vec![partition.clone()], JobStatus::JobCompleted),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = JobsRepository::new(query_engine);
+        let repo = JobsRepository::new(mock_log);
         
         let result = repo.show(&job_label.label).await.unwrap();
         assert!(result.is_some());
@@ -457,8 +456,8 @@ mod tests {
 
     #[tokio::test]
     async fn test_jobs_repository_show_nonexistent() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::new().await.unwrap());
-        let repo = JobsRepository::new(query_engine);
+        let repo = JobsRepository::new(mock_log);
         
         let result = repo.show("//:nonexistent_job").await.unwrap();
         assert!(result.is_none());
@@ -483,8 +482,8 @@ mod tests {
             test_events::job_event(Some(build_id.clone()), Some("run-3".to_string()), job_label.clone(), vec![partition.clone()], JobStatus::JobCancelled),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = JobsRepository::new(query_engine);
+        let repo = JobsRepository::new(mock_log);
         
         let result = repo.show(&job_label.label).await.unwrap();
         assert!(result.is_some());

databuild/repositories/partitions/mod.rs

@@ -1,6 +1,5 @@
 use crate::*;
-use crate::event_log::{BuildEventLogError, Result};
+use crate::event_log::{BuildEventLog, BuildEventLogError, Result};
-use crate::event_log::query_engine::BELQueryEngine;
 use crate::status_utils::list_response_helpers;
 use std::sync::Arc;
 use std::collections::HashMap;
@@ -8,7 +7,7 @@ use serde::Serialize;
 
 /// Repository for querying partition data from the build event log
 pub struct PartitionsRepository {
-    query_engine: Arc<BELQueryEngine>,
+    event_log: Arc<dyn BuildEventLog>,
 }
 
 /// Summary of a partition's current state and history
@@ -34,139 +33,171 @@ pub struct PartitionStatusEvent {
 
 impl PartitionsRepository {
     /// Create a new PartitionsRepository
-    pub fn new(query_engine: Arc<BELQueryEngine>) -> Self {
+    pub fn new(event_log: Arc<dyn BuildEventLog>) -> Self {
-        Self { query_engine }
+        Self { event_log }
     }
 
     /// List all partitions with their current status
     /// 
     /// Returns a list of all partitions that have been referenced in the build event log,
     /// along with their current status and summary information.
-    pub async fn list(&self, _limit: Option<usize>) -> Result<Vec<PartitionInfo>> {
+    pub async fn list(&self, limit: Option<usize>) -> Result<Vec<PartitionInfo>> {
-        // Get all events to find unique partitions
+        // Get all partition events from the event log
-        let filter = EventFilter {
+        let events = self.event_log.get_events_in_range(0, i64::MAX).await?;
-            partition_refs: vec![],
+        
-            partition_patterns: vec![],
+        let mut partition_data: HashMap<String, Vec<PartitionStatusEvent>> = HashMap::new();
-            job_labels: vec![],
+        
-            job_run_ids: vec![],
+        // Collect all partition events
-            build_request_ids: vec![],
+        for event in events {
+            if let Some(build_event::EventType::PartitionEvent(p_event)) = &event.event_type {
+                if let Some(partition_ref) = &p_event.partition_ref {
+                    let status = match p_event.status_code {
+                        1 => PartitionStatus::PartitionRequested,
+                        2 => PartitionStatus::PartitionAnalyzed,
+                        3 => PartitionStatus::PartitionBuilding,
+                        4 => PartitionStatus::PartitionAvailable,
+                        5 => PartitionStatus::PartitionFailed,
+                        6 => PartitionStatus::PartitionDelegated,
+                        _ => PartitionStatus::PartitionUnknown,
                     };
                     
-        let events = self.query_engine.get_events_in_range(0, i64::MAX).await?;
+                    let status_event = PartitionStatusEvent {
+                        timestamp: event.timestamp,
+                        status,
+                        message: p_event.message.clone(),
+                        build_request_id: event.build_request_id.clone(),
+                        job_run_id: if p_event.job_run_id.is_empty() { None } else { Some(p_event.job_run_id.clone()) },
+                    };
                     
-        // Collect unique partition references
+                    partition_data.entry(partition_ref.str.clone())
-        let mut unique_partitions = std::collections::HashSet::new();
+                        .or_insert_with(Vec::new)
-        for event in &events {
+                        .push(status_event);
-            match &event.event_type {
-                Some(crate::build_event::EventType::PartitionEvent(p_event)) => {
-                    if let Some(partition_ref) = &p_event.partition_ref {
-                        unique_partitions.insert(partition_ref.str.clone());
-                    }
-                }
-                Some(crate::build_event::EventType::BuildRequestEvent(br_event)) => {
-                    for partition_ref in &br_event.requested_partitions {
-                        unique_partitions.insert(partition_ref.str.clone());
-                    }
-                }
-                Some(crate::build_event::EventType::JobEvent(j_event)) => {
-                    for partition_ref in &j_event.target_partitions {
-                        unique_partitions.insert(partition_ref.str.clone());
-                    }
-                }
-                _ => {}
                 }
             }
             
-        // Get status for each partition and count builds
+            // Also check for partition invalidation events
-        let mut partition_infos = Vec::new();
+            if let Some(build_event::EventType::PartitionInvalidationEvent(pi_event)) = &event.event_type {
-        for partition_ref in unique_partitions {
+                if let Some(partition_ref) = &pi_event.partition_ref {
-            if let Ok(Some((status, last_updated))) = self.query_engine.get_latest_partition_status(&partition_ref).await {
+                    let status_event = PartitionStatusEvent {
-                // Count builds that reference this partition by looking at BuildRequestEvents
+                        timestamp: event.timestamp,
-                let mut builds_count = 0;
+                        status: PartitionStatus::PartitionUnknown, // Invalidated
-                for event in &events {
+                        message: format!("Invalidated: {}", pi_event.reason),
-                    if let Some(crate::build_event::EventType::BuildRequestEvent(br_event)) = &event.event_type {
+                        build_request_id: event.build_request_id.clone(),
-                        if br_event.requested_partitions.iter().any(|p| p.str == partition_ref) {
+                        job_run_id: None,
-                            builds_count += 1;
+                    };
+                    
+                    partition_data.entry(partition_ref.str.clone())
+                        .or_insert_with(Vec::new)
+                        .push(status_event);
                 }
             }
         }
         
-                partition_infos.push(PartitionInfo {
+        // Convert to PartitionInfo structs
+        let mut partition_infos: Vec<PartitionInfo> = partition_data.into_iter()
+            .map(|(partition_ref, mut events)| {
+                // Sort events by timestamp
+                events.sort_by_key(|e| e.timestamp);
+                
+                // Get current status from latest event
+                let (current_status, last_updated) = events.last()
+                    .map(|e| (e.status.clone(), e.timestamp))
+                    .unwrap_or((PartitionStatus::PartitionUnknown, 0));
+                
+                // Count builds and find last successful build
+                let builds: std::collections::HashSet<String> = events.iter()
+                    .map(|e| e.build_request_id.clone())
+                    .collect();
+                
+                let last_successful_build = events.iter()
+                    .rev()
+                    .find(|e| e.status == PartitionStatus::PartitionAvailable)
+                    .map(|e| e.build_request_id.clone());
+                
+                // Count invalidations
+                let invalidation_count = events.iter()
+                    .filter(|e| e.message.starts_with("Invalidated:"))
+                    .count();
+                
+                PartitionInfo {
                     partition_ref: PartitionRef { str: partition_ref },
-                    current_status: status,
+                    current_status,
                     last_updated,
-                    builds_count,
+                    builds_count: builds.len(),
-                    last_successful_build: None, // TODO: Find last successful build
+                    last_successful_build,
-                    invalidation_count: 0, // TODO: Count invalidation events
+                    invalidation_count,
-                });
-            }
                 }
+            })
+            .collect();
         
-        // Sort by partition reference for consistent ordering
+        // Sort by most recently updated
-        partition_infos.sort_by(|a, b| a.partition_ref.str.cmp(&b.partition_ref.str));
+        partition_infos.sort_by(|a, b| b.last_updated.cmp(&a.last_updated));
+        
+        // Apply limit if specified
+        if let Some(limit) = limit {
+            partition_infos.truncate(limit);
+        }
         
         Ok(partition_infos)
     }
 
-    // TODO: Implement remaining methods for BELQueryEngine
-    /*
-    Legacy methods that need to be updated to use query_engine:
-    
-    pub async fn show(&self, partition_ref: &str) -> Result<Option<(PartitionInfo, Vec<PartitionStatusEvent>)>> { ... }
-    pub async fn invalidate(&self, partition_ref: &str, reason: String, build_request_id: String) -> Result<()> { ... }
-    pub async fn show_protobuf(&self, partition_ref: &str) -> Result<Option<PartitionDetailResponse>> { ... }
-    pub async fn list_protobuf(&self, request: PartitionsListRequest) -> Result<PartitionsListResponse> { ... }
-    */
-
     /// Show detailed information about a specific partition
     /// 
     /// Returns the complete timeline of status changes for the specified partition,
     /// including all builds that have referenced it.
     pub async fn show(&self, partition_ref: &str) -> Result<Option<(PartitionInfo, Vec<PartitionStatusEvent>)>> {
-        // Get partition events from query engine
+        // Get all events for this partition
-        let events = self.query_engine.get_partition_events(partition_ref, None).await?;
+        let events = self.event_log.get_partition_events(partition_ref, None).await?;
         
         if events.is_empty() {
             return Ok(None);
         }
         
-        // Get the latest partition status
+        let mut status_events = Vec::new();
-        let latest_status_result = self.query_engine.get_latest_partition_status(partition_ref).await?;
+        let mut builds = std::collections::HashSet::new();
-        let (status, last_updated) = latest_status_result.unwrap_or((PartitionStatus::PartitionUnknown, 0));
         
-        // Count builds that reference this partition
+        // Process partition events
-        let all_events = self.query_engine.get_events_in_range(0, i64::MAX).await?;
+        for event in &events {
-        let mut builds_count = 0;
+            if let Some(build_event::EventType::PartitionEvent(p_event)) = &event.event_type {
-        for event in &all_events {
+                let status = match p_event.status_code {
-            if let Some(crate::build_event::EventType::BuildRequestEvent(br_event)) = &event.event_type {
+                    1 => PartitionStatus::PartitionRequested,
-                if br_event.requested_partitions.iter().any(|p| p.str == partition_ref) {
+                    2 => PartitionStatus::PartitionAnalyzed,
-                    builds_count += 1;
+                    3 => PartitionStatus::PartitionBuilding,
-                }
+                    4 => PartitionStatus::PartitionAvailable,
-            }
+                    5 => PartitionStatus::PartitionFailed,
-        }
+                    6 => PartitionStatus::PartitionDelegated,
-        
+                    _ => PartitionStatus::PartitionUnknown,
-        // Create partition info
-        let partition_info = PartitionInfo {
-            partition_ref: PartitionRef { str: partition_ref.to_string() },
-            current_status: status,
-            last_updated,
-            builds_count,
-            last_successful_build: None, // TODO: Find last successful build
-            invalidation_count: 0, // TODO: Count invalidation events
                 };
                 
-        // Convert events to PartitionStatusEvent
-        let mut status_events = Vec::new();
-        for event in events {
-            if let Some(crate::build_event::EventType::PartitionEvent(p_event)) = &event.event_type {
-                if let Ok(event_status) = PartitionStatus::try_from(p_event.status_code) {
                 status_events.push(PartitionStatusEvent {
                     timestamp: event.timestamp,
-                        status: event_status,
+                    status,
                     message: p_event.message.clone(),
-                        build_request_id: event.build_request_id,
+                    build_request_id: event.build_request_id.clone(),
                     job_run_id: if p_event.job_run_id.is_empty() { None } else { Some(p_event.job_run_id.clone()) },
                 });
+                
+                builds.insert(event.build_request_id.clone());
+            }
+        }
+        
+        // Also check for invalidation events in all events
+        let all_events = self.event_log.get_events_in_range(0, i64::MAX).await?;
+        let mut invalidation_count = 0;
+        
+        for event in all_events {
+            if let Some(build_event::EventType::PartitionInvalidationEvent(pi_event)) = &event.event_type {
+                if let Some(partition) = &pi_event.partition_ref {
+                    if partition.str == partition_ref {
+                        status_events.push(PartitionStatusEvent {
+                            timestamp: event.timestamp,
+                            status: PartitionStatus::PartitionUnknown, // Invalidated
+                            message: format!("Invalidated: {}", pi_event.reason),
+                            build_request_id: event.build_request_id.clone(),
+                            job_run_id: None,
+                        });
+                        invalidation_count += 1;
+                    }
                 }
             }
         }
@@ -174,6 +205,26 @@ impl PartitionsRepository {
         // Sort events by timestamp
         status_events.sort_by_key(|e| e.timestamp);
         
+        // Get current status from latest event
+        let (current_status, last_updated) = status_events.last()
+            .map(|e| (e.status.clone(), e.timestamp))
+            .unwrap_or((PartitionStatus::PartitionUnknown, 0));
+        
+        // Find last successful build
+        let last_successful_build = status_events.iter()
+            .rev()
+            .find(|e| e.status == PartitionStatus::PartitionAvailable)
+            .map(|e| e.build_request_id.clone());
+        
+        let partition_info = PartitionInfo {
+            partition_ref: PartitionRef { str: partition_ref.to_string() },
+            current_status,
+            last_updated,
+            builds_count: builds.len(),
+            last_successful_build,
+            invalidation_count,
+        };
+        
         Ok(Some((partition_info, status_events)))
     }
 
@@ -182,82 +233,89 @@ impl PartitionsRepository {
     /// This method uses the EventWriter to write a partition invalidation event.
     /// It validates that the partition exists before invalidating it.
     pub async fn invalidate(&self, partition_ref: &str, reason: String, build_request_id: String) -> Result<()> {
-        // Check if the partition exists by looking for any events that reference it
+        // First check if the partition exists
-        let partition_events = self.query_engine.get_partition_events(partition_ref, None).await?;
+        let partition_exists = self.show(partition_ref).await?.is_some();
-        let all_events = self.query_engine.get_events_in_range(0, i64::MAX).await?;
-        
-        // Check if partition is referenced in any build request events
-        let mut partition_exists = !partition_events.is_empty();
-        if !partition_exists {
-            for event in &all_events {
-                if let Some(crate::build_event::EventType::BuildRequestEvent(br_event)) = &event.event_type {
-                    if br_event.requested_partitions.iter().any(|p| p.str == partition_ref) {
-                        partition_exists = true;
-                        break;
-                    }
-                }
-            }
-        }
         
         if !partition_exists {
-            return Err(crate::event_log::BuildEventLogError::QueryError(
+            return Err(BuildEventLogError::QueryError(
                 format!("Cannot invalidate non-existent partition: {}", partition_ref)
             ));
         }
         
-        // Create a partition invalidation event
+        // Use EventWriter to write the invalidation event
-        use crate::event_log::create_build_event;
+        let event_writer = crate::event_log::writer::EventWriter::new(self.event_log.clone());
+        let partition = PartitionRef { str: partition_ref.to_string() };
         
-        let invalidation_event = create_build_event(
+        event_writer.invalidate_partition(build_request_id, partition, reason).await
-            build_request_id,
-            crate::build_event::EventType::PartitionInvalidationEvent(crate::PartitionInvalidationEvent {
-                partition_ref: Some(crate::PartitionRef { str: partition_ref.to_string() }),
-                reason,
-            })
-        );
-        
-        // Append the invalidation event
-        self.query_engine.append_event(invalidation_event).await?;
-        
-        Ok(())
     }
 
     /// Show detailed information about a specific partition using protobuf response format
     /// 
     /// Returns the complete partition details with dual status fields and timeline events.
     pub async fn show_protobuf(&self, partition_ref: &str) -> Result<Option<PartitionDetailResponse>> {
-        // TODO: Implement with query engine - for now return None
+        // Get partition info and timeline using existing show method
+        if let Some((partition_info, timeline)) = self.show(partition_ref).await? {
+            // Convert timeline events to protobuf format
+            let protobuf_timeline: Vec<PartitionTimelineEvent> = timeline
+                .into_iter()
+                .map(|event| PartitionTimelineEvent {
+                    timestamp: event.timestamp,
+                    status_code: event.status as i32,
+                    status_name: event.status.to_display_string(),
+                    message: event.message,
+                    build_request_id: event.build_request_id,
+                    job_run_id: event.job_run_id,
+                })
+                .collect();
+
+            let response = PartitionDetailResponse {
+                partition_ref: Some(partition_info.partition_ref),
+                status_code: partition_info.current_status as i32,
+                status_name: partition_info.current_status.to_display_string(),
+                last_updated: partition_info.last_updated,
+                builds_count: partition_info.builds_count as u32,
+                last_successful_build: partition_info.last_successful_build,
+                invalidation_count: partition_info.invalidation_count as u32,
+                timeline: protobuf_timeline,
+            };
+
+            Ok(Some(response))
+        } else {
             Ok(None)
         }
+    }
 
     /// List partitions returning protobuf response format with dual status fields
     /// 
     /// This method provides the unified CLI/Service response format with both
     /// status codes (enum values) and status names (human-readable strings).
     pub async fn list_protobuf(&self, request: PartitionsListRequest) -> Result<PartitionsListResponse> {
-        // Get partition info using existing list method
+        // Get legacy format data
         let partition_infos = self.list(request.limit.map(|l| l as usize)).await?;
         
-        // Convert to protobuf format
+        // Convert to protobuf format with dual status fields
-        let protobuf_partitions: Vec<crate::PartitionSummary> = partition_infos
+        let partitions: Vec<PartitionSummary> = partition_infos.into_iter()
-            .into_iter()
+            .map(|info| {
-            .map(|info| crate::PartitionSummary {
+                list_response_helpers::create_partition_summary(
-                partition_ref: Some(info.partition_ref),
+                    info.partition_ref,
-                status_code: info.current_status as i32,
+                    info.current_status,
-                status_name: info.current_status.to_display_string(),
+                    info.last_updated,
-                last_updated: info.last_updated,
+                    info.builds_count,
-                builds_count: info.builds_count as u32,
+                    info.invalidation_count,
-                last_successful_build: info.last_successful_build,
+                    info.last_successful_build,
-                invalidation_count: info.invalidation_count as u32,
+                )
             })
             .collect();
 
-        let total_count = protobuf_partitions.len() as u32;
+        // TODO: Implement proper pagination with offset and has_more
+        // For now, return simple response without full pagination support
+        let total_count = partitions.len() as u32;
+        let has_more = false; // This would be calculated based on actual total vs returned
 
         Ok(PartitionsListResponse {
-            partitions: protobuf_partitions,
+            partitions,
             total_count,
-            has_more: false, // TODO: Implement pagination
+            has_more,
         })
     }
 }
@@ -265,12 +323,12 @@ impl PartitionsRepository {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::event_log::mock::{create_mock_bel_query_engine, create_mock_bel_query_engine_with_events, test_events};
+    use crate::event_log::mock::{MockBuildEventLog, test_events};
 
     #[tokio::test]
     async fn test_partitions_repository_list_empty() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::new().await.unwrap());
-        let repo = PartitionsRepository::new(query_engine);
+        let repo = PartitionsRepository::new(mock_log);
         
         let partitions = repo.list(None).await.unwrap();
         assert!(partitions.is_empty());
@@ -291,8 +349,8 @@ mod tests {
             test_events::partition_status(Some(build_id.clone()), partition2.clone(), PartitionStatus::PartitionFailed, None),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = PartitionsRepository::new(query_engine.clone());
+        let repo = PartitionsRepository::new(mock_log);
         
         let partitions = repo.list(None).await.unwrap();
         assert_eq!(partitions.len(), 2);
@@ -313,14 +371,13 @@ mod tests {
         let partition = PartitionRef { str: "analytics/metrics".to_string() };
         
         let events = vec![
-            test_events::build_request_received(Some(build_id.clone()), vec![partition.clone()]),
             test_events::partition_status(Some(build_id.clone()), partition.clone(), PartitionStatus::PartitionRequested, None),
             test_events::partition_status(Some(build_id.clone()), partition.clone(), PartitionStatus::PartitionBuilding, None),
             test_events::partition_status(Some(build_id.clone()), partition.clone(), PartitionStatus::PartitionAvailable, None),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = PartitionsRepository::new(query_engine);
+        let repo = PartitionsRepository::new(mock_log);
         
         let result = repo.show(&partition.str).await.unwrap();
         assert!(result.is_some());
@@ -339,8 +396,8 @@ mod tests {
 
     #[tokio::test]
     async fn test_partitions_repository_show_nonexistent() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::new().await.unwrap());
-        let repo = PartitionsRepository::new(query_engine);
+        let repo = PartitionsRepository::new(mock_log);
         
         let result = repo.show("nonexistent/partition").await.unwrap();
         assert!(result.is_none());
@@ -356,8 +413,8 @@ mod tests {
             test_events::partition_status(Some(build_id.clone()), partition.clone(), PartitionStatus::PartitionAvailable, None),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = PartitionsRepository::new(query_engine.clone());
+        let repo = PartitionsRepository::new(mock_log.clone());
         
         // Invalidate the partition
         repo.invalidate(&partition.str, "Test invalidation".to_string(), build_id.clone()).await.unwrap();

databuild/repositories/tasks/mod.rs

@@ -1,14 +1,13 @@
 use crate::*;
-use crate::event_log::{BuildEventLogError, Result};
+use crate::event_log::{BuildEventLog, BuildEventLogError, Result};
-use crate::event_log::query_engine::BELQueryEngine;
+use crate::{TaskDetailResponse, TaskTimelineEvent as ServiceTaskTimelineEvent};
-use crate::{JobRunDetailResponse, JobRunTimelineEvent as ServiceTaskTimelineEvent};
 use std::sync::Arc;
 use std::collections::HashMap;
 use serde::Serialize;
 
 /// Repository for querying task (job run) data from the build event log
 pub struct TasksRepository {
-    query_engine: Arc<BELQueryEngine>,
+    event_log: Arc<dyn BuildEventLog>,
 }
 
 /// Summary of a task's execution
@@ -42,8 +41,8 @@ pub struct TaskEvent {
 
 impl TasksRepository {
     /// Create a new TasksRepository
-    pub fn new(query_engine: Arc<BELQueryEngine>) -> Self {
+    pub fn new(event_log: Arc<dyn BuildEventLog>) -> Self {
-        Self { query_engine }
+        Self { event_log }
     }
 
     /// List all tasks with their current status
@@ -52,14 +51,14 @@ impl TasksRepository {
     /// including their current status and execution details.
     pub async fn list(&self, limit: Option<usize>) -> Result<Vec<TaskInfo>> {
         // Get all events from the event log
-        let events = self.query_engine.get_events_in_range(0, i64::MAX).await?;
+        let events = self.event_log.get_events_in_range(0, i64::MAX).await?;
         
         let mut task_data: HashMap<String, TaskInfo> = HashMap::new();
         let mut task_cancellations: HashMap<String, String> = HashMap::new();
         
         // First pass: collect all task cancel events
         for event in &events {
-            if let Some(build_event::EventType::JobRunCancelEvent(tc_event)) = &event.event_type {
+            if let Some(build_event::EventType::TaskCancelEvent(tc_event)) = &event.event_type {
                 task_cancellations.insert(tc_event.job_run_id.clone(), tc_event.reason.clone());
             }
         }
@@ -151,7 +150,7 @@ impl TasksRepository {
     /// including all status changes and any cancellation events.
     pub async fn show(&self, job_run_id: &str) -> Result<Option<(TaskInfo, Vec<TaskEvent>)>> {
         // Get all events for this specific job run
-        let job_events = self.query_engine.get_job_run_events(job_run_id).await?;
+        let job_events = self.event_log.get_job_run_events(job_run_id).await?;
         
         if job_events.is_empty() {
             return Ok(None);
@@ -233,9 +232,9 @@ impl TasksRepository {
         }
         
         // Also check for task cancel events in all events
-        let all_events = self.query_engine.get_events_in_range(0, i64::MAX).await?;
+        let all_events = self.event_log.get_events_in_range(0, i64::MAX).await?;
         for event in all_events {
-            if let Some(build_event::EventType::JobRunCancelEvent(tc_event)) = &event.event_type {
+            if let Some(build_event::EventType::TaskCancelEvent(tc_event)) = &event.event_type {
                 if tc_event.job_run_id == job_run_id {
                     if let Some(task) = task_info.as_mut() {
                         task.cancelled = true;
@@ -296,14 +295,14 @@ impl TasksRepository {
         }
         
         // Use EventWriter to write the cancellation event
-        let event_writer = crate::event_log::writer::EventWriter::new(self.query_engine.clone());
+        let event_writer = crate::event_log::writer::EventWriter::new(self.event_log.clone());
         event_writer.cancel_task(build_request_id, job_run_id.to_string(), reason).await
     }
 
     /// Show detailed information about a specific task using protobuf response format
     /// 
     /// Returns the complete task details with dual status fields and timeline events.
-    pub async fn show_protobuf(&self, job_run_id: &str) -> Result<Option<JobRunDetailResponse>> {
+    pub async fn show_protobuf(&self, job_run_id: &str) -> Result<Option<TaskDetailResponse>> {
         // Get task info and timeline using existing show method
         if let Some((task_info, timeline)) = self.show(job_run_id).await? {
             // Convert timeline events to protobuf format
@@ -319,7 +318,7 @@ impl TasksRepository {
                 })
                 .collect();
 
-            let response = JobRunDetailResponse {
+            let response = TaskDetailResponse {
                 job_run_id: task_info.job_run_id,
                 job_label: task_info.job_label,
                 build_request_id: task_info.build_request_id,
@@ -344,16 +343,16 @@ impl TasksRepository {
 
     /// List tasks using protobuf response format with dual status fields
     /// 
-    /// Returns JobRunsListResponse protobuf message with JobRunSummary objects containing
+    /// Returns TasksListResponse protobuf message with TaskSummary objects containing
     /// status_code and status_name fields.
-    pub async fn list_protobuf(&self, request: JobRunsListRequest) -> Result<JobRunsListResponse> {
+    pub async fn list_protobuf(&self, request: TasksListRequest) -> Result<TasksListResponse> {
         // Get task info using existing list method
         let tasks = self.list(request.limit.map(|l| l as usize)).await?;
         
         // Convert to protobuf format
-        let protobuf_tasks: Vec<crate::JobRunSummary> = tasks
+        let protobuf_tasks: Vec<crate::TaskSummary> = tasks
             .into_iter()
-            .map(|task| crate::JobRunSummary {
+            .map(|task| crate::TaskSummary {
                 job_run_id: task.job_run_id,
                 job_label: task.job_label,
                 build_request_id: task.build_request_id,
@@ -371,7 +370,7 @@ impl TasksRepository {
         
         let total_count = protobuf_tasks.len() as u32;
         
-        Ok(JobRunsListResponse {
+        Ok(TasksListResponse {
             tasks: protobuf_tasks,
             total_count,
         })
@@ -381,12 +380,12 @@ impl TasksRepository {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::event_log::mock::{create_mock_bel_query_engine, create_mock_bel_query_engine_with_events, test_events};
+    use crate::event_log::mock::{MockBuildEventLog, test_events};
 
     #[tokio::test]
     async fn test_tasks_repository_list_empty() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::new().await.unwrap());
-        let repo = TasksRepository::new(query_engine);
+        let repo = TasksRepository::new(mock_log);
         
         let tasks = repo.list(None).await.unwrap();
         assert!(tasks.is_empty());
@@ -406,8 +405,8 @@ mod tests {
             test_events::job_event(Some(build_id.clone()), Some("task-2".to_string()), job_label.clone(), vec![partition.clone()], JobStatus::JobFailed),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = TasksRepository::new(query_engine);
+        let repo = TasksRepository::new(mock_log);
         
         let tasks = repo.list(None).await.unwrap();
         assert_eq!(tasks.len(), 2);
@@ -437,8 +436,8 @@ mod tests {
             test_events::job_event(Some(build_id.clone()), Some("task-123".to_string()), job_label.clone(), vec![partition.clone()], JobStatus::JobCompleted),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = TasksRepository::new(query_engine);
+        let repo = TasksRepository::new(mock_log);
         
         let result = repo.show("task-123").await.unwrap();
         assert!(result.is_some());
@@ -457,8 +456,8 @@ mod tests {
 
     #[tokio::test]
     async fn test_tasks_repository_show_nonexistent() {
-        let query_engine = create_mock_bel_query_engine().await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::new().await.unwrap());
-        let repo = TasksRepository::new(query_engine);
+        let repo = TasksRepository::new(mock_log);
         
         let result = repo.show("nonexistent-task").await.unwrap();
         assert!(result.is_none());
@@ -476,14 +475,14 @@ mod tests {
             test_events::job_event(Some(build_id.clone()), Some("task-456".to_string()), job_label.clone(), vec![partition.clone()], JobStatus::JobRunning),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = TasksRepository::new(query_engine.clone());
+        let repo = TasksRepository::new(mock_log.clone());
         
         // Cancel the task
         repo.cancel("task-456", "User requested cancellation".to_string(), build_id.clone()).await.unwrap();
         
         // Verify the cancellation was recorded
-        // Note: This test demonstrates the pattern, but the MockBELStorage would need
+        // Note: This test demonstrates the pattern, but the MockBuildEventLog would need
         // to be enhanced to properly store task cancel events for full verification
         
         // Try to cancel a non-existent task
@@ -503,8 +502,8 @@ mod tests {
             test_events::job_event(Some(build_id.clone()), Some("completed-task".to_string()), job_label.clone(), vec![partition.clone()], JobStatus::JobCompleted),
         ];
         
-        let query_engine = create_mock_bel_query_engine_with_events(events).await.unwrap();
+        let mock_log = Arc::new(MockBuildEventLog::with_events(events).await.unwrap());
-        let repo = TasksRepository::new(query_engine);
+        let repo = TasksRepository::new(mock_log);
         
         // Try to cancel the completed task - should fail
         let result = repo.cancel("completed-task", "Should fail".to_string(), build_id).await;

databuild/service/handlers.rs

@@ -79,7 +79,7 @@ pub async fn submit_build_request(
         .collect();
     
     let orchestrator = BuildOrchestrator::new(
-        service.query_engine.clone(),
+        service.event_log.clone(),
         build_request_id.clone(),
         requested_partitions,
     );
@@ -121,7 +121,7 @@ pub async fn get_build_status(
     State(service): State<ServiceState>,
     Path(BuildStatusRequest { build_request_id }): Path<BuildStatusRequest>,
 ) -> Result<Json<BuildDetailResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = crate::repositories::builds::BuildsRepository::new(service.query_engine.clone());
+    let repository = crate::repositories::builds::BuildsRepository::new(service.event_log.clone());
     
     match repository.show_protobuf(&build_request_id).await {
         Ok(Some(build_detail)) => {
@@ -183,7 +183,7 @@ pub async fn cancel_build_request(
         }),
     );
     
-    if let Err(e) = service.query_engine.append_event(event).await {
+    if let Err(e) = service.event_log.append_event(event).await {
         error!("Failed to log build request cancelled event: {}", e);
     }
     
@@ -205,7 +205,7 @@ pub async fn get_partition_status(
     Path(PartitionStatusRequest { partition_ref }): Path<PartitionStatusRequest>,
 ) -> Result<Json<PartitionStatusResponse>, (StatusCode, Json<ErrorResponse>)> {
     // Get latest partition status
-    let (status, last_updated) = match service.query_engine.get_latest_partition_status(&partition_ref).await {
+    let (status, last_updated) = match service.event_log.get_latest_partition_status(&partition_ref).await {
         Ok(Some((status, timestamp))) => (status, Some(timestamp)),
         Ok(None) => {
             // No partition events found - this is a legitimate 404
@@ -228,7 +228,7 @@ pub async fn get_partition_status(
     };
     
     // Get active builds for this partition
-    let build_requests = match service.query_engine.get_active_builds_for_partition(&partition_ref).await {
+    let build_requests = match service.event_log.get_active_builds_for_partition(&partition_ref).await {
         Ok(builds) => builds,
         Err(e) => {
             error!("Failed to get active builds for partition: {}", e);
@@ -261,7 +261,7 @@ pub async fn get_partition_events(
 ) -> Result<Json<PartitionEventsResponse>, (StatusCode, Json<ErrorResponse>)> {
     let decoded_partition_ref = base64_url_decode(&partition_ref).unwrap();
 
-    let events = match service.query_engine.get_partition_events(&decoded_partition_ref, None).await {
+    let events = match service.event_log.get_partition_events(&decoded_partition_ref, None).await {
         Ok(events) => events.into_iter().map(|e| {
             let (job_label, partition_ref, delegated_build_id) = extract_navigation_data(&e.event_type);
             BuildEventSummary {
@@ -344,7 +344,7 @@ async fn execute_build_request(
         .collect();
     
     let orchestrator = BuildOrchestrator::new(
-        service.query_engine.clone(),
+        service.event_log.clone(),
         build_request_id.clone(),
         requested_partitions,
     );
@@ -503,7 +503,7 @@ fn event_type_to_string(event_type: &Option<crate::build_event::EventType>) -> S
         Some(crate::build_event::EventType::DelegationEvent(_)) => "delegation".to_string(),
         Some(crate::build_event::EventType::JobGraphEvent(_)) => "job_graph".to_string(),
         Some(crate::build_event::EventType::PartitionInvalidationEvent(_)) => "partition_invalidation".to_string(),
-        Some(crate::build_event::EventType::JobRunCancelEvent(_)) => "task_cancel".to_string(),
+        Some(crate::build_event::EventType::TaskCancelEvent(_)) => "task_cancel".to_string(),
         Some(crate::build_event::EventType::BuildCancelEvent(_)) => "build_cancel".to_string(),
         None => "INVALID_EVENT_TYPE".to_string(), // Make this obvious rather than hiding it
     }
@@ -517,7 +517,7 @@ fn event_to_message(event_type: &Option<crate::build_event::EventType>) -> Strin
         Some(crate::build_event::EventType::DelegationEvent(event)) => event.message.clone(),
         Some(crate::build_event::EventType::JobGraphEvent(event)) => event.message.clone(),
         Some(crate::build_event::EventType::PartitionInvalidationEvent(event)) => event.reason.clone(),
-        Some(crate::build_event::EventType::JobRunCancelEvent(event)) => event.reason.clone(),
+        Some(crate::build_event::EventType::TaskCancelEvent(event)) => event.reason.clone(),
         Some(crate::build_event::EventType::BuildCancelEvent(event)) => event.reason.clone(),
         None => "INVALID_EVENT_NO_MESSAGE".to_string(), // Make this obvious
     }
@@ -549,7 +549,7 @@ fn extract_navigation_data(event_type: &Option<crate::build_event::EventType>) -
             let partition_ref = event.partition_ref.as_ref().map(|r| r.str.clone());
             (None, partition_ref, None)
         },
-        Some(crate::build_event::EventType::JobRunCancelEvent(_event)) => {
+        Some(crate::build_event::EventType::TaskCancelEvent(_event)) => {
             // Task cancel events reference job run IDs, which we could potentially navigate to
             (None, None, None)
         },
@@ -575,7 +575,7 @@ pub async fn list_build_requests(
         .min(100); // Cap at 100
     
     // Use repository with protobuf format
-    let builds_repo = BuildsRepository::new(service.query_engine.clone());
+    let builds_repo = BuildsRepository::new(service.event_log.clone());
     match builds_repo.list_protobuf(Some(limit as usize)).await {
         Ok(builds) => {
             let total_count = builds.len() as u32;
@@ -608,21 +608,27 @@ pub async fn list_partitions(
         .min(100); // Cap at 100
     
     // Use repository with protobuf format  
-    // TODO: Update PartitionsRepository to work with BELQueryEngine
+    let partitions_repo = PartitionsRepository::new(service.event_log.clone());
-    // let partitions_repo = PartitionsRepository::new(service.query_engine.clone());
     let request = PartitionsListRequest { 
         limit: Some(limit),
         offset: None,
         status_filter: None,
     };
     
-    // TODO: Implement with PartitionsRepository using BELQueryEngine
+    match partitions_repo.list_protobuf(request).await {
-    let response = PartitionsListResponse {
+        Ok(response) => {
-        partitions: vec![],
-        total_count: 0,
-        has_more: false,
-    };
             Ok(Json(response))
+        },
+        Err(e) => {
+            error!("Failed to list partitions: {}", e);
+            Err((
+                StatusCode::INTERNAL_SERVER_ERROR,
+                Json(ErrorResponse {
+                    error: format!("Failed to list partitions: {}", e),
+                }),
+            ))
+        }
+    }
 }
 
 // New unified protobuf-based handler for future migration
@@ -643,8 +649,7 @@ pub async fn list_partitions_unified(
         .and_then(|s| crate::PartitionStatus::from_display_string(s));
     
     // Use repository with protobuf response format
-    // TODO: Update PartitionsRepository to work with BELQueryEngine  
+    let repository = crate::repositories::partitions::PartitionsRepository::new(service.event_log.clone());
-    // let repository = crate::repositories::partitions::PartitionsRepository::new(service.query_engine.clone());
     
     let request = crate::PartitionsListRequest {
         limit: Some(limit),
@@ -652,22 +657,28 @@ pub async fn list_partitions_unified(
         status_filter: status_filter.map(|s| s.to_display_string()),
     };
     
-    // TODO: Implement with PartitionsRepository using BELQueryEngine
+    match repository.list_protobuf(request).await {
-    let response = PartitionsListResponse {
+        Ok(response) => {
-        partitions: vec![],
-        total_count: 0,
-        has_more: false,
-    };
             Ok(Json(response))
+        }
+        Err(e) => {
+            error!("Failed to list partitions: {}", e);
+            Err((
+                StatusCode::INTERNAL_SERVER_ERROR,
+                Json(ErrorResponse {
+                    error: format!("Failed to list partitions: {}", e),
+                }),
+            ))
+        }
+    }
 }
 
 pub async fn get_activity_summary(
     State(service): State<ServiceState>,
 ) -> Result<Json<ActivityApiResponse>, (StatusCode, Json<ErrorResponse>)> {
     // Build activity response using repositories to get dual status fields
-    let builds_repo = BuildsRepository::new(service.query_engine.clone());
+    let builds_repo = BuildsRepository::new(service.event_log.clone());
-    // TODO: Update PartitionsRepository to work with BELQueryEngine
+    let partitions_repo = PartitionsRepository::new(service.event_log.clone());
-    let partitions_repo = PartitionsRepository::new(service.query_engine.clone());
     
     // Get recent builds and partitions with dual status fields
     let recent_builds = builds_repo.list_protobuf(Some(5)).await.unwrap_or_else(|_| vec![]);
@@ -684,7 +695,7 @@ pub async fn get_activity_summary(
         });
     
     // Get activity counts (fallback to event log method for now)
-    let summary = service.query_engine.get_activity_summary().await.unwrap_or_else(|_| {
+    let summary = service.event_log.get_activity_summary().await.unwrap_or_else(|_| {
         crate::event_log::ActivitySummary {
             active_builds_count: 0,
             recent_builds: vec![],
@@ -734,7 +745,7 @@ pub async fn list_jobs(
     let search = params.get("search").map(|s| s.to_string());
     
     // Use repository with protobuf format
-    let jobs_repo = JobsRepository::new(service.query_engine.clone());
+    let jobs_repo = JobsRepository::new(service.event_log.clone());
     let request = JobsListRequest { 
         limit: Some(limit),
         search,
@@ -796,7 +807,7 @@ pub async fn get_job_metrics(
         LEFT JOIN job_run_durations jrd ON be.build_request_id = jrd.build_request_id
         WHERE je.job_label = ?";
     
-    let (success_rate, total_runs, avg_duration_ms) = match service.query_engine.execute_query(&metrics_query.replace("?", &format!("'{}'", decoded_label)).replace("?", &format!("'{}'", decoded_label))).await {
+    let (success_rate, total_runs, avg_duration_ms) = match service.event_log.execute_query(&metrics_query.replace("?", &format!("'{}'", decoded_label)).replace("?", &format!("'{}'", decoded_label))).await {
         Ok(result) if !result.rows.is_empty() => {
             let row = &result.rows[0];
             let completed_count: u32 = row[0].parse().unwrap_or(0);
@@ -838,7 +849,7 @@ pub async fn get_job_metrics(
         ORDER BY started_at DESC
         LIMIT 50";
     
-    let recent_runs = match service.query_engine.execute_query(&recent_runs_query.replace("?", &format!("'{}'", decoded_label)).replace("?", &format!("'{}'", decoded_label))).await {
+    let recent_runs = match service.event_log.execute_query(&recent_runs_query.replace("?", &format!("'{}'", decoded_label)).replace("?", &format!("'{}'", decoded_label))).await {
         Ok(result) => {
             result.rows.into_iter().map(|row| {
                 let build_request_id = row[0].clone();
@@ -910,7 +921,7 @@ pub async fn get_job_metrics(
         GROUP BY date(be.timestamp/1000000000, 'unixepoch')
         ORDER BY date DESC";
     
-    let daily_stats = match service.query_engine.execute_query(&daily_stats_query.replace("?", &format!("'{}'", decoded_label)).replace("?", &format!("'{}'", decoded_label))).await {
+    let daily_stats = match service.event_log.execute_query(&daily_stats_query.replace("?", &format!("'{}'", decoded_label)).replace("?", &format!("'{}'", decoded_label))).await {
         Ok(result) => {
             result.rows.into_iter().map(|row| {
                 let date = row[0].clone();
@@ -964,7 +975,7 @@ pub async fn get_partition_detail(
     State(service): State<ServiceState>,
     Path(PartitionDetailRequest { partition_ref }): Path<PartitionDetailRequest>,
 ) -> Result<Json<PartitionDetailResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = PartitionsRepository::new(service.query_engine.clone());
+    let repository = PartitionsRepository::new(service.event_log.clone());
     let decoded_partition_ref = base64_url_decode(&partition_ref).unwrap();
     
     match repository.show_protobuf(&decoded_partition_ref).await {
@@ -1027,7 +1038,7 @@ pub async fn invalidate_partition(
     Path(PartitionInvalidatePathRequest { partition_ref }): Path<PartitionInvalidatePathRequest>,
     Json(request): Json<InvalidatePartitionRequest>,
 ) -> Result<Json<PartitionInvalidateResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = PartitionsRepository::new(service.query_engine.clone());
+    let repository = PartitionsRepository::new(service.event_log.clone());
     
     match repository.invalidate(&partition_ref, request.reason.clone(), request.build_request_id).await {
         Ok(()) => Ok(Json(PartitionInvalidateResponse {
@@ -1052,7 +1063,7 @@ pub async fn list_partitions_repository(
     State(service): State<ServiceState>,
     Query(params): Query<HashMap<String, String>>,
 ) -> Result<Json<PartitionsListApiResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = PartitionsRepository::new(service.query_engine.clone());
+    let repository = PartitionsRepository::new(service.event_log.clone());
     let limit = params.get("limit").and_then(|s| s.parse().ok());
     
     let request = PartitionsListRequest { 
@@ -1094,17 +1105,17 @@ pub async fn list_partitions_repository(
 pub async fn list_tasks_repository(
     State(service): State<ServiceState>,
     Query(params): Query<HashMap<String, String>>,
-) -> Result<Json<JobRunsListApiResponse>, (StatusCode, Json<ErrorResponse>)> {
+) -> Result<Json<TasksListApiResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = TasksRepository::new(service.query_engine.clone());
+    let repository = TasksRepository::new(service.event_log.clone());
     let limit = params.get("limit").and_then(|s| s.parse().ok());
     
-    let request = JobRunsListRequest { limit };
+    let request = TasksListRequest { limit };
     
     match repository.list_protobuf(request).await {
         Ok(protobuf_response) => {
             let total_count = protobuf_response.total_count;
             
-            let api_response = JobRunsListApiResponse {
+            let api_response = TasksListApiResponse {
                 data: protobuf_response,
                 request_id: None, // TODO: add request ID tracking
                 pagination: Some(PaginationInfo {
@@ -1133,7 +1144,7 @@ pub async fn list_jobs_repository(
     State(service): State<ServiceState>,
     Query(params): Query<HashMap<String, String>>,
 ) -> Result<Json<JobsListApiResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = JobsRepository::new(service.query_engine.clone());
+    let repository = JobsRepository::new(service.event_log.clone());
     let limit = params.get("limit").and_then(|s| s.parse().ok());
     let search = params.get("search").map(|s| s.to_string());
     
@@ -1182,7 +1193,7 @@ pub async fn get_job_detail(
     Path(JobDetailRequest { label }): Path<JobDetailRequest>,
 ) -> Result<Json<JobDetailResponse>, (StatusCode, Json<ErrorResponse>)> {
     let job_label = base64_url_decode(&label).unwrap();
-    let repository = JobsRepository::new(service.query_engine.clone());
+    let repository = JobsRepository::new(service.event_log.clone());
     
     match repository.show_protobuf(&job_label).await {
         Ok(Some(protobuf_response)) => {
@@ -1236,11 +1247,11 @@ pub async fn get_job_detail(
 pub async fn list_tasks(
     State(service): State<ServiceState>,
     Query(params): Query<HashMap<String, String>>,
-) -> Result<Json<crate::JobRunsListResponse>, (StatusCode, Json<ErrorResponse>)> {
+) -> Result<Json<crate::TasksListResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = TasksRepository::new(service.query_engine.clone());
+    let repository = TasksRepository::new(service.event_log.clone());
     let limit = params.get("limit").and_then(|s| s.parse().ok());
     
-    let request = JobRunsListRequest { limit };
+    let request = TasksListRequest { limit };
     
     match repository.list_protobuf(request).await {
         Ok(response) => {
@@ -1268,13 +1279,13 @@ pub struct TaskDetailRequest {
 pub async fn get_task_detail(
     State(service): State<ServiceState>,
     Path(TaskDetailRequest { job_run_id }): Path<TaskDetailRequest>,
-) -> Result<Json<JobRunDetailResponse>, (StatusCode, Json<ErrorResponse>)> {
+) -> Result<Json<TaskDetailResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = TasksRepository::new(service.query_engine.clone());
+    let repository = TasksRepository::new(service.event_log.clone());
     
     match repository.show_protobuf(&job_run_id).await {
         Ok(Some(protobuf_response)) => {
-            let timeline_events: Vec<JobRunTimelineEvent> = protobuf_response.timeline.into_iter().map(|event| {
+            let timeline_events: Vec<TaskTimelineEvent> = protobuf_response.timeline.into_iter().map(|event| {
-                JobRunTimelineEvent {
+                TaskTimelineEvent {
                     timestamp: event.timestamp,
                     status_code: event.status_code,
                     status_name: event.status_name,
@@ -1284,7 +1295,7 @@ pub async fn get_task_detail(
                 }
             }).collect();
             
-            Ok(Json(JobRunDetailResponse {
+            Ok(Json(TaskDetailResponse {
                 job_run_id: protobuf_response.job_run_id,
                 job_label: protobuf_response.job_label,
                 build_request_id: protobuf_response.build_request_id,
@@ -1337,7 +1348,7 @@ pub async fn cancel_task(
     Path(TaskCancelPathRequest { job_run_id }): Path<TaskCancelPathRequest>,
     Json(request): Json<CancelTaskRequest>,
 ) -> Result<Json<TaskCancelResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = TasksRepository::new(service.query_engine.clone());
+    let repository = TasksRepository::new(service.event_log.clone());
     
     match repository.cancel(&job_run_id, request.reason.clone(), request.build_request_id).await {
         Ok(()) => Ok(Json(TaskCancelResponse {
@@ -1362,7 +1373,7 @@ pub async fn list_builds_repository(
     State(service): State<ServiceState>,
     Query(params): Query<HashMap<String, String>>,
 ) -> Result<Json<BuildsListApiResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = BuildsRepository::new(service.query_engine.clone());
+    let repository = BuildsRepository::new(service.event_log.clone());
     let limit = params.get("limit").and_then(|s| s.parse().ok());
     
     match repository.list_protobuf(limit).await {
@@ -1409,7 +1420,7 @@ pub async fn get_build_detail(
     State(service): State<ServiceState>,
     Path(BuildDetailRequest { build_request_id }): Path<BuildDetailRequest>,
 ) -> Result<Json<BuildDetailResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = BuildsRepository::new(service.query_engine.clone());
+    let repository = BuildsRepository::new(service.event_log.clone());
     
     match repository.show_protobuf(&build_request_id).await {
         Ok(Some(protobuf_response)) => {
@@ -1473,7 +1484,7 @@ pub async fn cancel_build_repository(
     Path(BuildCancelPathRequest { build_request_id }): Path<BuildCancelPathRequest>,
     Json(request): Json<CancelBuildRepositoryRequest>,
 ) -> Result<Json<BuildCancelRepositoryResponse>, (StatusCode, Json<ErrorResponse>)> {
-    let repository = BuildsRepository::new(service.query_engine.clone());
+    let repository = BuildsRepository::new(service.event_log.clone());
     
     match repository.cancel(&build_request_id, request.reason.clone()).await {
         Ok(()) => Ok(Json(BuildCancelRepositoryResponse {
@@ -1690,7 +1701,7 @@ pub async fn get_build_mermaid_diagram(
     info!("Generating mermaid diagram for build request {}", build_request_id);
     
     // Get build events for this build request
-    let events = match service.query_engine.get_build_request_events(&build_request_id, None).await {
+    let events = match service.event_log.get_build_request_events(&build_request_id, None).await {
         Ok(events) => events,
         Err(e) => {
             error!("Failed to get build events for {}: {}", build_request_id, e);

databuild/service/mod.rs

@@ -1,5 +1,5 @@
 use crate::*;
-use crate::event_log::BuildEventLogError;
+use crate::event_log::{BuildEventLog, BuildEventLogError, create_build_event_log};
 use aide::{
     axum::{
         routing::{get, post, delete},
@@ -20,7 +20,7 @@ pub mod handlers;
 
 #[derive(Clone)]
 pub struct BuildGraphService {
-    pub query_engine: Arc<crate::event_log::query_engine::BELQueryEngine>,
+    pub event_log: Arc<dyn BuildEventLog>,
     pub event_log_uri: String,
     pub active_builds: Arc<RwLock<HashMap<String, BuildRequestState>>>,
     pub graph_label: String,
@@ -150,8 +150,8 @@ pub struct JobsListApiResponse {
 }
 
 #[derive(Debug, Serialize, Deserialize, JsonSchema)]
-pub struct JobRunsListApiResponse {
+pub struct TasksListApiResponse {
-    pub data: crate::JobRunsListResponse,
+    pub data: crate::TasksListResponse,
     pub request_id: Option<String>,
     pub pagination: Option<PaginationInfo>,
 }
@@ -214,10 +214,10 @@ impl BuildGraphService {
         job_lookup_path: String,
         candidate_jobs: HashMap<String, String>,
     ) -> Result<Self, BuildEventLogError> {
-        let query_engine = crate::event_log::storage::create_bel_query_engine(event_log_uri).await?;
+        let event_log = create_build_event_log(event_log_uri).await?;
         
         Ok(Self {
-            query_engine,
+            event_log: Arc::from(event_log),
             event_log_uri: event_log_uri.to_string(),
             active_builds: Arc::new(RwLock::new(HashMap::new())),
             graph_label,
@@ -447,7 +447,7 @@ pub struct JobRepositorySummary {
     pub recent_builds: Vec<String>,
 }
 
-// Removed: JobDetailResponse, JobRunDetail, JobRunsListResponse, JobRunSummary (use crate:: proto versions)
+// Removed: JobDetailResponse, JobRunDetail, TasksListResponse, TaskSummary (use crate:: proto versions)
 
 // Removed: TaskDetailResponse and TaskTimelineEvent (use crate:: proto versions)
 

databuild/status_utils.rs

@@ -174,8 +174,8 @@ pub mod list_response_helpers {
         duration_ms: Option<i64>,
         cancelled: bool,
         message: String,
-    ) -> JobRunSummary {
+    ) -> TaskSummary {
-        JobRunSummary {
+        TaskSummary {
             job_run_id,
             job_label,
             build_request_id,

databuild/test/app/BUILD.bazel

@@ -1,15 +1,9 @@
 py_library(
     name = "job_src",
-    srcs = glob(["**/*.py"], exclude=["e2e_test_common.py"]),
+    srcs = glob(["**/*.py"]),
     visibility = ["//visibility:public"],
     deps = [
         "//databuild:py_proto",
         "//databuild/dsl/python:dsl",
     ],
 )
-
-py_library(
-    name = "e2e_test_common",
-    srcs = ["e2e_test_common.py"],
-    visibility = ["//visibility:public"],
-)

databuild/test/app/bazel/BUILD.bazel

@@ -65,14 +65,6 @@ py_test(
     ],
 )
 
-py_test(
-    name = "test_e2e",
-    srcs = ["test_e2e.py"],
-    data = [":bazel_graph.build"],
-    main = "test_e2e.py",
-    deps = ["//databuild/test/app:e2e_test_common"],
-)
-
 # Bazel-defined
 ## Graph
 databuild_graph(

databuild/test/app/bazel/graph/lookup.py

@@ -4,7 +4,7 @@ from collections import defaultdict
 import sys
 import json
 
-LABEL_BASE = "//databuild/test/app/bazel"
+LABEL_BASE = "//databuild/test/app"
 
 
 def lookup(raw_ref: str):

databuild/test/app/bazel/test_e2e.py

@@ -1,37 +0,0 @@
-#!/usr/bin/env python3
-"""
-End-to-end test for the bazel-defined test app.
-
-Tests the full pipeline: build execution -> output verification -> JSON validation.
-"""
-
-import os
-from databuild.test.app.e2e_test_common import DataBuildE2ETestBase
-
-
-class BazelE2ETest(DataBuildE2ETestBase):
-    """End-to-end test for the bazel-defined test app."""
-    
-    def test_end_to_end_execution(self):
-        """Test full end-to-end execution of the bazel graph."""
-        # Build possible paths for the bazel graph build binary
-        possible_paths = self.get_standard_runfiles_paths(
-            'databuild/test/app/bazel/bazel_graph.build'
-        )
-        
-        # Add fallback paths for local testing
-        possible_paths.extend([
-            'bazel-bin/databuild/test/app/bazel/bazel_graph.build',
-            './bazel_graph.build'
-        ])
-        
-        # Find the graph build binary
-        graph_build_path = self.find_graph_build_binary(possible_paths)
-        
-        # Execute and verify the graph build
-        self.execute_and_verify_graph_build(graph_build_path)
-
-
-if __name__ == '__main__':
-    import unittest
-    unittest.main()

databuild/test/app/dsl/BUILD.bazel

@@ -22,33 +22,3 @@ databuild_dsl_generator(
     deps = [":dsl_src"],
     visibility = ["//visibility:public"],
 )
-
-# Generate fresh DSL output for comparison testing
-genrule(
-    name = "generate_fresh_dsl",
-    outs = ["generated_fresh.tar"],
-    cmd_bash = """
-        # Create temporary directory for generation
-        mkdir -p temp_workspace/databuild/test/app/dsl
-        
-        # Set environment to generate to temp directory
-        export BUILD_WORKSPACE_DIRECTORY="temp_workspace"
-        
-        # Run the generator
-        $(location :graph.generate)
-        
-        # Create tar archive of generated files
-        if [ -d "temp_workspace/databuild/test/app/dsl/generated" ]; then
-            find temp_workspace/databuild/test/app/dsl/generated -exec touch -t 197001010000 {} +
-            tar -cf $@ -C temp_workspace/databuild/test/app/dsl/generated .
-        else
-            # Create empty tar if no files generated
-            tar -cf $@ -T /dev/null
-        fi
-        
-        # Clean up
-        rm -rf temp_workspace
-    """,
-    tools = [":graph.generate"],
-    visibility = ["//visibility:public"],
-)

databuild/test/app/dsl/claude-generated-dsl-test.md

@@ -1,9 +0,0 @@
-
-We can't write a direct `bazel test` for the DSL generated graph, because:
-
-1. Bazel doesn't allow you to `bazel run graph.generate` to generate a BUILD.bazel that will be used in the same build.
-2. We don't want to leak test generation into the graph generation code (since tests here are app specific)
-
-Instead, we need to use a two phase process, where we rely on the graph to already be generated here, which will contain a test, such that `bazel test //...` will give us recall over generated source as well. This implies that this generated source is going to be checked in to git (gasp, I know), and we need a mechanism to ensure it stays up to date. To achieve this, we'll create a test that asserts that the contents of the `generated` dir is the exact same as the output of a new run of `graph.generate`.
-
-Our task is to implement this test that asserts equality between the two, e.g. the target could depend on `graph.generate`, and in the test run it and md5 the results, comparing it to the md5 of the existing generated dir.

databuild/test/app/dsl/generated/BUILD.bazel

@@ -1,71 +0,0 @@
-load("@databuild//databuild:rules.bzl", "databuild_job", "databuild_graph")
-
-# Generated by DataBuild DSL - do not edit manually
-# This file is generated in a subdirectory to avoid overwriting the original BUILD.bazel
-
-py_binary(
-    name = "aggregate_color_votes_binary",
-    srcs = ["aggregate_color_votes.py"],
-    main = "aggregate_color_votes.py",
-    deps = ["@@//databuild/test/app/dsl:dsl_src"],
-)
-
-databuild_job(
-    name = "aggregate_color_votes",
-    binary = ":aggregate_color_votes_binary",
-)
-py_binary(
-    name = "color_vote_report_calc_binary",
-    srcs = ["color_vote_report_calc.py"],
-    main = "color_vote_report_calc.py",
-    deps = ["@@//databuild/test/app/dsl:dsl_src"],
-)
-
-databuild_job(
-    name = "color_vote_report_calc",
-    binary = ":color_vote_report_calc_binary",
-)
-py_binary(
-    name = "ingest_color_votes_binary",
-    srcs = ["ingest_color_votes.py"],
-    main = "ingest_color_votes.py",
-    deps = ["@@//databuild/test/app/dsl:dsl_src"],
-)
-
-databuild_job(
-    name = "ingest_color_votes",
-    binary = ":ingest_color_votes_binary",
-)
-py_binary(
-    name = "trailing_color_votes_binary",
-    srcs = ["trailing_color_votes.py"],
-    main = "trailing_color_votes.py",
-    deps = ["@@//databuild/test/app/dsl:dsl_src"],
-)
-
-databuild_job(
-    name = "trailing_color_votes",
-    binary = ":trailing_color_votes_binary",
-)
-
-py_binary(
-    name = "dsl_job_lookup",
-    srcs = ["dsl_job_lookup.py"],
-    deps = ["@@//databuild/test/app/dsl:dsl_src"],
-)
-
-databuild_graph(
-    name = "dsl_graph",
-    jobs = ["aggregate_color_votes", "color_vote_report_calc", "ingest_color_votes", "trailing_color_votes"],
-    lookup = ":dsl_job_lookup",
-    visibility = ["//visibility:public"],
-)
-
-# Create tar archive of generated files for testing
-genrule(
-    name = "existing_generated",
-    srcs = glob(["*.py", "BUILD.bazel"]),
-    outs = ["existing_generated.tar"],
-    cmd = "mkdir -p temp && cp $(SRCS) temp/ && find temp -exec touch -t 197001010000 {} + && tar -cf $@ -C temp .",
-    visibility = ["//visibility:public"],
-)

databuild/test/app/dsl/generated/aggregate_color_votes.py

@@ -1,58 +0,0 @@
-#!/usr/bin/env python3
-"""
-Generated job script for AggregateColorVotes.
-"""
-
-import sys
-import json
-from databuild.test.app.dsl.graph import AggregateColorVotes
-from databuild.proto import PartitionRef, JobConfigureResponse, to_dict
-
-
-def parse_outputs_from_args(args: list[str]) -> list:
-    """Parse partition output references from command line arguments."""
-    outputs = []
-    for arg in args:
-        # Find which output type can deserialize this partition reference
-        for output_type in AggregateColorVotes.output_types:
-            try:
-                partition = output_type.deserialize(arg)
-                outputs.append(partition)
-                break
-            except ValueError:
-                continue
-        else:
-            raise ValueError(f"No output type in AggregateColorVotes can deserialize partition ref: {arg}")
-    
-    return outputs
-
-
-if __name__ == "__main__":
-    if len(sys.argv) < 2:
-        raise Exception(f"Invalid command usage")
-
-    command = sys.argv[1]
-    job_instance = AggregateColorVotes()
-    
-    if command == "config":
-        # Parse output partition references as PartitionRef objects (for Rust wrapper)
-        output_refs = [PartitionRef(str=raw_ref) for raw_ref in sys.argv[2:]]
-        
-        # Also parse them into DSL partition objects (for DSL job.config())
-        outputs = parse_outputs_from_args(sys.argv[2:])
-        
-        # Call job's config method - returns list[JobConfig]
-        configs = job_instance.config(outputs)
-        
-        # Wrap in JobConfigureResponse and serialize using to_dict()
-        response = JobConfigureResponse(configs=configs)
-        print(json.dumps(to_dict(response)))
-        
-    elif command == "exec":
-        # The exec method expects a JobConfig but the Rust wrapper passes args
-        # For now, let the DSL job handle the args directly 
-        # TODO: This needs to be refined based on actual Rust wrapper interface
-        job_instance.exec(*sys.argv[2:])
-        
-    else:
-        raise Exception(f"Invalid command `{sys.argv[1]}`")

databuild/test/app/dsl/generated/color_vote_report_calc.py

@@ -1,58 +0,0 @@
-#!/usr/bin/env python3
-"""
-Generated job script for ColorVoteReportCalc.
-"""
-
-import sys
-import json
-from databuild.test.app.dsl.graph import ColorVoteReportCalc
-from databuild.proto import PartitionRef, JobConfigureResponse, to_dict
-
-
-def parse_outputs_from_args(args: list[str]) -> list:
-    """Parse partition output references from command line arguments."""
-    outputs = []
-    for arg in args:
-        # Find which output type can deserialize this partition reference
-        for output_type in ColorVoteReportCalc.output_types:
-            try:
-                partition = output_type.deserialize(arg)
-                outputs.append(partition)
-                break
-            except ValueError:
-                continue
-        else:
-            raise ValueError(f"No output type in ColorVoteReportCalc can deserialize partition ref: {arg}")
-    
-    return outputs
-
-
-if __name__ == "__main__":
-    if len(sys.argv) < 2:
-        raise Exception(f"Invalid command usage")
-
-    command = sys.argv[1]
-    job_instance = ColorVoteReportCalc()
-    
-    if command == "config":
-        # Parse output partition references as PartitionRef objects (for Rust wrapper)
-        output_refs = [PartitionRef(str=raw_ref) for raw_ref in sys.argv[2:]]
-        
-        # Also parse them into DSL partition objects (for DSL job.config())
-        outputs = parse_outputs_from_args(sys.argv[2:])
-        
-        # Call job's config method - returns list[JobConfig]
-        configs = job_instance.config(outputs)
-        
-        # Wrap in JobConfigureResponse and serialize using to_dict()
-        response = JobConfigureResponse(configs=configs)
-        print(json.dumps(to_dict(response)))
-        
-    elif command == "exec":
-        # The exec method expects a JobConfig but the Rust wrapper passes args
-        # For now, let the DSL job handle the args directly 
-        # TODO: This needs to be refined based on actual Rust wrapper interface
-        job_instance.exec(*sys.argv[2:])
-        
-    else:
-        raise Exception(f"Invalid command `{sys.argv[1]}`")

databuild/test/app/dsl/generated/dsl_job_lookup.py

@@ -1,53 +0,0 @@
-#!/usr/bin/env python3
-"""
-Generated job lookup for DataBuild DSL graph.
-Maps partition patterns to job targets.
-"""
-
-import sys
-import re
-import json
-from collections import defaultdict
-
-
-# Mapping from partition patterns to job targets
-JOB_MAPPINGS = {
-    r"daily_color_votes/(?P<data_date>\d{4}-\d{2}-\d{2})/(?P<color>[^/]+)": "//databuild/test/app/dsl/generated:ingest_color_votes",
-    r"color_votes_1m/(?P<data_date>\d{4}-\d{2}-\d{2})/(?P<color>[^/]+)": "//databuild/test/app/dsl/generated:trailing_color_votes",
-    r"color_votes_1w/(?P<data_date>\d{4}-\d{2}-\d{2})/(?P<color>[^/]+)": "//databuild/test/app/dsl/generated:trailing_color_votes",
-    r"daily_votes/(?P<data_date>\d{4}-\d{2}-\d{2})": "//databuild/test/app/dsl/generated:aggregate_color_votes",
-    r"votes_1w/(?P<data_date>\d{4}-\d{2}-\d{2})": "//databuild/test/app/dsl/generated:aggregate_color_votes",
-    r"votes_1m/(?P<data_date>\d{4}-\d{2}-\d{2})": "//databuild/test/app/dsl/generated:aggregate_color_votes",
-    r"color_vote_report/(?P<data_date>\d{4}-\d{2}-\d{2})/(?P<color>[^/]+)": "//databuild/test/app/dsl/generated:color_vote_report_calc",
-}
-
-
-def lookup_job_for_partition(partition_ref: str) -> str:
-    """Look up which job can build the given partition reference."""
-    for pattern, job_target in JOB_MAPPINGS.items():
-        if re.match(pattern, partition_ref):
-            return job_target
-    
-    raise ValueError(f"No job found for partition: {partition_ref}")
-
-
-def main():
-    if len(sys.argv) < 2:
-        print("Usage: job_lookup.py <partition_ref> [partition_ref...]", file=sys.stderr)
-        sys.exit(1)
-    
-    results = defaultdict(list)
-    try:
-        for partition_ref in sys.argv[1:]:
-            job_target = lookup_job_for_partition(partition_ref)
-            results[job_target].append(partition_ref)
-        
-        # Output the results as JSON (matching existing lookup format)
-        print(json.dumps(dict(results)))
-    except ValueError as e:
-        print(f"ERROR: {e}", file=sys.stderr)
-        sys.exit(1)
-
-
-if __name__ == "__main__":
-    main()

databuild/test/app/dsl/generated/ingest_color_votes.py

@@ -1,58 +0,0 @@
-#!/usr/bin/env python3
-"""
-Generated job script for IngestColorVotes.
-"""
-
-import sys
-import json
-from databuild.test.app.dsl.graph import IngestColorVotes
-from databuild.proto import PartitionRef, JobConfigureResponse, to_dict
-
-
-def parse_outputs_from_args(args: list[str]) -> list:
-    """Parse partition output references from command line arguments."""
-    outputs = []
-    for arg in args:
-        # Find which output type can deserialize this partition reference
-        for output_type in IngestColorVotes.output_types:
-            try:
-                partition = output_type.deserialize(arg)
-                outputs.append(partition)
-                break
-            except ValueError:
-                continue
-        else:
-            raise ValueError(f"No output type in IngestColorVotes can deserialize partition ref: {arg}")
-    
-    return outputs
-
-
-if __name__ == "__main__":
-    if len(sys.argv) < 2:
-        raise Exception(f"Invalid command usage")
-
-    command = sys.argv[1]
-    job_instance = IngestColorVotes()
-    
-    if command == "config":
-        # Parse output partition references as PartitionRef objects (for Rust wrapper)
-        output_refs = [PartitionRef(str=raw_ref) for raw_ref in sys.argv[2:]]
-        
-        # Also parse them into DSL partition objects (for DSL job.config())
-        outputs = parse_outputs_from_args(sys.argv[2:])
-        
-        # Call job's config method - returns list[JobConfig]
-        configs = job_instance.config(outputs)
-        
-        # Wrap in JobConfigureResponse and serialize using to_dict()
-        response = JobConfigureResponse(configs=configs)
-        print(json.dumps(to_dict(response)))
-        
-    elif command == "exec":
-        # The exec method expects a JobConfig but the Rust wrapper passes args
-        # For now, let the DSL job handle the args directly 
-        # TODO: This needs to be refined based on actual Rust wrapper interface
-        job_instance.exec(*sys.argv[2:])
-        
-    else:
-        raise Exception(f"Invalid command `{sys.argv[1]}`")

databuild/test/app/dsl/generated/trailing_color_votes.py

@@ -1,58 +0,0 @@
-#!/usr/bin/env python3
-"""
-Generated job script for TrailingColorVotes.
-"""
-
-import sys
-import json
-from databuild.test.app.dsl.graph import TrailingColorVotes
-from databuild.proto import PartitionRef, JobConfigureResponse, to_dict
-
-
-def parse_outputs_from_args(args: list[str]) -> list:
-    """Parse partition output references from command line arguments."""
-    outputs = []
-    for arg in args:
-        # Find which output type can deserialize this partition reference
-        for output_type in TrailingColorVotes.output_types:
-            try:
-                partition = output_type.deserialize(arg)
-                outputs.append(partition)
-                break
-            except ValueError:
-                continue
-        else:
-            raise ValueError(f"No output type in TrailingColorVotes can deserialize partition ref: {arg}")
-    
-    return outputs
-
-
-if __name__ == "__main__":
-    if len(sys.argv) < 2:
-        raise Exception(f"Invalid command usage")
-
-    command = sys.argv[1]
-    job_instance = TrailingColorVotes()
-    
-    if command == "config":
-        # Parse output partition references as PartitionRef objects (for Rust wrapper)
-        output_refs = [PartitionRef(str=raw_ref) for raw_ref in sys.argv[2:]]
-        
-        # Also parse them into DSL partition objects (for DSL job.config())
-        outputs = parse_outputs_from_args(sys.argv[2:])
-        
-        # Call job's config method - returns list[JobConfig]
-        configs = job_instance.config(outputs)
-        
-        # Wrap in JobConfigureResponse and serialize using to_dict()
-        response = JobConfigureResponse(configs=configs)
-        print(json.dumps(to_dict(response)))
-        
-    elif command == "exec":
-        # The exec method expects a JobConfig but the Rust wrapper passes args
-        # For now, let the DSL job handle the args directly 
-        # TODO: This needs to be refined based on actual Rust wrapper interface
-        job_instance.exec(*sys.argv[2:])
-        
-    else:
-        raise Exception(f"Invalid command `{sys.argv[1]}`")

databuild/test/app/dsl/generated_test/BUILD.bazel

@@ -1,7 +0,0 @@
-py_test(
-    name = "test_e2e",
-    srcs = ["test_e2e.py"],
-    data = ["//databuild/test/app/dsl/generated:dsl_graph.build"],
-    main = "test_e2e.py",
-    deps = ["//databuild/test/app:e2e_test_common"],
-)

databuild/test/app/dsl/generated_test/test_e2e.py

@@ -1,37 +0,0 @@
-#!/usr/bin/env python3
-"""
-End-to-end test for the DSL-generated test app.
-
-Tests the full pipeline: build execution -> output verification -> JSON validation.
-"""
-
-import os
-from databuild.test.app.e2e_test_common import DataBuildE2ETestBase
-
-
-class DSLGeneratedE2ETest(DataBuildE2ETestBase):
-    """End-to-end test for the DSL-generated test app."""
-    
-    def test_end_to_end_execution(self):
-        """Test full end-to-end execution of the DSL-generated graph."""
-        # Build possible paths for the DSL-generated graph build binary
-        possible_paths = self.get_standard_runfiles_paths(
-            'databuild/test/app/dsl/generated/dsl_graph.build'
-        )
-        
-        # Add fallback paths for local testing
-        possible_paths.extend([
-            'bazel-bin/databuild/test/app/dsl/generated/dsl_graph.build',
-            './dsl_graph.build'
-        ])
-        
-        # Find the graph build binary
-        graph_build_path = self.find_graph_build_binary(possible_paths)
-        
-        # Execute and verify the graph build
-        self.execute_and_verify_graph_build(graph_build_path)
-
-
-if __name__ == '__main__':
-    import unittest
-    unittest.main()

databuild/test/app/dsl/test/BUILD.bazel

@@ -73,15 +73,3 @@ py_test(
         "//databuild/test/app/dsl:dsl_src",
     ],
 )
-
-# DSL generation consistency test
-py_test(
-    name = "test_dsl_generation_consistency",
-    srcs = ["test_dsl_generation_consistency.py"],
-    main = "test_dsl_generation_consistency.py",
-    data = [
-        "//databuild/test/app/dsl:generate_fresh_dsl",
-        "//databuild/test/app/dsl/generated:existing_generated",
-    ],
-    deps = [],
-)

databuild/test/app/dsl/test/test_dsl_generation_consistency.py

@@ -1,105 +0,0 @@
-#!/usr/bin/env python3
-"""
-Test that verifies the generated DSL code is up-to-date.
-
-This test ensures that the checked-in generated directory contents match
-exactly what would be produced by a fresh run of graph.generate.
-"""
-
-import hashlib
-import os
-import subprocess
-import tempfile
-import unittest
-from pathlib import Path
-
-
-class TestDSLGenerationConsistency(unittest.TestCase):
-    def setUp(self):
-        # Find the test runfiles directory to locate tar files
-        runfiles_dir = os.environ.get("RUNFILES_DIR")
-        if runfiles_dir:
-            self.runfiles_root = Path(runfiles_dir) / "_main"
-        else:
-            # Fallback for development - not expected to work in this case
-            self.fail("RUNFILES_DIR not set - test must be run via bazel test")
-
-    def _compute_tar_hash(self, tar_path: Path) -> str:
-        """Compute MD5 hash of a tar file's contents."""
-        if not tar_path.exists():
-            self.fail(f"Tar file not found: {tar_path}")
-            
-        with open(tar_path, "rb") as f:
-            content = f.read()
-            return hashlib.md5(content).hexdigest()
-
-    def _extract_and_list_tar(self, tar_path: Path) -> set:
-        """Extract tar file and return set of file paths and their content hashes."""
-        if not tar_path.exists():
-            return set()
-            
-        result = subprocess.run([
-            "tar", "-tf", str(tar_path)
-        ], capture_output=True, text=True)
-        
-        if result.returncode != 0:
-            self.fail(f"Failed to list tar contents: {result.stderr}")
-            
-        return set(result.stdout.strip().split('\n')) if result.stdout.strip() else set()
-
-    def test_generated_code_is_up_to_date(self):
-        """Test that the existing generated tar matches the fresh generated tar."""
-        
-        # Find the tar files from data dependencies
-        existing_tar = self.runfiles_root / "databuild/test/app/dsl/generated/existing_generated.tar"
-        fresh_tar = self.runfiles_root / "databuild/test/app/dsl/generated_fresh.tar"
-        
-        # Compute hashes of both tar files
-        existing_hash = self._compute_tar_hash(existing_tar)
-        fresh_hash = self._compute_tar_hash(fresh_tar)
-        
-        # Compare hashes
-        if existing_hash != fresh_hash:
-            # Provide detailed diff information
-            existing_files = self._extract_and_list_tar(existing_tar)
-            fresh_files = self._extract_and_list_tar(fresh_tar)
-            
-            only_in_existing = existing_files - fresh_files
-            only_in_fresh = fresh_files - existing_files
-            
-            error_msg = [
-                "Generated DSL code is out of date!",
-                f"Existing tar hash: {existing_hash}",
-                f"Fresh tar hash: {fresh_hash}",
-                "",
-                "To fix this, run:",
-                "  bazel run //databuild/test/app/dsl:graph.generate",
-                ""
-            ]
-            
-            if only_in_existing:
-                error_msg.extend([
-                    "Files only in existing generated code:",
-                    *[f"  - {f}" for f in sorted(only_in_existing)],
-                    ""
-                ])
-            
-            if only_in_fresh:
-                error_msg.extend([
-                    "Files only in fresh generated code:",
-                    *[f"  + {f}" for f in sorted(only_in_fresh)],
-                    ""
-                ])
-                
-            common_files = existing_files & fresh_files
-            if common_files:
-                error_msg.extend([
-                    f"Common files: {len(common_files)}",
-                    "This suggests files have different contents.",
-                ])
-            
-            self.fail("\n".join(error_msg))
-
-
-if __name__ == "__main__":
-    unittest.main()

databuild/test/app/e2e_test_common.py

@@ -1,103 +0,0 @@
-#!/usr/bin/env python3
-"""
-Common end-to-end test logic for DataBuild test apps.
-
-Provides shared functionality for testing both bazel-defined and DSL-generated graphs.
-"""
-
-import json
-import os
-import shutil
-import subprocess
-import time
-import unittest
-from pathlib import Path
-from typing import List, Optional
-
-
-class DataBuildE2ETestBase(unittest.TestCase):
-    """Base class for DataBuild end-to-end tests."""
-    
-    def setUp(self):
-        """Set up test environment."""
-        self.output_dir = Path("/tmp/data/color_votes_1w/2025-09-01/red")
-        self.output_file = self.output_dir / "data.json"
-        self.partition_ref = "color_votes_1w/2025-09-01/red"
-        
-        # Clean up any existing test data
-        if self.output_dir.exists():
-            shutil.rmtree(self.output_dir)
-    
-    def tearDown(self):
-        """Clean up test environment."""
-        if self.output_dir.exists():
-            shutil.rmtree(self.output_dir)
-    
-    def find_graph_build_binary(self, possible_paths: List[str]) -> str:
-        """Find the graph.build binary from a list of possible paths."""
-        graph_build_path = None
-        for path in possible_paths:
-            if os.path.exists(path):
-                graph_build_path = path
-                break
-        
-        self.assertIsNotNone(graph_build_path, 
-                           f"Graph build binary not found in any of: {possible_paths}")
-        return graph_build_path
-    
-    def execute_and_verify_graph_build(self, graph_build_path: str) -> None:
-        """Execute the graph build and verify the results."""
-        # Record start time for file modification check
-        start_time = time.time()
-        
-        # Execute the graph build (shell script)
-        result = subprocess.run(
-            ["bash", graph_build_path, self.partition_ref],
-            capture_output=True,
-            text=True
-        )
-        
-        # Verify execution succeeded
-        self.assertEqual(result.returncode, 0, 
-                        f"Graph build failed with stderr: {result.stderr}")
-        
-        # Verify output file was created
-        self.assertTrue(self.output_file.exists(), 
-                       f"Output file {self.output_file} was not created")
-        
-        # Verify file was created recently (within 60 seconds)
-        file_mtime = os.path.getmtime(self.output_file)
-        time_diff = file_mtime - start_time
-        self.assertGreaterEqual(time_diff, -1,  # Allow 1 second clock skew
-                               f"File appears to be too old: {time_diff} seconds")
-        self.assertLessEqual(time_diff, 60,
-                            f"File creation took too long: {time_diff} seconds")
-        
-        # Verify file contains valid JSON
-        with open(self.output_file, 'r') as f:
-            content = f.read()
-        
-        try:
-            data = json.loads(content)
-        except json.JSONDecodeError as e:
-            self.fail(f"Output file does not contain valid JSON: {e}")
-        
-        # Basic sanity check on JSON structure
-        self.assertIsInstance(data, (dict, list), 
-                             "JSON should be an object or array")
-    
-    def get_standard_runfiles_paths(self, relative_path: str) -> List[str]:
-        """Get standard list of possible runfiles paths for a binary."""
-        runfiles_dir = os.environ.get("RUNFILES_DIR")
-        test_srcdir = os.environ.get("TEST_SRCDIR")
-        
-        possible_paths = []
-        if runfiles_dir:
-            possible_paths.append(os.path.join(runfiles_dir, '_main', relative_path))
-            possible_paths.append(os.path.join(runfiles_dir, relative_path))
-        
-        if test_srcdir:
-            possible_paths.append(os.path.join(test_srcdir, '_main', relative_path))
-            possible_paths.append(os.path.join(test_srcdir, relative_path))
-        
-        return possible_paths

design/build-event-log.md

@@ -1,72 +1,34 @@
 
 # Build Event Log (BEL)
-Purpose: Store build events and provide efficient cross-graph coordination via a minimal, append-only event stream.
+Purpose: Store build events and define views summarizing databuild application state, like partition catalog, build
+status summary, job run statistics, etc.
 
 ## Architecture
 - Uses [event sourcing](https://martinfowler.com/eaaDev/EventSourcing.html) / 
   [CQRS](https://www.wikipedia.org/wiki/cqrs) philosophy.
 - BELs are only ever written to by graph processes (e.g. CLI or service), not the jobs themselves.
-- **Three-layer architecture:**
+- BEL uses only two types of tables:
-  1. **Storage Layer**: Append-only event storage with sequential scanning
+  - The root event table, with event ID, timestamp, message, event type, and ID fields for related event types.
-  2. **Query Engine Layer**: App-layer aggregation for entity queries (partition status, build summaries, etc.)
+  - Type-specific event tables (e.g. task even, partition event, build request event, etc).
-  3. **Client Layer**: CLI, Service, Dashboard consuming aggregated views
+- This makes it easy to support multiple backends (SQLite, postgres, and delta tables are supported initially).
-- **Cross-graph coordination** via minimal `GraphService` API that supports event streaming since a given index
+- Exposes an access layer that mediates writes, and which exposes entity-specific repositories for reads.
-- Storage backends focus on efficient append + sequential scan operations (file-based, SQLite, Postgres, Delta Lake)
 
 ## Correctness Strategy
 - Access layer will evaluate events requested to be written, returning an error if the event is not a correct next. 
   state based on the involved component's governing state diagram.
 - Events are versioned, with each versions' schemas stored in [`databuild.proto`](../databuild/databuild.proto).
 
-## Storage Layer Interface
+## Write Interface
-Minimal append-only interface optimized for sequential scanning:
+See [trait definition](../databuild/event_log/mod.rs).
 
-```rust
+## Read Repositories
-#[async_trait]
+There are repositories for the following entities:
-trait BELStorage {
+- Builds
-    async fn append_event(&self, event: BuildEvent) -> Result<i64>; // returns event index
+- Jobs
-    async fn list_events(&self, since_idx: i64, filter: EventFilter) -> Result<EventPage>;
+- Partitions
-}
+- Tasks
-```
 
-Where `EventFilter` is defined in `databuild.proto` as:
+Generally the following verbs are available for each:
-```protobuf
+- Show
-message EventFilter {
+- List
-  repeated string partition_refs = 1;        // Exact partition matches
+- Cancel
-  repeated string partition_patterns = 2;    // Glob patterns like "data/users/*"
-  repeated string job_labels = 3;           // Job-specific events
-  repeated string task_ids = 4;             // Task run events
-  repeated string build_request_ids = 5;    // Build-specific events
-}
-```
-
-## Query Engine Interface  
-App-layer aggregation that scans storage layer events:
-
-```rust
-struct BELQueryEngine {
-    storage: Box<dyn BELStorage>,
-    partition_status_cache: Option<PartitionStatusCache>,
-}
-
-impl BELQueryEngine {
-    async fn get_latest_partition_status(&self, partition_ref: &str) -> Result<Option<PartitionStatus>>;
-    async fn get_active_builds_for_partition(&self, partition_ref: &str) -> Result<Vec<String>>;
-    async fn get_build_request_summary(&self, build_id: &str) -> Result<BuildRequestSummary>;
-    async fn list_build_requests(&self, limit: u32, offset: u32, status_filter: Option<BuildRequestStatus>) -> Result<Vec<BuildRequestSummary>>;
-}
-```
-
-## Cross-Graph Coordination
-Graphs coordinate via the `GraphService` API for efficient event streaming:
-
-```rust
-trait GraphService {
-    async fn list_events(&self, since_idx: i64, filter: EventFilter) -> Result<EventPage>;
-}
-```
-
-This enables:
-- **Event-driven reactivity**: Downstream graphs react within seconds of upstream partition availability
-- **Efficient subscriptions**: Only scan events for relevant partitions
-- **Reliable coordination**: HTTP polling avoids event-loss issues of streaming APIs

design/service.md

@@ -1,11 +1,6 @@
 
 # Service
-Purpose: Enable centrally hostable and human-consumable interface for databuild applications, plus efficient cross-graph coordination.
+Purpose: Enable centrally hostable and human-consumable interface for databuild applications.
-
-## Architecture
-The service provides two primary capabilities:
-1. **Human Interface**: Web dashboard and HTTP API for build management and monitoring
-2. **Cross-Graph Coordination**: `GraphService` API enabling efficient event-driven coordination between DataBuild instances
 
 ## Correctness Strategy
 - Rely on databuild.proto, call same shared code in core
@@ -13,48 +8,6 @@ The service provides two primary capabilities:
   - Core -- databuild.proto --> service -- openapi --> web app
 - No magic strings (how? protobuf doesn't have consts. enums values? code gen over yaml?)
 
-## Cross-Graph Coordination
-Services expose the `GraphService` API for cross-graph dependency management:
-
-```rust
-trait GraphService {
-    async fn list_events(&self, since_idx: i64, filter: EventFilter) -> Result<EventPage>;
-}
-```
-
-### Cross-Graph Usage Pattern
-```rust
-// Downstream graph subscribing to upstream partitions
-struct UpstreamDependency {
-    service_url: String,           // e.g., "https://upstream-databuild.corp.com"
-    partition_patterns: Vec<String>, // e.g., ["data/users/*", "ml/models/prod/*"]
-    last_sync_idx: i64,
-}
-
-// Periodic sync of relevant upstream events
-async fn sync_upstream_events(upstream: &UpstreamDependency) -> Result<()> {
-    let client = GraphServiceClient::new(&upstream.service_url);
-    let filter = EventFilter {
-        partition_patterns: upstream.partition_patterns.clone(),
-        ..Default::default()
-    };
-    
-    let events = client.list_events(upstream.last_sync_idx, filter).await?;
-    
-    // Process partition availability events for immediate job triggering
-    for event in events.events {
-        if let EventType::PartitionEvent(pe) = event.event_type {
-            if pe.status_code == PartitionStatus::PartitionAvailable {
-                trigger_dependent_jobs(&pe.partition_ref).await?;
-            }
-        }
-    }
-    
-    upstream.last_sync_idx = events.next_idx;
-    Ok(())
-}
-```
-
 ## API
 The purpose of the API is to enable remote, programmatic interaction with databuild applications, and to host endpoints 
 needed by the [web app](#web-app).

design/triggers.md

@@ -0,0 +1,56 @@
+
+# Triggers
+Purpose: to enable simple but powerful declarative specification of what data should be built.
+
+## Correctness Strategy
+- Wants + TTLs
+- ...?
+
+## Wants
+Wants cause graphs to try to build the wanted partitions until a) the partitions are live or b) the TTL runs out. Wants 
+can trigger a callback on TTL expiry, enabling SLA-like behavior. Wants are recorded in the [BEL](./build-event-log.md),
+so they can be queried and viewed in the web app, linking to build requests triggered by a given want, enabling
+answering of the "why doesn't this partition exist yet?" question.
+
+### Unwants
+You can also unwant partitions, which overrides all wants of those partitions prior to the unwant timestamp. This is 
+primarily to enable the "data source is now disabled" style feature practically necessary in many data platforms. 
+
+### Virtual Partitions & External Data
+Essentially all data teams consume some external data source, and late arriving data is the rule more than the 
+exception. Virtual partitions are a way to model external data that is not produced by a graph. For all intents and 
+purposes, these are standard partitions, the only difference is that the job that "produces" them doesn't actually 
+do any ETL, it just assesses external data sufficiency and emits a "partition live" event when its ready to be consumed.
+
+## Triggers
+
+## Taints
+- Mechanism for invalidating existing partitions (e.g. we know bad data went into this, need to stop consumers from
+  using it)
+
+---
+
+- Purpose
+  - Every useful data application has triggering to ensure data is built on schedule
+- Philosophy
+  - Opinionated strategy plus escape hatches
+- Taints
+  
+- Two strategies
+  - Basic: cron triggered scripts that return partitions
+    - Bazel: target with `cron`, `executable` fields, optional `partition_patterns` field to constrain 
+  - Declarative: want-based, wants cause build requests to be continually retried until the wanted 
+    partitions are live, or running a `want_failed` script if it times out (e.g. SLA breach) 
+    - +want and -want
+      - +want declares want for 1+ partitions with a timeout, recorded to the [build event log](./build-event-log.md)
+      - -want invalidates all past wants of specified partitions (but not future; doesn't impact non-specified 
+        partitions)
+        - Their primary purpose is to prevent an SLA breach alarm when a datasource is disabled, etc.
+      - Need graph preconditions? And concept of external/virtual partitions or readiness probes?
+        - Virtual partitions: allow graphs to say "precondition failed"; can be created in BEL, created via want or
+          cron trigger? (e.g. want strategy continually tries to resolve the external data, creating a virtual 
+          partition once it can find it; cron just runs the script when its triggered)
+        - Readiness probes don't fit the paradigm, feel too imperative.
+
+
+

design/wants.md

@@ -1,287 +0,0 @@
-# Wants System
-
-Purpose: Enable declarative specification of data requirements with SLA tracking, cross-graph coordination, and efficient build triggering while maintaining atomic build semantics.
-
-## Overview
-
-The wants system unifies all build requests (manual, scheduled, triggered) under a single declarative model where:
-- **Wants declare intent** via events in the [build event log](./build-event-log.md)
-- **Builds reactively satisfy** what's currently possible with atomic semantics
-- **Monitoring identifies gaps** between declared wants and delivered partitions
-- **Cross-graph coordination** happens via the `GraphService` API
-
-## Architecture
-
-### Core Components
-
-1. **PartitionWantEvent**: Declarative specification of data requirements
-2. **Build Evaluation**: Reactive logic that attempts to satisfy wants when possible
-3. **SLA Monitoring**: External system that queries for expired wants
-4. **Cross-Graph Coordination**: Event-driven dependency management across DataBuild instances
-
-### Want Event Schema
-
-Defined in `databuild.proto`:
-
-```protobuf
-message PartitionWantEvent {
-  string partition_ref = 1;               // Partition being requested
-  int64 created_at = 2;                   // Server time when want registered
-  int64 data_timestamp = 3;               // Business time this partition represents
-  optional uint64 ttl_seconds = 4;        // Give up after this long (from created_at)
-  optional uint64 sla_seconds = 5;        // SLA violation after this long (from data_timestamp)
-  repeated string external_dependencies = 6; // Cross-graph dependencies
-  string want_id = 7;                     // Unique identifier
-  WantSource source = 8;                  // How this want was created
-}
-
-message WantSource {
-  oneof source_type {
-    CliManual cli_manual = 1;             // Manual CLI request
-    DashboardManual dashboard_manual = 2;  // Manual dashboard request  
-    Scheduled scheduled = 3;              // Scheduled/triggered job
-    ApiRequest api_request = 4;           // External API call
-  }
-}
-```
-
-## Want Lifecycle
-
-### 1. Want Registration
-
-All build requests become wants:
-
-```rust
-// CLI: databuild build data/users/2024-01-01
-PartitionWantEvent {
-    partition_ref: "data/users/2024-01-01",
-    created_at: now(),
-    data_timestamp: None, // These must be set explicitly in the request
-    ttl_seconds: None,
-    sla_seconds: None,
-    external_dependencies: vec![], // no externally sourced data necessary
-    want_id: generate_uuid(),
-    source: WantSource { ... },
-}
-
-// Scheduled pipeline: Daily analytics
-PartitionWantEvent {
-    partition_ref: "analytics/daily/2024-01-01", 
-    created_at: now(),
-    data_timestamp: parse_date("2024-01-01"),
-    ttl_seconds: Some(365 * 24 * 3600), // Keep trying for 1 year
-    sla_seconds: Some(9 * 3600),        // Expected by 9am (9hrs after data_timestamp)
-    external_dependencies: vec!["data/users/2024-01-01"],
-    want_id: "daily-analytics-2024-01-01",
-    source: WantSource { ... },
-}
-```
-
-### 2. Build Evaluation
-
-DataBuild continuously evaluates build opportunities:
-
-```rust
-async fn evaluate_build_opportunities(&self) -> Result<Option<BuildRequest>> {
-    let now = current_timestamp_nanos();
-    
-    // Get wants that haven't exceeded TTL
-    let active_wants = self.get_non_expired_wants(now).await?;
-    
-    // Filter to wants where external dependencies are satisfied
-    let buildable_partitions = active_wants.into_iter()
-        .filter(|want| self.external_dependencies_satisfied(want))
-        .map(|want| want.partition_ref)
-        .collect();
-        
-    if buildable_partitions.is_empty() { return Ok(None); }
-    
-    // Create atomic build request for all currently buildable partitions
-    Ok(Some(BuildRequest { 
-        requested_partitions: buildable_partitions,
-        reason: "satisfying_active_wants".to_string(),
-    }))
-}
-```
-
-### 3. Build Triggers
-
-Builds are triggered on:
-- **New want registration**: Check if newly wanted partitions are immediately buildable
-- **External partition availability**: Check if any blocked wants are now unblocked
-- **Manual trigger**: Force re-evaluation (for debugging)
-
-## Cross-Graph Coordination
-### GraphService API
-Graphs expose events for cross-graph coordination:
-
-```rust
-trait GraphService {
-    async fn list_events(&self, since_idx: i64, filter: EventFilter) -> Result<EventPage>;
-}
-```
-
-Where `EventFilter` supports partition patterns for efficient subscriptions:
-
-```protobuf
-message EventFilter {
-  repeated string partition_refs = 1;        // Exact partition matches  
-  repeated string partition_patterns = 2;    // Glob patterns like "data/users/*"
-  repeated string job_labels = 3;           // Job-specific events
-  repeated string task_ids = 4;             // Task run events
-  repeated string build_request_ids = 5;    // Build-specific events
-}
-```
-
-### Upstream Dependencies
-
-Downstream graphs subscribe to upstream events:
-
-```rust
-struct UpstreamDependency {
-    service_url: String,           // "https://upstream-databuild.corp.com"
-    partition_patterns: Vec<String>, // ["data/users/*", "ml/models/prod/*"]
-    last_sync_idx: i64,
-}
-
-// Periodic sync of upstream events
-async fn sync_upstream_events(upstream: &UpstreamDependency) -> Result<()> {
-    let client = GraphServiceClient::new(&upstream.service_url);
-    let filter = EventFilter {
-        partition_patterns: upstream.partition_patterns.clone(),
-        ..Default::default()
-    };
-    
-    let events = client.list_events(upstream.last_sync_idx, filter).await?;
-    
-    // Process partition availability events
-    for event in events.events {
-        if let EventType::PartitionEvent(pe) = event.event_type {
-            if pe.status_code == PartitionStatus::PartitionAvailable {
-                // Trigger local build evaluation
-                trigger_build_evaluation().await?;
-            }
-        }
-    }
-    
-    upstream.last_sync_idx = events.next_idx;
-    Ok(())
-}
-```
-
-## SLA Monitoring and TTL Management
-
-### SLA Violations
-
-External monitoring systems query for SLA violations:
-
-```sql
--- Find SLA violations (for alerting)
-SELECT * FROM partition_want_events w
-WHERE w.sla_seconds IS NOT NULL 
-AND (w.data_timestamp + (w.sla_seconds * 1000000000)) < ?  -- now
-AND NOT EXISTS (
-    SELECT 1 FROM partition_events p 
-    WHERE p.partition_ref = w.partition_ref 
-    AND p.status_code = ?  -- PartitionAvailable
-)
-```
-
-### TTL Expiration
-
-Wants with expired TTLs are excluded from build evaluation:
-
-```sql
--- Get active (non-expired) wants
-SELECT * FROM partition_want_events w
-WHERE (w.ttl_seconds IS NULL OR w.created_at + (w.ttl_seconds * 1000000000) > ?) -- now
-AND NOT EXISTS (
-    SELECT 1 FROM partition_events p 
-    WHERE p.partition_ref = w.partition_ref 
-    AND p.status_code = ?  -- PartitionAvailable
-)
-```
-
-## Example Scenarios
-
-### Scenario 1: Daily Analytics Pipeline
-
-```
-1. 6:00 AM: Daily trigger creates want for analytics/daily/2024-01-01
-   - SLA: 9:00 AM (3 hours after data_timestamp of midnight)
-   - TTL: 1 year (keep trying for historical data)
-   - External deps: ["data/users/2024-01-01"]
-
-2. 6:01 AM: Build evaluation runs, data/users/2024-01-01 missing
-   - No build request generated
-
-3. 8:30 AM: Upstream publishes data/users/2024-01-01
-   - Cross-graph sync detects availability
-   - Build evaluation triggered
-   - BuildRequest[analytics/daily/2024-01-01] succeeds
-
-4. Result: Analytics available at 8:45 AM, within SLA
-```
-
-### Scenario 2: Late Data with SLA Miss
-
-```  
-1. 6:00 AM: Want created for analytics/daily/2024-01-01 (SLA: 9:00 AM)
-2. 9:30 AM: SLA monitoring detects violation, sends alert
-3. 11:00 AM: Upstream data finally arrives
-4. 11:01 AM: Build evaluation triggers, analytics built
-5. Result: Late delivery logged, but data still processed
-```
-
-### Scenario 3: Manual CLI Build
-
-```
-1. User: databuild build data/transform/urgent
-2. Want created with short TTL (30 min) and SLA (5 min)  
-3. Build evaluation: dependencies available, immediate build
-4. Result: Fast feedback for interactive use
-```
-
-## Benefits
-
-### Unified Build Model
-- All builds (manual, scheduled, triggered) use same want mechanism
-- Complete audit trail in build event log
-- Consistent SLA tracking across all build types
-
-### Event-Driven Efficiency  
-- Builds only triggered when dependencies change
-- Cross-graph coordination via efficient event streaming
-- No polling for task readiness within builds
-
-### Atomic Build Semantics
-- Individual build requests remain all-or-nothing
-- Fast failure provides immediate feedback
-- Partial progress via multiple build requests over time
-
-### Flexible SLA Management
-- Separate business expectations (SLA) from operational limits (TTL)
-- External monitoring with clear blame assignment
-- Automatic cleanup of stale wants
-
-### Cross-Graph Scalability
-- Reliable HTTP-based coordination (no message loss)
-- Efficient filtering via partition patterns
-- Decentralized architecture with clear boundaries
-
-## Implementation Notes
-
-### Build Event Log Integration
-- Wants are stored as events in the BEL for consistency
-- Same query interfaces used for wants and build coordination
-- Event-driven architecture throughout
-
-### Service Integration  
-- GraphService API exposed via HTTP for cross-graph coordination
-- Dashboard integration for manual want creation
-- External SLA monitoring via BEL queries
-
-### CLI Integration
-- CLI commands create manual wants with appropriate TTLs
-- Immediate build evaluation for interactive feedback  
-- Standard build request execution path

examples/podcast_reviews/BUILD.bazel

@@ -178,6 +178,12 @@ py_binary(
     ],
 )
 
+# Legacy test job (kept for compatibility)
+databuild_job(
+    name = "test_job",
+    binary = ":test_job_binary",
+)
+
 # Test target
 py_binary(
     name = "test_jobs",

plans/18-bel-refactor.md

@@ -1,304 +0,0 @@
-# BEL Refactoring to 3-Tier Architecture
-
-## Overview
-
-This plan restructures DataBuild's Build Event Log (BEL) access layer from the current monolithic trait to a clean 3-tier architecture as described in [design/build-event-log.md](../design/build-event-log.md). This refactoring creates clear separation of concerns and simplifies the codebase by removing complex storage backends.
-
-## Current State Analysis
-
-The current BEL implementation (`databuild/event_log/mod.rs`) has a single `BuildEventLog` trait that mixes:
-- Low-level storage operations (`append_event`, `get_events_in_range`)
-- High-level aggregation queries (`list_build_requests`, `get_activity_summary`) 
-- Application-specific logic (`get_latest_partition_status`, `get_active_builds_for_partition`)
-
-This creates several problems:
-- Storage backends must implement complex aggregation logic
-- No clear separation between storage and business logic
-- Difficult to extend with new query patterns
-- Delta Lake implementation adds unnecessary complexity
-
-## Target Architecture
-
-### 1. Storage Layer: `BELStorage` Trait
-Minimal append-only interface optimized for sequential scanning:
-
-```rust
-#[async_trait]
-pub trait BELStorage: Send + Sync {
-    /// Append a single event, returns the sequential index
-    async fn append_event(&self, event: BuildEvent) -> Result<i64>;
-    
-    /// List events with filtering, starting from a given index
-    async fn list_events(&self, since_idx: i64, filter: EventFilter) -> Result<EventPage>;
-    
-    /// Initialize storage backend (create tables, etc.)
-    async fn initialize(&self) -> Result<()>;
-}
-
-#[derive(Debug, Clone)]
-pub struct EventPage {
-    pub events: Vec<BuildEvent>,
-    pub next_idx: i64,
-    pub has_more: bool,
-}
-```
-
-### 2. Query Engine Layer: `BELQueryEngine` 
-App-layer aggregation that scans storage events:
-
-```rust
-pub struct BELQueryEngine {
-    storage: Arc<dyn BELStorage>,
-}
-
-impl BELQueryEngine {
-    pub fn new(storage: Arc<dyn BELStorage>) -> Self {
-        Self { storage }
-    }
-    
-    /// Get latest status for a partition by scanning recent events
-    pub async fn get_latest_partition_status(&self, partition_ref: &str) -> Result<Option<PartitionStatus>>;
-    
-    /// Get all build requests that are currently building a partition
-    pub async fn get_active_builds_for_partition(&self, partition_ref: &str) -> Result<Vec<String>>;
-    
-    /// Get summary of a build request by aggregating its events
-    pub async fn get_build_request_summary(&self, build_id: &str) -> Result<BuildRequestSummary>;
-    
-    /// List build requests with pagination and filtering
-    pub async fn list_build_requests(&self, request: BuildsListRequest) -> Result<BuildsListResponse>;
-    
-    /// Get activity summary for dashboard
-    pub async fn get_activity_summary(&self) -> Result<ActivityResponse>;
-}
-```
-
-### 3. Client Layer: Repository Pattern
-Clean interfaces for CLI, Service, and Dashboard (unchanged from current):
-
-```rust
-// Existing repositories continue to work, but now use BELQueryEngine
-pub struct PartitionsRepository {
-    query_engine: Arc<BELQueryEngine>,
-}
-
-pub struct BuildsRepository {
-    query_engine: Arc<BELQueryEngine>,
-}
-```
-
-## Implementation Plan
-
-### Phase 1: Create Storage Layer Interface
-
-1. **Define New Storage Trait**
-   ```rust
-   // In databuild/event_log/storage.rs
-   pub trait BELStorage { /* as defined above */ }
-   
-   pub fn create_bel_storage(uri: &str) -> Result<Box<dyn BELStorage>>;
-   ```
-
-2. **Add EventFilter to Protobuf**
-   ```protobuf
-   // In databuild/databuild.proto
-   message EventFilter {
-     repeated string partition_refs = 1;
-     repeated string partition_patterns = 2;  
-     repeated string job_labels = 3;
-     repeated string task_ids = 4;
-     repeated string build_request_ids = 5;
-   }
-   
-   message EventPage {
-     repeated BuildEvent events = 1;
-     int64 next_idx = 2;
-     bool has_more = 3;
-   }
-   ```
-
-3. **Implement SQLite Storage Backend**
-   ```rust
-   // In databuild/event_log/sqlite_storage.rs
-   pub struct SqliteBELStorage {
-       pool: sqlx::SqlitePool,
-   }
-   
-   impl BELStorage for SqliteBELStorage {
-       async fn append_event(&self, event: BuildEvent) -> Result<i64> {
-           // Simple INSERT returning rowid
-           let serialized = serde_json::to_string(&event)?;
-           let row_id = sqlx::query("INSERT INTO build_events (event_data) VALUES (?)")
-               .bind(serialized)
-               .execute(&self.pool)
-               .await?
-               .last_insert_rowid();
-           Ok(row_id)
-       }
-       
-       async fn list_events(&self, since_idx: i64, filter: EventFilter) -> Result<EventPage> {
-           // Efficient sequential scan with filtering
-           // Build WHERE clause based on filter criteria
-           // Return paginated results
-       }
-   }
-   ```
-
-### Phase 2: Create Query Engine Layer
-
-1. **Implement BELQueryEngine**
-   ```rust
-   // In databuild/event_log/query_engine.rs
-   impl BELQueryEngine {
-       pub async fn get_latest_partition_status(&self, partition_ref: &str) -> Result<Option<PartitionStatus>> {
-           // Scan recent partition events to determine current status
-           let filter = EventFilter {
-               partition_refs: vec![partition_ref.to_string()],
-               ..Default::default()
-           };
-           
-           let events = self.storage.list_events(0, filter).await?;
-           self.aggregate_partition_status(&events.events)
-       }
-       
-       async fn aggregate_partition_status(&self, events: &[BuildEvent]) -> Result<Option<PartitionStatus>> {
-           // Walk through events chronologically to determine final partition status
-           // Return the most recent status
-       }
-   }
-   ```
-
-2. **Implement All Current Query Methods**
-   - Port all methods from current `BuildEventLog` trait
-   - Use event scanning and aggregation instead of complex SQL queries
-   - Keep same return types for compatibility
-
-### Phase 3: Migrate Existing Code
-
-1. **Update Repository Constructors**
-   ```rust
-   // Old: PartitionsRepository::new(Arc<dyn BuildEventLog>)
-   // New: PartitionsRepository::new(Arc<BELQueryEngine>)
-   
-   impl PartitionsRepository {
-       pub fn new(query_engine: Arc<BELQueryEngine>) -> Self {
-           Self { query_engine }
-       }
-       
-       pub async fn list_protobuf(&self, request: PartitionsListRequest) -> Result<PartitionsListResponse> {
-           self.query_engine.list_build_requests(request).await
-       }
-   }
-   ```
-
-2. **Update CLI and Service Initialization**
-   ```rust
-   // In CLI main.rs and service mod.rs
-   let storage = create_bel_storage(&event_log_uri).await?;
-   let query_engine = Arc::new(BELQueryEngine::new(storage));
-   
-   let partitions_repo = PartitionsRepository::new(query_engine.clone());
-   let builds_repo = BuildsRepository::new(query_engine.clone());
-   ```
-
-### Phase 4: Remove Legacy Components
-
-1. **Remove Delta Lake Implementation**
-   ```rust
-   // Delete databuild/event_log/delta.rs
-   // Remove delta dependencies from MODULE.bazel
-   // Remove delta:// support from create_build_event_log()
-   ```
-
-2. **Deprecate Old BuildEventLog Trait**
-   ```rust
-   // Mark as deprecated, keep for backwards compatibility during transition
-   #[deprecated(note = "Use BELQueryEngine and BELStorage instead")]
-   pub trait BuildEventLog { /* existing implementation */ }
-   ```
-
-3. **Update Factory Function**
-   ```rust
-   // In databuild/event_log/mod.rs
-   pub async fn create_build_event_log(uri: &str) -> Result<Arc<BELQueryEngine>> {
-       let storage = if uri == "stdout" {
-           Arc::new(stdout::StdoutBELStorage::new()) as Arc<dyn BELStorage>
-       } else if uri.starts_with("sqlite://") {
-           let path = &uri[9..];
-           let storage = sqlite_storage::SqliteBELStorage::new(path).await?;
-           storage.initialize().await?;
-           Arc::new(storage) as Arc<dyn BELStorage>
-       } else if uri.starts_with("postgres://") {
-           let storage = postgres_storage::PostgresBELStorage::new(uri).await?;
-           storage.initialize().await?;
-           Arc::new(storage) as Arc<dyn BELStorage>
-       } else {
-           return Err(BuildEventLogError::ConnectionError(
-               format!("Unsupported build event log URI: {}", uri)
-           ));
-       };
-       
-       Ok(Arc::new(BELQueryEngine::new(storage)))
-   }
-   ```
-
-### Phase 5: Final Cleanup
-
-1. **Remove Legacy Implementations**
-   - Delete complex aggregation logic from existing storage backends
-   - Simplify remaining backends to implement only new `BELStorage` trait
-   - Remove deprecated `BuildEventLog` trait
-
-2. **Update Documentation**
-   - Update design docs to reflect new architecture
-   - Create migration guide for external users
-   - Update code examples and README
-
-## Benefits of 3-Tier Architecture
-
-### ✅ **Simplified Codebase**
-- Removes complex Delta Lake dependencies
-- Storage backends focus only on append + scan operations
-- Clear separation between storage and business logic
-
-### ✅ **Better Maintainability** 
-- Single SQLite implementation for most use cases
-- Query logic centralized in one place
-- Easier to debug and test each layer independently
-
-### ✅ **Future-Ready Foundation**
-- Clean foundation for wants system (next phase)
-- Easy to add new storage backends when needed
-- Query engine ready for cross-graph coordination APIs
-
-### ✅ **Performance Benefits**
-- Eliminates complex SQL joins in storage layer
-- Enables sequential scanning optimizations
-- Cleaner separation allows targeted optimizations
-
-## Success Criteria
-
-### Phase 1-2: Foundation
-- [ ] Storage layer trait compiles and tests pass
-- [ ] SQLite storage backend supports append + list operations
-- [ ] Query engine provides same functionality as current BEL trait
-- [ ] EventFilter protobuf types generate correctly
-
-### Phase 3-4: Migration  
-- [ ] All repositories work with new query engine
-- [ ] CLI and service use new architecture
-- [ ] Existing functionality unchanged from user perspective
-- [ ] Delta Lake implementation removed
-
-### Phase 5: Completion
-- [ ] Legacy BEL trait removed
-- [ ] Performance meets or exceeds current implementation
-- [ ] Documentation updated for new architecture
-- [ ] Codebase simplified and maintainable
-
-## Risk Mitigation
-
-1. **Gradual Migration**: Implement new architecture alongside existing code
-2. **Feature Parity**: Ensure all existing functionality works before removing old code  
-3. **Performance Testing**: Benchmark new implementation against current performance
-4. **Simple First**: Start with SQLite-only implementation, add complexity later as needed

plans/19-client-server-cli.md

@@ -1,183 +0,0 @@
-# Client-Server CLI Architecture
-
-## Overview
-
-This plan transforms DataBuild's CLI from a monolithic in-process execution model to a Bazel-style client-server architecture. The CLI becomes a thin client that delegates all operations to a persistent service process, enabling better resource management and build coordination.
-
-## Current State Analysis
-
-The current CLI (`databuild/cli/main.rs`) directly:
-- Creates event log connections
-- Runs analysis and execution in-process  
-- Spawns bazel processes directly
-- No coordination between concurrent CLI invocations
-
-This creates several limitations:
-- No coordination between concurrent builds
-- Multiple BEL connections from concurrent CLI calls
-- Each CLI process spawns separate bazel execution
-- No shared execution environment for builds
-
-## Target Architecture
-
-### Bazel-Style Client-Server Model
-
-**CLI (Thin Client)**:
-- Auto-starts service if not running
-- Delegates all operations to service via HTTP
-- Streams progress back to user
-- Auto-shuts down idle service
-
-**Service (Persistent Process)**:
-- Maintains single BEL connection
-- Coordinates builds across multiple CLI calls
-- Manages bazel execution processes
-- Auto-shuts down after idle timeout
-
-## Implementation Plan
-
-### Phase 1: Service Foundation
-
-1. **Extend Current Service for CLI Operations**
-   - Add new endpoints to handle CLI build requests
-   - Move analysis and execution logic from CLI to service
-   - Service maintains orchestrator state and coordinates builds
-   - Add real-time progress streaming for CLI consumption
-
-2. **Add CLI-Specific API Endpoints**
-   - `/api/v1/cli/build` - Handle build requests from CLI
-   - `/api/v1/cli/builds/{id}/progress` - Stream build progress via Server-Sent Events
-   - Request/response types for CLI build operations
-   - Background vs foreground build support
-
-3. **Add Service Auto-Management**
-   - Service tracks last activity timestamp
-   - Configurable auto-shutdown timeout (default: 5 minutes)
-   - Service monitors for idle state and gracefully shuts down
-   - Activity tracking includes API calls and active builds
-
-4. **Service Port Management**
-   - Service attempts to bind to preferred port (e.g., 8080)
-   - If port unavailable, tries next available port in range
-   - Service writes actual port to lockfile/pidfile for CLI discovery
-   - CLI reads port from lockfile to connect to running service
-   - Cleanup lockfile on service shutdown
-
-### Phase 2: Thin CLI Implementation
-
-1. **New CLI Main Function**
-   - Replace existing main with service delegation logic
-   - Parse arguments and determine target service operation
-   - Handle service connection and auto-start logic
-   - Preserve existing CLI interface and help text
-
-2. **Service Client Implementation**
-   - HTTP client for communicating with service
-   - Auto-start service if not already running
-   - Health check and connection retry logic
-   - Progress streaming for real-time build feedback
-
-3. **Build Command via Service**
-   - Parse build arguments and create service request
-   - Submit build request to service endpoint
-   - Stream progress updates for foreground builds
-   - Return immediately for background builds with build ID
-
-### Phase 3: Repository Commands via Service
-
-1. **Delegate Repository Commands to Service**
-   - Partition, build, job, and task commands go through service
-   - Use existing service API endpoints where available
-   - Maintain same output formats (table, JSON) as current CLI
-   - Preserve all existing functionality and options
-
-2. **Service Client Repository Methods**
-   - Client methods for each repository operation
-   - Handle pagination, filtering, and formatting options
-   - Error handling and appropriate HTTP status code handling
-   - URL encoding for partition references and other parameters
-
-### Phase 4: Complete Migration
-
-1. **Remove Old CLI Implementation**
-   - Delete existing `databuild/cli/main.rs` implementation
-   - Remove in-process analysis and execution logic
-   - Clean up CLI-specific dependencies that are no longer needed
-   - Update build configuration to use new thin client only
-
-2. **Service Integration Testing**
-   - End-to-end testing of CLI-to-service communication
-   - Verify all existing CLI functionality works through service
-   - Performance testing to ensure no regression
-   - Error handling validation for various failure modes
-
-### Phase 5: Integration and Testing
-
-1. **Environment Variable Support**
-   - `DATABUILD_SERVICE_URL` for custom service locations
-   - `DATABUILD_SERVICE_TIMEOUT` for auto-shutdown configuration
-   - Existing BEL environment variables passed to service
-   - Clear precedence rules for configuration sources
-
-2. **Error Handling and User Experience**
-   - Service startup timeout and clear error messages
-   - Connection failure handling with fallback suggestions
-   - Health check logic to verify service readiness
-   - Graceful handling of service unavailability
-
-## Benefits of Client-Server Architecture
-
-### ✅ **Build Coordination**
-- Multiple CLI calls share same service instance
-- Coordination between concurrent builds
-- Single BEL connection eliminates connection conflicts
-
-### ✅ **Resource Management**
-- Auto-shutdown prevents resource leaks
-- Service manages persistent connections
-- Better isolation between CLI and build execution
-- Shared bazel execution environment
-
-### ✅ **Improved User Experience**
-- Background builds with `--background` flag
-- Real-time progress streaming
-- Consistent build execution environment
-
-### ✅ **Simplified Architecture**
-- Single execution path through service
-- Cleaner separation of concerns
-- Reduced code duplication
-
-### ✅ **Future-Ready Foundation**
-- Service architecture prepared for additional coordination features
-- HTTP API foundation for programmatic access
-- Clear separation of concerns between client and execution
-
-## Success Criteria
-
-### Phase 1-2: Service Foundation
-- [ ] Service can handle CLI build requests
-- [ ] Service auto-shutdown works correctly
-- [ ] Service port management and discovery works
-- [ ] New CLI can start and connect to service
-- [ ] Build requests execute with same functionality as current CLI
-
-### Phase 3-4: Complete Migration
-- [ ] All CLI commands work via service delegation
-- [ ] Repository commands (partitions, builds, etc.) work via HTTP API
-- [ ] Old CLI implementation completely removed
-- [ ] Error handling provides clear user feedback
-
-### Phase 5: Polish
-- [ ] Multiple concurrent CLI calls work correctly
-- [ ] Background builds work as expected
-- [ ] Performance meets or exceeds current CLI
-- [ ] Service management is reliable and transparent
-
-## Risk Mitigation
-
-1. **Thorough Testing**: Comprehensive testing before removing old CLI
-2. **Feature Parity**: Ensure all existing functionality works via service
-3. **Performance Validation**: Benchmark new implementation against current performance
-4. **Simple Protocol**: Use HTTP/JSON for service communication (not gRPC initially)
-5. **Clear Error Messages**: Service startup and connection failures should be obvious to users

plans/20-wants-initial.md

@@ -1,163 +0,0 @@
-# Wants System Implementation
-
-## Overview
-
-This plan implements the wants system described in [design/wants.md](../design/wants.md), transitioning DataBuild from direct build requests to a declarative want-based model with cross-graph coordination and SLA tracking. This builds on the 3-tier BEL architecture and client-server CLI established in the previous phases.
-
-## Prerequisites
-
-This plan assumes completion of:
-- **Phase 18**: 3-tier BEL architecture with storage/query/client layers
-- **Phase 19**: Client-server CLI architecture with service delegation
-
-## Implementation Phases
-
-### Phase 1: Extend BEL Storage for Wants
-
-1. **Add PartitionWantEvent to databuild.proto**
-   - Want event schema as defined in design/wants.md
-   - Want source tracking (CLI, dashboard, scheduled, API)
-   - TTL and SLA timestamp fields
-   - External dependency specifications
-
-2. **Extend BELStorage Interface**
-   - Add `append_want()` method for want events
-   - Extend `EventFilter` to support want filtering
-   - Add want-specific query capabilities to storage layer
-
-3. **Implement in SQLite Storage Backend**
-   - Add wants table with appropriate indexes
-   - Implement want filtering in list_events()
-   - Schema migration logic for existing databases
-
-### Phase 2: Basic Want API in Service
-
-1. **Implement Want Management in Service**
-   - Service methods for creating and querying wants
-   - Want lifecycle management (creation, expiration, satisfaction)
-   - Integration with existing service auto-management
-
-2. **Add Want HTTP Endpoints**
-   - `POST /api/v1/wants` - Create new want
-   - `GET /api/v1/wants` - List active wants with filtering
-   - `GET /api/v1/wants/{id}` - Get want details
-   - `DELETE /api/v1/wants/{id}` - Cancel want
-
-3. **CLI Want Commands**
-   - `./bazel-bin/my_graph.build want create <partition-ref>` with SLA/TTL options
-   - `./bazel-bin/my_graph.build want list` with filtering options
-   - `./bazel-bin/my_graph.build want status <partition-ref>` for want status
-   - Modify build commands to create wants via service
-
-### Phase 3: Want-Driven Build Evaluation
-
-1. **Implement Build Evaluator in Service**
-   - Continuous evaluation loop that checks for buildable wants
-   - External dependency satisfaction checking
-   - TTL expiration filtering for active wants
-
-2. **Replace Build Request Handling**
-   - Graph build commands create wants instead of direct build requests
-   - Service background loop evaluates wants and triggers builds
-   - Maintain atomic build semantics while satisfying multiple wants
-
-3. **Build Coordination Logic**
-   - Aggregate wants that can be satisfied by same build
-   - Priority handling for urgent wants (short SLA)
-   - Resource coordination across concurrent want evaluation
-
-### Phase 4: Cross-Graph Coordination
-
-1. **Implement GraphService API**
-   - HTTP API for cross-graph event streaming as defined in design/wants.md
-   - Event filtering for efficient partition pattern subscriptions
-   - Service-to-service communication for upstream dependencies
-
-2. **Upstream Dependency Configuration**
-   - Service configuration for upstream DataBuild instances
-   - Partition pattern subscriptions to upstream graphs
-   - Automatic want evaluation when upstream partitions become available
-
-3. **Cross-Graph Event Sync**
-   - Background sync process for upstream events
-   - Triggering local build evaluation on upstream availability
-   - Reliable HTTP-based coordination to avoid message loss
-
-### Phase 5: SLA Monitoring and Dashboard Integration
-
-1. **SLA Violation Tracking**
-   - External monitoring endpoints for SLA violations
-   - Want timeline and status tracking
-   - Integration with existing dashboard for want visualization
-
-2. **Want Dashboard Features**
-   - Want creation and monitoring UI
-   - Cross-graph dependency visualization
-   - SLA violation dashboard and alerting
-
-3. **Migration from Direct Builds**
-   - All build requests go through want system
-   - Remove direct build request pathways
-   - Update documentation for new build model
-
-## Benefits of Want-Based Architecture
-
-### ✅ **Unified Build Model**
-- All builds (manual, scheduled, triggered) use same want mechanism
-- Complete audit trail in build event log
-- Consistent SLA tracking across all build types
-
-### ✅ **Event-Driven Efficiency**
-- Builds only triggered when dependencies change
-- Cross-graph coordination via efficient event streaming
-- No polling for task readiness within builds
-
-### ✅ **Atomic Build Semantics Preserved**
-- Individual build requests remain all-or-nothing
-- Fast failure provides immediate feedback
-- Partial progress via multiple build requests over time
-
-### ✅ **Flexible SLA Management**
-- Separate business expectations (SLA) from operational limits (TTL)
-- External monitoring with clear blame assignment
-- Automatic cleanup of stale wants
-
-### ✅ **Cross-Graph Scalability**
-- Reliable HTTP-based coordination
-- Efficient filtering via partition patterns
-- Decentralized architecture with clear boundaries
-
-## Success Criteria
-
-### Phase 1: Storage Foundation
-- [ ] Want events can be stored and queried in BEL storage
-- [ ] EventFilter supports want-specific filtering
-- [ ] SQLite backend handles want operations efficiently
-
-### Phase 2: Basic Want API
-- [ ] Service can create and query wants via HTTP API
-- [ ] Graph build commands work for want management
-- [ ] Build commands create wants instead of direct builds
-
-### Phase 3: Want-Driven Builds
-- [ ] Service background loop evaluates wants continuously
-- [ ] Build evaluation triggers on want creation and external events
-- [ ] TTL expiration and external dependency checking work correctly
-
-### Phase 4: Cross-Graph Coordination
-- [ ] GraphService API returns filtered events for cross-graph coordination
-- [ ] Upstream partition availability triggers downstream want evaluation
-- [ ] Service-to-service communication is reliable and efficient
-
-### Phase 5: Complete Migration
-- [ ] All builds go through want system
-- [ ] Dashboard supports want creation and monitoring
-- [ ] SLA violation endpoints provide monitoring integration
-- [ ] Documentation reflects new want-based build model
-
-## Risk Mitigation
-
-1. **Incremental Migration**: Implement wants alongside existing build system initially
-2. **Performance Validation**: Ensure want evaluation doesn't introduce significant latency
-3. **Backwards Compatibility**: Maintain existing build semantics during transition
-4. **Monitoring Integration**: Provide clear observability into want lifecycle and performance