From b27a249b0932f2cd35934390b391fe7e72fc2970 Mon Sep 17 00:00:00 2001
From: Stuart Axelbrooke <stuart@axelbrooke.com>
Date: Mon, 7 Jul 2025 22:42:59 -0700
Subject: [PATCH] Add testing details

---
 CLAUDE.md | 40 +++++++++++++++++++++++++++++++++++++++-
 README.md | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 90 insertions(+), 1 deletion(-)

diff --git a/CLAUDE.md b/CLAUDE.md
index a5ab040..1f8d4c4 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -8,7 +8,10 @@ DataBuild is a bazel-based data build system. Key files:
 
 ## Build & Test
 ```bash
-# Run all tests
+# Run comprehensive end-to-end tests (validates CLI vs Service consistency)
+./run_e2e_tests.sh
+
+# Run all core unit tests
 ./scripts/bb_test_all
 
 # Remote testing
@@ -17,6 +20,41 @@ DataBuild is a bazel-based data build system. Key files:
 # Do not try to `bazel test //examples/basic_graph/...`, as this will not work.
 ```
 
+## End-to-End Testing
+The project includes comprehensive end-to-end tests that validate CLI and Service build consistency:
+
+### Test Suite Structure
+- `tests/end_to_end/simple_test.sh` - Basic CLI vs Service validation
+- `tests/end_to_end/podcast_simple_test.sh` - Podcast reviews CLI vs Service validation  
+- `tests/end_to_end/basic_graph_test.sh` - Comprehensive basic graph testing
+- `tests/end_to_end/podcast_reviews_test.sh` - Comprehensive podcast testing
+
+### Event Validation
+Tests ensure CLI and Service emit identical build events:
+- **Build request events**: Orchestration lifecycle (received, planning, executing, completed)
+- **Job events**: Job execution tracking
+- **Partition events**: Partition build status
+
+### CLI vs Service Event Alignment
+Recent improvements ensure both paths emit identical events:
+- CLI: Enhanced with orchestration events to match Service behavior
+- Service: HTTP API orchestration events + core build events
+- Validation: Tests fail if event counts or types differ between CLI and Service
+
+### Running Individual Tests
+```bash
+# Test basic graph
+tests/end_to_end/simple_test.sh \
+  examples/basic_graph/bazel-bin/basic_graph.build \
+  examples/basic_graph/bazel-bin/basic_graph.service
+
+# Test podcast reviews (run from correct directory)
+cd examples/podcast_reviews
+../../tests/end_to_end/podcast_simple_test.sh \
+  bazel-bin/podcast_reviews_graph.build \
+  bazel-bin/podcast_reviews_graph.service
+```
+
 ## Project Structure
 - `databuild/` - Core system (Rust/Proto)
 - `examples/` - Example implementations
diff --git a/README.md b/README.md
index f03f797..9c0e486 100644
--- a/README.md
+++ b/README.md
@@ -4,3 +4,54 @@
 A bazel-based data build system.
 
 For important context, check out [the manifesto](./manifesto.md), and [core concepts](./core-concepts.md). Also, check out [`databuild.proto`](./databuild/databuild.proto) for key system interfaces.
+
+## Testing
+
+### Quick Test
+Run the comprehensive end-to-end test suite:
+```bash
+./run_e2e_tests.sh
+```
+
+### Core Unit Tests
+```bash
+# Run all core DataBuild tests
+./scripts/bb_test_all
+
+# Remote testing
+./scripts/bb_remote_test_all
+```
+
+### Manual Testing
+```bash
+# Test basic graph CLI build
+cd examples/basic_graph
+bazel run //:basic_graph.build -- "generated_number/pippin"
+
+# Test podcast reviews CLI build  
+cd examples/podcast_reviews
+bazel run //:podcast_reviews_graph.build -- "reviews/date=2020-01-01"
+
+# Test service builds
+bazel run //:basic_graph.service -- --port=8080
+# Then in another terminal:
+curl -X POST -H "Content-Type: application/json" \
+  -d '{"partitions": ["generated_number/pippin"]}' \
+  http://localhost:8080/api/v1/builds
+```
+
+### Event Validation Tests
+The end-to-end tests validate that CLI and Service builds emit identical events:
+- **Event count alignment**: CLI and Service must generate the same total event count
+- **Event type breakdown**: Job, partition, and build_request events must match exactly
+- **Event consistency**: Both interfaces represent the same logical build process
+
+Example test output:
+```
+Event breakdown:
+  Job events:       CLI=2, Service=2
+  Partition events: CLI=3, Service=3
+  Request events:   CLI=9, Service=9
+✅ All build events (job, partition, and request) are identical
+✅ Total event counts are identical: 14 events each
+```