This commit is contained in:
parent
39effe5b2b
commit
b27a249b09
2 changed files with 90 additions and 1 deletions
40
CLAUDE.md
40
CLAUDE.md
|
|
@ -8,7 +8,10 @@ DataBuild is a bazel-based data build system. Key files:
|
|||
|
||||
## Build & Test
|
||||
```bash
|
||||
# Run all tests
|
||||
# Run comprehensive end-to-end tests (validates CLI vs Service consistency)
|
||||
./run_e2e_tests.sh
|
||||
|
||||
# Run all core unit tests
|
||||
./scripts/bb_test_all
|
||||
|
||||
# Remote testing
|
||||
|
|
@ -17,6 +20,41 @@ DataBuild is a bazel-based data build system. Key files:
|
|||
# Do not try to `bazel test //examples/basic_graph/...`, as this will not work.
|
||||
```
|
||||
|
||||
## End-to-End Testing
|
||||
The project includes comprehensive end-to-end tests that validate CLI and Service build consistency:
|
||||
|
||||
### Test Suite Structure
|
||||
- `tests/end_to_end/simple_test.sh` - Basic CLI vs Service validation
|
||||
- `tests/end_to_end/podcast_simple_test.sh` - Podcast reviews CLI vs Service validation
|
||||
- `tests/end_to_end/basic_graph_test.sh` - Comprehensive basic graph testing
|
||||
- `tests/end_to_end/podcast_reviews_test.sh` - Comprehensive podcast testing
|
||||
|
||||
### Event Validation
|
||||
Tests ensure CLI and Service emit identical build events:
|
||||
- **Build request events**: Orchestration lifecycle (received, planning, executing, completed)
|
||||
- **Job events**: Job execution tracking
|
||||
- **Partition events**: Partition build status
|
||||
|
||||
### CLI vs Service Event Alignment
|
||||
Recent improvements ensure both paths emit identical events:
|
||||
- CLI: Enhanced with orchestration events to match Service behavior
|
||||
- Service: HTTP API orchestration events + core build events
|
||||
- Validation: Tests fail if event counts or types differ between CLI and Service
|
||||
|
||||
### Running Individual Tests
|
||||
```bash
|
||||
# Test basic graph
|
||||
tests/end_to_end/simple_test.sh \
|
||||
examples/basic_graph/bazel-bin/basic_graph.build \
|
||||
examples/basic_graph/bazel-bin/basic_graph.service
|
||||
|
||||
# Test podcast reviews (run from correct directory)
|
||||
cd examples/podcast_reviews
|
||||
../../tests/end_to_end/podcast_simple_test.sh \
|
||||
bazel-bin/podcast_reviews_graph.build \
|
||||
bazel-bin/podcast_reviews_graph.service
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
- `databuild/` - Core system (Rust/Proto)
|
||||
- `examples/` - Example implementations
|
||||
|
|
|
|||
51
README.md
51
README.md
|
|
@ -4,3 +4,54 @@
|
|||
A bazel-based data build system.
|
||||
|
||||
For important context, check out [the manifesto](./manifesto.md), and [core concepts](./core-concepts.md). Also, check out [`databuild.proto`](./databuild/databuild.proto) for key system interfaces.
|
||||
|
||||
## Testing
|
||||
|
||||
### Quick Test
|
||||
Run the comprehensive end-to-end test suite:
|
||||
```bash
|
||||
./run_e2e_tests.sh
|
||||
```
|
||||
|
||||
### Core Unit Tests
|
||||
```bash
|
||||
# Run all core DataBuild tests
|
||||
./scripts/bb_test_all
|
||||
|
||||
# Remote testing
|
||||
./scripts/bb_remote_test_all
|
||||
```
|
||||
|
||||
### Manual Testing
|
||||
```bash
|
||||
# Test basic graph CLI build
|
||||
cd examples/basic_graph
|
||||
bazel run //:basic_graph.build -- "generated_number/pippin"
|
||||
|
||||
# Test podcast reviews CLI build
|
||||
cd examples/podcast_reviews
|
||||
bazel run //:podcast_reviews_graph.build -- "reviews/date=2020-01-01"
|
||||
|
||||
# Test service builds
|
||||
bazel run //:basic_graph.service -- --port=8080
|
||||
# Then in another terminal:
|
||||
curl -X POST -H "Content-Type: application/json" \
|
||||
-d '{"partitions": ["generated_number/pippin"]}' \
|
||||
http://localhost:8080/api/v1/builds
|
||||
```
|
||||
|
||||
### Event Validation Tests
|
||||
The end-to-end tests validate that CLI and Service builds emit identical events:
|
||||
- **Event count alignment**: CLI and Service must generate the same total event count
|
||||
- **Event type breakdown**: Job, partition, and build_request events must match exactly
|
||||
- **Event consistency**: Both interfaces represent the same logical build process
|
||||
|
||||
Example test output:
|
||||
```
|
||||
Event breakdown:
|
||||
Job events: CLI=2, Service=2
|
||||
Partition events: CLI=3, Service=3
|
||||
Request events: CLI=9, Service=9
|
||||
✅ All build events (job, partition, and request) are identical
|
||||
✅ Total event counts are identical: 14 events each
|
||||
```
|
||||
|
|
|
|||
Loading…
Reference in a new issue