Add testing details
Some checks are pending
/ setup (push) Waiting to run

This commit is contained in:
Stuart Axelbrooke 2025-07-07 22:42:59 -07:00
parent 39effe5b2b
commit b27a249b09
2 changed files with 90 additions and 1 deletions

View file

@ -8,7 +8,10 @@ DataBuild is a bazel-based data build system. Key files:
## Build & Test
```bash
# Run all tests
# Run comprehensive end-to-end tests (validates CLI vs Service consistency)
./run_e2e_tests.sh
# Run all core unit tests
./scripts/bb_test_all
# Remote testing
@ -17,6 +20,41 @@ DataBuild is a bazel-based data build system. Key files:
# Do not try to `bazel test //examples/basic_graph/...`, as this will not work.
```
## End-to-End Testing
The project includes comprehensive end-to-end tests that validate CLI and Service build consistency:
### Test Suite Structure
- `tests/end_to_end/simple_test.sh` - Basic CLI vs Service validation
- `tests/end_to_end/podcast_simple_test.sh` - Podcast reviews CLI vs Service validation
- `tests/end_to_end/basic_graph_test.sh` - Comprehensive basic graph testing
- `tests/end_to_end/podcast_reviews_test.sh` - Comprehensive podcast testing
### Event Validation
Tests ensure CLI and Service emit identical build events:
- **Build request events**: Orchestration lifecycle (received, planning, executing, completed)
- **Job events**: Job execution tracking
- **Partition events**: Partition build status
### CLI vs Service Event Alignment
Recent improvements ensure both paths emit identical events:
- CLI: Enhanced with orchestration events to match Service behavior
- Service: HTTP API orchestration events + core build events
- Validation: Tests fail if event counts or types differ between CLI and Service
### Running Individual Tests
```bash
# Test basic graph
tests/end_to_end/simple_test.sh \
examples/basic_graph/bazel-bin/basic_graph.build \
examples/basic_graph/bazel-bin/basic_graph.service
# Test podcast reviews (run from correct directory)
cd examples/podcast_reviews
../../tests/end_to_end/podcast_simple_test.sh \
bazel-bin/podcast_reviews_graph.build \
bazel-bin/podcast_reviews_graph.service
```
## Project Structure
- `databuild/` - Core system (Rust/Proto)
- `examples/` - Example implementations

View file

@ -4,3 +4,54 @@
A bazel-based data build system.
For important context, check out [the manifesto](./manifesto.md), and [core concepts](./core-concepts.md). Also, check out [`databuild.proto`](./databuild/databuild.proto) for key system interfaces.
## Testing
### Quick Test
Run the comprehensive end-to-end test suite:
```bash
./run_e2e_tests.sh
```
### Core Unit Tests
```bash
# Run all core DataBuild tests
./scripts/bb_test_all
# Remote testing
./scripts/bb_remote_test_all
```
### Manual Testing
```bash
# Test basic graph CLI build
cd examples/basic_graph
bazel run //:basic_graph.build -- "generated_number/pippin"
# Test podcast reviews CLI build
cd examples/podcast_reviews
bazel run //:podcast_reviews_graph.build -- "reviews/date=2020-01-01"
# Test service builds
bazel run //:basic_graph.service -- --port=8080
# Then in another terminal:
curl -X POST -H "Content-Type: application/json" \
-d '{"partitions": ["generated_number/pippin"]}' \
http://localhost:8080/api/v1/builds
```
### Event Validation Tests
The end-to-end tests validate that CLI and Service builds emit identical events:
- **Event count alignment**: CLI and Service must generate the same total event count
- **Event type breakdown**: Job, partition, and build_request events must match exactly
- **Event consistency**: Both interfaces represent the same logical build process
Example test output:
```
Event breakdown:
Job events: CLI=2, Service=2
Partition events: CLI=3, Service=3
Request events: CLI=9, Service=9
✅ All build events (job, partition, and request) are identical
✅ Total event counts are identical: 14 events each
```