62 lines
2 KiB
Markdown
62 lines
2 KiB
Markdown
|
|
# DataBuild
|
|
|
|
Orchestration logic changes frequently, so just don't write it.
|
|
|
|
DataBuild is a trivially-deployable, partition-oriented, declarative data build system.
|
|
|
|
For important context, check out [DESIGN.md](./DESIGN.md). Also, check out [`databuild.proto`](./databuild/databuild.proto) for key system interfaces.
|
|
|
|
## Usage
|
|
|
|
See the [podcast example BUILD file](examples/podcast_reviews/BUILD.bazel).
|
|
|
|
## Development
|
|
|
|
### Intellij
|
|
|
|
Run these to allow intellij to understand the rust source:
|
|
|
|
```bash
|
|
# Generate a Cargo.toml file so intellij can link rust src
|
|
python3 scripts/generate_cargo_toml.py
|
|
# Generate a gitignore'd rust file representing the protobuf interfaces
|
|
scripts/generate_proto_for_ide.sh
|
|
```
|
|
|
|
### Compiling
|
|
```bash
|
|
bazel build //...
|
|
```
|
|
|
|
**Bullet-proof compile-time correctness** is essential for production reliability. Backend protobuf changes must cause predictable frontend compilation failures, preventing runtime errors. Our three-pronged approach ensures this:
|
|
|
|
1. **Complete Type Chain**: Proto → Rust → OpenAPI → TypeScript → Components
|
|
- Each step uses generated types, maintaining accuracy across the entire pipeline
|
|
- Breaking changes at any layer cause compilation failures in dependent layers
|
|
|
|
2. **Consistent Data Transformation**: Service boundary layer transforms API responses to dashboard types
|
|
- Canonical frontend interfaces isolated from backend implementation details
|
|
- Transformations handle protobuf nullability and normalize data shapes
|
|
- Components never directly access generated API types
|
|
|
|
3. **Strict TypeScript Configuration**: Enforces explicit null handling and prevents implicit `any` types
|
|
- `strictNullChecks` catches undefined property access patterns
|
|
- `noImplicitAny` surfaces type safety gaps
|
|
- Runtime type errors become compile-time failures
|
|
|
|
This system guarantees that backend interface changes are caught during TypeScript compilation, not in production.
|
|
|
|
### Testing
|
|
|
|
DataBuild core testing:
|
|
|
|
````bash
|
|
bazel test //...
|
|
````
|
|
|
|
End to end testing:
|
|
|
|
```bash
|
|
./run_e2e_tests.sh
|
|
```
|