# DataBuild

Orchestration logic changes frequently, so just don't write it.

DataBuild is a trivially-deployable, partition-oriented, declarative data build system.

For important context, check out [DESIGN.md](./DESIGN.md). Also, check out [`databuild.proto`](./databuild/databuild.proto) for key system interfaces.

## Usage

See the [podcast example BUILD file](examples/podcast_reviews/BUILD.bazel).

## Development

### Intellij

Run these to allow intellij to understand the rust source:

```bash
# Generate a Cargo.toml file so intellij can link rust src
python3 scripts/generate_cargo_toml.py
# Generate a gitignore'd rust file representing the protobuf interfaces
scripts/generate_proto_for_ide.sh
```

### Compiling
```bash
bazel build //...
```

**Bullet-proof compile-time correctness** is essential for production reliability. Backend protobuf changes must cause predictable frontend compilation failures, preventing runtime errors. Our three-pronged approach ensures this:

1. **Complete Type Chain**: Proto → Rust → OpenAPI → TypeScript → Components
   - Each step uses generated types, maintaining accuracy across the entire pipeline
   - Breaking changes at any layer cause compilation failures in dependent layers

2. **Consistent Data Transformation**: Service boundary layer transforms API responses to dashboard types
   - Canonical frontend interfaces isolated from backend implementation details  
   - Transformations handle protobuf nullability and normalize data shapes
   - Components never directly access generated API types

3. **Strict TypeScript Configuration**: Enforces explicit null handling and prevents implicit `any` types
   - `strictNullChecks` catches undefined property access patterns
   - `noImplicitAny` surfaces type safety gaps
   - Runtime type errors become compile-time failures

This system guarantees that backend interface changes are caught during TypeScript compilation, not in production.

### Testing

DataBuild core testing:

````bash
bazel test //...
````

End to end testing:

```bash
./run_e2e_tests.sh
```