From 07cd3a3e8f76c924bbe35feb05fbd9b6695a4c69 Mon Sep 17 00:00:00 2001 From: Stuart Axelbrooke Date: Tue, 8 Jul 2025 18:38:33 -0700 Subject: [PATCH] Create phased web app plan --- plans/build-graph-dashboard.md | 29 ++- plans/roadmap.md | 2 +- plans/webapp_v1/chunk-1-client-generation.md | 102 ++++++++ plans/webapp_v1/chunk-2-hello-world-app.md | 110 +++++++++ plans/webapp_v1/chunk-3-routing-framework.md | 130 +++++++++++ plans/webapp_v1/chunk-4-recent-activity.md | 148 ++++++++++++ plans/webapp_v1/chunk-5-build-status.md | 157 +++++++++++++ plans/webapp_v1/chunk-6-partition-pages.md | 224 ++++++++++++++++++ plans/webapp_v1/chunk-7-jobs-pages.md | 230 +++++++++++++++++++ plans/webapp_v1/chunk-8-graph-analysis.md | 209 +++++++++++++++++ plans/webapp_v1/chunk-9-polish.md | 128 +++++++++++ 11 files changed, 1467 insertions(+), 2 deletions(-) create mode 100644 plans/webapp_v1/chunk-1-client-generation.md create mode 100644 plans/webapp_v1/chunk-2-hello-world-app.md create mode 100644 plans/webapp_v1/chunk-3-routing-framework.md create mode 100644 plans/webapp_v1/chunk-4-recent-activity.md create mode 100644 plans/webapp_v1/chunk-5-build-status.md create mode 100644 plans/webapp_v1/chunk-6-partition-pages.md create mode 100644 plans/webapp_v1/chunk-7-jobs-pages.md create mode 100644 plans/webapp_v1/chunk-8-graph-analysis.md create mode 100644 plans/webapp_v1/chunk-9-polish.md diff --git a/plans/build-graph-dashboard.md b/plans/build-graph-dashboard.md index ed54739..4877bc1 100644 --- a/plans/build-graph-dashboard.md +++ b/plans/build-graph-dashboard.md @@ -1,6 +1,12 @@ # Build Graph Dashboard +**Parent:** [DataBuild Roadmap](./roadmap.md) + +**Status:** In Progress + +**Implementation:** Broken into 9 incremental chunks (see [Chunk Plans](#chunk-implementation-plan)) + - Simplicity is absolutely critical here - Use mithril - Use typescript @@ -86,7 +92,7 @@ Interactive build graph visualization. **UI Elements:** - Partition input form -- Generated job graph (mermaid.js rendering - simple visualization) +- Generated job graph (mermaid.js rendering - simple visualization, rendered in the client) - Execution plan table ### Technical Implementation @@ -186,3 +192,24 @@ npm run typecheck # Bundled with build graph service ./scripts/build_dashboard ``` + +## Chunk Implementation Plan + +This dashboard implementation is broken into 9 incremental, testable chunks: + +### Phase 1: Foundation & Infrastructure +- **[Chunk 1: TypeScript Client Generation](./webapp_v1/chunk-1-client-generation.md)** - Generate typed client from proto +- **[Chunk 2: Hello World App](./webapp_v1/chunk-2-hello-world-app.md)** - Bazel-built TypeScript + Mithril app +- **[Chunk 3: Routing Framework](./webapp_v1/chunk-3-routing-framework.md)** - Multi-page routing and layout + +### Phase 2: Core Dashboard Pages +- **[Chunk 4: Recent Activity](./webapp_v1/chunk-4-recent-activity.md)** - Dashboard home page +- **[Chunk 5: Build Status](./webapp_v1/chunk-5-build-status.md)** - Real-time build monitoring +- **[Chunk 6: Partition Pages](./webapp_v1/chunk-6-partition-pages.md)** - Partition status and history +- **[Chunk 7: Jobs Pages](./webapp_v1/chunk-7-jobs-pages.md)** - Job metrics and monitoring + +### Phase 3: Advanced Features +- **[Chunk 8: Graph Analysis](./webapp_v1/chunk-8-graph-analysis.md)** - Interactive graph visualization +- **[Chunk 9: Polish](./webapp_v1/chunk-9-polish.md)** - Final styling and optimization + +Each chunk delivers working functionality that can be tested independently while building toward the complete dashboard vision. diff --git a/plans/roadmap.md b/plans/roadmap.md index 9129eb3..afecf45 100644 --- a/plans/roadmap.md +++ b/plans/roadmap.md @@ -60,7 +60,7 @@ Uses the [basic graph](../examples/basic_graph/README.md) and [podcast reviews]( [**Design Doc**](./build-graph-dashboard.md) -Status: Not Started +Status: In Progress A UI that relies on the Build Graph Service, showing things like build activity and partition liveness information. There are a few key pages: diff --git a/plans/webapp_v1/chunk-1-client-generation.md b/plans/webapp_v1/chunk-1-client-generation.md new file mode 100644 index 0000000..7b2308d --- /dev/null +++ b/plans/webapp_v1/chunk-1-client-generation.md @@ -0,0 +1,102 @@ +# Chunk 1: TypeScript Client Generation + +**Parent:** [Build Graph Dashboard](../build-graph-dashboard.md) +**Next:** [Chunk 2: Hello World App](./chunk-2-hello-world-app.md) + +## Overview + +Generate TypeScript client code from the DataBuild protobuf definitions to provide type-safe API access for the dashboard. + +## Scope + +### In Scope +- Generate TypeScript interfaces from `databuild.proto` +- Create typed client for Build Graph Service HTTP API endpoints +- Set up Bazel rules for protobuf-to-TypeScript generation +- Implement simple custom protobuf-to-TypeScript tooling (if off-the-shelf solutions are complex) + +### Out of Scope +- Full protobuf runtime features (only need type definitions) +- Complex protobuf features (focus on basic message types) +- Client-side validation (rely on server-side validation) + +## Technical Approach + +### Proto Analysis +Key messages to generate TypeScript for: +- `BuildEvent` and related event types +- `PartitionRef`, `JobLabel`, `GraphLabel` +- `BuildRequestStatus`, `PartitionStatus`, `JobStatus` +- `JobGraph`, `Task`, `JobConfig` +- Service request/response types + +### Generated Client Structure +```typescript +// Generated types +interface PartitionRef { + str: string; +} + +interface BuildStatusResponse { + build_request_id: string; + status: BuildRequestStatus; + requested_partitions: string[]; + created_at: number; + updated_at: number; + events: BuildEventSummary[]; +} + +// Client class +class BuildGraphClient { + constructor(baseUrl: string); + + async submitBuild(partitions: string[]): Promise<{build_request_id: string}>; + async getBuildStatus(id: string): Promise; + async cancelBuild(id: string): Promise<{cancelled: boolean}>; + async getPartitionStatus(ref: string): Promise; + async getPartitionEvents(ref: string): Promise; + async analyzeGraph(partitions: string[]): Promise<{job_graph: JobGraph}>; +} +``` + +### Bazel Integration +- Create `//databuild/client:typescript` target +- Generate TypeScript files from `.proto` sources +- Ensure hermetic build process +- Output client code to `databuild/client/typescript/` + +## Implementation Strategy + +1. **Assess Off-the-Shelf Solutions** + - Quick evaluation of existing protobuf-to-TypeScript tools + - If complex setup required, proceed with custom implementation + +2. **Custom Generator (if needed)** + - Simple Python/Rust script to parse proto and generate TypeScript + - Focus only on message types and basic field mapping + - No complex protobuf features needed + +3. **Bazel Rules** + - Create `typescript_proto_library` rule + - Generate client code during build + - Ensure proper dependency management + +## Deliverables + +- [ ] TypeScript interfaces for all relevant protobuf messages +- [ ] Typed HTTP client for Build Graph Service endpoints +- [ ] Bazel rules for client generation +- [ ] Documentation for using the generated client + +## Success Criteria + +- Generated TypeScript compiles without errors +- Client provides type safety for all API endpoints +- Bazel build integrates seamlessly +- Ready for use in Chunk 2 (Hello World App) + +## Testing + +- Verify generated TypeScript compiles +- Test client against running Build Graph Service +- Validate type safety with TypeScript compiler \ No newline at end of file diff --git a/plans/webapp_v1/chunk-2-hello-world-app.md b/plans/webapp_v1/chunk-2-hello-world-app.md new file mode 100644 index 0000000..2ac5024 --- /dev/null +++ b/plans/webapp_v1/chunk-2-hello-world-app.md @@ -0,0 +1,110 @@ +# Chunk 2: Hello World App + +**Parent:** [Build Graph Dashboard](../build-graph-dashboard.md) +**Previous:** [Chunk 1: TypeScript Client Generation](./chunk-1-client-generation.md) +**Next:** [Chunk 3: Routing Framework](./chunk-3-routing-framework.md) + +## Overview + +Create a minimal "Hello World" single-page application using TypeScript and Mithril, fully integrated with the Bazel build system and served by the Build Graph Service. + +## Scope + +### In Scope +- Basic TypeScript + Mithril SPA setup +- Bazel BUILD rules for TypeScript compilation +- Integration with Build Graph Service (served by same process) +- Hermetic development workflow +- Basic bundling and minification + +### Out of Scope +- Complex routing (handled in Chunk 3) +- Styling framework (basic CSS only) +- Real API integration (use generated client from Chunk 1 for basic connectivity test) + +## Technical Approach + +### Application Structure +``` +databuild/ +├── dashboard/ +│ ├── BUILD +│ ├── src/ +│ │ ├── main.ts # Application entry point +│ │ ├── components/ +│ │ │ └── HelloWorld.ts # Basic component +│ │ └── api/ +│ │ └── client.ts # Generated client import +│ ├── static/ +│ │ └── index.html # HTML template +│ └── tsconfig.json # TypeScript configuration +``` + +### Bazel Integration +- Create `//databuild/dashboard:app` target +- Use `rules_nodejs` for TypeScript compilation +- Bundle with webpack or similar tool +- Serve static files from Build Graph Service + +### Service Integration +- Update Build Graph Service to serve static files +- Add route for `/*` to serve dashboard +- Ensure API routes take precedence over static files +- Dashboard should self-configure service URL + +### Development Workflow +```bash +# Build dashboard +bazel build //databuild/dashboard:app + +# Development mode (with file watching) +bazel run //databuild/dashboard:dev + +# Type checking +bazel run //databuild/dashboard:typecheck +``` + +## Implementation Strategy + +1. **Set Up Bazel Rules** + - Configure `rules_nodejs` for TypeScript + - Create build targets for development and production + - Set up bundling pipeline + +2. **Create Basic App** + - Simple Mithril application with single component + - Import and test generated TypeScript client + - Basic connectivity test to service API + +3. **Service Integration** + - Update Build Graph Service to serve static files + - Add dashboard route configuration + - Ensure proper content-type headers + +4. **Development Tooling** + - File watching for development + - TypeScript compilation + - Error reporting + +## Deliverables + +- [ ] Working TypeScript + Mithril "Hello World" app +- [ ] Bazel BUILD rules for compilation and bundling +- [ ] Integration with Build Graph Service +- [ ] Development workflow scripts +- [ ] Basic connectivity test with generated client + +## Success Criteria + +- App compiles and runs without errors +- Served by Build Graph Service on same port as API +- TypeScript client successfully connects to service +- Development workflow supports rapid iteration +- Hermetic build process + +## Testing + +- Build app with `bazel build //databuild/dashboard:app` +- Start service and verify dashboard loads +- Test API connectivity from dashboard +- Verify TypeScript compilation and type checking work \ No newline at end of file diff --git a/plans/webapp_v1/chunk-3-routing-framework.md b/plans/webapp_v1/chunk-3-routing-framework.md new file mode 100644 index 0000000..0f4b931 --- /dev/null +++ b/plans/webapp_v1/chunk-3-routing-framework.md @@ -0,0 +1,130 @@ +# Chunk 3: Routing Framework + +**Parent:** [Build Graph Dashboard](../build-graph-dashboard.md) +**Previous:** [Chunk 2: Hello World App](./chunk-2-hello-world-app.md) +**Next:** [Chunk 4: Recent Activity](./chunk-4-recent-activity.md) + +## Overview + +Implement multi-page routing using Mithril's routing system, create the base layout with navigation, and handle URL encoding/decoding for partition references. + +## Scope + +### In Scope +- Mithril routing for all planned dashboard pages +- Base layout with navigation header +- URL encoding/decoding for partition references +- Initial page scaffolding for all routes +- Basic Tailwind + DaisyUI styling setup + +### Out of Scope +- Full page implementations (handled in subsequent chunks) +- Complex state management +- Advanced styling (minimal styling only) + +## Technical Approach + +### Routing Structure +```typescript +const routes = { + '/': RecentActivity, + '/builds/:id': BuildStatus, + '/partitions': PartitionsList, + '/partitions/:base64_ref': PartitionStatus, + '/jobs': JobsList, + '/jobs/:label': JobMetrics, + '/analyze': GraphAnalysis, +}; +``` + +### URL Encoding Utilities +```typescript +// Partition reference URL encoding +function encodePartitionRef(ref: string): string { + return btoa(ref).replace(/\+/g, '-').replace(/\//g, '_').replace(/=/g, ''); +} + +function decodePartitionRef(encoded: string): string { + // Add padding if needed + const padded = encoded.replace(/-/g, '+').replace(/_/g, '/'); + return atob(padded); +} +``` + +### Base Layout +```typescript +const Layout = { + view: (vnode: any) => [ + m('header.navbar', [ + m('nav', [ + m('a[href="/"]', 'Dashboard'), + m('a[href="/partitions"]', 'Partitions'), + m('a[href="/jobs"]', 'Jobs'), + m('a[href="/analyze"]', 'Analyze'), + ]) + ]), + m('main', vnode.children) + ] +}; +``` + +### Page Scaffolding +Create placeholder components for each route: +- `RecentActivity` - Dashboard home +- `BuildStatus` - Build request status +- `PartitionsList` - Partition listing +- `PartitionStatus` - Individual partition status +- `JobsList` - Jobs listing +- `JobMetrics` - Job metrics and history +- `GraphAnalysis` - Graph analysis tool + +## Implementation Strategy + +1. **Set Up Routing** + - Configure Mithril routing + - Create route definitions + - Implement navigation handlers + +2. **Create Base Layout** + - Navigation header with links + - Main content area + - Basic responsive design + +3. **Implement URL Encoding** + - Partition reference encoding/decoding + - URL parameter handling + - Error handling for invalid refs + +4. **Add Tailwind + DaisyUI** + - Configure build system for CSS processing + - Add basic styling to layout + - Set up design tokens + +5. **Create Page Scaffolds** + - Placeholder components for each route + - Basic page structure + - Navigation between pages + +## Deliverables + +- [ ] Working multi-page routing system +- [ ] Base layout with navigation +- [ ] URL encoding/decoding for partition refs +- [ ] Scaffold pages for all planned routes +- [ ] Basic Tailwind + DaisyUI styling setup + +## Success Criteria + +- All routes load without errors +- Navigation between pages works correctly +- Partition reference encoding/decoding handles edge cases +- Layout is responsive and functional +- Ready for page implementations in subsequent chunks + +## Testing + +- Navigate to all routes and verify they load +- Test partition reference encoding/decoding with various inputs +- Verify browser back/forward navigation works +- Test responsive layout on different screen sizes +- Validate URL parameter handling \ No newline at end of file diff --git a/plans/webapp_v1/chunk-4-recent-activity.md b/plans/webapp_v1/chunk-4-recent-activity.md new file mode 100644 index 0000000..716ce82 --- /dev/null +++ b/plans/webapp_v1/chunk-4-recent-activity.md @@ -0,0 +1,148 @@ +# Chunk 4: Recent Activity + +**Parent:** [Build Graph Dashboard](../build-graph-dashboard.md) +**Previous:** [Chunk 3: Routing Framework](./chunk-3-routing-framework.md) +**Next:** [Chunk 5: Build Status](./chunk-5-build-status.md) + +## Overview + +Implement the dashboard home page showing recent build activity and system status with real-time updates via polling. + +## Scope + +### In Scope +- Recent build requests display +- Active builds count and status +- Recent partition builds +- System health indicators +- Basic polling for real-time updates +- Tailwind + DaisyUI styling + +### Out of Scope +- Complex filtering or searching +- Historical data beyond recent activity +- Advanced visualizations +- User preferences/settings + +## Technical Approach + +### Data Sources +From Build Graph Service API: +- Recent build requests (from event log) +- Active builds count +- Recent partition builds +- System status + +### Component Structure +```typescript +const RecentActivity = { + oninit: () => { + // Start polling for updates + this.pollInterval = setInterval(this.loadData, 5000); + this.loadData(); + }, + + onremove: () => { + // Clean up polling + clearInterval(this.pollInterval); + }, + + view: () => [ + m('.dashboard-header', [ + m('h1', 'DataBuild Dashboard'), + m('.stats', [ + m('.stat-item', `Active Builds: ${this.activeBuilds}`), + m('.stat-item', `Recent Builds: ${this.recentBuilds.length}`), + ]) + ]), + + m('.dashboard-content', [ + m('.recent-builds', [ + m('h2', 'Recent Build Requests'), + m('table', this.recentBuilds.map(build => + m('tr', [ + m('td', m('a', { href: `/builds/${build.id}` }, build.id)), + m('td', build.status), + m('td', formatTime(build.created_at)), + ]) + )) + ]), + + m('.recent-partitions', [ + m('h2', 'Recent Partition Builds'), + m('table', this.recentPartitions.map(partition => + m('tr', [ + m('td', m('a', { + href: `/partitions/${encodePartitionRef(partition.ref)}` + }, partition.ref)), + m('td', partition.status), + m('td', formatTime(partition.updated_at)), + ]) + )) + ]) + ]) + ] +}; +``` + +### Polling Strategy +- Poll every 5 seconds for updates +- Use Page Visibility API to pause when tab inactive +- Show loading states during updates +- Handle connection errors gracefully + +### Styling +- Use DaisyUI stat components for metrics +- Table layout for recent items +- Responsive grid for dashboard sections +- Status badges for build states + +## Implementation Strategy + +1. **Create Data Layer** + - API calls for recent activity data + - Polling manager with visibility detection + - Error handling and retries + +2. **Build UI Components** + - Dashboard header with metrics + - Recent builds table + - Recent partitions table + - Loading and error states + +3. **Implement Real-time Updates** + - Set up polling with proper cleanup + - Page Visibility API integration + - Optimistic updates for better UX + +4. **Add Styling** + - DaisyUI components for consistent look + - Responsive layout + - Status indicators and badges + +## Deliverables + +- [ ] Dashboard home page with recent activity +- [ ] Real-time polling with visibility detection +- [ ] Recent build requests table with links +- [ ] Recent partition builds display +- [ ] System metrics and health indicators +- [ ] Responsive styling with DaisyUI + +## Success Criteria + +- Page loads and displays recent activity +- Real-time updates work correctly +- Links navigate to appropriate detail pages +- Polling stops when tab is inactive +- Layout is responsive and well-styled +- Error states are handled gracefully + +## Testing + +- Verify recent builds and partitions display +- Test real-time updates with running builds +- Validate links to build and partition detail pages +- Check polling behavior with tab visibility changes +- Test error handling with service unavailable +- Verify responsive layout on different screen sizes \ No newline at end of file diff --git a/plans/webapp_v1/chunk-5-build-status.md b/plans/webapp_v1/chunk-5-build-status.md new file mode 100644 index 0000000..8707bc0 --- /dev/null +++ b/plans/webapp_v1/chunk-5-build-status.md @@ -0,0 +1,157 @@ +# Chunk 5: Build Status + +**Parent:** [Build Graph Dashboard](../build-graph-dashboard.md) +**Previous:** [Chunk 4: Recent Activity](./chunk-4-recent-activity.md) +**Next:** [Chunk 6: Partition Pages](./chunk-6-partition-pages.md) + +## Overview + +Implement the core operational build request status page with real-time updates, job/partition status visualization, and execution logs. + +## Scope + +### In Scope +- Build request status display with real-time updates +- Job and partition status visualization +- Execution timeline and logs from build events +- Delegation indicators for shared builds +- Auto-refresh with Page Visibility API +- Expandable job details + +### Out of Scope +- Complex graph visualizations (simple status indicators) +- Historical build comparisons +- Advanced filtering of events +- User interactions beyond viewing + +## Technical Approach + +### Data Sources +From Build Graph Service API `/api/v1/builds/:id`: +- Build request metadata (ID, status, timestamps) +- Requested partitions +- Build events with job execution details +- Delegation information + +### Component Structure +```typescript +const BuildStatus = { + oninit: (vnode) => { + this.buildId = vnode.attrs.id; + this.build = null; + this.pollInterval = null; + this.loadBuild(); + this.startPolling(); + }, + + onremove: () => { + this.stopPolling(); + }, + + view: () => [ + m('.build-header', [ + m('h1', `Build ${this.buildId}`), + m('.build-meta', [ + m('.badge', this.build?.status), + m('.timestamp', formatTime(this.build?.created_at)), + m('.partitions', `${this.build?.requested_partitions.length} partitions`), + ]) + ]), + + m('.build-content', [ + m('.partition-status', [ + m('h2', 'Partition Status'), + m('.partition-grid', + this.build?.requested_partitions.map(partition => + m('.partition-card', [ + m('.partition-ref', partition), + m('.partition-status', this.getPartitionStatus(partition)), + m('.partition-job', this.getPartitionJob(partition)), + ]) + ) + ) + ]), + + m('.execution-timeline', [ + m('h2', 'Execution Timeline'), + m('.timeline', this.build?.events.map(event => + m('.timeline-item', [ + m('.timestamp', formatTime(event.timestamp)), + m('.event-type', event.event_type), + m('.message', event.message), + this.isJobEvent(event) ? m('.expandable-logs', [ + m('button', { onclick: () => this.toggleLogs(event) }, 'Show Logs'), + this.logsExpanded[event.event_id] ? + m('.logs', this.formatJobLogs(event)) : null + ]) : null + ]) + )) + ]) + ]) + ] +}; +``` + +### Real-time Updates +- Poll every 2 seconds when build is active +- Poll every 10 seconds when build is completed +- Stop polling when tab is not visible +- Visual indicators for live updates + +### Status Visualization +- Color-coded status badges (green/yellow/red) +- Progress indicators for running builds +- Delegation indicators with links to other builds +- Timeline visualization for build events + +## Implementation Strategy + +1. **Create Data Layer** + - API integration for build status + - Event parsing and categorization + - Polling manager with different intervals + +2. **Build Status Components** + - Build header with metadata + - Partition status grid + - Execution timeline + - Expandable log sections + +3. **Real-time Updates** + - Intelligent polling based on build state + - Page Visibility API integration + - Loading states and error handling + +4. **Status Visualization** + - Color-coded status indicators + - Progress bars for active builds + - Timeline layout for events + - Delegation indicators + +## Deliverables + +- [ ] Build request status page with real-time updates +- [ ] Partition status grid with visual indicators +- [ ] Execution timeline with build events +- [ ] Expandable job logs and details +- [ ] Auto-refresh with visibility detection +- [ ] Delegation indicators and links + +## Success Criteria + +- Real-time updates show build progress accurately +- All partition statuses are clearly displayed +- Job logs are accessible and readable +- Polling behaves correctly based on build state +- Delegation to other builds is clearly indicated +- Page is responsive and performs well + +## Testing + +- Test with running builds to verify real-time updates +- Verify partition status changes are reflected +- Test job log expansion and readability +- Validate polling behavior with tab visibility +- Test with delegated builds +- Verify error handling with invalid build IDs +- Check performance with large build requests \ No newline at end of file diff --git a/plans/webapp_v1/chunk-6-partition-pages.md b/plans/webapp_v1/chunk-6-partition-pages.md new file mode 100644 index 0000000..3566d5e --- /dev/null +++ b/plans/webapp_v1/chunk-6-partition-pages.md @@ -0,0 +1,224 @@ +# Chunk 6: Partition Pages + +**Parent:** [Build Graph Dashboard](../build-graph-dashboard.md) +**Previous:** [Chunk 5: Build Status](./chunk-5-build-status.md) +**Next:** [Chunk 7: Jobs Pages](./chunk-7-jobs-pages.md) + +## Overview + +Implement partition listing and individual partition status pages with build history, "Build Now" functionality, and related partition discovery. + +## Scope + +### In Scope +- Partition listing page with search functionality +- Individual partition status pages +- Build history for each partition +- "Build Now" button with force rebuild option +- Related partitions (upstream/downstream dependencies) +- Partition reference URL handling with base64 encoding + +### Out of Scope +- Complex dependency graph visualization +- Partition metadata beyond build history +- Advanced filtering beyond basic search +- Batch operations on multiple partitions + +## Technical Approach + +### Data Sources +From Build Graph Service API: +- `/api/v1/partitions/:ref/status` - Partition status and metadata +- `/api/v1/partitions/:ref/events` - Build events for partition +- `/api/v1/builds` - Submit new build requests +- Recent partitions from build event log + +### URL Encoding +```typescript +// Handle partition references in URLs +function encodePartitionRef(ref: string): string { + return btoa(ref).replace(/\+/g, '-').replace(/\//g, '_').replace(/=/g, ''); +} + +function decodePartitionRef(encoded: string): string { + // Add padding if needed + const padded = encoded.replace(/-/g, '+').replace(/_/g, '/'); + return atob(padded); +} +``` + +### Component Structure +```typescript +const PartitionsList = { + oninit: () => { + this.partitions = []; + this.searchTerm = ''; + this.loadPartitions(); + }, + + view: () => [ + m('.partitions-header', [ + m('h1', 'Partitions'), + m('input', { + placeholder: 'Search partitions...', + oninput: (e) => this.searchTerm = e.target.value, + }) + ]), + + m('.partitions-table', [ + m('table.table', [ + m('thead', [ + m('tr', [ + m('th', 'Partition Reference'), + m('th', 'Status'), + m('th', 'Last Updated'), + m('th', 'Actions'), + ]) + ]), + m('tbody', this.filteredPartitions().map(partition => + m('tr', [ + m('td', m('a', { + href: `/partitions/${encodePartitionRef(partition.ref)}` + }, partition.ref)), + m('td', m('.badge', partition.status)), + m('td', formatTime(partition.last_updated)), + m('td', [ + m('button.btn.btn-sm', { + onclick: () => this.buildPartition(partition.ref) + }, 'Build') + ]) + ]) + )) + ]) + ]) + ] +}; + +const PartitionStatus = { + oninit: (vnode) => { + this.partitionRef = decodePartitionRef(vnode.attrs.base64_ref); + this.partition = null; + this.buildHistory = []; + this.relatedPartitions = []; + this.loadPartition(); + }, + + view: () => [ + m('.partition-header', [ + m('h1', this.partitionRef), + m('.partition-meta', [ + m('.badge', this.partition?.status), + m('.timestamp', formatTime(this.partition?.last_updated)), + ]), + m('.partition-actions', [ + m('button.btn.btn-primary', { + onclick: () => this.buildPartition(false) + }, 'Build Now'), + m('button.btn.btn-secondary', { + onclick: () => this.buildPartition(true) + }, 'Force Rebuild'), + ]) + ]), + + m('.partition-content', [ + m('.build-history', [ + m('h2', 'Build History'), + m('table.table', [ + m('thead', [ + m('tr', [ + m('th', 'Build Request'), + m('th', 'Status'), + m('th', 'Started'), + m('th', 'Completed'), + ]) + ]), + m('tbody', this.buildHistory.map(build => + m('tr', [ + m('td', m('a', { href: `/builds/${build.id}` }, build.id)), + m('td', m('.badge', build.status)), + m('td', formatTime(build.started_at)), + m('td', formatTime(build.completed_at)), + ]) + )) + ]) + ]), + + m('.related-partitions', [ + m('h2', 'Related Partitions'), + m('.partition-deps', [ + m('h3', 'Dependencies'), + m('ul', this.relatedPartitions.dependencies?.map(dep => + m('li', m('a', { + href: `/partitions/${encodePartitionRef(dep)}` + }, dep)) + )) + ]), + m('.partition-dependents', [ + m('h3', 'Dependents'), + m('ul', this.relatedPartitions.dependents?.map(dep => + m('li', m('a', { + href: `/partitions/${encodePartitionRef(dep)}` + }, dep)) + )) + ]) + ]) + ]) + ] +}; +``` + +### Build Now Functionality +- Submit build request for specific partition +- Handle force rebuild option +- Redirect to build status page +- Show loading states during submission + +## Implementation Strategy + +1. **Create Data Layer** + - API integration for partition data + - Search and filtering logic + - Build request submission + +2. **Build List Page** + - Partition table with search + - Status indicators + - Quick build actions + +3. **Individual Partition Pages** + - Partition status display + - Build history table + - Related partitions discovery + +4. **Build Actions** + - "Build Now" functionality + - Force rebuild option + - Error handling and feedback + +## Deliverables + +- [ ] Partition listing page with search +- [ ] Individual partition status pages +- [ ] Build history display +- [ ] "Build Now" and force rebuild functionality +- [ ] Related partitions discovery +- [ ] URL encoding/decoding for partition references + +## Success Criteria + +- Partition list loads and search works correctly +- Individual partition pages display complete information +- Build history shows all relevant builds +- "Build Now" successfully submits builds +- Related partitions are discoverable +- URL encoding handles all partition reference formats + +## Testing + +- Test partition list search with various terms +- Verify individual partition pages load correctly +- Test build history display and links +- Submit builds and verify they start correctly +- Test force rebuild functionality +- Validate URL encoding with complex partition references +- Check related partitions discovery \ No newline at end of file diff --git a/plans/webapp_v1/chunk-7-jobs-pages.md b/plans/webapp_v1/chunk-7-jobs-pages.md new file mode 100644 index 0000000..848a398 --- /dev/null +++ b/plans/webapp_v1/chunk-7-jobs-pages.md @@ -0,0 +1,230 @@ +# Chunk 7: Jobs Pages + +**Parent:** [Build Graph Dashboard](../build-graph-dashboard.md) +**Previous:** [Chunk 6: Partition Pages](./chunk-6-partition-pages.md) +**Next:** [Chunk 8: Graph Analysis](./chunk-8-graph-analysis.md) + +## Overview + +Implement job listing and individual job metrics pages with performance data, success rates, and execution history. + +## Scope + +### In Scope +- Jobs listing page with high-level metadata +- Individual job metrics and performance pages +- Success rate tracking and trends +- Recent job runs with execution details +- Average duration and timing analysis +- Job label URL encoding for safe navigation + +### Out of Scope +- Complex performance analytics +- Historical trend analysis beyond recent runs +- Job configuration editing +- Advanced filtering beyond basic search + +## Technical Approach + +### Data Sources +From Build Graph Service API and build event log: +- Job list from graph analysis +- Job execution history from build events +- Performance metrics aggregated from event data +- Success/failure rates from job events + +### URL Encoding +```typescript +// Handle job labels in URLs (similar to partition refs) +function encodeJobLabel(label: string): string { + return btoa(label).replace(/\+/g, '-').replace(/\//g, '_').replace(/=/g, ''); +} + +function decodeJobLabel(encoded: string): string { + const padded = encoded.replace(/-/g, '+').replace(/_/g, '/'); + return atob(padded); +} +``` + +### Component Structure +```typescript +const JobsList = { + oninit: () => { + this.jobs = []; + this.searchTerm = ''; + this.loadJobs(); + }, + + view: () => [ + m('.jobs-header', [ + m('h1', 'Jobs'), + m('input', { + placeholder: 'Search jobs...', + oninput: (e) => this.searchTerm = e.target.value, + }) + ]), + + m('.jobs-table', [ + m('table.table', [ + m('thead', [ + m('tr', [ + m('th', 'Job Label'), + m('th', 'Success Rate'), + m('th', 'Avg Duration'), + m('th', 'Recent Runs'), + m('th', 'Last Run'), + ]) + ]), + m('tbody', this.filteredJobs().map(job => + m('tr', [ + m('td', m('a', { + href: `/jobs/${encodeJobLabel(job.label)}` + }, job.label)), + m('td', [ + m('.badge', job.success_rate >= 0.9 ? 'success' : 'warning'), + ` ${Math.round(job.success_rate * 100)}%` + ]), + m('td', formatDuration(job.avg_duration)), + m('td', job.recent_runs), + m('td', formatTime(job.last_run)), + ]) + )) + ]) + ]) + ] +}; + +const JobMetrics = { + oninit: (vnode) => { + this.jobLabel = decodeJobLabel(vnode.attrs.label); + this.metrics = null; + this.recentRuns = []; + this.loadJobMetrics(); + }, + + view: () => [ + m('.job-header', [ + m('h1', this.jobLabel), + m('.job-stats', [ + m('.stat', [ + m('.stat-title', 'Success Rate'), + m('.stat-value', `${Math.round(this.metrics?.success_rate * 100)}%`), + ]), + m('.stat', [ + m('.stat-title', 'Avg Duration'), + m('.stat-value', formatDuration(this.metrics?.avg_duration)), + ]), + m('.stat', [ + m('.stat-title', 'Total Runs'), + m('.stat-value', this.metrics?.total_runs), + ]), + ]) + ]), + + m('.job-content', [ + m('.performance-chart', [ + m('h2', 'Performance Trends'), + m('.chart-placeholder', 'Success rate and duration trends over time'), + // Simple trend visualization or table + m('table.table', [ + m('thead', [ + m('tr', [ + m('th', 'Date'), + m('th', 'Success Rate'), + m('th', 'Avg Duration'), + ]) + ]), + m('tbody', this.metrics?.daily_stats?.map(stat => + m('tr', [ + m('td', formatDate(stat.date)), + m('td', `${Math.round(stat.success_rate * 100)}%`), + m('td', formatDuration(stat.avg_duration)), + ]) + )) + ]) + ]), + + m('.recent-runs', [ + m('h2', 'Recent Runs'), + m('table.table', [ + m('thead', [ + m('tr', [ + m('th', 'Build Request'), + m('th', 'Partitions'), + m('th', 'Status'), + m('th', 'Duration'), + m('th', 'Started'), + ]) + ]), + m('tbody', this.recentRuns.map(run => + m('tr', [ + m('td', m('a', { href: `/builds/${run.build_id}` }, run.build_id)), + m('td', run.partitions.join(', ')), + m('td', m('.badge', run.status)), + m('td', formatDuration(run.duration)), + m('td', formatTime(run.started_at)), + ]) + )) + ]) + ]) + ]) + ] +}; +``` + +### Metrics Calculation +From build event log: +- Success rate: completed jobs / total jobs +- Average duration: mean time from SCHEDULED to COMPLETED/FAILED +- Recent runs: last 50 job executions +- Daily aggregations for trend analysis + +## Implementation Strategy + +1. **Create Data Layer** + - API integration for job data + - Metrics calculation from build events + - Search and filtering logic + +2. **Build Jobs List Page** + - Job table with search + - High-level metrics display + - Performance indicators + +3. **Individual Job Pages** + - Detailed metrics display + - Performance trends + - Recent runs history + +4. **Performance Visualization** + - Simple trend charts or tables + - Success rate indicators + - Duration analysis + +## Deliverables + +- [ ] Jobs listing page with search and metrics +- [ ] Individual job metrics pages +- [ ] Success rate tracking and display +- [ ] Performance trends visualization +- [ ] Recent job runs history +- [ ] Job label URL encoding/decoding + +## Success Criteria + +- Jobs list loads with accurate metrics +- Individual job pages show detailed performance data +- Success rates are calculated correctly +- Performance trends are meaningful +- Recent runs link to build details +- URL encoding handles all job label formats + +## Testing + +- Test jobs list search functionality +- Verify individual job pages load correctly +- Validate metrics calculations against known data +- Test performance trend accuracy +- Check recent runs links and data +- Validate URL encoding with complex job labels +- Test with jobs that have no runs yet \ No newline at end of file diff --git a/plans/webapp_v1/chunk-8-graph-analysis.md b/plans/webapp_v1/chunk-8-graph-analysis.md new file mode 100644 index 0000000..f30d760 --- /dev/null +++ b/plans/webapp_v1/chunk-8-graph-analysis.md @@ -0,0 +1,209 @@ +# Chunk 8: Graph Analysis + +**Parent:** [Build Graph Dashboard](../build-graph-dashboard.md) +**Previous:** [Chunk 7: Jobs Pages](./chunk-7-jobs-pages.md) +**Next:** [Chunk 9: Polish](./chunk-9-polish.md) + +## Overview + +Implement interactive build graph analysis with partition input forms, Mermaid.js visualization, and execution plan display. + +## Scope + +### In Scope +- Partition input form for analysis requests +- Mermaid.js visualization of job graphs +- Execution plan table with job details +- Interactive graph exploration +- Error handling for invalid partition references + +### Out of Scope +- Complex graph editing capabilities +- Advanced graph algorithms or analysis +- Performance optimization for large graphs +- Real-time graph updates + +## Technical Approach + +### Data Sources +From Build Graph Service API: +- `/api/v1/analyze` - Analyze build graph for partitions +- Returns `JobGraph` with tasks and dependencies + +### Component Structure +```typescript +const GraphAnalysis = { + oninit: () => { + this.partitions = ['']; + this.jobGraph = null; + this.loading = false; + this.error = null; + this.mermaidRendered = false; + }, + + view: () => [ + m('.analysis-header', [ + m('h1', 'Graph Analysis'), + m('p', 'Analyze the build graph for specific partitions') + ]), + + m('.analysis-form', [ + m('h2', 'Partition References'), + m('.partition-inputs', [ + this.partitions.map((partition, index) => + m('.input-group', [ + m('input', { + value: partition, + placeholder: 'Enter partition reference...', + oninput: (e) => this.partitions[index] = e.target.value, + }), + m('button.btn.btn-outline', { + onclick: () => this.removePartition(index), + disabled: this.partitions.length <= 1 + }, 'Remove') + ]) + ), + m('button.btn.btn-outline', { + onclick: () => this.addPartition() + }, 'Add Partition') + ]), + + m('.form-actions', [ + m('button.btn.btn-primary', { + onclick: () => this.analyzeGraph(), + disabled: this.loading || !this.hasValidPartitions() + }, this.loading ? 'Analyzing...' : 'Analyze Graph') + ]) + ]), + + this.error ? m('.error-message', [ + m('.alert.alert-error', this.error) + ]) : null, + + this.jobGraph ? m('.analysis-results', [ + m('.graph-visualization', [ + m('h2', 'Job Graph'), + m('#mermaid-graph', { + oncreate: () => this.renderMermaid(), + onupdate: () => this.renderMermaid() + }) + ]), + + m('.execution-plan', [ + m('h2', 'Execution Plan'), + m('table.table', [ + m('thead', [ + m('tr', [ + m('th', 'Job'), + m('th', 'Outputs'), + m('th', 'Inputs'), + m('th', 'Arguments'), + ]) + ]), + m('tbody', this.jobGraph.nodes.map(task => + m('tr', [ + m('td', m('a', { + href: `/jobs/${encodeJobLabel(task.job.label)}` + }, task.job.label)), + m('td', m('ul', task.config.outputs.map(output => + m('li', m('a', { + href: `/partitions/${encodePartitionRef(output.str)}` + }, output.str)) + ))), + m('td', m('ul', task.config.inputs.map(input => + m('li', input.partition_ref.str) + ))), + m('td', m('code', task.config.args.join(' '))), + ]) + )) + ]) + ]) + ]) : null + ] +}; +``` + +### Mermaid.js Integration +```typescript +// Generate Mermaid diagram from JobGraph +function generateMermaidDiagram(jobGraph: JobGraph): string { + const nodes = jobGraph.nodes.map(task => + `${task.job.label}["${task.job.label}"]` + ).join('\n '); + + const edges = jobGraph.nodes.flatMap(task => + task.config.inputs.map(input => { + const sourceJob = findJobForPartition(input.partition_ref.str); + return sourceJob ? `${sourceJob} --> ${task.job.label}` : null; + }).filter(Boolean) + ).join('\n '); + + return `graph TD\n ${nodes}\n ${edges}`; +} + +// Render with Mermaid +function renderMermaid() { + if (!this.jobGraph || this.mermaidRendered) return; + + const diagram = generateMermaidDiagram(this.jobGraph); + mermaid.render('graph', diagram, (svgCode) => { + document.getElementById('mermaid-graph').innerHTML = svgCode; + this.mermaidRendered = true; + }); +} +``` + +### Form Management +- Dynamic partition input fields +- Add/remove partition functionality +- Input validation +- Error handling for invalid references + +## Implementation Strategy + +1. **Create Form Interface** + - Dynamic partition input management + - Form validation and submission + - Loading states and error handling + +2. **Integrate Graph Analysis API** + - API calls to analyze endpoint + - Error handling for analysis failures + - JobGraph data processing + +3. **Add Mermaid.js Visualization** + - Include Mermaid.js library + - Generate diagrams from JobGraph + - Handle rendering lifecycle + +4. **Build Execution Plan Display** + - Table layout for job details + - Links to job and partition pages + - Clear display of dependencies + +## Deliverables + +- [ ] Interactive partition input form +- [ ] Mermaid.js graph visualization +- [ ] Execution plan table with job details +- [ ] Error handling for invalid inputs +- [ ] Links to related job and partition pages + +## Success Criteria + +- Form allows adding/removing partition inputs +- Graph analysis API integration works correctly +- Mermaid diagrams render accurately +- Execution plan shows complete job details +- Error handling provides useful feedback +- Links navigate to appropriate detail pages + +## Testing + +- Test form with various partition combinations +- Verify graph analysis API calls work correctly +- Test Mermaid diagram generation and rendering +- Validate execution plan accuracy +- Test error handling with invalid partitions +- Check links to job and partition detail pages +- Test with complex multi-job graphs \ No newline at end of file diff --git a/plans/webapp_v1/chunk-9-polish.md b/plans/webapp_v1/chunk-9-polish.md new file mode 100644 index 0000000..3ffb881 --- /dev/null +++ b/plans/webapp_v1/chunk-9-polish.md @@ -0,0 +1,128 @@ +# Chunk 9: Polish + +**Parent:** [Build Graph Dashboard](../build-graph-dashboard.md) +**Previous:** [Chunk 8: Graph Analysis](./chunk-8-graph-analysis.md) + +## Overview + +Final polish phase focusing on complete Tailwind + DaisyUI styling implementation, performance optimization, user experience improvements, and production readiness. + +## Scope + +### In Scope +- Complete Tailwind + DaisyUI styling across all pages +- Performance optimization to meet bundle size targets +- Error handling and user experience improvements +- Loading states and feedback mechanisms +- Responsive design refinements +- Production build optimization + +### Out of Scope +- New functionality or features +- Complex animations or transitions +- Advanced accessibility features beyond basics +- Internationalization + +## Technical Approach + +### Styling Implementation +Complete DaisyUI component integration: +- Consistent color scheme and typography +- Proper spacing and layout +- Status badges and indicators +- Form styling and validation feedback +- Table and data display improvements + +### Performance Optimization +Target: < 50KB gzipped bundle +- Code splitting for better loading +- Tree shaking unused code +- Optimize CSS bundle size +- Lazy loading for non-critical components +- Bundle analysis and optimization + +### User Experience Improvements +- Consistent loading states across all pages +- Error boundaries and graceful error handling +- Toast notifications for user actions +- Confirmation dialogs for destructive actions +- Keyboard navigation support +- Mobile-responsive design + +### Production Readiness +- Environment-specific configuration +- Error logging and monitoring hooks +- Performance monitoring +- Cache headers and static asset optimization +- Security headers and CSP + +## Implementation Strategy + +1. **Complete Styling System** + - Apply DaisyUI components consistently + - Create shared style utilities + - Implement responsive design patterns + - Add loading and error states + +2. **Performance Optimization** + - Analyze bundle size and optimize + - Implement code splitting + - Optimize CSS and assets + - Add performance monitoring + +3. **User Experience Polish** + - Add feedback mechanisms + - Improve error handling + - Add loading indicators + - Polish interactions + +4. **Production Preparation** + - Environment configuration + - Monitoring and logging + - Security improvements + - Deployment optimization + +## Deliverables + +- [ ] Complete Tailwind + DaisyUI styling implementation +- [ ] Bundle size optimization (< 50KB gzipped) +- [ ] Comprehensive error handling and user feedback +- [ ] Loading states and progress indicators +- [ ] Mobile-responsive design +- [ ] Production-ready build configuration + +## Success Criteria + +- All pages have consistent, professional styling +- Bundle size meets target (< 50KB gzipped) +- Error handling provides helpful user feedback +- Loading states appear where appropriate +- Dashboard works well on mobile devices +- Production build is optimized and secure + +## Testing + +- Test all pages for consistent styling +- Verify bundle size meets targets +- Test error handling scenarios +- Validate loading states and feedback +- Test responsive design on various devices +- Verify production build works correctly +- Performance testing under load + +## Bundle Size Breakdown Target +- Mithril: ~10KB +- Custom code: ~20KB +- CSS (Tailwind + DaisyUI): ~5KB +- Protobuf client: ~15KB +- **Total: < 50KB gzipped** + +## Final Checklist +- [ ] All pages styled with DaisyUI +- [ ] Bundle size optimized +- [ ] Error handling complete +- [ ] Loading states implemented +- [ ] Mobile responsive +- [ ] Production build ready +- [ ] Performance targets met +- [ ] Security considerations addressed \ No newline at end of file