Add doc for workers architecture

This commit is contained in:
2025-11-07 15:34:13 +07:00
parent 2dc34f07d8
commit 5578d272fa
9 changed files with 672 additions and 0 deletions

View File

@@ -0,0 +1,52 @@
# Request Flow Sequence Diagram
This diagram shows the complete request flow from user submission to job completion and status polling.
```mermaid
sequenceDiagram
participant User
participant API as API Server<br/>(Orleans)
participant DB as PostgreSQL<br/>(Job Queue)
participant Worker as Compute Worker
participant Influx as InfluxDB
User->>API: POST /api/backtest/bundle
API->>API: Create BundleBacktestRequest
API->>API: Generate BacktestJobs from variants
API->>DB: INSERT BacktestJobs (Status: Pending)
API-->>User: 202 Accepted<br/>{bundleRequestId, status: "Queued"}
Note over Worker: Polling every 5 seconds
Worker->>DB: SELECT pending jobs<br/>(ORDER BY priority, createdAt)
DB-->>Worker: Return pending jobs
Worker->>DB: UPDATE job<br/>(Status: Running, AssignedWorkerId)
Worker->>Influx: Load candles for backtest
Influx-->>Worker: Return candles
loop Process each candle
Worker->>Worker: Run backtest logic
Worker->>DB: UPDATE job progress
end
Worker->>DB: UPDATE job<br/>(Status: Completed, ResultJson)
Worker->>DB: UPDATE BundleBacktestRequest<br/>(CompletedBacktests++)
User->>API: GET /api/backtest/bundle/{id}/status
API->>DB: SELECT BundleBacktestRequest + job stats
DB-->>API: Return status
API-->>User: {status, progress, completed/total}
```
## Flow Steps
1. **User Request**: User submits bundle backtest request
2. **API Processing**: API creates bundle request and generates individual backtest jobs
3. **Job Queue**: Jobs are inserted into database with `Pending` status
4. **Immediate Response**: API returns 202 Accepted with bundle request ID
5. **Worker Polling**: Compute workers poll database every 5 seconds
6. **Job Claiming**: Worker claims jobs using PostgreSQL advisory locks
7. **Candle Loading**: Worker loads candles from InfluxDB
8. **Backtest Processing**: Worker processes backtest with progress updates
9. **Result Storage**: Worker saves results and updates bundle progress
10. **Status Polling**: User polls API for status updates