Add doc for workers architecture
This commit is contained in:
52
assets/documentation/Workers processing/02-Request-Flow.md
Normal file
52
assets/documentation/Workers processing/02-Request-Flow.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# Request Flow Sequence Diagram
|
||||
|
||||
This diagram shows the complete request flow from user submission to job completion and status polling.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant API as API Server<br/>(Orleans)
|
||||
participant DB as PostgreSQL<br/>(Job Queue)
|
||||
participant Worker as Compute Worker
|
||||
participant Influx as InfluxDB
|
||||
|
||||
User->>API: POST /api/backtest/bundle
|
||||
API->>API: Create BundleBacktestRequest
|
||||
API->>API: Generate BacktestJobs from variants
|
||||
API->>DB: INSERT BacktestJobs (Status: Pending)
|
||||
API-->>User: 202 Accepted<br/>{bundleRequestId, status: "Queued"}
|
||||
|
||||
Note over Worker: Polling every 5 seconds
|
||||
Worker->>DB: SELECT pending jobs<br/>(ORDER BY priority, createdAt)
|
||||
DB-->>Worker: Return pending jobs
|
||||
Worker->>DB: UPDATE job<br/>(Status: Running, AssignedWorkerId)
|
||||
Worker->>Influx: Load candles for backtest
|
||||
Influx-->>Worker: Return candles
|
||||
|
||||
loop Process each candle
|
||||
Worker->>Worker: Run backtest logic
|
||||
Worker->>DB: UPDATE job progress
|
||||
end
|
||||
|
||||
Worker->>DB: UPDATE job<br/>(Status: Completed, ResultJson)
|
||||
Worker->>DB: UPDATE BundleBacktestRequest<br/>(CompletedBacktests++)
|
||||
|
||||
User->>API: GET /api/backtest/bundle/{id}/status
|
||||
API->>DB: SELECT BundleBacktestRequest + job stats
|
||||
DB-->>API: Return status
|
||||
API-->>User: {status, progress, completed/total}
|
||||
```
|
||||
|
||||
## Flow Steps
|
||||
|
||||
1. **User Request**: User submits bundle backtest request
|
||||
2. **API Processing**: API creates bundle request and generates individual backtest jobs
|
||||
3. **Job Queue**: Jobs are inserted into database with `Pending` status
|
||||
4. **Immediate Response**: API returns 202 Accepted with bundle request ID
|
||||
5. **Worker Polling**: Compute workers poll database every 5 seconds
|
||||
6. **Job Claiming**: Worker claims jobs using PostgreSQL advisory locks
|
||||
7. **Candle Loading**: Worker loads candles from InfluxDB
|
||||
8. **Backtest Processing**: Worker processes backtest with progress updates
|
||||
9. **Result Storage**: Worker saves results and updates bundle progress
|
||||
10. **Status Polling**: User polls API for status updates
|
||||
|
||||
Reference in New Issue
Block a user