managing-apps/assets/documentation/Workers processing/02-Request-Flow.md

Request Flow Sequence Diagram

This diagram shows the complete request flow from user submission to job completion and status polling.

sequenceDiagram
    participant User
    participant API as API Server<br/>(Orleans)
    participant DB as PostgreSQL<br/>(Job Queue)
    participant Worker as Compute Worker
    participant Influx as InfluxDB
    
    User->>API: POST /api/backtest/bundle
    API->>API: Create BundleBacktestRequest
    API->>API: Generate BacktestJobs from variants
    API->>DB: INSERT BacktestJobs (Status: Pending)
    API-->>User: 202 Accepted<br/>{bundleRequestId, status: "Queued"}
    
    Note over Worker: Polling every 5 seconds
    Worker->>DB: SELECT pending jobs<br/>(ORDER BY priority, createdAt)
    DB-->>Worker: Return pending jobs
    Worker->>DB: UPDATE job<br/>(Status: Running, AssignedWorkerId)
    Worker->>Influx: Load candles for backtest
    Influx-->>Worker: Return candles
    
    loop Process each candle
        Worker->>Worker: Run backtest logic
        Worker->>DB: UPDATE job progress
    end
    
    Worker->>DB: UPDATE job<br/>(Status: Completed, ResultJson)
    Worker->>DB: UPDATE BundleBacktestRequest<br/>(CompletedBacktests++)
    
    User->>API: GET /api/backtest/bundle/{id}/status
    API->>DB: SELECT BundleBacktestRequest + job stats
    DB-->>API: Return status
    API-->>User: {status, progress, completed/total}

Flow Steps

1. User Request: User submits bundle backtest request
2. API Processing: API creates bundle request and generates individual backtest jobs
3. Job Queue: Jobs are inserted into database with Pending status
4. Immediate Response: API returns 202 Accepted with bundle request ID
5. Worker Polling: Compute workers poll database every 5 seconds
6. Job Claiming: Worker claims jobs using PostgreSQL advisory locks
7. Candle Loading: Worker loads candles from InfluxDB
8. Backtest Processing: Worker processes backtest with progress updates
9. Result Storage: Worker saves results and updates bundle progress
10. Status Polling: User polls API for status updates