# Implementation Plan

## Phase 1: Database & Domain Setup

- [ ] Create `BacktestJob` entity in `Managing.Domain/Backtests/`
- [ ] Create `BacktestJobStatus` enum (Pending, Running, Completed, Failed)
- [ ] Create database migration for `BacktestJobs` table
- [ ] Add indexes: `idx_status_priority`, `idx_bundle_request`, `idx_assigned_worker`
- [ ] Create `IBacktestJobRepository` interface
- [ ] Implement `BacktestJobRepository` with advisory lock support

## Phase 2: Compute Worker Project

- [ ] Refactor `Managing.Workers.Api` project (or rename to `Managing.Compute`)
- [ ] Remove Orleans dependencies completely
- [ ] Add project references to shared projects (Application, Domain, Infrastructure)
- [ ] Configure DI container with all required services (NO Orleans)
- [ ] Create `BacktestComputeWorker` background service
- [ ] Implement job polling logic (every 5 seconds)
- [ ] Implement job claiming with PostgreSQL advisory locks
- [ ] Implement semaphore-based concurrency control
- [ ] Implement progress callback mechanism
- [ ] Implement heartbeat mechanism (every 30 seconds)
- [ ] Add configuration: `MaxConcurrentBacktests`, `JobPollIntervalSeconds`, `WorkerId`

## Phase 3: API Server Updates

- [ ] Update `BacktestController` to create jobs instead of calling grains directly
- [ ] Implement `CreateBundleBacktest` endpoint (returns immediately)
- [ ] Implement `GetJobStatus` endpoint (polls database for single job)
- [ ] Implement `GetBundleStatus` endpoint (polls database, aggregates job statuses)
- [ ] Update `Backtester.cs` to generate `BacktestJob` entities from bundle variants
- [ ] Remove all Orleans grain calls for backtests (direct replacement, no feature flags)
- [ ] Remove `IGrainFactory` dependency from `Backtester.cs`

## Phase 4: Shared Logic Extraction

- [ ] Create `BacktestExecutor.cs` service (new file)
- [ ] Extract backtest execution logic from `BacktestTradingBotGrain` to `BacktestExecutor`
- [ ] Make backtest logic Orleans-agnostic (no grain dependencies)
- [ ] Add progress callback support to execution method
- [ ] Ensure candle loading works in compute worker context
- [ ] Handle credit debiting/refunding in executor
- [ ] Handle user context resolution in executor

## Phase 5: Monitoring & Health Checks

- [ ] Add health check endpoint to compute worker (`/health` or `/healthz`)
- [ ] Add metrics: pending jobs, running jobs, completed/failed counts
- [ ] Add stale job detection (reclaim jobs from dead workers, LastHeartbeat > 5 min)
- [ ] Add comprehensive logging for job lifecycle events
- [ ] Include structured logging: JobId, BundleRequestId, UserId, WorkerId, Duration

## Phase 6: SignalR & Notifications

- [ ] Inject `IHubContext<BacktestHub>` into compute worker or executor
- [ ] Send SignalR progress updates during job execution
- [ ] Update `BacktestJob.ProgressPercentage` in database
- [ ] Update `BundleBacktestRequest` progress when jobs complete
- [ ] Send completion notifications via SignalR and Telegram

## Phase 7: Deployment

- [ ] Create Dockerfile for `Managing.Compute` (or update existing)
- [ ] Update `docker-compose.yml` to add compute worker service
- [ ] Configure environment variables: `MaxConcurrentBacktests`, `JobPollIntervalSeconds`, `WorkerId`
- [ ] Set up health check configuration in Docker
- [ ] Configure auto-scaling rules for compute workers (min: 1, max: 10)

## Phase 9: Testing & Validation

- [ ] Unit tests: BacktestJobRepository (advisory locks, job claiming, stale detection)
- [ ] Unit tests: BacktestExecutor (core logic, progress callbacks)
- [ ] Integration tests: Single backtest job processing
- [ ] Integration tests: Bundle backtest with multiple jobs
- [ ] Integration tests: Concurrent job processing (multiple workers)
- [ ] Integration tests: Job recovery after worker failure
- [ ] Integration tests: Priority queue ordering
- [ ] Load tests: 100+ concurrent users, 1000+ pending jobs, multiple workers

## Phase 8: Cleanup & Removal

- [ ] Remove or deprecate `BacktestTradingBotGrain.cs` (no longer used)
- [ ] Remove or deprecate `BundleBacktestGrain.cs` (replaced by compute workers)
- [ ] Remove Orleans grain interfaces for backtests (if not used elsewhere)
- [ ] Update `ApiBootstrap.cs` to remove Orleans backtest grain registrations
- [ ] Remove Orleans dependencies from `Backtester.cs` (keep for other operations)
- [ ] Update documentation to reflect new architecture