Add doc for workers architecture

assets/documentation/Workers processing/README.md

@@ -0,0 +1,75 @@
+# Workers Processing Architecture
+
+This folder contains documentation for the enterprise-grade backtest processing architecture using a database queue pattern with separate API and Compute worker clusters.
+
+## Overview
+
+The architecture separates concerns between:
+- **API Server**: Handles HTTP requests, creates jobs, returns immediately (fire-and-forget)
+- **Compute Workers**: Process CPU-intensive backtest jobs from the database queue
+- **Database Queue**: Central coordination point using PostgreSQL
+
+## Documentation Files
+
+1. **[01-Overall-Architecture.md](./01-Overall-Architecture.md)**
+   - Complete system architecture diagram
+   - Component relationships
+   - External service integrations
+
+2. **[02-Request-Flow.md](./02-Request-Flow.md)**
+   - Sequence diagram of request flow
+   - User request → Job creation → Processing → Status polling
+
+3. **[03-Job-Processing-Flow.md](./03-Job-Processing-Flow.md)**
+   - Detailed job processing workflow
+   - Worker polling, job claiming, semaphore control
+
+4. **[04-Database-Schema.md](./04-Database-Schema.md)**
+   - Entity relationship diagram
+   - Database schema for job queue
+   - Key indexes and relationships
+
+5. **[05-Deployment-Architecture.md](./05-Deployment-Architecture.md)**
+   - Production deployment topology
+   - Load balancing, clustering, monitoring
+
+6. **[06-Concurrency-Control.md](./06-Concurrency-Control.md)**
+   - Concurrency control mechanisms
+   - Semaphore-based limiting
+   - Capacity calculations
+
+7. **[07-Monorepo-Structure.md](./07-Monorepo-Structure.md)**
+   - Monorepo project organization
+   - Shared projects and dependencies
+
+## Key Features
+
+- ✅ **No Timeouts**: Fire-and-forget pattern with polling
+- ✅ **Scalable**: Horizontal scaling of both API and Compute clusters
+- ✅ **Reliable**: Jobs persist in database, survive restarts
+- ✅ **Efficient**: Dedicated CPU resources for compute work
+- ✅ **Enterprise-Grade**: Handles 1000+ users, priority queue, health checks
+
+## Architecture Principles
+
+1. **Separation of Concerns**: API handles requests, Compute handles CPU work
+2. **Database as Queue**: PostgreSQL serves as reliable job queue
+3. **Shared Codebase**: Monorepo with shared business logic
+4. **Resource Isolation**: Compute workers don't interfere with API responsiveness
+5. **Fault Tolerance**: Jobs survive worker failures, can be reclaimed
+
+## Capacity Planning
+
+- **Per Worker**: 6 concurrent backtests (8-core machine)
+- **3 Workers**: 18 concurrent backtests
+- **Throughput**: ~1,080 backtests/hour
+- **1000 Users × 10 backtests**: ~9 hours processing time
+
+## Next Steps
+
+1. Create `Managing.Compute` project
+2. Implement `BacktestJob` entity and repository
+3. Create `BacktestComputeWorker` background service
+4. Update API controllers to use job queue pattern
+5. Deploy compute workers to dedicated servers
+