managing-apps/docs/API_AND_WORKERS_PROCESSES.md

API and Workers Processes

This document lists all processes that run when the API and Workers are started.

Process Hierarchy

1. API Process (dotnet run for Managing.Api)

Parent Process:

• dotnet run - The .NET CLI process that starts the API
  • PID stored in: .task-pids/api-${TASK_ID}.pid

Child Process:

• Managing.Api executable - The actual API application
  • Location: src/Managing.Api/bin/Debug/net8.0/Managing.Api
  • This is the main ASP.NET Core application

Background Services (within the API process): All of these run as IHostedService within the same API process:

1. GrainInitializer - Initializes Orleans grains
2. DiscordService - Discord bot service
3. PricesFifteenMinutesWorker - Updates prices every 15 minutes (if enabled)
4. PricesOneHourWorker - Updates prices every hour (if enabled)
5. PricesFourHoursWorker - Updates prices every 4 hours (if enabled)
6. PricesOneDayWorker - Updates prices every day (if enabled)
7. PricesFiveMinutesWorker - Updates prices every 5 minutes (if enabled)
8. SpotlightWorker - Spotlight feature worker (if enabled)
9. TraderWatcher - Watches traders (if enabled)
10. LeaderboardWorker - Updates leaderboard (if enabled)
11. FundingRatesWatcher - Watches funding rates (if enabled)
12. GeneticAlgorithmWorker - Genetic algorithm worker (if enabled)
13. NotifyBundleBacktestWorker - Notifies about bundle backtests (if enabled)

Orleans Components (within the API process):

• Orleans Silo - Runs on port 11111 + (TASK_SLOT - 1) * 10
• Orleans Gateway - Runs on port 30000 + (TASK_SLOT - 1) * 10
• Orleans Dashboard - Runs on port 9999 + (TASK_SLOT - 1) (development only)

2. Workers Process (dotnet run for Managing.Workers)

Parent Process:

• dotnet run - The .NET CLI process that starts the Workers
  • PID stored in: .task-pids/workers-${TASK_ID}.pid

Child Process:

• Managing.Workers executable - The actual Workers application
  • Location: src/Managing.Workers/bin/Debug/net8.0/Managing.Workers
  • This is a .NET Host application

Background Services (within the Workers process): All of these run as BackgroundService within the same Workers process:

1. BacktestComputeWorker - Processes backtest jobs (if enabled)
2. GeneticComputeWorker - Processes genetic algorithm jobs (if enabled)
3. BundleBacktestHealthCheckWorker - Health check for bundle backtests (if enabled, only on TASK_SLOT=1)

Process Management

Starting Processes

Processes are started by scripts/start-api-and-workers.sh:

• API: cd src/Managing.Api && dotnet run &
• Workers: cd src/Managing.Workers && dotnet run &

Stopping Processes

Processes are stopped by scripts/stop-task-docker.sh or the cleanup script:

1. Read PID from .task-pids/api-${TASK_ID}.pid
2. Kill the parent dotnet run process
3. Kill any orphaned child processes
4. Read PID from .task-pids/workers-${TASK_ID}.pid
5. Kill the parent dotnet run process
6. Kill any orphaned child processes

Finding All Related Processes

To find all processes related to a specific task:

# Find by PID file
TASK_ID="YOUR_TASK_ID"
API_PID=$(cat .task-pids/api-${TASK_ID}.pid 2>/dev/null)
WORKERS_PID=$(cat .task-pids/workers-${TASK_ID}.pid 2>/dev/null)

# Find child processes
ps -ef | grep $API_PID
ps -ef | grep $WORKERS_PID

# Find by executable name
ps aux | grep "Managing.Api"
ps aux | grep "Managing.Workers"
ps aux | grep "dotnet run"

Finding Processes by Port

# Find API process by port
lsof -i :5000  # Default API port
lsof -i :$((5000 + PORT_OFFSET))  # With port offset

# Find Orleans processes by port
lsof -i :11111  # Orleans Silo (default)
lsof -i :30000  # Orleans Gateway (default)

Important Notes

1. Single Process Architecture: All background services run within the same process as the API or Workers. They are not separate processes.

2. PID Files: The PID files store the parent dotnet run process PID, not the child executable PID.

3. Orphaned Processes: If the parent dotnet run process is killed, the child Managing.Api or Managing.Workers process may become orphaned. The cleanup script should handle this.

4. Port Conflicts: Each task uses unique ports based on PORT_OFFSET:

   • API: 5000 + PORT_OFFSET
   • PostgreSQL: 5432 + PORT_OFFSET
   • Redis: 6379 + PORT_OFFSET
   • Orleans Silo: 11111 + (TASK_SLOT - 1) * 10
   • Orleans Gateway: 30000 + (TASK_SLOT - 1) * 10

5. Worker Consolidation: Most workers have been consolidated into the API process. The separate Managing.Workers project now only runs compute-intensive workers (BacktestComputeWorker, GeneticComputeWorker).