Update Vibe Kanban setup and scripts
- Added new entries to .gitignore for environment files and dynamically generated Docker Compose files. - Updated documentation to reflect the new path for the Vibe Kanban dev server script. - Enhanced task composition scripts to extract TASK_SLOT from TASK_ID, ensuring unique Orleans ports and preventing conflicts. - Removed the old vibe-dev-server script, consolidating functionality into the new structure.
This commit is contained in:
122
docs/API_AND_WORKERS_PROCESSES.md
Normal file
122
docs/API_AND_WORKERS_PROCESSES.md
Normal file
@@ -0,0 +1,122 @@
|
||||
# API and Workers Processes
|
||||
|
||||
This document lists all processes that run when the API and Workers are started.
|
||||
|
||||
## Process Hierarchy
|
||||
|
||||
### 1. API Process (`dotnet run` for Managing.Api)
|
||||
|
||||
**Parent Process:**
|
||||
- `dotnet run` - The .NET CLI process that starts the API
|
||||
- PID stored in: `.task-pids/api-${TASK_ID}.pid`
|
||||
|
||||
**Child Process:**
|
||||
- `Managing.Api` executable - The actual API application
|
||||
- Location: `src/Managing.Api/bin/Debug/net8.0/Managing.Api`
|
||||
- This is the main ASP.NET Core application
|
||||
|
||||
**Background Services (within the API process):**
|
||||
All of these run as `IHostedService` within the same API process:
|
||||
|
||||
1. **GrainInitializer** - Initializes Orleans grains
|
||||
2. **DiscordService** - Discord bot service
|
||||
3. **PricesFifteenMinutesWorker** - Updates prices every 15 minutes (if enabled)
|
||||
4. **PricesOneHourWorker** - Updates prices every hour (if enabled)
|
||||
5. **PricesFourHoursWorker** - Updates prices every 4 hours (if enabled)
|
||||
6. **PricesOneDayWorker** - Updates prices every day (if enabled)
|
||||
7. **PricesFiveMinutesWorker** - Updates prices every 5 minutes (if enabled)
|
||||
8. **SpotlightWorker** - Spotlight feature worker (if enabled)
|
||||
9. **TraderWatcher** - Watches traders (if enabled)
|
||||
10. **LeaderboardWorker** - Updates leaderboard (if enabled)
|
||||
11. **FundingRatesWatcher** - Watches funding rates (if enabled)
|
||||
12. **GeneticAlgorithmWorker** - Genetic algorithm worker (if enabled)
|
||||
13. **NotifyBundleBacktestWorker** - Notifies about bundle backtests (if enabled)
|
||||
|
||||
**Orleans Components (within the API process):**
|
||||
- Orleans Silo - Runs on port `11111 + (TASK_SLOT - 1) * 10`
|
||||
- Orleans Gateway - Runs on port `30000 + (TASK_SLOT - 1) * 10`
|
||||
- Orleans Dashboard - Runs on port `9999 + (TASK_SLOT - 1)` (development only)
|
||||
|
||||
### 2. Workers Process (`dotnet run` for Managing.Workers)
|
||||
|
||||
**Parent Process:**
|
||||
- `dotnet run` - The .NET CLI process that starts the Workers
|
||||
- PID stored in: `.task-pids/workers-${TASK_ID}.pid`
|
||||
|
||||
**Child Process:**
|
||||
- `Managing.Workers` executable - The actual Workers application
|
||||
- Location: `src/Managing.Workers/bin/Debug/net8.0/Managing.Workers`
|
||||
- This is a .NET Host application
|
||||
|
||||
**Background Services (within the Workers process):**
|
||||
All of these run as `BackgroundService` within the same Workers process:
|
||||
|
||||
1. **BacktestComputeWorker** - Processes backtest jobs (if enabled)
|
||||
2. **GeneticComputeWorker** - Processes genetic algorithm jobs (if enabled)
|
||||
3. **BundleBacktestHealthCheckWorker** - Health check for bundle backtests (if enabled, only on TASK_SLOT=1)
|
||||
|
||||
## Process Management
|
||||
|
||||
### Starting Processes
|
||||
Processes are started by `scripts/start-api-and-workers.sh`:
|
||||
- API: `cd src/Managing.Api && dotnet run &`
|
||||
- Workers: `cd src/Managing.Workers && dotnet run &`
|
||||
|
||||
### Stopping Processes
|
||||
Processes are stopped by `scripts/stop-task-docker.sh` or the cleanup script:
|
||||
1. Read PID from `.task-pids/api-${TASK_ID}.pid`
|
||||
2. Kill the parent `dotnet run` process
|
||||
3. Kill any orphaned child processes
|
||||
4. Read PID from `.task-pids/workers-${TASK_ID}.pid`
|
||||
5. Kill the parent `dotnet run` process
|
||||
6. Kill any orphaned child processes
|
||||
|
||||
### Finding All Related Processes
|
||||
|
||||
To find all processes related to a specific task:
|
||||
|
||||
```bash
|
||||
# Find by PID file
|
||||
TASK_ID="YOUR_TASK_ID"
|
||||
API_PID=$(cat .task-pids/api-${TASK_ID}.pid 2>/dev/null)
|
||||
WORKERS_PID=$(cat .task-pids/workers-${TASK_ID}.pid 2>/dev/null)
|
||||
|
||||
# Find child processes
|
||||
ps -ef | grep $API_PID
|
||||
ps -ef | grep $WORKERS_PID
|
||||
|
||||
# Find by executable name
|
||||
ps aux | grep "Managing.Api"
|
||||
ps aux | grep "Managing.Workers"
|
||||
ps aux | grep "dotnet run"
|
||||
```
|
||||
|
||||
### Finding Processes by Port
|
||||
|
||||
```bash
|
||||
# Find API process by port
|
||||
lsof -i :5000 # Default API port
|
||||
lsof -i :$((5000 + PORT_OFFSET)) # With port offset
|
||||
|
||||
# Find Orleans processes by port
|
||||
lsof -i :11111 # Orleans Silo (default)
|
||||
lsof -i :30000 # Orleans Gateway (default)
|
||||
```
|
||||
|
||||
## Important Notes
|
||||
|
||||
1. **Single Process Architecture**: All background services run within the same process as the API or Workers. They are not separate processes.
|
||||
|
||||
2. **PID Files**: The PID files store the parent `dotnet run` process PID, not the child executable PID.
|
||||
|
||||
3. **Orphaned Processes**: If the parent `dotnet run` process is killed, the child `Managing.Api` or `Managing.Workers` process may become orphaned. The cleanup script should handle this.
|
||||
|
||||
4. **Port Conflicts**: Each task uses unique ports based on `PORT_OFFSET`:
|
||||
- API: `5000 + PORT_OFFSET`
|
||||
- PostgreSQL: `5432 + PORT_OFFSET`
|
||||
- Redis: `6379 + PORT_OFFSET`
|
||||
- Orleans Silo: `11111 + (TASK_SLOT - 1) * 10`
|
||||
- Orleans Gateway: `30000 + (TASK_SLOT - 1) * 10`
|
||||
|
||||
5. **Worker Consolidation**: Most workers have been consolidated into the API process. The separate `Managing.Workers` project now only runs compute-intensive workers (BacktestComputeWorker, GeneticComputeWorker).
|
||||
|
||||
@@ -9,7 +9,7 @@ Vibe Kanban runs the dev server script from a different working directory than e
|
||||
In the Vibe Kanban dev server script field, use the **absolute path** to the wrapper script:
|
||||
|
||||
```bash
|
||||
bash /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-dev-server.sh
|
||||
bash /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-kanban/vibe-dev-server.sh
|
||||
```
|
||||
|
||||
## Alternative: If Relative Path is Required
|
||||
@@ -17,7 +17,7 @@ bash /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-dev-server.sh
|
||||
If Vibe Kanban requires a relative path and you know it runs from `/Users/oda/Desktop`, use:
|
||||
|
||||
```bash
|
||||
bash Projects/managing-apps/scripts/vibe-dev-server.sh
|
||||
bash Projects/managing-apps/scripts/vibe-kanban/vibe-dev-server.sh
|
||||
```
|
||||
|
||||
## What the Wrapper Script Does
|
||||
@@ -36,7 +36,7 @@ You can test the script manually:
|
||||
|
||||
```bash
|
||||
# From any directory
|
||||
bash /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-dev-server.sh TEST-001 0
|
||||
bash /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-kanban/vibe-dev-server.sh TEST-001 0
|
||||
```
|
||||
|
||||
## Debug Output
|
||||
@@ -54,17 +54,17 @@ This helps identify if Vibe Kanban is running from an unexpected directory.
|
||||
|
||||
1. Verify the script exists:
|
||||
```bash
|
||||
ls -la /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-dev-server.sh
|
||||
ls -la /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-kanban/vibe-dev-server.sh
|
||||
```
|
||||
|
||||
2. Check permissions:
|
||||
```bash
|
||||
chmod +x /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-dev-server.sh
|
||||
chmod +x /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-kanban/vibe-dev-server.sh
|
||||
```
|
||||
|
||||
3. Try running it directly:
|
||||
```bash
|
||||
bash /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-dev-server.sh TEST-001
|
||||
bash /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-kanban/vibe-dev-server.sh TEST-001
|
||||
```
|
||||
|
||||
### Error: "Cannot change to project root"
|
||||
@@ -76,7 +76,7 @@ This helps identify if Vibe Kanban is running from an unexpected directory.
|
||||
|
||||
**Dev Server Script Field:**
|
||||
```
|
||||
bash /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-dev-server.sh
|
||||
bash /Users/oda/Desktop/Projects/managing-apps/scripts/vibe-kanban/vibe-dev-server.sh
|
||||
```
|
||||
|
||||
**Health Check URL:**
|
||||
|
||||
@@ -29,12 +29,12 @@ In the **QA** or **Testing** section, configure:
|
||||
|
||||
**Dev Environment Script:**
|
||||
```
|
||||
managing-apps/scripts/vibe-dev-server.sh
|
||||
managing-apps/scripts/vibe-kanban/vibe-dev-server.sh
|
||||
```
|
||||
|
||||
**Or use absolute path:**
|
||||
```
|
||||
/Users/oda/Desktop/Projects/managing-apps/scripts/vibe-dev-server.sh
|
||||
/Users/oda/Desktop/Projects/managing-apps/scripts/vibe-kanban/vibe-dev-server.sh
|
||||
```
|
||||
|
||||
**Note:** This path is relative to `/Users/oda/Desktop/Projects` (the Projects folder root)
|
||||
|
||||
Reference in New Issue
Block a user