Update perf

This commit is contained in:
2025-11-11 14:19:41 +07:00
parent 903413692c
commit 583b35d209
4 changed files with 45 additions and 98 deletions

View File

@@ -122,30 +122,38 @@ Recent performance logging revealed the **true bottleneck** in backtest executio
3. **Optimize signal update logic** - Further reduce unnecessary updates
4. **Memory pooling** - Reuse objects to reduce GC pressure
## Major Optimization Success: Pre-Calculated Signals
## Major Optimization Attempt: Pre-Calculated Signals (REVERTED)
### **Optimization: Pre-Calculated Signals**
**What was implemented**: Pre-calculated all signals once upfront instead of calling `TradingBox.GetSignal()` ~1,932 times during backtest execution.
### **Optimization: Pre-Calculated Signals - REVERTED**
**What was attempted**: Pre-calculate all signals once upfront to avoid calling `TradingBox.GetSignal()` repeatedly.
**Technical Details**:
- Added `PreCalculateAllSignals()` method in `BacktestExecutor.cs`
- Pre-calculates signals for all candles using rolling window logic
- Modified `TradingBotBase.UpdateSignals()` to support pre-calculated signal lookup
- Updated backtest loop to use O(1) signal lookups instead of expensive calculations
**Why it failed**: The approach was fundamentally flawed because:
- Signal generation depends on the current rolling window state
- Pre-calculating signals upfront still required calling the expensive `TradingBox.GetSignal()` method N times
- The lookup mechanism failed due to date matching issues
- Net result: Double the work with no performance benefit
**Performance Impact** (Average of 3 runs):
- **Processing Rate**: 2,800 → **~5,800 candles/sec** (2.1x improvement!)
- **Execution Time**: 1.4-1.6s → **~1.0s** (35-50% faster!)
- **Signal Update Time**: ~1,417ms → **Eliminated** (no more repeated calculations)
- **Consistent Results**: 5,217 - 6,871 candles/sec range (expected system variance)
**Technical Issues**:
- Pre-calculated signals were not found during lookup (every candle fell back to on-the-fly calculation)
- Signal calculation depends on dynamic rolling window state that cannot be pre-calculated
- Added complexity without performance benefit
**Business Logic Validation**:
**Result**: Reverted to original `TradingBox.GetSignal()` approach with signal update frequency optimization.
**Takeaway**: Not all "optimizations" work. The signal generation logic is inherently dependent on current market state and cannot be effectively pre-calculated.
## Current Performance Status (Post-Reversion)
After reverting the flawed pre-calculated signals optimization, performance is **excellent**:
-**Processing Rate**: 3,000-7,000 candles/sec (excellent performance with expected system variance)
-**Execution Time**: 0.8-1.8s for 5760 candles (depends on system load)
-**Signal Update Efficiency**: 66.5% (reduces updates by 2.8x)
-**Memory Usage**: 23.73MB peak
- ✅ All validation tests passed
-Final PnL matches baseline (±0)
- ✅ Two-scenarios test includes baseline assertions for consistency over time (with proper win rate percentage handling)
- ✅ Live trading functionality preserved (no changes to live trading code)
-Business logic integrity maintained
**Takeaway**: The biggest performance gains come from eliminating redundant calculations. Pre-calculating expensive operations once upfront is far more effective than micro-optimizations.
The **signal update frequency optimization** remains in place and provides significant performance benefits without breaking business logic.
## Safe Optimization Strategies