92 lines
3.5 KiB
Markdown
92 lines
3.5 KiB
Markdown
# Benchmark Backtest Performance
|
||
|
||
This command runs the backtest performance tests and records the results in the performance benchmark CSV file.
|
||
|
||
## Usage
|
||
|
||
Run this command to benchmark backtest performance and update the tracking CSV:
|
||
|
||
```
|
||
/benchmark-backtest-performance
|
||
```
|
||
|
||
Or run the script directly:
|
||
|
||
```bash
|
||
./scripts/benchmark-backtest-performance.sh
|
||
```
|
||
|
||
## What it does
|
||
|
||
1. Runs the performance telemetry test (`ExecuteBacktest_With_Large_Dataset_Should_Show_Performance_Telemetry`)
|
||
2. **Validates Business Logic**: Compares Final PnL with the first run of the file to ensure optimizations don't break behavior
|
||
3. Extracts performance metrics from the test output
|
||
4. Appends a new row to `src/Managing.Workers.Tests/performance-benchmarks.csv`
|
||
5. **Never commits changes automatically**
|
||
|
||
## CSV Format
|
||
|
||
The CSV file contains clean numeric values for all telemetry metrics:
|
||
|
||
- `DateTime`: ISO 8601 timestamp when the benchmark was run
|
||
- `TestName`: Name of the test that was executed
|
||
- `CandlesCount`: Integer - Number of candles processed
|
||
- `ExecutionTimeSeconds`: Decimal - Total execution time in seconds
|
||
- `ProcessingRateCandlesPerSec`: Decimal - Candles processed per second
|
||
- `MemoryStartMB`: Decimal - Memory usage at start
|
||
- `MemoryEndMB`: Decimal - Memory usage at end
|
||
- `MemoryPeakMB`: Decimal - Peak memory usage
|
||
- `SignalUpdatesCount`: Decimal - Total signal updates performed
|
||
- `SignalUpdatesSkipped`: Integer - Number of signal updates skipped
|
||
- `SignalUpdateEfficiencyPercent`: Decimal - Percentage of signal updates that were skipped
|
||
- `BacktestStepsCount`: Decimal - Number of backtest steps executed
|
||
- `AverageSignalUpdateMs`: Decimal - Average time per signal update
|
||
- `AverageBacktestStepMs`: Decimal - Average time per backtest step
|
||
- `FinalPnL`: Decimal - Final profit and loss
|
||
- `WinRatePercent`: Integer - Win rate percentage
|
||
- `GrowthPercentage`: Decimal - Growth percentage
|
||
- `Score`: Decimal - Backtest score
|
||
- `CommitHash`: Git commit hash
|
||
- `GitBranch`: Git branch name
|
||
- `Environment`: Environment where test was run
|
||
|
||
## Implementation Details
|
||
|
||
The command uses regex patterns to extract metrics from the test console output and formats them into CSV rows. It detects the current git branch and commit hash for tracking purposes but **never commits changes automatically**.
|
||
|
||
## Example Output
|
||
|
||
```
|
||
🚀 Running backtest performance benchmark...
|
||
📊 Test Results:
|
||
• Processing Rate: 2686.2 candles/sec
|
||
• Execution Time: 2.115 seconds
|
||
• Memory Peak: 23.91 MB
|
||
• Signal Efficiency: 33.1%
|
||
• Candles Processed: 5760
|
||
• Score: 0.00
|
||
|
||
✅ Business Logic OK: Final PnL consistent (±0.00)
|
||
✅ Benchmark data recorded in performance-benchmarks.csv
|
||
```
|
||
|
||
### Business Logic Validation
|
||
|
||
The benchmark includes **business logic validation** to ensure performance optimizations don't break backtest behavior:
|
||
|
||
- **✅ Consistent**: Final PnL matches previous run (±0.01 tolerance)
|
||
- **✅ Baseline OK**: First run validates against expected baseline (24560.79)
|
||
- **⚠️ Warning**: Large PnL differences may indicate broken business logic
|
||
- **ℹ️ First Run**: Validates against established baseline
|
||
|
||
**Expected Baseline**: Final PnL should be **24560.79** for correct business logic.
|
||
|
||
**This prevents performance improvements from accidentally changing trading outcomes!**
|
||
|
||
## Files Modified
|
||
|
||
- `src/Managing.Workers.Tests/performance-benchmarks.csv` - **Modified** (new benchmark row added)
|
||
|
||
**Note**: Changes are **not committed automatically**. Review the results and commit manually if satisfied.
|
||
|