managing-apps/SQL_MONITORING_README.md

SQL Query Monitoring and Loop Detection System

Overview

This comprehensive SQL monitoring system has been implemented to identify and resolve the SQL script loop issue that was causing DDOS-like behavior on your server. The system provides detailed logging, performance monitoring, and automatic loop detection to help identify the root cause of problematic database operations.

Features

🔍 Comprehensive SQL Query Logging

• Detailed Query Tracking: Every SQL query is logged with timing, parameters, and execution context
• Performance Metrics: Automatic tracking of query execution times, row counts, and resource usage
• Connection State Monitoring: Tracks database connection open/close operations with timing
• Error Logging: Comprehensive error logging with stack traces and context information

🚨 Automatic Loop Detection

• Pattern Recognition: Identifies repeated query patterns that may indicate infinite loops
• Frequency Analysis: Monitors query execution frequency and detects abnormally high rates
• Performance Thresholds: Automatically flags slow queries and high-frequency operations
• Real-time Alerts: Immediate notification when potential loops are detected

📊 Performance Monitoring

• Query Execution Statistics: Tracks execution counts, average times, and performance trends
• Resource Usage Monitoring: Monitors memory, CPU, and I/O usage during database operations
• Connection Pool Monitoring: Tracks database connection pool health and usage
• Transaction Monitoring: Monitors transaction duration and rollback rates

🎯 Smart Alerting System

• Configurable Thresholds: Customizable thresholds for slow queries, high frequency, and error rates
• Multi-level Alerts: Different alert levels (Info, Warning, Error, Critical) based on severity
• Contextual Information: Alerts include repository name, method name, and query patterns
• Automatic Escalation: Critical issues are automatically escalated with detailed diagnostics

Components

1. SqlQueryLogger

Location: src/Managing.Infrastructure.Database/PostgreSql/SqlQueryLogger.cs

Provides comprehensive logging for individual database operations:

• Operation start/completion logging
• Query execution timing and parameters
• Connection state changes
• Error handling and exception logging
• Performance issue detection

2. SqlLoopDetectionService

Location: src/Managing.Infrastructure.Database/PostgreSql/SqlLoopDetectionService.cs

Advanced loop detection and performance monitoring:

• Real-time query pattern analysis
• Execution frequency tracking
• Performance threshold monitoring
• Automatic cleanup of old tracking data
• Configurable detection rules

3. BaseRepositoryWithLogging

Location: src/Managing.Infrastructure.Database/PostgreSql/BaseRepositoryWithLogging.cs

Base class for repositories with integrated monitoring:

• Automatic query execution tracking
• Performance monitoring for all database operations
• Error handling and logging
• Loop detection integration

4. Enhanced ManagingDbContext

Location: src/Managing.Infrastructure.Database/PostgreSql/ManagingDbContext.cs

Extended DbContext with monitoring capabilities:

• Query execution tracking
• Performance metrics collection
• Loop detection integration
• Statistics and health monitoring

5. SqlMonitoringController

Location: src/Managing.Api/Controllers/SqlMonitoringController.cs

REST API endpoints for monitoring and management:

• Real-time query statistics
• Alert management
• Performance metrics
• Health monitoring
• Configuration management

API Endpoints

Get Query Statistics

GET /api/SqlMonitoring/statistics

Returns comprehensive query execution statistics including:

• Loop detection statistics
• Context execution counts
• Active query patterns
• Performance metrics

Get Alerts

GET /api/SqlMonitoring/alerts

Returns current alerts and potential issues:

• High frequency queries
• Slow query patterns
• Performance issues
• Loop detection alerts

Clear Tracking Data

POST /api/SqlMonitoring/clear-tracking

Clears all tracking data and resets monitoring counters.

Get Query Details

GET /api/SqlMonitoring/query-details/{repositoryName}/{methodName}

Returns detailed information about specific query patterns.

Get Monitoring Health

GET /api/SqlMonitoring/health

Returns overall monitoring system health status.

Configuration

SqlMonitoringSettings

Location: src/Managing.Infrastructure.Database/PostgreSql/SqlMonitoringSettings.cs

Comprehensive configuration options:

• TrackingWindow: Time window for query tracking (default: 5 minutes)
• MaxExecutionsPerWindow: Maximum executions per window (default: 10)
• SlowQueryThresholdMs: Slow query threshold (default: 1000ms)
• HighFrequencyThreshold: High frequency threshold (default: 20 executions/minute)
• EnableDetailedLogging: Enable detailed SQL logging (default: true)
• EnableLoopDetection: Enable loop detection (default: true)
• EnablePerformanceMonitoring: Enable performance monitoring (default: true)

Usage Examples

1. Using Enhanced Repository

public class MyRepository : BaseRepositoryWithLogging, IMyRepository
{
    public MyRepository(ManagingDbContext context, ILogger<MyRepository> logger, SqlLoopDetectionService loopDetectionService) 
        : base(context, logger, loopDetectionService)
    {
    }

    public async Task<User> GetUserAsync(string name)
    {
        return await ExecuteWithLoggingAsync(async () =>
        {
            // Your database operation here
            return await _context.Users.FirstOrDefaultAsync(u => u.Name == name);
        }, nameof(GetUserAsync), ("name", name));
    }
}

2. Manual Query Tracking

// Track a specific query execution
_context.TrackQueryExecution("GetUserByName", TimeSpan.FromMilliseconds(150), "UserRepository", "GetUserAsync");

3. Monitoring API Usage

# Get current statistics
curl -X GET "https://your-api/api/SqlMonitoring/statistics"

# Get alerts
curl -X GET "https://your-api/api/SqlMonitoring/alerts"

# Clear tracking data
curl -X POST "https://your-api/api/SqlMonitoring/clear-tracking"

Logging Output Examples

Query Execution Log

[SQL-OP-START] a1b2c3d4 | PostgreSqlUserRepository.GetUserByNameAsync | Started at 14:30:15.123
[SQL-CONNECTION] a1b2c3d4 | PostgreSqlUserRepository.GetUserByNameAsync | Connection OPENED (took 5ms)
[SQL-QUERY] a1b2c3d4 | PostgreSqlUserRepository.GetUserByNameAsync | Executed in 25ms | Rows: 1
[SQL-CONNECTION] a1b2c3d4 | PostgreSqlUserRepository.GetUserByNameAsync | Connection CLOSED (took 2ms)
[SQL-OP-COMPLETE] a1b2c3d4 | PostgreSqlUserRepository.GetUserByNameAsync | Completed in 32ms | Queries: 1 | Result: User

Loop Detection Alert

[SQL-LOOP-DETECTED] e5f6g7h8 | PostgreSqlTradingRepository.GetPositionsAsync | Pattern 'GetPositionsAsync()' executed 15 times | Possible infinite loop!
[SQL-LOOP-ALERT] Potential infinite loop detected in PostgreSqlTradingRepository.GetPositionsAsync with pattern 'GetPositionsAsync()'

Performance Warning

[SQL-PERFORMANCE] PostgreSqlTradingRepository | GetPositionsAsync took 2500ms (threshold: 1000ms)
[SQL-QUERY-DETAILS] i9j0k1l2 | Query: SELECT * FROM Positions WHERE Status = @status | Parameters: {"status":"Active"}

Troubleshooting

Common Issues and Solutions

1. High Query Frequency

Symptoms: Multiple queries executing rapidly Detection: [SQL-LOOP-DETECTED] logs with high execution counts Solution:

• Check for recursive method calls
• Verify loop conditions in business logic
• Review async/await patterns

2. Slow Query Performance

Symptoms: Queries taking longer than expected Detection: [SQL-PERFORMANCE] warnings Solution:

• Review query execution plans
• Check database indexes
• Optimize query parameters

3. Connection Issues

Symptoms: Connection timeouts or pool exhaustion Detection: [SQL-CONNECTION] error logs Solution:

• Review connection management
• Check connection pool settings
• Verify proper connection disposal

4. Memory Issues

Symptoms: High memory usage during database operations Detection: Memory monitoring alerts Solution:

• Review query result set sizes
• Implement pagination
• Check for memory leaks in entity tracking

Integration Steps

1. Update Existing Repositories

Replace existing repository implementations with the enhanced base class:

// Before
public class MyRepository : IMyRepository
{
    private readonly ManagingDbContext _context;
    // ...
}

// After
public class MyRepository : BaseRepositoryWithLogging, IMyRepository
{
    public MyRepository(ManagingDbContext context, ILogger<MyRepository> logger, SqlLoopDetectionService loopDetectionService) 
        : base(context, logger, loopDetectionService)
    {
    }
    // ...
}

2. Update Dependency Injection

The services are automatically registered in Program.cs:

• SqlLoopDetectionService as Singleton
• Enhanced ManagingDbContext with monitoring
• All repositories with logging capabilities

3. Configure Monitoring Settings

Add configuration to appsettings.json:

{
  "SqlMonitoring": {
    "TrackingWindow": "00:05:00",
    "MaxExecutionsPerWindow": 10,
    "SlowQueryThresholdMs": 1000,
    "HighFrequencyThreshold": 20,
    "EnableDetailedLogging": true,
    "EnableLoopDetection": true,
    "EnablePerformanceMonitoring": true
  }
}

Monitoring Dashboard

Key Metrics to Monitor

1. Query Execution Count: Track total queries per minute
2. Average Execution Time: Monitor query performance trends
3. Error Rate: Track database error frequency
4. Connection Pool Usage: Monitor connection health
5. Loop Detection Alerts: Immediate notification of potential issues

Alert Thresholds

• Critical: >50 queries/minute, >5 second execution time
• Warning: >20 queries/minute, >1 second execution time
• Info: Normal operation metrics

Best Practices

1. Repository Design

• Always inherit from BaseRepositoryWithLogging
• Use ExecuteWithLoggingAsync for all database operations
• Include meaningful parameter names in logging calls
• Handle exceptions properly with logging

2. Performance Optimization

• Monitor slow queries regularly
• Implement proper indexing strategies
• Use pagination for large result sets
• Avoid N+1 query problems

3. Error Handling

• Log all database errors with context
• Implement proper retry mechanisms
• Use circuit breaker patterns for external dependencies
• Monitor error rates and trends

4. Security Considerations

• Avoid logging sensitive data in query parameters
• Use parameterized queries to prevent SQL injection
• Implement proper access controls for monitoring endpoints
• Regular security audits of database operations

Conclusion

This comprehensive SQL monitoring system provides the tools needed to identify and resolve the SQL script loop issue. The system offers:

• Real-time monitoring of all database operations
• Automatic loop detection with configurable thresholds
• Performance tracking with detailed metrics
• Comprehensive logging for debugging and analysis
• REST API endpoints for monitoring and management
• Configurable settings for different environments

The system is designed to be non-intrusive while providing maximum visibility into database operations, helping you quickly identify and resolve performance issues and potential infinite loops.