Refactor LlmController and GeminiProvider for improved message handling and redundant tool call detection
- Enhanced LlmController to detect and handle redundant tool calls, ensuring efficient processing and preventing unnecessary requests. - Updated message formatting in GeminiProvider to align with Gemini's expectations, improving the structure of requests sent to the API. - Improved logging in AiChat component to provide better insights into received responses and fallback mechanisms for empty content. - Adjusted handling of final responses in AiChat to ensure meaningful content is displayed, enhancing user experience during interactions.
This commit is contained in:
599
LLM_IMPROVEMENTS_TODO.md
Normal file
599
LLM_IMPROVEMENTS_TODO.md
Normal file
@@ -0,0 +1,599 @@
|
||||
# LLM Controller - Feature Improvements Roadmap
|
||||
|
||||
## 🎯 Quick Wins (1-2 days)
|
||||
|
||||
### ✅ Priority 1: Suggested Follow-up Questions
|
||||
**Status:** Not Started
|
||||
**Effort:** 4-6 hours
|
||||
**Impact:** High
|
||||
|
||||
**Description:**
|
||||
After each response, the LLM suggests 3-5 relevant follow-up questions to guide user exploration.
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Update `BuildSystemMessage()` to include follow-up question instruction
|
||||
- [ ] Add `SuggestedQuestions` property to `LlmProgressUpdate` class
|
||||
- [ ] Create `ExtractFollowUpQuestions()` method to parse questions from response
|
||||
- [ ] Update `ChatStreamInternal()` to extract and send suggested questions
|
||||
- [ ] Update frontend to display suggested questions as clickable chips
|
||||
- [ ] Test with various query types (backtest, indicator, general finance)
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
- `src/Managing.Application.Abstractions/Services/ILlmService.cs`
|
||||
- Frontend components (AiChat.tsx)
|
||||
|
||||
---
|
||||
|
||||
### ✅ Priority 2: Feedback & Rating System
|
||||
**Status:** Not Started
|
||||
**Effort:** 6-8 hours
|
||||
**Impact:** High (Quality tracking)
|
||||
|
||||
**Description:**
|
||||
Users can rate LLM responses (👍👎) with optional comments to track quality and improve prompts.
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `LlmFeedback` domain model (ResponseId, UserId, Rating, Comment, Timestamp)
|
||||
- [ ] Create `ILlmFeedbackRepository` interface
|
||||
- [ ] Implement `LlmFeedbackRepository` with MongoDB
|
||||
- [ ] Add `ResponseId` property to `LlmChatResponse`
|
||||
- [ ] Create new endpoint: `POST /Llm/Feedback`
|
||||
- [ ] Create new endpoint: `GET /Llm/Analytics/Feedback`
|
||||
- [ ] Update frontend to show 👍👎 buttons after each response
|
||||
- [ ] Create analytics dashboard to view feedback trends
|
||||
|
||||
**Files to Create:**
|
||||
- `src/Managing.Domain/Llm/LlmFeedback.cs`
|
||||
- `src/Managing.Application.Abstractions/Repositories/ILlmFeedbackRepository.cs`
|
||||
- `src/Managing.Infrastructure/Repositories/LlmFeedbackRepository.cs`
|
||||
- `src/Managing.Application/Services/LlmFeedbackService.cs`
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
- `src/Managing.Application.Abstractions/Services/ILlmService.cs`
|
||||
|
||||
---
|
||||
|
||||
### ✅ Priority 3: Export Conversations
|
||||
**Status:** Not Started
|
||||
**Effort:** 4-6 hours
|
||||
**Impact:** Medium
|
||||
|
||||
**Description:**
|
||||
Export conversation to Markdown, JSON, or PDF for reporting and sharing.
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `IConversationExportService` interface
|
||||
- [ ] Implement Markdown export (simple format with messages)
|
||||
- [ ] Implement JSON export (structured data)
|
||||
- [ ] Implement PDF export using QuestPDF or similar
|
||||
- [ ] Create new endpoint: `GET /Llm/Conversations/{id}/Export?format={md|json|pdf}`
|
||||
- [ ] Add "Export" button to conversation UI
|
||||
- [ ] Test with long conversations and special characters
|
||||
|
||||
**Files to Create:**
|
||||
- `src/Managing.Application/Services/ConversationExportService.cs`
|
||||
- `src/Managing.Application.Abstractions/Services/IConversationExportService.cs`
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
|
||||
---
|
||||
|
||||
### ✅ Priority 4: Query Categorization
|
||||
**Status:** Not Started
|
||||
**Effort:** 3-4 hours
|
||||
**Impact:** Medium (Better analytics)
|
||||
|
||||
**Description:**
|
||||
Automatically categorize queries (BacktestAnalysis, GeneralFinance, etc.) for analytics.
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `QueryCategory` enum (BacktestAnalysis, BundleAnalysis, IndicatorQuestion, GeneralFinance, HowTo, DataRetrieval, Comparison)
|
||||
- [ ] Add `QueryCategory` property to `LlmProgressUpdate`
|
||||
- [ ] Create `DetermineQueryCategory()` method using keyword matching
|
||||
- [ ] Update system prompt to include category in response
|
||||
- [ ] Send category in initial progress update
|
||||
- [ ] Track category distribution in analytics
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
- `src/Managing.Application.Abstractions/Services/ILlmService.cs`
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Medium Effort (3-5 days)
|
||||
|
||||
### ✅ Priority 5: Conversation Persistence
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** Very High (Core feature)
|
||||
|
||||
**Description:**
|
||||
Save conversation history to database so users can resume previous conversations across sessions.
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `ChatConversation` domain model (Id, UserId, Title, CreatedAt, UpdatedAt, LastMessageAt)
|
||||
- [ ] Create `ChatMessage` domain model (Id, ConversationId, Role, Content, Timestamp, TokenCount, ToolCalls)
|
||||
- [ ] Create `IChatConversationRepository` interface
|
||||
- [ ] Implement `ChatConversationRepository` with MongoDB
|
||||
- [ ] Create `IChatMessageRepository` interface
|
||||
- [ ] Implement `ChatMessageRepository` with MongoDB
|
||||
- [ ] Create new endpoint: `GET /Llm/Conversations` (list user's conversations)
|
||||
- [ ] Create new endpoint: `GET /Llm/Conversations/{id}` (get conversation with messages)
|
||||
- [ ] Create new endpoint: `POST /Llm/Conversations` (create new conversation)
|
||||
- [ ] Create new endpoint: `POST /Llm/Conversations/{id}/Messages` (add message to conversation)
|
||||
- [ ] Create new endpoint: `DELETE /Llm/Conversations/{id}` (delete conversation)
|
||||
- [ ] Update `ChatStream` to save messages automatically
|
||||
- [ ] Create conversation list UI component
|
||||
- [ ] Add "New Conversation" button
|
||||
- [ ] Add conversation sidebar with search/filter
|
||||
- [ ] Test with multiple concurrent conversations
|
||||
|
||||
**Files to Create:**
|
||||
- `src/Managing.Domain/Llm/ChatConversation.cs`
|
||||
- `src/Managing.Domain/Llm/ChatMessage.cs`
|
||||
- `src/Managing.Application.Abstractions/Repositories/IChatConversationRepository.cs`
|
||||
- `src/Managing.Application.Abstractions/Repositories/IChatMessageRepository.cs`
|
||||
- `src/Managing.Infrastructure/Repositories/ChatConversationRepository.cs`
|
||||
- `src/Managing.Infrastructure/Repositories/ChatMessageRepository.cs`
|
||||
- `src/Managing.Application/Services/ChatConversationService.cs`
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
|
||||
---
|
||||
|
||||
### ✅ Priority 6: Response Streaming (Token-by-Token)
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** High (UX improvement)
|
||||
|
||||
**Description:**
|
||||
Stream LLM response as tokens arrive instead of waiting for complete response.
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Update `ILlmService.ChatAsync()` to return `IAsyncEnumerable<LlmTokenChunk>`
|
||||
- [ ] Modify LLM provider implementations to support streaming
|
||||
- [ ] Update `ChatStreamInternal()` to stream tokens via SignalR
|
||||
- [ ] Add new progress update type: "token_stream"
|
||||
- [ ] Update frontend to display streaming tokens with typing animation
|
||||
- [ ] Handle tool calls during streaming (partial JSON parsing)
|
||||
- [ ] Add "Stop Generation" button in UI
|
||||
- [ ] Test with different LLM providers
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Application.Abstractions/Services/ILlmService.cs`
|
||||
- `src/Managing.Application/Services/LlmService.cs` (or provider-specific implementations)
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
- Frontend components (AiChat.tsx)
|
||||
|
||||
---
|
||||
|
||||
### ✅ Priority 7: Usage Analytics Dashboard
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** High (Cost monitoring)
|
||||
|
||||
**Description:**
|
||||
Track and visualize LLM usage metrics (tokens, cost, performance).
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `LlmUsageMetric` domain model (UserId, Timestamp, Provider, Model, PromptTokens, CompletionTokens, Cost, Duration, QueryCategory)
|
||||
- [ ] Create `ILlmUsageRepository` interface
|
||||
- [ ] Implement `LlmUsageRepository` with InfluxDB (time-series data)
|
||||
- [ ] Update `ChatStreamInternal()` to log usage metrics
|
||||
- [ ] Create new endpoint: `GET /Llm/Analytics/Usage` (token usage over time)
|
||||
- [ ] Create new endpoint: `GET /Llm/Analytics/PopularTools` (most called tools)
|
||||
- [ ] Create new endpoint: `GET /Llm/Analytics/AverageIterations` (performance metrics)
|
||||
- [ ] Create new endpoint: `GET /Llm/Analytics/ErrorRate` (quality metrics)
|
||||
- [ ] Create new endpoint: `GET /Llm/Analytics/CostEstimate` (current month cost)
|
||||
- [ ] Create analytics dashboard component with charts (Chart.js or Recharts)
|
||||
- [ ] Add filters: date range, category, provider
|
||||
- [ ] Display key metrics: total tokens, cost, avg response time
|
||||
- [ ] Test with large datasets
|
||||
|
||||
**Files to Create:**
|
||||
- `src/Managing.Domain/Llm/LlmUsageMetric.cs`
|
||||
- `src/Managing.Application.Abstractions/Repositories/ILlmUsageRepository.cs`
|
||||
- `src/Managing.Infrastructure/Repositories/LlmUsageRepository.cs`
|
||||
- `src/Managing.Application/Services/LlmAnalyticsService.cs`
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
|
||||
---
|
||||
|
||||
### ✅ Priority 8: Quick Actions / Shortcuts
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** Medium (Workflow improvement)
|
||||
|
||||
**Description:**
|
||||
Recognize patterns and offer action buttons (e.g., "Delete this backtest" after analysis).
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `QuickAction` model (Id, Label, Icon, Endpoint, Parameters)
|
||||
- [ ] Add `Actions` property to `LlmProgressUpdate`
|
||||
- [ ] Create `GenerateQuickActions()` method based on context
|
||||
- [ ] Update system prompt to suggest actions in structured format
|
||||
- [ ] Parse action suggestions from LLM response
|
||||
- [ ] Update frontend to display action buttons
|
||||
- [ ] Implement action handlers (call APIs)
|
||||
- [ ] Add confirmation dialogs for destructive actions
|
||||
- [ ] Test with various scenarios (backtest, bundle, indicator)
|
||||
|
||||
**Example Actions:**
|
||||
- After backtest analysis: "Delete this backtest", "Run similar backtest", "Export details"
|
||||
- After bundle analysis: "Delete bundle", "Run again with different params"
|
||||
- After list query: "Export to CSV", "Show details"
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
- `src/Managing.Application.Abstractions/Services/ILlmService.cs`
|
||||
- Frontend components (AiChat.tsx)
|
||||
|
||||
---
|
||||
|
||||
## 🎨 Long-term (1-2 weeks)
|
||||
|
||||
### ✅ Priority 9: Multi-Provider Fallback
|
||||
**Status:** Not Started
|
||||
**Effort:** 3-5 days
|
||||
**Impact:** High (Reliability)
|
||||
|
||||
**Description:**
|
||||
Automatically fallback to alternative LLM provider on failure or rate limit.
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `LlmProviderHealth` model to track provider status
|
||||
- [ ] Create `IProviderHealthMonitor` service
|
||||
- [ ] Implement health check mechanism (ping providers periodically)
|
||||
- [ ] Create provider priority list configuration
|
||||
- [ ] Update `LlmService.ChatAsync()` to implement fallback logic
|
||||
- [ ] Add retry logic with exponential backoff
|
||||
- [ ] Track provider failure rates
|
||||
- [ ] Send alert when provider is down
|
||||
- [ ] Update frontend to show current provider
|
||||
- [ ] Test failover scenarios
|
||||
|
||||
**Provider Priority Example:**
|
||||
1. Primary: OpenAI GPT-4
|
||||
2. Secondary: Anthropic Claude
|
||||
3. Tertiary: Google Gemini
|
||||
4. Fallback: Local model (if available)
|
||||
|
||||
**Files to Create:**
|
||||
- `src/Managing.Application/Services/ProviderHealthMonitor.cs`
|
||||
- `src/Managing.Domain/Llm/LlmProviderHealth.cs`
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Application/Services/LlmService.cs`
|
||||
|
||||
---
|
||||
|
||||
### ✅ Priority 10: Scheduled Queries / Alerts
|
||||
**Status:** Not Started
|
||||
**Effort:** 4-6 days
|
||||
**Impact:** High (Automation)
|
||||
|
||||
**Description:**
|
||||
Run queries on schedule and notify users of changes (e.g., "Alert when backtest scores > 80").
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `LlmAlert` domain model (Id, UserId, Query, Schedule, Condition, IsActive, LastRun, CreatedAt)
|
||||
- [ ] Create `ILlmAlertRepository` interface
|
||||
- [ ] Implement `LlmAlertRepository` with MongoDB
|
||||
- [ ] Create background service to process alerts (Hangfire or Quartz.NET)
|
||||
- [ ] Create new endpoint: `POST /Llm/Alerts` (create alert)
|
||||
- [ ] Create new endpoint: `GET /Llm/Alerts` (list user's alerts)
|
||||
- [ ] Create new endpoint: `PUT /Llm/Alerts/{id}` (update alert)
|
||||
- [ ] Create new endpoint: `DELETE /Llm/Alerts/{id}` (delete alert)
|
||||
- [ ] Implement notification system (SignalR, email, push)
|
||||
- [ ] Create alert management UI
|
||||
- [ ] Add schedule picker (cron expression builder)
|
||||
- [ ] Test with various schedules and conditions
|
||||
|
||||
**Example Alerts:**
|
||||
- "Notify me when a backtest scores > 80" (run every hour)
|
||||
- "Daily summary of new backtests" (run at 9am daily)
|
||||
- "Alert when bundle completes" (run every 5 minutes)
|
||||
|
||||
**Files to Create:**
|
||||
- `src/Managing.Domain/Llm/LlmAlert.cs`
|
||||
- `src/Managing.Application.Abstractions/Repositories/ILlmAlertRepository.cs`
|
||||
- `src/Managing.Infrastructure/Repositories/LlmAlertRepository.cs`
|
||||
- `src/Managing.Application/Services/LlmAlertService.cs`
|
||||
- `src/Managing.Application/BackgroundServices/LlmAlertProcessor.cs`
|
||||
|
||||
---
|
||||
|
||||
### ✅ Priority 11: Smart Context Window Management
|
||||
**Status:** Not Started
|
||||
**Effort:** 3-5 days
|
||||
**Impact:** Medium (Better conversations)
|
||||
|
||||
**Description:**
|
||||
Intelligently compress conversation history instead of simple truncation.
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Research and implement summarization approach (LLM-based or extractive)
|
||||
- [ ] Create `SummarizeConversation()` method
|
||||
- [ ] Update `TrimConversationContext()` to use summarization
|
||||
- [ ] Preserve key entities (IDs, numbers, dates)
|
||||
- [ ] Use embeddings to identify relevant context (optional, advanced)
|
||||
- [ ] Test with long conversations (50+ messages)
|
||||
- [ ] Measure token savings vs quality trade-off
|
||||
- [ ] Add configuration for compression strategy
|
||||
|
||||
**Approaches:**
|
||||
1. **Simple:** Summarize every N old messages into single message
|
||||
2. **Advanced:** Use embeddings to keep semantically relevant messages
|
||||
3. **Hybrid:** Keep recent messages + summarized older messages + key facts
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
|
||||
---
|
||||
|
||||
### ✅ Priority 12: Interactive Clarification Questions
|
||||
**Status:** Not Started
|
||||
**Effort:** 3-4 days
|
||||
**Impact:** Medium (Reduce back-and-forth)
|
||||
|
||||
**Description:**
|
||||
When ambiguous, LLM asks structured multiple-choice questions instead of open-ended text.
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `ClarificationOption` model (Id, Label, Description)
|
||||
- [ ] Add `Options` property to `LlmProgressUpdate`
|
||||
- [ ] Update system prompt to output clarification questions in structured format
|
||||
- [ ] Create `ExtractClarificationOptions()` method
|
||||
- [ ] Update `ChatStreamInternal()` to handle clarification state
|
||||
- [ ] Update frontend to display radio buttons/chips for options
|
||||
- [ ] Handle user selection (send as next message automatically)
|
||||
- [ ] Test with ambiguous queries
|
||||
|
||||
**Example:**
|
||||
User: "Show me the backtest"
|
||||
LLM: "Which backtest would you like to see?"
|
||||
- 🔘 Best performing backtest
|
||||
- 🔘 Most recent backtest
|
||||
- 🔘 Specific backtest by name
|
||||
|
||||
**Files to Create:**
|
||||
- `src/Managing.Domain/Llm/ClarificationOption.cs`
|
||||
|
||||
**Files to Modify:**
|
||||
- `src/Managing.Api/Controllers/LlmController.cs`
|
||||
- `src/Managing.Application.Abstractions/Services/ILlmService.cs`
|
||||
- Frontend components (AiChat.tsx)
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Additional Features (Nice to Have)
|
||||
|
||||
### Voice Input Support
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** Medium
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create new endpoint: `POST /Llm/VoiceChat` (accept audio file)
|
||||
- [ ] Integrate speech-to-text service (Azure Speech, OpenAI Whisper)
|
||||
- [ ] Process transcribed text as normal chat
|
||||
- [ ] Add microphone button in frontend
|
||||
- [ ] Handle audio recording in browser
|
||||
- [ ] Test with various audio formats and accents
|
||||
|
||||
---
|
||||
|
||||
### Smart Conversation Titling
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 hours
|
||||
**Impact:** Low (QoL)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] After first response, send summary request to LLM
|
||||
- [ ] Update conversation title in background
|
||||
- [ ] Don't block user while generating title
|
||||
- [ ] Test with various conversation types
|
||||
|
||||
---
|
||||
|
||||
### Tool Call Caching
|
||||
**Status:** Not Started
|
||||
**Effort:** 1-2 days
|
||||
**Impact:** Medium (Performance)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create cache key hash function (toolName + arguments)
|
||||
- [ ] Implement cache wrapper around `ExecuteToolAsync()`
|
||||
- [ ] Configure cache duration per tool type
|
||||
- [ ] Invalidate cache on data mutations
|
||||
- [ ] Test cache hit/miss rates
|
||||
|
||||
---
|
||||
|
||||
### Conversation Branching
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** Low (Power user feature)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create new endpoint: `POST /Llm/Conversations/{id}/Branch?fromMessageId={id}`
|
||||
- [ ] Copy conversation history up to branch point
|
||||
- [ ] Create new conversation with copied history
|
||||
- [ ] Update UI to show branch option on messages
|
||||
- [ ] Test branching at various points
|
||||
|
||||
---
|
||||
|
||||
### LLM Model Selection
|
||||
**Status:** Not Started
|
||||
**Effort:** 1-2 days
|
||||
**Impact:** Medium (Cost control)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Add `PreferredModel` property to `LlmChatRequest`
|
||||
- [ ] Create model configuration (pricing, speed, quality scores)
|
||||
- [ ] Update frontend with model selector dropdown
|
||||
- [ ] Display model info (cost, speed, quality)
|
||||
- [ ] Test with different models
|
||||
|
||||
---
|
||||
|
||||
### Debug Mode
|
||||
**Status:** Not Started
|
||||
**Effort:** 4-6 hours
|
||||
**Impact:** Low (Developer tool)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Add `Debug` property to `LlmChatRequest`
|
||||
- [ ] Return full prompt, raw response, token breakdown when debug=true
|
||||
- [ ] Create debug panel in UI
|
||||
- [ ] Add toggle to enable/disable debug mode
|
||||
- [ ] Test with various queries
|
||||
|
||||
---
|
||||
|
||||
### PII Detection & Redaction
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** Medium (Security)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Implement PII detection regex (email, phone, SSN, credit card)
|
||||
- [ ] Scan messages before sending to LLM
|
||||
- [ ] Warn user about detected PII
|
||||
- [ ] Option to redact or anonymize
|
||||
- [ ] Test with various PII patterns
|
||||
|
||||
---
|
||||
|
||||
### Rate Limiting Per User
|
||||
**Status:** Not Started
|
||||
**Effort:** 1-2 days
|
||||
**Impact:** Medium (Cost control)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create rate limit configuration (requests/hour, tokens/day)
|
||||
- [ ] Implement rate limit middleware
|
||||
- [ ] Track usage per user
|
||||
- [ ] Return 429 with quota info when exceeded
|
||||
- [ ] Display quota usage in UI
|
||||
|
||||
---
|
||||
|
||||
### Request Queueing
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** Medium (Reliability)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Implement request queue with priority
|
||||
- [ ] Queue requests when rate limited
|
||||
- [ ] Send position-in-queue updates via SignalR
|
||||
- [ ] Process queue when rate limit resets
|
||||
- [ ] Test with high load
|
||||
|
||||
---
|
||||
|
||||
### Prompt Version Control
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** Low (Optimization)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create `SystemPrompt` model (Version, Content, CreatedAt, IsActive, SuccessRate)
|
||||
- [ ] Store multiple prompt versions
|
||||
- [ ] A/B test prompts (rotate per conversation)
|
||||
- [ ] Track success metrics per prompt version
|
||||
- [ ] UI to manage prompt versions
|
||||
|
||||
---
|
||||
|
||||
### LLM Playground
|
||||
**Status:** Not Started
|
||||
**Effort:** 3-4 days
|
||||
**Impact:** Low (Developer tool)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Create playground UI component
|
||||
- [ ] System prompt editor with syntax highlighting
|
||||
- [ ] Message history builder
|
||||
- [ ] Tool selector
|
||||
- [ ] Temperature/token controls
|
||||
- [ ] Side-by-side comparison
|
||||
- [ ] Test various configurations
|
||||
|
||||
---
|
||||
|
||||
### Collaborative Filtering
|
||||
**Status:** Not Started
|
||||
**Effort:** 3-5 days
|
||||
**Impact:** Low (Discovery)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Track query patterns per user
|
||||
- [ ] Implement collaborative filtering algorithm
|
||||
- [ ] Suggest related queries after response
|
||||
- [ ] Display "Users also asked" section
|
||||
- [ ] Test recommendation quality
|
||||
|
||||
---
|
||||
|
||||
### Conversation Encryption
|
||||
**Status:** Not Started
|
||||
**Effort:** 2-3 days
|
||||
**Impact:** Medium (Security)
|
||||
|
||||
**Implementation Tasks:**
|
||||
- [ ] Implement encryption/decryption service
|
||||
- [ ] Generate user-specific encryption keys
|
||||
- [ ] Encrypt messages before storing
|
||||
- [ ] Decrypt on retrieval
|
||||
- [ ] Test performance impact
|
||||
|
||||
---
|
||||
|
||||
## 📊 Progress Tracker
|
||||
|
||||
**Quick Wins:** 0/4 completed (0%)
|
||||
**Medium Effort:** 0/4 completed (0%)
|
||||
**Long-term:** 0/4 completed (0%)
|
||||
**Additional Features:** 0/15 completed (0%)
|
||||
|
||||
**Overall Progress:** 0/27 completed (0%)
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Recommended Implementation Order
|
||||
|
||||
1. **Conversation Persistence** - Foundation for other features
|
||||
2. **Suggested Follow-up Questions** - Quick UX win
|
||||
3. **Feedback & Rating System** - Quality tracking
|
||||
4. **Usage Analytics Dashboard** - Monitor costs
|
||||
5. **Response Streaming** - Better UX
|
||||
6. **Export Conversations** - User requested feature
|
||||
7. **Quick Actions** - Workflow optimization
|
||||
8. **Multi-Provider Fallback** - Reliability
|
||||
9. **Query Categorization** - Better analytics
|
||||
10. **Smart Context Management** - Better conversations
|
||||
|
||||
---
|
||||
|
||||
## 📝 Notes
|
||||
|
||||
- All features should follow the Controller → Application → Repository pattern
|
||||
- Regenerate `ManagingApi.ts` after adding new endpoints: `cd src/Managing.Nswag && dotnet build`
|
||||
- Use MongoDB for document storage, InfluxDB for time-series metrics
|
||||
- Test all features with real user scenarios
|
||||
- Consider token costs when implementing LLM-heavy features (summarization, titling)
|
||||
- Ensure all features respect user privacy and data security
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-01-07
|
||||
**Maintained By:** Development Team
|
||||
@@ -116,28 +116,32 @@ public class LlmController : BaseController
|
||||
var scopedUser = await userService.GetUserByIdAsync(userId);
|
||||
if (scopedUser == null)
|
||||
{
|
||||
await hubContext.Clients.Client(request.ConnectionId).SendAsync("ProgressUpdate", new LlmProgressUpdate
|
||||
{
|
||||
Type = "error",
|
||||
Message = "User not found",
|
||||
Error = "Unable to authenticate user"
|
||||
});
|
||||
await hubContext.Clients.Client(request.ConnectionId).SendAsync("ProgressUpdate",
|
||||
new LlmProgressUpdate
|
||||
{
|
||||
Type = "error",
|
||||
Message = "User not found",
|
||||
Error = "Unable to authenticate user"
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
await ChatStreamInternal(request, scopedUser, request.ConnectionId, llmService, mcpService, cache, hubContext, logger);
|
||||
await ChatStreamInternal(request, scopedUser, request.ConnectionId, llmService, mcpService, cache,
|
||||
hubContext, logger);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Error processing chat stream for connection {ConnectionId}", request.ConnectionId);
|
||||
_logger.LogError(ex, "Error processing chat stream for connection {ConnectionId}",
|
||||
request.ConnectionId);
|
||||
try
|
||||
{
|
||||
await _hubContext.Clients.Client(request.ConnectionId).SendAsync("ProgressUpdate", new LlmProgressUpdate
|
||||
{
|
||||
Type = "error",
|
||||
Message = $"Error processing chat: {ex.Message}",
|
||||
Error = ex.Message
|
||||
});
|
||||
await _hubContext.Clients.Client(request.ConnectionId).SendAsync("ProgressUpdate",
|
||||
new LlmProgressUpdate
|
||||
{
|
||||
Type = "error",
|
||||
Message = $"Error processing chat: {ex.Message}",
|
||||
Error = ex.Message
|
||||
});
|
||||
}
|
||||
catch (Exception hubEx)
|
||||
{
|
||||
@@ -215,9 +219,11 @@ public class LlmController : BaseController
|
||||
// Iterative tool calling: keep looping until we get a final answer without tool calls
|
||||
int maxIterations = DetermineMaxIterations(chatRequest);
|
||||
int iteration = 0;
|
||||
int redundantCallDetections = 0;
|
||||
LlmChatResponse? finalResponse = null;
|
||||
const int DelayBetweenIterationsMs = 2000; // Increased from 500ms to 2s to respect rate limits
|
||||
const int DelayAfterToolCallsMs = 1000; // Additional delay after tool calls before next LLM call
|
||||
const int MaxRedundantDetections = 2; // Maximum times we'll detect redundant calls before forcing final response
|
||||
|
||||
await SendProgressUpdate(connectionId, hubContext, logger, new LlmProgressUpdate
|
||||
{
|
||||
@@ -229,6 +235,9 @@ public class LlmController : BaseController
|
||||
{
|
||||
iteration++;
|
||||
|
||||
// Get the last user question once per iteration to avoid scope conflicts
|
||||
var lastUserQuestion = chatRequest.Messages.LastOrDefault(m => m.Role == "user")?.Content ?? "the user's question";
|
||||
|
||||
await SendProgressUpdate(connectionId, hubContext, logger, new LlmProgressUpdate
|
||||
{
|
||||
Type = "iteration_start",
|
||||
@@ -270,10 +279,13 @@ public class LlmController : BaseController
|
||||
{
|
||||
response = await llmService.ChatAsync(user, chatRequest);
|
||||
}
|
||||
catch (HttpRequestException httpEx) when (httpEx.Message.Contains("429") || httpEx.Message.Contains("TooManyRequests") || httpEx.Message.Contains("RESOURCE_EXHAUSTED"))
|
||||
catch (HttpRequestException httpEx) when (httpEx.Message.Contains("429") ||
|
||||
httpEx.Message.Contains("TooManyRequests") ||
|
||||
httpEx.Message.Contains("RESOURCE_EXHAUSTED"))
|
||||
{
|
||||
// Rate limit hit - wait longer before retrying
|
||||
logger.LogWarning("Rate limit hit (429) in iteration {Iteration}. Waiting 10 seconds before retry...", iteration);
|
||||
logger.LogWarning("Rate limit hit (429) in iteration {Iteration}. Waiting 10 seconds before retry...",
|
||||
iteration);
|
||||
await SendProgressUpdate(connectionId, hubContext, logger, new LlmProgressUpdate
|
||||
{
|
||||
Type = "thinking",
|
||||
@@ -293,7 +305,8 @@ public class LlmController : BaseController
|
||||
catch (Exception retryEx)
|
||||
{
|
||||
logger.LogError(retryEx, "Retry after rate limit also failed in iteration {Iteration}", iteration);
|
||||
throw new HttpRequestException($"Rate limit error persists after retry: {retryEx.Message}", retryEx);
|
||||
throw new HttpRequestException($"Rate limit error persists after retry: {retryEx.Message}",
|
||||
retryEx);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -315,7 +328,108 @@ public class LlmController : BaseController
|
||||
break;
|
||||
}
|
||||
|
||||
// LLM wants to call tools - execute them
|
||||
// LLM wants to call tools - check for redundant calls first
|
||||
var redundantCalls = DetectRedundantToolCalls(chatRequest.Messages, response.ToolCalls);
|
||||
if (redundantCalls.Any())
|
||||
{
|
||||
redundantCallDetections++;
|
||||
logger.LogWarning("LLM requested {Count} redundant tool calls in iteration {Iteration}: {Tools} (Detection #{DetectionCount})",
|
||||
redundantCalls.Count, iteration, string.Join(", ", redundantCalls.Select(t => t.Name)), redundantCallDetections);
|
||||
|
||||
await SendProgressUpdate(connectionId, hubContext, logger, new LlmProgressUpdate
|
||||
{
|
||||
Type = "thinking",
|
||||
Message = "Detected redundant tool calls. Using cached data...",
|
||||
Iteration = iteration,
|
||||
MaxIterations = maxIterations
|
||||
});
|
||||
|
||||
// If we've detected redundant calls multiple times, force a final response by removing tools
|
||||
if (redundantCallDetections >= MaxRedundantDetections)
|
||||
{
|
||||
logger.LogWarning("Reached maximum redundant call detections ({MaxDetections}). Removing tools to force final response.",
|
||||
MaxRedundantDetections);
|
||||
|
||||
await SendProgressUpdate(connectionId, hubContext, logger, new LlmProgressUpdate
|
||||
{
|
||||
Type = "thinking",
|
||||
Message = "Multiple redundant tool calls detected. Forcing final response...",
|
||||
Iteration = iteration,
|
||||
MaxIterations = maxIterations
|
||||
});
|
||||
|
||||
// Remove tools to force a text-only response
|
||||
chatRequest.Tools = null;
|
||||
|
||||
// Add explicit user message for final answer (user messages are always seen in Gemini)
|
||||
var toolResultsCount = chatRequest.Messages.Count(m => m.Role == "tool");
|
||||
chatRequest.Messages.Add(new LlmMessage
|
||||
{
|
||||
Role = "user",
|
||||
Content =
|
||||
$"FINAL RESPONSE REQUIRED: You have already executed tools and received {toolResultsCount} tool result(s). The data is available in the conversation above. You have attempted to call the same tools multiple times. Based ONLY on the existing data, provide a complete answer to: {lastUserQuestion}. DO NOT request any tool calls."
|
||||
});
|
||||
|
||||
// Get final response without tools
|
||||
try
|
||||
{
|
||||
finalResponse = await llmService.ChatAsync(user, chatRequest);
|
||||
break;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
logger.LogError(ex, "Error getting final response after redundant call detection for user {UserId}", user.Id);
|
||||
|
||||
// Create a fallback response instead of falling through
|
||||
finalResponse = new LlmChatResponse
|
||||
{
|
||||
Content =
|
||||
"I apologize, but I encountered an issue providing a final response. Based on the data retrieved, I was unable to complete the analysis. Please try rephrasing your question or ask for specific details.",
|
||||
RequiresToolExecution = false
|
||||
};
|
||||
break;
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
// Add a user message (not system - Gemini only sees first system message) to force final response
|
||||
chatRequest.Messages.Add(new LlmMessage
|
||||
{
|
||||
Role = "user",
|
||||
Content =
|
||||
$"You just attempted to call the same tools again ({string.Join(", ", redundantCalls.Select(t => t.Name))}), but this data is ALREADY available in the conversation above. Please analyze the existing tool results and provide your final answer to: {lastUserQuestion}. Do not request any tool calls - use only the data you already have."
|
||||
});
|
||||
|
||||
// Temporarily remove tools to prevent the LLM from requesting them again
|
||||
var originalTools = chatRequest.Tools;
|
||||
chatRequest.Tools = null;
|
||||
|
||||
// Continue to next iteration without tools
|
||||
try
|
||||
{
|
||||
response = await llmService.ChatAsync(user, chatRequest);
|
||||
|
||||
// If we got a response without tools, use it as final
|
||||
if (!response.RequiresToolExecution || response.ToolCalls == null || !response.ToolCalls.Any())
|
||||
{
|
||||
finalResponse = response;
|
||||
break;
|
||||
}
|
||||
|
||||
// Restore tools if LLM still wants to call different tools
|
||||
chatRequest.Tools = originalTools;
|
||||
continue;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
logger.LogError(ex, "Error getting response after removing tools for redundant calls");
|
||||
// Restore tools and continue
|
||||
chatRequest.Tools = originalTools;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
logger.LogInformation("LLM requested {Count} tool calls in iteration {Iteration} for user {UserId}",
|
||||
response.ToolCalls.Count, iteration, user.Id);
|
||||
|
||||
@@ -341,11 +455,13 @@ public class LlmController : BaseController
|
||||
ToolArguments = toolCall.Arguments
|
||||
});
|
||||
|
||||
var (success, result, error) = await ExecuteToolSafely(user, toolCall.Name, toolCall.Arguments, toolCall.Id, iteration, maxIterations, mcpService, logger);
|
||||
var (success, result, error) = await ExecuteToolSafely(user, toolCall.Name, toolCall.Arguments,
|
||||
toolCall.Id, iteration, maxIterations, mcpService, logger);
|
||||
|
||||
if (success && result != null)
|
||||
{
|
||||
logger.LogInformation("Successfully executed tool {ToolName} in iteration {Iteration} for user {UserId}",
|
||||
logger.LogInformation(
|
||||
"Successfully executed tool {ToolName} in iteration {Iteration} for user {UserId}",
|
||||
toolCall.Name, iteration, user.Id);
|
||||
|
||||
var resultMessage = GenerateToolResultMessage(toolCall.Name, result);
|
||||
@@ -367,7 +483,8 @@ public class LlmController : BaseController
|
||||
}
|
||||
else
|
||||
{
|
||||
logger.LogError("Error executing tool {ToolName} in iteration {Iteration} for user {UserId}: {Error}",
|
||||
logger.LogError(
|
||||
"Error executing tool {ToolName} in iteration {Iteration} for user {UserId}: {Error}",
|
||||
toolCall.Name, iteration, user.Id, error);
|
||||
|
||||
await SendProgressUpdate(connectionId, hubContext, logger, new LlmProgressUpdate
|
||||
@@ -408,11 +525,12 @@ public class LlmController : BaseController
|
||||
// Add tool results to conversation history
|
||||
chatRequest.Messages.AddRange(toolResults);
|
||||
|
||||
// Add a system reminder to prevent redundant tool calls
|
||||
// Add a user message to guide the LLM (user messages are always seen, unlike system messages in Gemini)
|
||||
// This ensures the LLM understands it should analyze the tool results and provide a final answer
|
||||
chatRequest.Messages.Add(new LlmMessage
|
||||
{
|
||||
Role = "system",
|
||||
Content = "You now have all the data from the tool calls above. Analyze this data and provide your final answer to the user. DO NOT call the same tools again with the same arguments."
|
||||
Role = "user",
|
||||
Content = $"Based on the tool results above, please analyze the data and provide a comprehensive answer to: {lastUserQuestion}. Do not call any tools again - use only the data you have already retrieved."
|
||||
});
|
||||
|
||||
// Add delay after tool calls before next LLM call to avoid rate limits
|
||||
@@ -424,13 +542,14 @@ public class LlmController : BaseController
|
||||
// If we hit max iterations, return the last response (even if it has tool calls)
|
||||
if (finalResponse == null)
|
||||
{
|
||||
logger.LogWarning("Reached max iterations ({MaxIterations}) for user {UserId}. Forcing final response without tools.",
|
||||
logger.LogWarning(
|
||||
"Reached max iterations ({MaxIterations}) for user {UserId}. Forcing final response without tools.",
|
||||
maxIterations, user.Id);
|
||||
|
||||
await SendProgressUpdate(connectionId, hubContext, logger, new LlmProgressUpdate
|
||||
{
|
||||
Type = "thinking",
|
||||
Message = "Reached maximum iterations. Getting final response...",
|
||||
Message = "Reached maximum iterations. Preparing final response with available data...",
|
||||
Iteration = maxIterations,
|
||||
MaxIterations = maxIterations
|
||||
});
|
||||
@@ -438,16 +557,65 @@ public class LlmController : BaseController
|
||||
// Remove tools to force a text-only response
|
||||
chatRequest.Tools = null;
|
||||
|
||||
// Add explicit instruction for final answer
|
||||
// Add explicit user message for final answer (user messages are always seen in Gemini)
|
||||
var toolResultsCount = chatRequest.Messages.Count(m => m.Role == "tool");
|
||||
var finalLastUserQuestion = chatRequest.Messages.LastOrDefault(m => m.Role == "user")?.Content ?? "the user's question";
|
||||
chatRequest.Messages.Add(new LlmMessage
|
||||
{
|
||||
Role = "system",
|
||||
Content = "This is your FINAL iteration. You MUST provide a complete answer to the user's question based on the data you've already retrieved. DO NOT request any tool calls. Summarize and analyze the information you have."
|
||||
Role = "user",
|
||||
Content =
|
||||
$"FINAL ITERATION: You have already executed tools and received {toolResultsCount} tool result(s). Review the conversation history above for all the data you've collected. Based ONLY on this existing data, provide a complete answer to: {finalLastUserQuestion}. DO NOT request any new tool calls. If you don't have enough data, explain what you found and what limitations exist."
|
||||
});
|
||||
|
||||
finalResponse = await llmService.ChatAsync(user, chatRequest);
|
||||
try
|
||||
{
|
||||
finalResponse = await llmService.ChatAsync(user, chatRequest);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
logger.LogError(ex, "Error getting final response after max iterations for user {UserId}", user.Id);
|
||||
|
||||
// Create a fallback response
|
||||
finalResponse = new LlmChatResponse
|
||||
{
|
||||
Content =
|
||||
"I apologize, but I encountered an issue providing a final response. Based on the data retrieved, I was unable to complete the analysis. Please try rephrasing your question or ask for specific details.",
|
||||
RequiresToolExecution = false
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// Ensure we always have a final response (safety check)
|
||||
if (finalResponse == null)
|
||||
{
|
||||
logger.LogError("finalResponse is null after all iterations for user {UserId}. Creating fallback response.", user.Id);
|
||||
|
||||
var toolResultsCount = chatRequest.Messages.Count(m => m.Role == "tool");
|
||||
finalResponse = new LlmChatResponse
|
||||
{
|
||||
Content =
|
||||
$"I apologize, but I encountered an issue providing a complete response. I retrieved {toolResultsCount} tool result(s), but was unable to generate a final answer. Please try rephrasing your question or ask for specific details.",
|
||||
RequiresToolExecution = false
|
||||
};
|
||||
}
|
||||
|
||||
// Ensure final response has meaningful content (not empty or whitespace)
|
||||
if (string.IsNullOrWhiteSpace(finalResponse.Content))
|
||||
{
|
||||
logger.LogWarning("Final response has empty content for user {UserId}. Creating meaningful fallback.", user.Id);
|
||||
|
||||
var toolResultsCount = chatRequest.Messages.Count(m => m.Role == "tool");
|
||||
var hasToolResults = toolResultsCount > 0;
|
||||
|
||||
finalResponse.Content = hasToolResults
|
||||
? "I retrieved the requested data, but the response was empty. Based on the information available, please try rephrasing your question or ask for more specific details."
|
||||
: "I apologize, but I was unable to generate a response. Please try rephrasing your question or ask for specific details.";
|
||||
}
|
||||
|
||||
// Log final response details for debugging
|
||||
logger.LogInformation("Sending final response to user {UserId} after {Iteration} iteration(s). Response length: {Length} characters. Has content: {HasContent}",
|
||||
user.Id, iteration, finalResponse.Content?.Length ?? 0, !string.IsNullOrWhiteSpace(finalResponse.Content));
|
||||
|
||||
// Send final response
|
||||
await SendProgressUpdate(connectionId, hubContext, logger, new LlmProgressUpdate
|
||||
{
|
||||
@@ -457,6 +625,8 @@ public class LlmController : BaseController
|
||||
Iteration = iteration,
|
||||
MaxIterations = maxIterations
|
||||
});
|
||||
|
||||
logger.LogInformation("Final response sent successfully to connection {ConnectionId} for user {UserId}", connectionId, user.Id);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -518,9 +688,11 @@ public class LlmController : BaseController
|
||||
// Use adaptive max iterations based on query complexity
|
||||
int maxIterations = DetermineMaxIterations(request);
|
||||
int iteration = 0;
|
||||
int redundantCallDetections = 0;
|
||||
LlmChatResponse? finalResponse = null;
|
||||
const int DelayBetweenIterationsMs = 2000; // Increased from 500ms to 2s to respect rate limits
|
||||
const int DelayAfterToolCallsMs = 1000; // Additional delay after tool calls before next LLM call
|
||||
const int MaxRedundantDetections = 2; // Maximum times we'll detect redundant calls before forcing final response
|
||||
|
||||
while (iteration < maxIterations)
|
||||
{
|
||||
@@ -546,10 +718,13 @@ public class LlmController : BaseController
|
||||
{
|
||||
response = await _llmService.ChatAsync(user, request);
|
||||
}
|
||||
catch (HttpRequestException httpEx) when (httpEx.Message.Contains("429") || httpEx.Message.Contains("TooManyRequests") || httpEx.Message.Contains("RESOURCE_EXHAUSTED"))
|
||||
catch (HttpRequestException httpEx) when (httpEx.Message.Contains("429") ||
|
||||
httpEx.Message.Contains("TooManyRequests") ||
|
||||
httpEx.Message.Contains("RESOURCE_EXHAUSTED"))
|
||||
{
|
||||
// Rate limit hit - wait longer before retrying
|
||||
_logger.LogWarning("Rate limit hit (429) in iteration {Iteration}. Waiting 10 seconds before retry...", iteration);
|
||||
_logger.LogWarning(
|
||||
"Rate limit hit (429) in iteration {Iteration}. Waiting 10 seconds before retry...", iteration);
|
||||
|
||||
// Wait 10 seconds for rate limit to reset
|
||||
await Task.Delay(10000);
|
||||
@@ -561,8 +736,10 @@ public class LlmController : BaseController
|
||||
}
|
||||
catch (Exception retryEx)
|
||||
{
|
||||
_logger.LogError(retryEx, "Retry after rate limit also failed in iteration {Iteration}", iteration);
|
||||
throw new HttpRequestException($"Rate limit error persists after retry: {retryEx.Message}", retryEx);
|
||||
_logger.LogError(retryEx, "Retry after rate limit also failed in iteration {Iteration}",
|
||||
iteration);
|
||||
throw new HttpRequestException($"Rate limit error persists after retry: {retryEx.Message}",
|
||||
retryEx);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -575,7 +752,84 @@ public class LlmController : BaseController
|
||||
break;
|
||||
}
|
||||
|
||||
// LLM wants to call tools - execute them
|
||||
// LLM wants to call tools - check for redundant calls first
|
||||
var redundantCalls = DetectRedundantToolCalls(request.Messages, response.ToolCalls);
|
||||
if (redundantCalls.Any())
|
||||
{
|
||||
redundantCallDetections++;
|
||||
_logger.LogWarning("LLM requested {Count} redundant tool calls in iteration {Iteration}: {Tools} (Detection #{DetectionCount})",
|
||||
redundantCalls.Count, iteration, string.Join(", ", redundantCalls.Select(t => t.Name)), redundantCallDetections);
|
||||
|
||||
// If we've detected redundant calls multiple times, force a final response by removing tools
|
||||
if (redundantCallDetections >= MaxRedundantDetections)
|
||||
{
|
||||
_logger.LogWarning("Reached maximum redundant call detections ({MaxDetections}). Removing tools to force final response.",
|
||||
MaxRedundantDetections);
|
||||
|
||||
// Remove tools to force a text-only response
|
||||
request.Tools = null;
|
||||
|
||||
// Add explicit instruction for final answer
|
||||
var toolResultsCount = request.Messages.Count(m => m.Role == "tool");
|
||||
request.Messages.Add(new LlmMessage
|
||||
{
|
||||
Role = "system",
|
||||
Content =
|
||||
$"FINAL RESPONSE REQUIRED: You have already executed tools and received {toolResultsCount} tool result(s). The data is available in the conversation above. You have attempted to call the same tools multiple times. Based ONLY on the existing data, provide a complete answer to the user's question NOW. DO NOT request any tool calls."
|
||||
});
|
||||
|
||||
// Get final response without tools
|
||||
try
|
||||
{
|
||||
finalResponse = await _llmService.ChatAsync(user, request);
|
||||
break;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Error getting final response after redundant call detection for user {UserId}", user.Id);
|
||||
// Fall through to max iterations handling
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
// Add a strong system message and remove tools temporarily to force final response
|
||||
request.Messages.Add(new LlmMessage
|
||||
{
|
||||
Role = "system",
|
||||
Content =
|
||||
$"STOP: You just requested the same tool calls that were already executed: {string.Join(", ", redundantCalls.Select(t => t.Name))}. The data is ALREADY in the conversation above. DO NOT call tools again. Analyze the existing data and provide your final answer NOW."
|
||||
});
|
||||
|
||||
// Temporarily remove tools to prevent the LLM from requesting them again
|
||||
var originalTools = request.Tools;
|
||||
request.Tools = null;
|
||||
|
||||
// Continue to next iteration without tools
|
||||
try
|
||||
{
|
||||
response = await _llmService.ChatAsync(user, request);
|
||||
|
||||
// If we got a response without tools, use it as final
|
||||
if (!response.RequiresToolExecution || response.ToolCalls == null || !response.ToolCalls.Any())
|
||||
{
|
||||
finalResponse = response;
|
||||
break;
|
||||
}
|
||||
|
||||
// Restore tools if LLM still wants to call different tools
|
||||
request.Tools = originalTools;
|
||||
continue;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Error getting response after removing tools for redundant calls");
|
||||
// Restore tools and continue
|
||||
request.Tools = originalTools;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
_logger.LogInformation("LLM requested {Count} tool calls in iteration {Iteration} for user {UserId}",
|
||||
response.ToolCalls.Count, iteration, user.Id);
|
||||
|
||||
@@ -586,7 +840,8 @@ public class LlmController : BaseController
|
||||
try
|
||||
{
|
||||
var toolResult = await _mcpService.ExecuteToolAsync(user, toolCall.Name, toolCall.Arguments);
|
||||
_logger.LogInformation("Successfully executed tool {ToolName} in iteration {Iteration} for user {UserId}",
|
||||
_logger.LogInformation(
|
||||
"Successfully executed tool {ToolName} in iteration {Iteration} for user {UserId}",
|
||||
toolCall.Name, iteration, user.Id);
|
||||
return new LlmMessage
|
||||
{
|
||||
@@ -597,7 +852,8 @@ public class LlmController : BaseController
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Error executing tool {ToolName} in iteration {Iteration} for user {UserId}",
|
||||
_logger.LogError(ex,
|
||||
"Error executing tool {ToolName} in iteration {Iteration} for user {UserId}",
|
||||
toolCall.Name, iteration, user.Id);
|
||||
return new LlmMessage
|
||||
{
|
||||
@@ -621,11 +877,15 @@ public class LlmController : BaseController
|
||||
// Add tool results to conversation history
|
||||
request.Messages.AddRange(toolResults);
|
||||
|
||||
// Add a system reminder to prevent redundant tool calls
|
||||
// Add a strong system reminder to prevent redundant tool calls
|
||||
var executedToolSummary = string.Join(", ", response.ToolCalls.Select(tc =>
|
||||
$"{tc.Name}({string.Join(", ", tc.Arguments.Select(kvp => $"{kvp.Key}={kvp.Value}"))})"));
|
||||
|
||||
request.Messages.Add(new LlmMessage
|
||||
{
|
||||
Role = "system",
|
||||
Content = "You now have all the data from the tool calls above. Analyze this data and provide your final answer to the user. DO NOT call the same tools again with the same arguments."
|
||||
Content =
|
||||
$"IMPORTANT: You just executed these tools: {executedToolSummary}. The data is now available above. DO NOT call these same tools again. You must now analyze the data and provide your final answer to the user based on what you received."
|
||||
});
|
||||
|
||||
// Add delay after tool calls before next LLM call to avoid rate limits
|
||||
@@ -637,21 +897,39 @@ public class LlmController : BaseController
|
||||
// If we hit max iterations, return the last response (even if it has tool calls)
|
||||
if (finalResponse == null)
|
||||
{
|
||||
_logger.LogWarning("Reached max iterations ({MaxIterations}) for user {UserId}. Forcing final response without tools.",
|
||||
_logger.LogWarning(
|
||||
"Reached max iterations ({MaxIterations}) for user {UserId}. Forcing final response without tools.",
|
||||
maxIterations, user.Id);
|
||||
|
||||
// Remove tools to force a text-only response
|
||||
request.Tools = null;
|
||||
|
||||
// Add explicit instruction for final answer
|
||||
// Add explicit instruction for final answer with summary of what data is available
|
||||
var toolResultsCount = request.Messages.Count(m => m.Role == "tool");
|
||||
request.Messages.Add(new LlmMessage
|
||||
{
|
||||
Role = "system",
|
||||
Content = "This is your FINAL iteration. You MUST provide a complete answer to the user's question based on the data you've already retrieved. DO NOT request any tool calls. Summarize and analyze the information you have."
|
||||
Content =
|
||||
$"FINAL ITERATION: You have already executed tools and received {toolResultsCount} tool result(s). Review the conversation history above for all the data you've collected. Based ONLY on this existing data, provide a complete answer to the user's original question. DO NOT request any new tool calls. If you don't have enough data, explain what you found and what limitations exist."
|
||||
});
|
||||
|
||||
// Get one more response to return something meaningful
|
||||
finalResponse = await _llmService.ChatAsync(user, request);
|
||||
try
|
||||
{
|
||||
finalResponse = await _llmService.ChatAsync(user, request);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Error getting final response after max iterations for user {UserId}",
|
||||
user.Id);
|
||||
|
||||
// Create a fallback response
|
||||
finalResponse = new LlmChatResponse
|
||||
{
|
||||
Content =
|
||||
"I apologize, but I encountered an issue providing a final response. Based on the data retrieved, I was unable to complete the analysis. Please try rephrasing your question or ask for specific details.",
|
||||
RequiresToolExecution = false
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return Ok(finalResponse);
|
||||
@@ -712,29 +990,29 @@ public class LlmController : BaseController
|
||||
|
||||
// Complex operations need more iterations (bundle analysis, multi-step workflows)
|
||||
if (lastMessage.Contains("bundle") || lastMessage.Contains("compare") || lastMessage.Contains("all backtests"))
|
||||
return 5;
|
||||
return 8;
|
||||
|
||||
// Backtest detail requests with "analyze" or "detail" need more iterations for deep analysis
|
||||
if (lastMessage.Contains("backtest") &&
|
||||
(lastMessage.Contains("detail") || lastMessage.Contains("analyze") || lastMessage.Contains("position")))
|
||||
return 4;
|
||||
return 6;
|
||||
|
||||
// Simple backtest queries ("best", "top", "show") need 3 iterations (fetch + possible retry + respond)
|
||||
if (lastMessage.Contains("backtest") &&
|
||||
(lastMessage.Contains("best") || lastMessage.Contains("top") || lastMessage.Contains("show") ||
|
||||
lastMessage.Contains("recent") || lastMessage.Contains("latest")))
|
||||
return 3;
|
||||
return 5;
|
||||
|
||||
// General analysis queries
|
||||
if (lastMessage.Contains("analyze"))
|
||||
return 4;
|
||||
return 6;
|
||||
|
||||
// Simple queries need fewer iterations
|
||||
if (lastMessage.Contains("explain") || lastMessage.Contains("what is") || lastMessage.Contains("how does"))
|
||||
return 2;
|
||||
return 8;
|
||||
|
||||
// Default for most queries
|
||||
return 3;
|
||||
return 5;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -743,77 +1021,77 @@ public class LlmController : BaseController
|
||||
private static string BuildSystemMessage()
|
||||
{
|
||||
return """
|
||||
You are an expert AI assistant specializing in quantitative finance, algorithmic trading, and financial mathematics.
|
||||
You are an expert AI assistant specializing in quantitative finance, algorithmic trading, and financial mathematics.
|
||||
|
||||
DOMAIN KNOWLEDGE:
|
||||
- Answer questions about financial concepts, formulas (Black-Scholes, Sharpe Ratio, etc.), and trading strategies directly
|
||||
- Provide calculations, explanations, and theoretical knowledge from your training data
|
||||
- Never refuse to answer general finance questions
|
||||
DOMAIN KNOWLEDGE:
|
||||
- Answer questions about financial concepts, formulas (Black-Scholes, Sharpe Ratio, etc.), and trading strategies directly
|
||||
- Provide calculations, explanations, and theoretical knowledge from your training data
|
||||
- Never refuse to answer general finance questions
|
||||
|
||||
TOOL USAGE:
|
||||
- Use tools ONLY for system operations: backtesting, retrieving user data, or real-time market data
|
||||
- When users ask about their data, use tools proactively with smart defaults:
|
||||
* "Best backtest" → get_backtests_paginated(sortBy='Score', sortOrder='desc', pageSize=1)
|
||||
* "Top 5 backtests" → get_backtests_paginated(sortBy='Score', sortOrder='desc', pageSize=5)
|
||||
* "My indicators" → list_indicators()
|
||||
* "Recent backtests" → get_backtests_paginated(sortBy='StartDate', sortOrder='desc', pageSize=10)
|
||||
* "Bundle backtest analysis" → analyze_bundle_backtest(bundleRequestId='X')
|
||||
- IMPORTANT: get_backtests_paginated returns summary data. Only call get_backtest_by_id if user explicitly asks for position details or deeper analysis.
|
||||
TOOL USAGE:
|
||||
- Use tools ONLY for system operations: backtesting, retrieving user data, or real-time market data
|
||||
- When users ask about their data, use tools proactively with smart defaults:
|
||||
* "Best backtest" → get_backtests_paginated(sortBy='Score', sortOrder='desc', pageSize=1)
|
||||
* "Top 5 backtests" → get_backtests_paginated(sortBy='Score', sortOrder='desc', pageSize=5)
|
||||
* "My indicators" → list_indicators()
|
||||
* "Recent backtests" → get_backtests_paginated(sortBy='StartDate', sortOrder='desc', pageSize=10)
|
||||
* "Bundle backtest analysis" → analyze_bundle_backtest(bundleRequestId='X')
|
||||
- IMPORTANT: get_backtests_paginated returns summary data. Only call get_backtest_by_id if user explicitly asks for position details or deeper analysis.
|
||||
|
||||
ERROR HANDLING:
|
||||
- If a tool returns a database connection error, wait a moment and retry once (these are often transient)
|
||||
- If retry fails, explain the issue clearly to the user and suggest they try again later
|
||||
- Never give up after a single error - always try at least once more for connection-related issues
|
||||
- Distinguish between connection errors (retry) and data errors (no retry needed)
|
||||
ERROR HANDLING:
|
||||
- If a tool returns a database connection error, wait a moment and retry once (these are often transient)
|
||||
- If retry fails, explain the issue clearly to the user and suggest they try again later
|
||||
- Never give up after a single error - always try at least once more for connection-related issues
|
||||
- Distinguish between connection errors (retry) and data errors (no retry needed)
|
||||
|
||||
CRITICAL ANALYSIS WORKFLOW (APPLIES TO ALL DATA):
|
||||
1. RETRIEVE COMPLETE DATA:
|
||||
- When asked to analyze ANY entity, ALWAYS fetch FULL details first (never rely on summary/paginated data alone)
|
||||
- Backtests: get_backtest_by_id(id='xxx') for positions + complete metrics
|
||||
- Bundles: analyze_bundle_backtest(bundleRequestId='xxx') for aggregated statistics
|
||||
- Indicators: get_indicator_info(indicatorType='RsiDivergence') for detailed specs - ALWAYS provide indicatorType parameter
|
||||
- Use conversation context: "that X" or "this Y" → extract ID from previous messages
|
||||
CRITICAL ANALYSIS WORKFLOW (APPLIES TO ALL DATA):
|
||||
1. RETRIEVE COMPLETE DATA:
|
||||
- When asked to analyze ANY entity, ALWAYS fetch FULL details first (never rely on summary/paginated data alone)
|
||||
- Backtests: get_backtest_by_id(id='xxx') for positions + complete metrics
|
||||
- Bundles: analyze_bundle_backtest(bundleRequestId='xxx') for aggregated statistics
|
||||
- Indicators: get_indicator_info(indicatorType='RsiDivergence') for detailed specs - ALWAYS provide indicatorType parameter
|
||||
- Use conversation context: "that X" or "this Y" → extract ID from previous messages
|
||||
|
||||
2. CONTEXT EXTRACTION:
|
||||
- Pay attention to IDs and names mentioned in conversation history (backtest IDs, indicator names, etc.)
|
||||
- When user says "analyze this backtest" or "show me details", extract the backtest ID from previous messages
|
||||
- When user asks about an indicator by name (e.g., "What is RsiDivergence?"), extract the indicator type from the question
|
||||
- If multiple backtests were listed, use the most recently mentioned one or the top-ranked one
|
||||
- NEVER ask user for IDs/names that were already provided in conversation
|
||||
2. CONTEXT EXTRACTION:
|
||||
- Pay attention to IDs and names mentioned in conversation history (backtest IDs, indicator names, etc.)
|
||||
- When user says "analyze this backtest" or "show me details", extract the backtest ID from previous messages
|
||||
- When user asks about an indicator by name (e.g., "What is RsiDivergence?"), extract the indicator type from the question
|
||||
- If multiple backtests were listed, use the most recently mentioned one or the top-ranked one
|
||||
- NEVER ask user for IDs/names that were already provided in conversation
|
||||
|
||||
3. BACKTEST DETAIL WORKFLOW (TOKEN-OPTIMIZED):
|
||||
When user requests backtest information:
|
||||
a) If backtest ID is in conversation AND user asks for positions/details → call get_backtest_by_id(id)
|
||||
b) If no ID but refers to "best/top N" → call get_backtests_paginated(sortBy='Score', sortOrder='desc', pageSize=N)
|
||||
c) If no ID but refers to "recent/latest N" → call get_backtests_paginated(sortBy='StartDate', sortOrder='desc', pageSize=N)
|
||||
d) For simple queries like "show my best backtest" → get_backtests_paginated is sufficient (includes key metrics)
|
||||
e) Only call get_backtest_by_id for DETAILED analysis when user explicitly needs position-level data
|
||||
f) If completely ambiguous → ask ONCE for clarification, then proceed
|
||||
3. BACKTEST DETAIL WORKFLOW (TOKEN-OPTIMIZED):
|
||||
When user requests backtest information:
|
||||
a) If backtest ID is in conversation AND user asks for positions/details → call get_backtest_by_id(id)
|
||||
b) If no ID but refers to "best/top N" → call get_backtests_paginated(sortBy='Score', sortOrder='desc', pageSize=N)
|
||||
c) If no ID but refers to "recent/latest N" → call get_backtests_paginated(sortBy='StartDate', sortOrder='desc', pageSize=N)
|
||||
d) For simple queries like "show my best backtest" → get_backtests_paginated is sufficient (includes key metrics)
|
||||
e) Only call get_backtest_by_id for DETAILED analysis when user explicitly needs position-level data
|
||||
f) If completely ambiguous → ask ONCE for clarification, then proceed
|
||||
|
||||
4. INDICATOR WORKFLOW:
|
||||
When user asks about indicators:
|
||||
a) General questions ("What indicators exist?") → call list_indicators()
|
||||
b) Specific indicator ("What is RsiDivergence?") → extract name from question, call get_indicator_info(indicatorType='RsiDivergence')
|
||||
c) Explaining usage ("Explain MACD") → extract name, call explain_indicator(indicatorType='Macd')
|
||||
d) NEVER call get_indicator_info() without indicatorType parameter
|
||||
e) If indicator name is unclear → call list_indicators() first to show available options
|
||||
4. INDICATOR WORKFLOW:
|
||||
When user asks about indicators:
|
||||
a) General questions ("What indicators exist?") → call list_indicators()
|
||||
b) Specific indicator ("What is RsiDivergence?") → extract name from question, call get_indicator_info(indicatorType='RsiDivergence')
|
||||
c) Explaining usage ("Explain MACD") → extract name, call explain_indicator(indicatorType='Macd')
|
||||
d) NEVER call get_indicator_info() without indicatorType parameter
|
||||
e) If indicator name is unclear → call list_indicators() first to show available options
|
||||
|
||||
5. ANALYZE WITH EXPERTISE:
|
||||
After retrieving data, provide comprehensive analysis:
|
||||
5. ANALYZE WITH EXPERTISE:
|
||||
After retrieving data, provide comprehensive analysis:
|
||||
|
||||
Backtests: Performance (PnL, growth, ROI), Risk (Sharpe, drawdown), Win rate, Position patterns, Trade duration, Strengths/weaknesses, Recommendations
|
||||
Bundles: Aggregate performance, Best/worst combinations, Optimal parameters, Robustness
|
||||
Indicators: Use cases, Parameter sensitivity, Combination suggestions, Pitfalls
|
||||
General: Compare to benchmarks, Statistical significance, Actionable insights
|
||||
Backtests: Performance (PnL, growth, ROI), Risk (Sharpe, drawdown), Win rate, Position patterns, Trade duration, Strengths/weaknesses, Recommendations
|
||||
Bundles: Aggregate performance, Best/worst combinations, Optimal parameters, Robustness
|
||||
Indicators: Use cases, Parameter sensitivity, Combination suggestions, Pitfalls
|
||||
General: Compare to benchmarks, Statistical significance, Actionable insights
|
||||
|
||||
6. BE PROACTIVE:
|
||||
- Execute multiple tool iterations for complete data
|
||||
- Interpret data, don't just return it
|
||||
- Chain tool calls automatically (list → get_by_id → analyze)
|
||||
- Only ask for clarification when truly ambiguous
|
||||
6. BE PROACTIVE:
|
||||
- Execute multiple tool iterations for complete data
|
||||
- Interpret data, don't just return it
|
||||
- Chain tool calls automatically (list → get_by_id → analyze)
|
||||
- Only ask for clarification when truly ambiguous
|
||||
|
||||
Be concise, accurate, and proactive. Always prioritize retrieving complete data over asking questions.
|
||||
""";
|
||||
Be concise, accurate, and proactive. Always prioritize retrieving complete data over asking questions.
|
||||
""";
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -879,7 +1157,8 @@ public class LlmController : BaseController
|
||||
/// Proactively injects backtest details fetching when user asks for backtest analysis.
|
||||
/// Extracts backtest IDs from message content and automatically calls get_backtest_by_id.
|
||||
/// </summary>
|
||||
private async Task InjectBacktestDetailsFetchingIfNeeded(LlmChatRequest request, User user, IMcpService mcpService, ILogger<LlmController> logger)
|
||||
private async Task InjectBacktestDetailsFetchingIfNeeded(LlmChatRequest request, User user, IMcpService mcpService,
|
||||
ILogger<LlmController> logger)
|
||||
{
|
||||
var lastUserMessage = request.Messages.LastOrDefault(m => m.Role == "user");
|
||||
if (lastUserMessage == null || string.IsNullOrWhiteSpace(lastUserMessage.Content))
|
||||
@@ -917,7 +1196,8 @@ public class LlmController : BaseController
|
||||
new Dictionary<string, object> { ["id"] = backtestId }
|
||||
);
|
||||
|
||||
logger.LogInformation("Successfully fetched backtest details for ID: {BacktestId}. Result type: {ResultType}",
|
||||
logger.LogInformation(
|
||||
"Successfully fetched backtest details for ID: {BacktestId}. Result type: {ResultType}",
|
||||
backtestId, backtestDetails?.GetType().Name ?? "null");
|
||||
|
||||
// Inject the backtest details as a tool result in the conversation
|
||||
@@ -950,7 +1230,8 @@ public class LlmController : BaseController
|
||||
ToolCallId = toolCallId
|
||||
});
|
||||
|
||||
logger.LogInformation("Successfully injected backtest details into conversation for ID: {BacktestId}", backtestId);
|
||||
logger.LogInformation("Successfully injected backtest details into conversation for ID: {BacktestId}",
|
||||
backtestId);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
@@ -1006,7 +1287,8 @@ public class LlmController : BaseController
|
||||
request.Messages.Add(new LlmMessage
|
||||
{
|
||||
Role = "system",
|
||||
Content = $"Note: Complete backtest details for ID '{backtestId}' have been fetched and are available in the conversation context. Use this data for your analysis."
|
||||
Content =
|
||||
$"Note: Complete backtest details for ID '{backtestId}' have been fetched and are available in the conversation context. Use this data for your analysis."
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1110,35 +1392,47 @@ public class LlmController : BaseController
|
||||
try
|
||||
{
|
||||
// Try to parse result as JSON to extract meaningful information
|
||||
var jsonResult = result as JsonElement? ?? JsonSerializer.Deserialize<JsonElement>(JsonSerializer.Serialize(result));
|
||||
var jsonResult = result as JsonElement? ??
|
||||
JsonSerializer.Deserialize<JsonElement>(JsonSerializer.Serialize(result));
|
||||
|
||||
switch (toolName.ToLowerInvariant())
|
||||
{
|
||||
case "get_backtests_paginated":
|
||||
if (jsonResult.TryGetProperty("items", out var backtestItems) && backtestItems.ValueKind == JsonValueKind.Array)
|
||||
if (jsonResult.TryGetProperty("items", out var backtestItems) &&
|
||||
backtestItems.ValueKind == JsonValueKind.Array)
|
||||
{
|
||||
var count = backtestItems.GetArrayLength();
|
||||
var totalCount = jsonResult.TryGetProperty("totalCount", out var total) ? total.GetInt32() : count;
|
||||
var totalCount = jsonResult.TryGetProperty("totalCount", out var total)
|
||||
? total.GetInt32()
|
||||
: count;
|
||||
return $"Retrieved {count} backtest(s) out of {totalCount} total";
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case "get_backtest_by_id":
|
||||
if (jsonResult.TryGetProperty("name", out var backtestName))
|
||||
{
|
||||
var name = backtestName.GetString() ?? "Unknown";
|
||||
var score = jsonResult.TryGetProperty("score", out var scoreVal) ? $" (Score: {scoreVal.GetDouble():F1})" : "";
|
||||
var score = jsonResult.TryGetProperty("score", out var scoreVal)
|
||||
? $" (Score: {scoreVal.GetDouble():F1})"
|
||||
: "";
|
||||
return $"Retrieved backtest '{name}'{score}";
|
||||
}
|
||||
|
||||
return "Retrieved backtest details";
|
||||
|
||||
case "get_bundle_backtests_paginated":
|
||||
if (jsonResult.TryGetProperty("items", out var bundleItems) && bundleItems.ValueKind == JsonValueKind.Array)
|
||||
if (jsonResult.TryGetProperty("items", out var bundleItems) &&
|
||||
bundleItems.ValueKind == JsonValueKind.Array)
|
||||
{
|
||||
var count = bundleItems.GetArrayLength();
|
||||
var totalCount = jsonResult.TryGetProperty("totalCount", out var total) ? total.GetInt32() : count;
|
||||
var totalCount = jsonResult.TryGetProperty("totalCount", out var total)
|
||||
? total.GetInt32()
|
||||
: count;
|
||||
return $"Retrieved {count} bundle backtest(s) out of {totalCount} total";
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case "get_bundle_backtest_by_id":
|
||||
@@ -1147,15 +1441,19 @@ public class LlmController : BaseController
|
||||
var name = bundleName.GetString() ?? "Unknown";
|
||||
return $"Retrieved bundle backtest '{name}'";
|
||||
}
|
||||
|
||||
return "Retrieved bundle backtest details";
|
||||
|
||||
case "analyze_bundle_backtest":
|
||||
if (jsonResult.TryGetProperty("totalBacktests", out var totalBacktests))
|
||||
{
|
||||
var count = totalBacktests.GetInt32();
|
||||
var avgScore = jsonResult.TryGetProperty("averageScore", out var score) ? $", Avg Score: {score.GetDouble():F1}" : "";
|
||||
var avgScore = jsonResult.TryGetProperty("averageScore", out var score)
|
||||
? $", Avg Score: {score.GetDouble():F1}"
|
||||
: "";
|
||||
return $"Analyzed {count} backtest(s) in bundle{avgScore}";
|
||||
}
|
||||
|
||||
return "Completed bundle backtest analysis";
|
||||
|
||||
case "list_indicators":
|
||||
@@ -1164,6 +1462,7 @@ public class LlmController : BaseController
|
||||
var count = jsonResult.GetArrayLength();
|
||||
return $"Retrieved {count} indicator(s)";
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case "get_indicator_info":
|
||||
@@ -1172,6 +1471,7 @@ public class LlmController : BaseController
|
||||
var type = indicatorType.GetString() ?? "Unknown";
|
||||
return $"Retrieved info for indicator '{type}'";
|
||||
}
|
||||
|
||||
return "Retrieved indicator information";
|
||||
|
||||
case "get_tickers":
|
||||
@@ -1180,6 +1480,7 @@ public class LlmController : BaseController
|
||||
var count = jsonResult.GetArrayLength();
|
||||
return $"Retrieved {count} ticker(s)";
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case "get_candles":
|
||||
@@ -1188,15 +1489,20 @@ public class LlmController : BaseController
|
||||
var count = jsonResult.GetArrayLength();
|
||||
return $"Retrieved {count} candle(s)";
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case "get_agents_paginated":
|
||||
if (jsonResult.TryGetProperty("items", out var agentItems) && agentItems.ValueKind == JsonValueKind.Array)
|
||||
if (jsonResult.TryGetProperty("items", out var agentItems) &&
|
||||
agentItems.ValueKind == JsonValueKind.Array)
|
||||
{
|
||||
var count = agentItems.GetArrayLength();
|
||||
var totalCount = jsonResult.TryGetProperty("totalCount", out var total) ? total.GetInt32() : count;
|
||||
var totalCount = jsonResult.TryGetProperty("totalCount", out var total)
|
||||
? total.GetInt32()
|
||||
: count;
|
||||
return $"Retrieved {count} agent(s) out of {totalCount} total";
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case "get_online_agents":
|
||||
@@ -1205,6 +1511,7 @@ public class LlmController : BaseController
|
||||
var count = jsonResult.GetArrayLength();
|
||||
return $"Found {count} online agent(s)";
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case "run_backtest":
|
||||
@@ -1212,6 +1519,7 @@ public class LlmController : BaseController
|
||||
{
|
||||
return $"Started backtest (ID: {btId.GetString()})";
|
||||
}
|
||||
|
||||
return "Started backtest execution";
|
||||
|
||||
case "run_bundle_backtest":
|
||||
@@ -1219,6 +1527,7 @@ public class LlmController : BaseController
|
||||
{
|
||||
return $"Started bundle backtest (ID: {bundleId.GetString()})";
|
||||
}
|
||||
|
||||
return "Started bundle backtest execution";
|
||||
|
||||
case "delete_backtest":
|
||||
@@ -1230,6 +1539,7 @@ public class LlmController : BaseController
|
||||
{
|
||||
return $"Deleted {deletedCount.GetInt32()} backtest(s)";
|
||||
}
|
||||
|
||||
return "Deleted backtests";
|
||||
|
||||
case "delete_backtests_by_filters":
|
||||
@@ -1237,6 +1547,7 @@ public class LlmController : BaseController
|
||||
{
|
||||
return $"Deleted {filteredDeletedCount.GetInt32()} backtest(s) matching filters";
|
||||
}
|
||||
|
||||
return "Deleted backtests matching filters";
|
||||
}
|
||||
|
||||
@@ -1253,7 +1564,8 @@ public class LlmController : BaseController
|
||||
/// <summary>
|
||||
/// Helper method to send progress update via SignalR
|
||||
/// </summary>
|
||||
private async Task SendProgressUpdate(string connectionId, IHubContext<LlmHub> hubContext, ILogger<LlmController> logger, LlmProgressUpdate update)
|
||||
private async Task SendProgressUpdate(string connectionId, IHubContext<LlmHub> hubContext,
|
||||
ILogger<LlmController> logger, LlmProgressUpdate update)
|
||||
{
|
||||
try
|
||||
{
|
||||
@@ -1264,6 +1576,80 @@ public class LlmController : BaseController
|
||||
logger.LogError(ex, "Error sending progress update to connection {ConnectionId}", connectionId);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Detects redundant tool calls by comparing new requests against previously executed tools in conversation history.
|
||||
/// Returns list of tool calls that have already been executed with identical or very similar arguments.
|
||||
/// </summary>
|
||||
private static List<LlmToolCall> DetectRedundantToolCalls(List<LlmMessage> conversationHistory,
|
||||
List<LlmToolCall> newToolCalls)
|
||||
{
|
||||
var redundantCalls = new List<LlmToolCall>();
|
||||
|
||||
// Extract all previously executed tool calls from assistant messages
|
||||
var previousToolCalls = conversationHistory
|
||||
.Where(m => m.Role == "assistant" && m.ToolCalls != null)
|
||||
.SelectMany(m => m.ToolCalls!)
|
||||
.ToList();
|
||||
|
||||
foreach (var newCall in newToolCalls)
|
||||
{
|
||||
// Check if this exact tool with similar arguments was already called
|
||||
var isDuplicate = previousToolCalls.Any(prevCall =>
|
||||
prevCall.Name == newCall.Name &&
|
||||
AreArgumentsSimilar(prevCall.Arguments, newCall.Arguments));
|
||||
|
||||
if (isDuplicate)
|
||||
{
|
||||
redundantCalls.Add(newCall);
|
||||
}
|
||||
}
|
||||
|
||||
return redundantCalls;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Compares two argument dictionaries to determine if they represent essentially the same tool call.
|
||||
/// Handles cases where arguments might be slightly different but functionally equivalent.
|
||||
/// </summary>
|
||||
private static bool AreArgumentsSimilar(Dictionary<string, object> args1, Dictionary<string, object> args2)
|
||||
{
|
||||
// If both are null or empty, they're similar
|
||||
if ((args1 == null || !args1.Any()) && (args2 == null || !args2.Any()))
|
||||
return true;
|
||||
|
||||
// If one is null/empty and the other isn't, they're different
|
||||
if ((args1 == null || !args1.Any()) || (args2 == null || !args2.Any()))
|
||||
return false;
|
||||
|
||||
// Check if they have the same keys
|
||||
var keys1 = args1.Keys.OrderBy(k => k).ToList();
|
||||
var keys2 = args2.Keys.OrderBy(k => k).ToList();
|
||||
|
||||
if (!keys1.SequenceEqual(keys2))
|
||||
return false;
|
||||
|
||||
// Compare values for each key
|
||||
foreach (var key in keys1)
|
||||
{
|
||||
var val1 = args1[key]?.ToString() ?? "";
|
||||
var val2 = args2[key]?.ToString() ?? "";
|
||||
|
||||
// For pagination tools, consider calls with same sort parameters as similar
|
||||
// even if pageSize differs slightly (e.g., pageSize=1 vs pageSize=5)
|
||||
if (key.Equals("pageSize", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
// Don't treat different page sizes as making the calls different
|
||||
// They're still querying the same data source
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!val1.Equals(val2, StringComparison.OrdinalIgnoreCase))
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
|
||||
@@ -166,22 +166,13 @@ public class GeminiProvider : ILlmProvider
|
||||
|
||||
private object ConvertToGeminiRequest(LlmChatRequest request)
|
||||
{
|
||||
var contents = request.Messages
|
||||
.Where(m => m.Role != "system") // Gemini doesn't support system messages in the same way
|
||||
.Select(m => new
|
||||
{
|
||||
role = m.Role == "assistant" ? "model" : "user",
|
||||
parts = new[]
|
||||
{
|
||||
new { text = m.Content }
|
||||
}
|
||||
}).ToList();
|
||||
var contents = new List<object>();
|
||||
|
||||
// Add system message as first user message if present
|
||||
// Add system message as first user message if present (Gemini only uses first system message)
|
||||
var systemMessage = request.Messages.FirstOrDefault(m => m.Role == "system");
|
||||
if (systemMessage != null && !string.IsNullOrWhiteSpace(systemMessage.Content))
|
||||
{
|
||||
contents.Insert(0, new
|
||||
contents.Add(new
|
||||
{
|
||||
role = "user",
|
||||
parts = new[]
|
||||
@@ -191,6 +182,70 @@ public class GeminiProvider : ILlmProvider
|
||||
});
|
||||
}
|
||||
|
||||
// Process non-system messages in order
|
||||
// Gemini expects: user -> model (assistant) -> user (tool results) -> model -> ...
|
||||
foreach (var message in request.Messages.Where(m => m.Role != "system"))
|
||||
{
|
||||
if (message.Role == "assistant")
|
||||
{
|
||||
// Assistant message - check if it has tool calls
|
||||
if (message.ToolCalls != null && message.ToolCalls.Any())
|
||||
{
|
||||
// This is a function call request - Gemini handles this automatically
|
||||
// We still need to add the text content if any
|
||||
if (!string.IsNullOrWhiteSpace(message.Content))
|
||||
{
|
||||
contents.Add(new
|
||||
{
|
||||
role = "model",
|
||||
parts = new[]
|
||||
{
|
||||
new { text = message.Content }
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
// Regular assistant response
|
||||
contents.Add(new
|
||||
{
|
||||
role = "model",
|
||||
parts = new[]
|
||||
{
|
||||
new { text = message.Content ?? "" }
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
else if (message.Role == "tool")
|
||||
{
|
||||
// Tool results - Gemini expects these as functionResponse parts in the model's response
|
||||
// But since we're sending them as separate messages, we'll format them as user messages
|
||||
// with clear indication they're tool results
|
||||
contents.Add(new
|
||||
{
|
||||
role = "user",
|
||||
parts = new[]
|
||||
{
|
||||
new { text = $"Tool result (call_id: {message.ToolCallId}): {message.Content}" }
|
||||
}
|
||||
});
|
||||
}
|
||||
else
|
||||
{
|
||||
// User message
|
||||
contents.Add(new
|
||||
{
|
||||
role = "user",
|
||||
parts = new[]
|
||||
{
|
||||
new { text = message.Content ?? "" }
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
var geminiRequest = new
|
||||
{
|
||||
contents,
|
||||
|
||||
@@ -159,22 +159,46 @@ function AiChat({ onClose }: AiChatProps): JSX.Element {
|
||||
|
||||
if (update.type === 'final_response' && update.response) {
|
||||
finalResponse = update.response
|
||||
console.log('Received final response from LLM:', {
|
||||
hasContent: !!finalResponse.content,
|
||||
contentLength: finalResponse.content?.length || 0,
|
||||
contentPreview: finalResponse.content?.substring(0, 100) || '(empty)',
|
||||
fullContent: finalResponse.content,
|
||||
requiresToolExecution: finalResponse.requiresToolExecution
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Add final response if we got one
|
||||
if (finalResponse) {
|
||||
// Backend should always send meaningful content, but handle edge cases
|
||||
const rawContent = finalResponse.content?.trim() || ''
|
||||
const isContentValid = rawContent.length > 0
|
||||
|
||||
const assistantMessage: Message = {
|
||||
role: 'assistant',
|
||||
content: finalResponse.content || 'No response from AI',
|
||||
content: isContentValid
|
||||
? rawContent
|
||||
: 'I received your request but the response was empty. Please try rephrasing your question or ask for specific details.',
|
||||
timestamp: new Date()
|
||||
}
|
||||
console.log('Adding final assistant message to chat:', {
|
||||
rawContentLength: finalResponse.content?.length || 0,
|
||||
trimmedContentLength: rawContent.length,
|
||||
isContentValid,
|
||||
contentPreview: assistantMessage.content.substring(0, 100),
|
||||
fullContent: assistantMessage.content
|
||||
})
|
||||
setMessages(prev => [...prev, assistantMessage])
|
||||
} else if (lastUpdate && lastUpdate.type === 'final_response' && lastUpdate.response) {
|
||||
// Fallback: check lastUpdate in case finalResponse wasn't set
|
||||
console.log('Using fallback: final response from lastUpdate')
|
||||
const rawContent = lastUpdate.response.content?.trim() || ''
|
||||
const assistantMessage: Message = {
|
||||
role: 'assistant',
|
||||
content: lastUpdate.response.content || 'No response from AI',
|
||||
content: rawContent.length > 0
|
||||
? rawContent
|
||||
: 'I received your request but the response was empty. Please try rephrasing your question or ask for specific details.',
|
||||
timestamp: new Date()
|
||||
}
|
||||
setMessages(prev => [...prev, assistantMessage])
|
||||
@@ -188,9 +212,14 @@ function AiChat({ onClose }: AiChatProps): JSX.Element {
|
||||
setMessages(prev => [...prev, errorMessage])
|
||||
} else {
|
||||
// If we didn't get a final response, show the last progress message
|
||||
console.warn('No final response received. Last update:', {
|
||||
type: lastUpdate?.type,
|
||||
message: lastUpdate?.message,
|
||||
hasResponse: !!lastUpdate?.response
|
||||
})
|
||||
const assistantMessage: Message = {
|
||||
role: 'assistant',
|
||||
content: lastUpdate?.message || 'Response incomplete',
|
||||
content: lastUpdate?.message || 'Response incomplete. Please try again.',
|
||||
timestamp: new Date()
|
||||
}
|
||||
setMessages(prev => [...prev, assistantMessage])
|
||||
|
||||
Reference in New Issue
Block a user