Implement LLM provider configuration and update user settings
- Added functionality to update the default LLM provider for users via a new endpoint in UserController. - Introduced LlmProvider enum to manage available LLM options: Auto, Gemini, OpenAI, and Claude. - Updated User and UserEntity models to include DefaultLlmProvider property. - Enhanced database context and migrations to support the new LLM provider configuration. - Integrated LLM services into the application bootstrap for dependency injection. - Updated TypeScript API client to include methods for managing LLM providers and chat requests.
This commit is contained in:
282
assets/documentation/MCP-Configuration-Models.md
Normal file
282
assets/documentation/MCP-Configuration-Models.md
Normal file
@@ -0,0 +1,282 @@
|
||||
# MCP LLM Model Configuration
|
||||
|
||||
## Overview
|
||||
|
||||
All LLM provider models are now configured exclusively through `appsettings.json` - **no hardcoded values in the code**. This allows you to easily change models without recompiling the application.
|
||||
|
||||
## Configuration Location
|
||||
|
||||
All model settings are in: `src/Managing.Api/appsettings.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"Llm": {
|
||||
"Gemini": {
|
||||
"ApiKey": "", // Add your key here or via user secrets
|
||||
"DefaultModel": "gemini-3-flash-preview"
|
||||
},
|
||||
"OpenAI": {
|
||||
"ApiKey": "",
|
||||
"DefaultModel": "gpt-4o"
|
||||
},
|
||||
"Claude": {
|
||||
"ApiKey": "",
|
||||
"DefaultModel": "claude-haiku-4-5-20251001"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Current Models (from appsettings.json)
|
||||
|
||||
- **Gemini**: `gemini-3-flash-preview`
|
||||
- **OpenAI**: `gpt-4o`
|
||||
- **Claude**: `claude-haiku-4-5-20251001`
|
||||
|
||||
## Fallback Models (in code)
|
||||
|
||||
If `DefaultModel` is not specified in configuration, the providers use these fallback models:
|
||||
|
||||
- **Gemini**: `gemini-2.0-flash-exp`
|
||||
- **OpenAI**: `gpt-4o`
|
||||
- **Claude**: `claude-3-5-sonnet-20241022`
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. Configuration Reading
|
||||
|
||||
When the application starts, `LlmService` reads the model configuration:
|
||||
|
||||
```csharp
|
||||
var geminiModel = _configuration["Llm:Gemini:DefaultModel"];
|
||||
var openaiModel = _configuration["Llm:OpenAI:DefaultModel"];
|
||||
var claudeModel = _configuration["Llm:Claude:DefaultModel"];
|
||||
```
|
||||
|
||||
### 2. Provider Initialization
|
||||
|
||||
Each provider is initialized with the configured model:
|
||||
|
||||
```csharp
|
||||
_providers["gemini"] = new GeminiProvider(geminiApiKey, geminiModel, httpClientFactory, _logger);
|
||||
_providers["openai"] = new OpenAiProvider(openaiApiKey, openaiModel, httpClientFactory, _logger);
|
||||
_providers["claude"] = new ClaudeProvider(claudeApiKey, claudeModel, httpClientFactory, _logger);
|
||||
```
|
||||
|
||||
### 3. Model Usage
|
||||
|
||||
The provider uses the configured model for all API calls:
|
||||
|
||||
```csharp
|
||||
public async Task<LlmChatResponse> ChatAsync(LlmChatRequest request)
|
||||
{
|
||||
var model = _defaultModel; // From configuration
|
||||
var url = $"{BaseUrl}/models/{model}:generateContent?key={_apiKey}";
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
## Changing Models
|
||||
|
||||
### Method 1: Edit appsettings.json
|
||||
|
||||
```json
|
||||
{
|
||||
"Llm": {
|
||||
"Claude": {
|
||||
"DefaultModel": "claude-3-5-sonnet-20241022" // Change to Sonnet
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Method 2: Environment Variables
|
||||
|
||||
```bash
|
||||
export Llm__Claude__DefaultModel="claude-3-5-sonnet-20241022"
|
||||
```
|
||||
|
||||
### Method 3: User Secrets (Development)
|
||||
|
||||
```bash
|
||||
cd src/Managing.Api
|
||||
dotnet user-secrets set "Llm:Claude:DefaultModel" "claude-3-5-sonnet-20241022"
|
||||
```
|
||||
|
||||
## Available Models
|
||||
|
||||
### Gemini Models
|
||||
|
||||
- `gemini-2.0-flash-exp` - Latest Flash (experimental)
|
||||
- `gemini-3-flash-preview` - Flash preview
|
||||
- `gemini-1.5-pro` - Pro model
|
||||
- `gemini-1.5-flash` - Fast and efficient
|
||||
|
||||
### OpenAI Models
|
||||
|
||||
- `gpt-4o` - GPT-4 Optimized (recommended)
|
||||
- `gpt-4o-mini` - Smaller, faster
|
||||
- `gpt-4-turbo` - GPT-4 Turbo
|
||||
- `gpt-3.5-turbo` - Cheaper, faster
|
||||
|
||||
### Claude Models
|
||||
|
||||
- `claude-haiku-4-5-20251001` - Haiku 4.5 (fastest, cheapest)
|
||||
- `claude-3-5-sonnet-20241022` - Sonnet 3.5 (balanced, recommended)
|
||||
- `claude-3-opus-20240229` - Opus (most capable)
|
||||
- `claude-3-sonnet-20240229` - Sonnet 3
|
||||
- `claude-3-haiku-20240307` - Haiku 3
|
||||
|
||||
## Model Selection Guide
|
||||
|
||||
### For Development/Testing
|
||||
- **Gemini**: `gemini-2.0-flash-exp` (free tier)
|
||||
- **Claude**: `claude-haiku-4-5-20251001` (cheapest)
|
||||
- **OpenAI**: `gpt-4o-mini` (cheapest)
|
||||
|
||||
### For Production (Balanced)
|
||||
- **Claude**: `claude-3-5-sonnet-20241022` ✅ Recommended
|
||||
- **OpenAI**: `gpt-4o`
|
||||
- **Gemini**: `gemini-1.5-pro`
|
||||
|
||||
### For Maximum Capability
|
||||
- **Claude**: `claude-3-opus-20240229` (best reasoning)
|
||||
- **OpenAI**: `gpt-4-turbo`
|
||||
- **Gemini**: `gemini-1.5-pro`
|
||||
|
||||
### For Speed/Cost Efficiency
|
||||
- **Claude**: `claude-haiku-4-5-20251001`
|
||||
- **OpenAI**: `gpt-4o-mini`
|
||||
- **Gemini**: `gemini-2.0-flash-exp`
|
||||
|
||||
## Cost Comparison (Approximate)
|
||||
|
||||
### Claude
|
||||
- **Haiku 4.5**: ~$0.50 per 1M tokens (cheapest)
|
||||
- **Sonnet 3.5**: ~$9 per 1M tokens (recommended)
|
||||
- **Opus**: ~$45 per 1M tokens (most expensive)
|
||||
|
||||
### OpenAI
|
||||
- **GPT-4o-mini**: ~$0.30 per 1M tokens
|
||||
- **GPT-4o**: ~$10 per 1M tokens
|
||||
- **GPT-4-turbo**: ~$30 per 1M tokens
|
||||
|
||||
### Gemini
|
||||
- **Free tier**: 15 requests/minute (development)
|
||||
- **Paid**: ~$0.50 per 1M tokens
|
||||
|
||||
## Logging
|
||||
|
||||
When providers are initialized, you'll see log messages indicating which model is being used:
|
||||
|
||||
```
|
||||
[Information] Gemini provider initialized with model: gemini-3-flash-preview
|
||||
[Information] OpenAI provider initialized with model: gpt-4o
|
||||
[Information] Claude provider initialized with model: claude-haiku-4-5-20251001
|
||||
```
|
||||
|
||||
If no model is configured, it will show:
|
||||
|
||||
```
|
||||
[Information] Gemini provider initialized with model: default
|
||||
```
|
||||
|
||||
And the fallback model will be used.
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use environment variables** for production to keep configuration flexible
|
||||
2. **Test with cheaper models** during development
|
||||
3. **Monitor costs** in provider dashboards
|
||||
4. **Update models** as new versions are released
|
||||
5. **Document changes** when switching models for your team
|
||||
|
||||
## Example Configurations
|
||||
|
||||
### Development (Cost-Optimized)
|
||||
```json
|
||||
{
|
||||
"Llm": {
|
||||
"Claude": {
|
||||
"ApiKey": "your-key",
|
||||
"DefaultModel": "claude-haiku-4-5-20251001"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Production (Balanced)
|
||||
```json
|
||||
{
|
||||
"Llm": {
|
||||
"Claude": {
|
||||
"ApiKey": "your-key",
|
||||
"DefaultModel": "claude-3-5-sonnet-20241022"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### High-Performance (Maximum Capability)
|
||||
```json
|
||||
{
|
||||
"Llm": {
|
||||
"Claude": {
|
||||
"ApiKey": "your-key",
|
||||
"DefaultModel": "claude-3-opus-20240229"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
To verify which model is being used:
|
||||
|
||||
1. Check application logs on startup
|
||||
2. Look for provider initialization messages
|
||||
3. Check LLM response metadata (includes model name)
|
||||
4. Monitor provider dashboards for API usage
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Model not found error
|
||||
|
||||
**Issue**: "Model not found" or "Invalid model name"
|
||||
|
||||
**Solution**:
|
||||
1. Verify model name spelling in `appsettings.json`
|
||||
2. Check provider documentation for available models
|
||||
3. Ensure model is available in your region/tier
|
||||
4. Try removing `DefaultModel` to use the fallback
|
||||
|
||||
### Wrong model being used
|
||||
|
||||
**Issue**: Application uses fallback instead of configured model
|
||||
|
||||
**Solution**:
|
||||
1. Check configuration path: `Llm:ProviderName:DefaultModel`
|
||||
2. Verify no typos in JSON (case-sensitive)
|
||||
3. Restart application after configuration changes
|
||||
4. Check logs for which model was loaded
|
||||
|
||||
### Configuration not loading
|
||||
|
||||
**Issue**: Changes to `appsettings.json` not taking effect
|
||||
|
||||
**Solution**:
|
||||
1. Restart the application
|
||||
2. Clear build artifacts: `dotnet clean`
|
||||
3. Check file is in correct location: `src/Managing.Api/appsettings.json`
|
||||
4. Verify JSON syntax is valid
|
||||
|
||||
## Summary
|
||||
|
||||
✅ All models configured in `appsettings.json`
|
||||
✅ No hardcoded model names in code
|
||||
✅ Easy to change without recompiling
|
||||
✅ Fallback models in case of missing configuration
|
||||
✅ Full flexibility for different environments
|
||||
✅ Logged on startup for verification
|
||||
|
||||
This design allows maximum flexibility while maintaining sensible defaults!
|
||||
Reference in New Issue
Block a user