OBS: Detta är liveutgåvan anteckningar från gitHub.

Inledning

Bara utgåvorna från det här nuget-paketet.

Release Notes

v2.1.0 (2025-01-06) - Quality & Validation Release

Focus: Enhanced reliability, comprehensive testing, and improved developer experience

This release focuses on improving chunking reliability, providing comprehensive validation tooling, and streamlining configuration management. All features from v2.0 remain fully compatible.

Major Improvements

1. Comprehensive HTTP Validation Suite

70+ Test Cases: Complete validation coverage in LLMApi.http
- OpenAPI management (load, list, get, unload, describe)
- API context management (create, update, pause, resume, delete)
- gRPC proto management (upload, list, get, services, call)
- Continuous SSE streaming validation
- Chunking tests with large arrays
- Context history and custom descriptions
- Backend selection (X-LLM-Backend header)
- Schema validation (includeSchema + X-Response-Schema)
- Combined feature validation
- Error simulation (comprehensive HTTP error codes)

2. Enhanced Chunking Reliability

Explicit Array Formatting: Enhanced prompts with ultra-explicit instructions for JSON array generation
- Added critical formatting rules in PromptBuilder.cs: "Your FIRST character MUST be: ["
- Reinforced array formatting in ChunkingCoordinator.cs chunk context
- Prevents comma-separated object output (e.g., {...},{...} instead of [{...},{...}])
Improved Instruction Following: Better guidance for LLMs during multi-chunk requests
Known Limitations: Documented model/temperature recommendations for optimal chunking (see docs/OLLAMA_MODELS.md)

3. Configuration Streamlining

Clean appsettings.json: Removed verbose model comments, cleaner structure
docs/OLLAMA_MODELS.md: New comprehensive reference guide (285 lines)
- 10+ model configurations with hardware requirements
- Temperature guidelines for different use cases
- Context window sizing recommendations
- Multi-backend configuration examples
- Chunking troubleshooting guide
- Model installation instructions

4. Documentation Improvements

Full Swagger Documentation: All 25+ management endpoints now have complete Swagger docs
- Tags organization (OpenAPI Management, API Contexts, gRPC, etc.)
- Summaries and detailed descriptions
- Request/response examples
Backend API Reference: New docs/BACKEND_API_REFERENCE.md (600+ lines)
- Complete endpoint documentation
- Query parameters and headers reference
- Error response formats
- SignalR hub documentation

Bug Fixes

Fixed URL encoding issue in HTTP test for OpenAPI endpoint descriptions (:, /, {, } characters)

Files Modified

Code Changes:

mostlylucid.mockllmapi/Services/PromptBuilder.cs: Enhanced array formatting instructions (lines 82-94)
mostlylucid.mockllmapi/Services/ChunkingCoordinator.cs: Added array formatting to chunk context (line 447)
LLMApi/appsettings.json: Streamlined configuration

New Documentation:

docs/OLLAMA_MODELS.md: Comprehensive model configuration guide
docs/BACKEND_API_REFERENCE.md: Complete management API reference

Updated Files:

LLMApi/LLMApi.http: Expanded from 448 to 847 lines (70+ new validation tests)

Known Limitations

Chunking at High Temperature:

LLMs may generate comma-separated objects instead of arrays at temperature 1.2+
Workarounds:
1. Lower temperature to 0.8-1.0 for chunked requests
2. Use llama3.2:3b or mistral-nemo (better instruction following)
3. Reduce item count to avoid chunking
4. Disable auto-chunking with ?autoChunk=false

See docs/OLLAMA_MODELS.md for detailed troubleshooting.

Compatibility

Fully backward compatible with v2.0.0
All existing features continue to work
No breaking changes

v2.0.0 (2025-01-06) - MAJOR RELEASE

NO BREAKING CHANGES - Despite the major version bump, all existing code continues to work!

This is a major milestone release that transforms LLMock API into a comprehensive, production-ready mocking platform. Version 2.0 adds realistic SSE streaming modes, multi-backend load balancing, comprehensive backend selection, and extensive documentation.

Major Features

1. Realistic SSE Streaming Modes (MAJOR)

Three distinct SSE streaming modes for testing different real-world API patterns:

LlmTokens Mode (Default - Backward Compatible)

Token-by-token streaming for AI chat interfaces
Format: {"chunk":"text","accumulated":"fulltext","done":false}
Use case: Testing chatbot UIs, LLM applications

CompleteObjects Mode (NEW)

Complete JSON objects as separate SSE events
Format: {"data":{object},"index":0,"total":10,"done":false}
Use case: Twitter/X API, stock tickers, real-time feeds, IoT sensors

ArrayItems Mode (NEW)

Array items with rich metadata
Format: {"item":{object},"index":0,"total":100,"arrayName":"users","hasMore":true,"done":false}
Use case: Paginated results, bulk exports, search results

Configuration:

{
  "MockLlmApi": {
    "SseMode": "CompleteObjects"  // LlmTokens | CompleteObjects | ArrayItems
  }
}

Per-Request Override:

GET /api/mock/stream/users?sseMode=CompleteObjects
GET /api/mock/stream/data?sseMode=ArrayItems
GET /api/mock/stream/chat?sseMode=LlmTokens

Client Example (CompleteObjects):

const eventSource = new EventSource('/api/mock/stream/users?sseMode=CompleteObjects');

eventSource.onmessage = (event) => {
    const response = JSON.parse(event.data);

    if (response.done) {
        console.log('Complete!');
        eventSource.close();
    } else {
        console.log(`User ${response.index + 1}/${response.total}:`, response.data);
        // response.data contains the complete user object
    }
};

2. Multi-Backend Load Balancing (MAJOR)

Distribute requests across multiple LLM backends for high throughput:

Configuration:

{
  "MockLlmApi": {
    "Backends": [
      {
        "Name": "ollama-llama3",
        "Provider": "ollama",
        "Weight": 3,
        "Enabled": true
      },
      {
        "Name": "ollama-mistral",
        "Provider": "ollama",
        "Weight": 2,
        "Enabled": true
      },
      {
        "Name": "lmstudio-default",
        "Provider": "lmstudio",
        "Weight": 1,
        "Enabled": true
      }
    ]
  }
}

SignalR Hub with Load Balancing:

{
  "HubContexts": [
    {
      "Name": "high-throughput-data",
      "BackendNames": ["ollama-llama3", "ollama-mistral", "lmstudio-default"]
    }
  ]
}

Features:

Weighted round-robin distribution
Per-request backend selection
Automatic failover to default if backends unavailable
Works with SignalR, SSE, REST, and GraphQL

3. Comprehensive Backend Selection

Per-Request Selection (Multiple Methods):

# Via query parameter
GET /api/mock/users?backend=openai-gpt4

# Via header
GET /api/mock/users
X-LLM-Backend: openai-gpt4

# SignalR hub context
{
  "HubContexts": [
    {
      "Name": "analytics",
      "BackendName": "openai-gpt4-turbo"
    }
  ]
}

Multiple Providers Simultaneously:

Ollama (local models)
OpenAI (cloud API)
LM Studio (local server)
Mistral-Nemo (128k context)

4. Mistral-Nemo 128k Context Support

Configuration for Mistral-Nemo with massive 128k context window:

{
  "Backends": [
    {
      "Name": "ollama-mistral-nemo",
      "Provider": "ollama",
      "ModelName": "mistral-nemo",
      "MaxTokens": 128000,
      "Enabled": true
    }
  ]
}

Use Cases:

Generating thousands of detailed records in a single request
Complex nested data structures with deep relationships
Large batch operations (use with MaxItems=10000+)
Comprehensive test datasets
GraphQL queries with extensive nested fields

SignalR Example:

{
  "HubContexts": [
    {
      "Name": "massive-dataset-128k",
      "Description": "Massive dataset generation with 128k context",
      "BackendName": "ollama-mistral-nemo"
    }
  ]
}

5. Swagger/OpenAPI UI

Interactive API documentation with Swagger UI:

Swagger UI: Available at /swagger
OpenAPI Specification: Auto-generated from endpoints
Interactive Testing: Try endpoints directly in browser
Comprehensive Descriptions: Full documentation for all endpoints
Navigation: Linked from demo page header

Enable in Program.cs:

builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

app.UseSwagger();
app.UseSwaggerUI();

Documentation Overhaul

New Documentation Files

docs/SSE_STREAMING_MODES.md (2,500+ lines) - Complete SSE guide
- Detailed explanation of all three modes
- Client code examples for each mode
- Use case decision matrix
- Comparison charts
- Migration guide
- Troubleshooting section
LLMApi/SSE_Streaming.http - 30+ HTTP examples for SSE modes
- Examples for all three SSE modes
- Combined feature examples (backend selection, context, errors)
- Practical use cases (stock tickers, bulk exports, logs)
- Client JavaScript code examples

Updated Documentation

docs/CONFIGURATION_REFERENCE.md - Added SSE modes section
docs/MULTIPLE_LLM_BACKENDS.md - Enhanced with load balancing
appsettings.Full.json - Added SSE mode examples and Mistral-Nemo

🧪 Testing Improvements

New Test Coverage:

22 SSE mode tests (all passing)
- Enum value validation
- Configuration parsing
- Query parameter override
- Event format validation for all modes
- Case-insensitive parsing

Total Test Suite:

218 tests (213 passing, 5 gRPC tests skipped)
Zero compilation errors or warnings
Full backward compatibility verified

🔧 Configuration Enhancements

New Configuration Options:

{
  "MockLlmApi": {
    // SSE Streaming Modes (NEW)
    "SseMode": "LlmTokens",  // LlmTokens | CompleteObjects | ArrayItems

    // Multiple LLM Backends with Load Balancing (v1.8.0)
    "Backends": [
      {
        "Name": "backend-name",
        "Provider": "ollama",  // ollama | openai | lmstudio
        "BaseUrl": "http://localhost:11434/v1/",
        "ModelName": "llama3",
        "ApiKey": null,
        "MaxTokens": 8192,
        "Enabled": true,
        "Weight": 3,  // For load balancing
        "Priority": 10
      }
    ],

    // Legacy Single Backend (Still Supported)
    "BaseUrl": "http://localhost:11434/v1/",
    "ModelName": "llama3",

    // Auto-Chunking (v1.8.0)
    "EnableAutoChunking": true,
    "MaxInputTokens": 4096,
    "MaxOutputTokens": 2048,
    "MaxItems": 1000,

    // Streaming Configuration
    "StreamingChunkDelayMinMs": 0,
    "StreamingChunkDelayMaxMs": 0,

    // Cache Configuration
    "CacheSlidingExpirationMinutes": 15,
    "CacheAbsoluteExpirationMinutes": 60,
    "MaxCachePerKey": 5
  }
}

🌍 Environment Variables

Comprehensive environment variable support with full documentation:

# SSE Mode
export MockLlmApi__SseMode="CompleteObjects"

# Backend Selection
export MockLlmApi__Backends__0__Name="ollama-llama3"
export MockLlmApi__Backends__0__Provider="ollama"
export MockLlmApi__Backends__0__BaseUrl="http://localhost:11434/v1/"
export MockLlmApi__Backends__0__ModelName="llama3"
export MockLlmApi__Backends__0__Enabled="true"
export MockLlmApi__Backends__0__Weight="3"

# SignalR Hub Contexts with Backend Selection
export MockLlmApi__HubContexts__0__Name="analytics"
export MockLlmApi__HubContexts__0__BackendName="openai-gpt4-turbo"

# Or load balancing
export MockLlmApi__HubContexts__1__BackendNames__0="ollama-llama3"
export MockLlmApi__HubContexts__1__BackendNames__1="ollama-mistral"

📦 New Files

Core Implementation:

mostlylucid.mockllmapi/Models/SseMode.cs - SSE mode enum
mostlylucid.mockllmapi/Services/Providers/ILlmProvider.cs - Provider interface
mostlylucid.mockllmapi/Services/Providers/OllamaProvider.cs - Ollama provider
mostlylucid.mockllmapi/Services/Providers/OpenAIProvider.cs - OpenAI provider
mostlylucid.mockllmapi/Services/Providers/LMStudioProvider.cs - LM Studio provider
mostlylucid.mockllmapi/Services/Providers/LlmProviderFactory.cs - Provider factory
mostlylucid.mockllmapi/Services/LlmBackendSelector.cs - Backend selection logic

Testing:

LLMApi.Tests/SseModeTests.cs - 22 SSE mode tests

Documentation:

docs/SSE_STREAMING_MODES.md - Complete SSE guide (2,500+ lines)
LLMApi/SSE_Streaming.http - 30+ SSE examples

📝 Updated Files

Core Services:

mostlylucid.mockllmapi/LLMockApiOptions.cs - Added SseMode property, LlmBackends array
mostlylucid.mockllmapi/RequestHandlers/StreamingRequestHandler.cs - Added SSE mode routing
mostlylucid.mockllmapi/Services/LlmClient.cs - Added backend selection overloads
mostlylucid.mockllmapi/Services/MockDataBackgroundService.cs - Added SignalR backend selection
mostlylucid.mockllmapi/Models/HubContextConfig.cs - Added BackendName and BackendNames

Documentation:

README.md - Updated to v2.0, comprehensive feature list
docs/CONFIGURATION_REFERENCE.md - Added SSE modes and backend selection
docs/MULTIPLE_LLM_BACKENDS.md - Enhanced with load balancing examples
appsettings.Full.json - Added comprehensive examples

Demo Application:

LLMApi/Program.cs - Added Swagger configuration
LLMApi/Pages/_Layout.cshtml - Added Swagger UI link

🎨 Use Case Examples

Stock Market Feed (CompleteObjects):

GET /api/mock/stream/stocks?sseMode=CompleteObjects&shape={"ticker":"AAPL","price":150.25,"change":2.5}

Bulk Customer Export (ArrayItems):

GET /api/mock/stream/export-customers?sseMode=ArrayItems&shape={"customers":[{"id":"string","name":"string"}]}

AI Chat Interface (LlmTokens):

GET /api/mock/stream/chat?sseMode=LlmTokens&shape={"message":"Hello!"}

High-Throughput IoT Sensors (Load Balanced):

{
  "HubContexts": [
    {
      "Name": "iot-sensors",
      "BackendNames": ["ollama-llama3", "ollama-mistral", "lmstudio-default"]
    }
  ]
}

Massive Dataset with 128k Context:

GET /api/mock/stream/bulk-data?sseMode=ArrayItems&backend=ollama-mistral-nemo

Migration from v1.x

No Code Changes Required!

Version 2.0 is 100% backward compatible with v1.x:

// v1.x code - still works exactly the same
builder.Services.AddLLMockApi(builder.Configuration);
app.MapLLMockApi("/api/mock", includeStreaming: true);

// SSE streaming defaults to LlmTokens mode (original behavior)
// Legacy single backend config (BaseUrl/ModelName) still works

Opt-In to New Features:

// Same setup, just update appsettings.json
{
  "MockLlmApi": {
    "SseMode": "CompleteObjects",  // Switch to realistic streaming
    "Backends": [...]  // Add multiple backends
  }
}

Breaking Changes

NONE!

Despite the major version bump to 2.0, there are zero breaking changes:

Default SSE mode is LlmTokens (original behavior)
Legacy BaseUrl/ModelName config still supported
All existing endpoints work unchanged
All existing tests pass
Full backward compatibility maintained

📈 Performance Improvements

Multi-Backend Load Balancing: Distribute load across multiple LLM instances
Weighted Round-Robin: Smart distribution based on backend capabilities
Backend-Specific Token Limits: Optimize for each model's capabilities
CompleteObjects Mode: Fewer SSE events = lower overhead than token-by-token

Why Version 2.0?

This release represents a fundamental transformation:

v1.x: Mock API with LLM-powered generation

Single LLM backend
One SSE streaming mode
Basic configuration

v2.0: Production-Ready Mock Platform

Multiple LLM backends with load balancing
Three realistic SSE streaming modes
Comprehensive backend selection
Production-scale configuration (128k contexts)
Enterprise-ready documentation
Interactive API documentation (Swagger)

Version 2.0 positions LLMock API as a comprehensive mocking platform capable of handling production-scale testing requirements across diverse use cases.

Complete Documentation

SSE Streaming Modes - Complete guide to SSE modes
Multiple LLM Backends - Backend configuration guide
Configuration Reference - All configuration options
HTTP Examples - Ready-to-run SSE examples
Chunking and Caching - Auto-chunking guide
API Contexts - Context management guide
gRPC Support - gRPC mocking guide
README - Quick start and overview

🙏 Thank You

Thank you to all users and contributors who have helped shape LLMock API into a comprehensive mocking platform. Your feedback and use cases have driven these improvements!

v1.8.0 (2025-01-06)

See previous release notes for v1.8.0 features (Multiple LLM Backend Support, Automatic Request Chunking, Enhanced Cache Configuration).

Previous Releases

See full release history below for v1.7.x, v1.6.x, v1.5.x, and earlier versions.

LLMAPI: Anmärkningar till utgåvan

Inledning

Release Notes

v2.1.0 (2025-01-06) - Quality & Validation Release

Major Improvements

1. Comprehensive HTTP Validation Suite

2. Enhanced Chunking Reliability

3. Configuration Streamlining

4. Documentation Improvements

Bug Fixes

Files Modified

Known Limitations

Compatibility

v2.0.0 (2025-01-06) - MAJOR RELEASE

Major Features

1. Realistic SSE Streaming Modes (MAJOR)

2. Multi-Backend Load Balancing (MAJOR)

3. Comprehensive Backend Selection

4. Mistral-Nemo 128k Context Support

5. Swagger/OpenAPI UI

Documentation Overhaul

New Documentation Files

Updated Documentation

🧪 Testing Improvements

🔧 Configuration Enhancements

🌍 Environment Variables

📦 New Files

📝 Updated Files

🎨 Use Case Examples

Migration from v1.x

Breaking Changes

📈 Performance Improvements

Why Version 2.0?

Complete Documentation

🙏 Thank You

v1.8.0 (2025-01-06)

Previous Releases