.
INTRODUCTION
The landscape of software development has been transformed by Large Language Models. For .NET developers working on Windows systems, integrating LLM capabilities into applications opens up unprecedented possibilities for creating intelligent, context-aware systems. This article provides a comprehensive guide to building LLM-powered applications using the .NET ecosystem, with a focus on practical implementation strategies and real-world examples.
Throughout this guide, we will construct a research chatbot that leverages both Retrieval Augmented Generation and Graph-based Retrieval Augmented Generation techniques. This running example will demonstrate how to combine traditional .NET development practices with cutting-edge AI capabilities. We will explore both local and remote LLM deployment scenarios, understanding when each approach is appropriate and how to implement them effectively.
The .NET ecosystem provides several pathways for LLM integration. Microsoft's Semantic Kernel framework offers a comprehensive orchestration layer, while direct API integration provides fine-grained control. Additionally, libraries like LangChain for .NET and ML.NET extensions enable various integration patterns. Understanding these options empowers you to choose the right architecture for your specific requirements.
SETTING UP YOUR DEVELOPMENT ENVIRONMENT
Before diving into LLM application development, you need to establish a properly configured development environment on your Windows system. The foundation begins with ensuring you have the latest .NET SDK installed. At the time of writing, .NET 8 represents the current long-term support version and provides the best foundation for LLM application development.
Download and install the .NET 8 SDK from the official Microsoft website. After installation, verify your setup by opening a command prompt and executing the following command:
dotnet --version
This should display version 8.0 or higher. Next, you will need a capable Integrated Development Environment. Visual Studio 2022 Community Edition or higher provides excellent support for .NET development with built-in debugging, IntelliSense, and project management capabilities. Alternatively, Visual Studio Code with the C# extension offers a lightweight but powerful development experience.
For working with local LLMs, you will need additional tools. Ollama represents one of the most accessible options for running LLMs locally on Windows. Download Ollama from the official website and install it. Ollama provides a simple command-line interface for downloading and running various open-source models. After installation, you can pull a model like Llama 2 by executing:
ollama pull llama2
Another excellent option for local LLM deployment is LM Studio, which provides a graphical user interface for managing and running models. LM Studio supports a wide range of models in GGUF format and offers an OpenAI-compatible API endpoint, making integration straightforward.
For remote LLM access, you will need API keys from your chosen provider. Azure OpenAI Service requires an Azure subscription and resource provisioning through the Azure Portal. OpenAI's API requires registration at platform.openai.com and API key generation. Store these keys securely, preferably using environment variables or Azure Key Vault rather than hardcoding them in your source code.
UNDERSTANDING LLM INTEGRATION APPROACHES IN .NET
The .NET ecosystem offers multiple pathways for integrating LLM capabilities into your applications. Each approach has distinct characteristics, advantages, and appropriate use cases. Understanding these options enables you to make informed architectural decisions.
The first approach involves direct HTTP API integration. This method uses HttpClient to make REST API calls to LLM endpoints, whether local or remote. This approach provides maximum control and minimal dependencies but requires you to handle serialization, error handling, and retry logic manually. Direct API integration works well when you need fine-grained control over every aspect of the LLM interaction or when working with proprietary or custom LLM endpoints.
The second approach leverages Microsoft's Semantic Kernel framework. Semantic Kernel provides an orchestration layer that abstracts away many low-level details while offering powerful features like prompt templating, function calling, planning, and memory management. Semantic Kernel supports multiple LLM providers through a plugin architecture, making it easy to switch between different models or providers. This framework excels in scenarios requiring complex multi-step reasoning, function calling, or integration with multiple AI services.
A third option involves using LangChain for .NET, a port of the popular Python framework. LangChain provides abstractions for chains, agents, and tools, enabling you to build complex LLM-powered workflows. While the .NET version is less mature than its Python counterpart, it offers familiar patterns for developers coming from the Python ecosystem.
For our research chatbot example, we will primarily use Semantic Kernel due to its robust .NET integration, excellent documentation, and powerful features for RAG implementation. However, we will also demonstrate direct API integration to illustrate the underlying mechanics.
WORKING WITH REMOTE LLMS
Remote LLMs, hosted by providers like OpenAI or Azure OpenAI Service, offer powerful capabilities without requiring local computational resources. Let us begin by creating a simple console application that interacts with Azure OpenAI Service.
First, create a new console application:
dotnet new console -n ResearchChatbot
cd ResearchChatbot
Add the necessary NuGet packages:
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.Extensions.Configuration
dotnet add package Microsoft.Extensions.Configuration.EnvironmentVariables
dotnet add package Microsoft.Extensions.Configuration.UserSecrets
Now, let us create a basic program that connects to Azure OpenAI Service. Open Program.cs and replace its contents with the following code:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.Extensions.Configuration;
namespace ResearchChatbot
{
class Program
{
static async Task Main(string[] args)
{
// Build configuration to read from environment variables and user secrets
var configuration = new ConfigurationBuilder()
.AddEnvironmentVariables()
.AddUserSecrets<Program>()
.Build();
// Retrieve Azure OpenAI configuration from secure storage
string endpoint = configuration["AzureOpenAI:Endpoint"]
?? throw new InvalidOperationException("Azure OpenAI endpoint not configured");
string apiKey = configuration["AzureOpenAI:ApiKey"]
?? throw new InvalidOperationException("Azure OpenAI API key not configured");
string deploymentName = configuration["AzureOpenAI:DeploymentName"]
?? throw new InvalidOperationException("Azure OpenAI deployment name not configured");
// Initialize the Semantic Kernel with Azure OpenAI
var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.AddAzureOpenAIChatCompletion(
deploymentName: deploymentName,
endpoint: endpoint,
apiKey: apiKey
);
var kernel = kernelBuilder.Build();
// Create a simple chat loop
Console.WriteLine("Research Chatbot initialized. Type 'exit' to quit.");
Console.WriteLine("Ask me anything about your research topics!");
Console.WriteLine();
while (true)
{
Console.Write("You: ");
string userInput = Console.ReadLine() ?? string.Empty;
if (userInput.ToLower() == "exit")
{
break;
}
try
{
// Get the chat completion service from the kernel
var chatService = kernel.GetRequiredService<IChatCompletionService>();
// Create a chat history to maintain conversation context
var chatHistory = new ChatHistory();
chatHistory.AddSystemMessage(
"You are a helpful research assistant. Provide accurate, " +
"well-researched answers to academic questions."
);
chatHistory.AddUserMessage(userInput);
// Get the response from the LLM
var response = await chatService.GetChatMessageContentAsync(
chatHistory,
kernel: kernel
);
Console.WriteLine($"Assistant: {response.Content}");
Console.WriteLine();
}
catch (Exception ex)
{
Console.WriteLine($"Error: {ex.Message}");
Console.WriteLine();
}
}
}
}
}
This code establishes the foundation for our research chatbot. The configuration system reads sensitive information like API keys from user secrets rather than hardcoding them. This follows security best practices by keeping credentials out of source control.
To configure user secrets, execute the following commands in your project directory:
dotnet user-secrets init
dotnet user-secrets set "AzureOpenAI:Endpoint" "https://your-resource.openai.azure.com/"
dotnet user-secrets set "AzureOpenAI:ApiKey" "your-api-key-here"
dotnet user-secrets set "AzureOpenAI:DeploymentName" "gpt-4"
The Semantic Kernel initialization creates a kernel object that serves as the central orchestrator for all AI operations. The AddAzureOpenAIChatCompletion method registers the Azure OpenAI service with the kernel, enabling it to route chat completion requests to your deployed model.
The chat loop demonstrates a basic interaction pattern. The ChatHistory object maintains conversation context, allowing the model to reference previous messages. The system message sets the behavior and personality of the assistant, while user messages contain the actual queries. This separation enables fine-grained control over the assistant's responses.
IMPLEMENTING DIRECT API INTEGRATION
While Semantic Kernel provides excellent abstractions, understanding direct API integration helps you appreciate what happens under the hood and enables custom implementations when needed. Let us create a simple wrapper class that directly calls the Azure OpenAI API using HttpClient.
Create a new file called DirectAzureOpenAIClient.cs:
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace ResearchChatbot
{
/// <summary>
/// Direct HTTP client for Azure OpenAI API without using Semantic Kernel
/// </summary>
public class DirectAzureOpenAIClient
{
private readonly HttpClient _httpClient;
private readonly string _endpoint;
private readonly string _deploymentName;
private readonly string _apiVersion;
public DirectAzureOpenAIClient(string endpoint, string apiKey, string deploymentName)
{
_httpClient = new HttpClient();
_httpClient.DefaultRequestHeaders.Add("api-key", apiKey);
_endpoint = endpoint.TrimEnd('/');
_deploymentName = deploymentName;
_apiVersion = "2024-02-15-preview";
}
/// <summary>
/// Sends a chat completion request to Azure OpenAI
/// </summary>
public async Task<string> GetChatCompletionAsync(string userMessage, string systemMessage = "")
{
// Construct the request URL
string url = $"{_endpoint}/openai/deployments/{_deploymentName}/chat/completions?api-version={_apiVersion}";
// Build the request payload
var messages = new List<ChatMessage>();
if (!string.IsNullOrEmpty(systemMessage))
{
messages.Add(new ChatMessage { Role = "system", Content = systemMessage });
}
messages.Add(new ChatMessage { Role = "user", Content = userMessage });
var requestBody = new ChatCompletionRequest
{
Messages = messages,
Temperature = 0.7,
MaxTokens = 800,
TopP = 0.95,
FrequencyPenalty = 0,
PresencePenalty = 0
};
// Serialize the request to JSON
string jsonRequest = JsonSerializer.Serialize(requestBody, new JsonSerializerOptions
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
});
var content = new StringContent(jsonRequest, Encoding.UTF8, "application/json");
// Send the HTTP POST request
HttpResponseMessage response = await _httpClient.PostAsync(url, content);
if (!response.IsSuccessStatusCode)
{
string errorContent = await response.Content.ReadAsStringAsync();
throw new HttpRequestException(
$"Azure OpenAI API request failed with status {response.StatusCode}: {errorContent}"
);
}
// Parse the response
string jsonResponse = await response.Content.ReadAsStringAsync();
var completionResponse = JsonSerializer.Deserialize<ChatCompletionResponse>(
jsonResponse,
new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }
);
return completionResponse?.Choices?[0]?.Message?.Content
?? throw new InvalidOperationException("No content in response");
}
}
// Request and response model classes
public class ChatCompletionRequest
{
public List<ChatMessage> Messages { get; set; } = new();
public double Temperature { get; set; }
public int MaxTokens { get; set; }
public double TopP { get; set; }
public double FrequencyPenalty { get; set; }
public double PresencePenalty { get; set; }
}
public class ChatMessage
{
public string Role { get; set; } = string.Empty;
public string Content { get; set; } = string.Empty;
}
public class ChatCompletionResponse
{
public List<Choice> Choices { get; set; } = new();
}
public class Choice
{
public ChatMessage Message { get; set; } = new();
}
}
This implementation demonstrates the underlying mechanics of LLM API communication. The GetChatCompletionAsync method constructs an HTTP POST request with a JSON payload containing the conversation messages and generation parameters. The Temperature parameter controls randomness in the output, with lower values producing more deterministic responses. MaxTokens limits the response length, preventing excessively long outputs. TopP implements nucleus sampling, another technique for controlling output diversity.
The request and response classes use System.Text.Json for serialization, which provides excellent performance and is the recommended JSON library for modern .NET applications. The PropertyNamingPolicy converts C# PascalCase property names to the camelCase format expected by the API.
To use this direct client in your application, you would instantiate it with your credentials and call the method:
var directClient = new DirectAzureOpenAIClient(endpoint, apiKey, deploymentName);
string response = await directClient.GetChatCompletionAsync(
"What are the key principles of quantum computing?",
"You are a helpful research assistant."
);
Console.WriteLine(response);
This direct approach provides maximum flexibility but requires more boilerplate code compared to Semantic Kernel. It becomes valuable when you need to implement custom retry logic, handle streaming responses differently, or integrate with LLM providers not yet supported by higher-level frameworks.
WORKING WITH LOCAL LLMS
Local LLMs offer several advantages including data privacy, no API costs, and offline operation. However, they require sufficient computational resources and typically provide lower quality outputs compared to large commercial models. Let us explore how to integrate local LLMs using Ollama.
Ollama exposes an OpenAI-compatible API, making integration straightforward. First, ensure Ollama is running and you have pulled a model:
ollama pull llama2
ollama serve
By default, Ollama runs on localhost port 11434. We can modify our Semantic Kernel setup to use Ollama instead of Azure OpenAI. Create a new file called LocalLLMExample.cs:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.OpenAI;
namespace ResearchChatbot
{
/// <summary>
/// Example demonstrating integration with local LLMs via Ollama
/// </summary>
public class LocalLLMExample
{
public static async Task RunLocalChatbot()
{
// Configure Semantic Kernel to use Ollama's OpenAI-compatible endpoint
var kernelBuilder = Kernel.CreateBuilder();
// Ollama uses OpenAI-compatible API, so we use the OpenAI connector
// but point it to the local Ollama endpoint
kernelBuilder.AddOpenAIChatCompletion(
modelId: "llama2",
apiKey: "not-needed", // Ollama doesn't require an API key
endpoint: new Uri("http://localhost:11434/v1")
);
var kernel = kernelBuilder.Build();
var chatService = kernel.GetRequiredService<IChatCompletionService>();
Console.WriteLine("Local LLM Chatbot (Ollama) initialized.");
Console.WriteLine("Type 'exit' to quit.");
Console.WriteLine();
var chatHistory = new ChatHistory();
chatHistory.AddSystemMessage(
"You are a helpful research assistant specializing in academic topics. " +
"Provide clear, accurate, and well-structured answers."
);
while (true)
{
Console.Write("You: ");
string userInput = Console.ReadLine() ?? string.Empty;
if (userInput.ToLower() == "exit")
{
break;
}
chatHistory.AddUserMessage(userInput);
try
{
// Request completion from local LLM
var response = await chatService.GetChatMessageContentAsync(
chatHistory,
kernel: kernel
);
chatHistory.AddAssistantMessage(response.Content ?? string.Empty);
Console.WriteLine($"Assistant: {response.Content}");
Console.WriteLine();
}
catch (Exception ex)
{
Console.WriteLine($"Error: {ex.Message}");
Console.WriteLine();
}
}
}
}
}
The key difference when working with local LLMs is the endpoint configuration. We point the OpenAI connector to the local Ollama server rather than a cloud endpoint. The API key parameter is required by the connector but ignored by Ollama, so we pass a placeholder value.
Local LLMs typically have slower inference times and may produce lower quality outputs compared to large commercial models. However, they excel in scenarios requiring data privacy, offline operation, or cost control. For the research chatbot, you might use a local model for initial query processing or classification, then route complex queries to a more powerful remote model.
Another popular option for local LLM deployment is LM Studio. LM Studio provides a graphical interface for downloading and running models in GGUF format. It also exposes an OpenAI-compatible API on port 1234 by default. To use LM Studio with Semantic Kernel, simply change the endpoint:
kernelBuilder.AddOpenAIChatCompletion(
modelId: "local-model",
apiKey: "not-needed",
endpoint: new Uri("http://localhost:1234/v1")
);
LM Studio offers advantages including a user-friendly interface for model management, support for quantized models that run efficiently on consumer hardware, and the ability to load multiple models simultaneously. The quantization techniques used in GGUF format models significantly reduce memory requirements while maintaining reasonable output quality.
INTRODUCING RETRIEVAL AUGMENTED GENERATION
Retrieval Augmented Generation represents a powerful technique for grounding LLM responses in specific knowledge bases. Rather than relying solely on the model's training data, RAG retrieves relevant documents or passages and includes them in the prompt context. This approach dramatically improves accuracy for domain-specific questions and reduces hallucinations.
The RAG pipeline consists of several stages. First, documents are chunked into smaller segments that fit within the LLM's context window. Second, these chunks are converted into vector embeddings using an embedding model. Third, these embeddings are stored in a vector database. When a user asks a question, the question is also embedded, and similar document chunks are retrieved using vector similarity search. Finally, the retrieved chunks are included in the prompt sent to the LLM.
Let us implement a basic RAG system for our research chatbot. We will use Semantic Kernel's memory capabilities combined with a vector store. First, add the required packages:
dotnet add package Microsoft.SemanticKernel.Connectors.Memory.Qdrant
dotnet add package Microsoft.SemanticKernel.Plugins.Memory
For this example, we will use Qdrant as our vector database. Qdrant can run locally via Docker:
docker run -p 6333:6333 qdrant/qdrant
Now, let us create a document ingestion system. Create a file called DocumentIngestionService.cs:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Embeddings;
using Microsoft.SemanticKernel.Memory;
using System.Text;
namespace ResearchChatbot
{
/// <summary>
/// Service responsible for ingesting documents into the vector store
/// </summary>
public class DocumentIngestionService
{
private readonly ISemanticTextMemory _memory;
private readonly ITextEmbeddingGenerationService _embeddingService;
private const int ChunkSize = 1000;
private const int ChunkOverlap = 200;
public DocumentIngestionService(
ISemanticTextMemory memory,
ITextEmbeddingGenerationService embeddingService)
{
_memory = memory;
_embeddingService = embeddingService;
}
/// <summary>
/// Ingests a document by chunking it and storing embeddings in the vector store
/// </summary>
public async Task IngestDocumentAsync(string documentContent, string documentId, string collectionName)
{
// Split the document into chunks with overlap
var chunks = ChunkDocument(documentContent);
Console.WriteLine($"Processing document {documentId} into {chunks.Count} chunks...");
// Process each chunk
for (int i = 0; i < chunks.Count; i++)
{
string chunkId = $"{documentId}_chunk_{i}";
string chunk = chunks[i];
// Store the chunk in semantic memory
// The memory system will automatically generate embeddings
await _memory.SaveInformationAsync(
collection: collectionName,
id: chunkId,
text: chunk,
description: $"Chunk {i} of document {documentId}"
);
Console.WriteLine($"Stored chunk {i + 1}/{chunks.Count}");
}
Console.WriteLine($"Document {documentId} ingestion complete.");
}
/// <summary>
/// Chunks a document into overlapping segments
/// </summary>
private List<string> ChunkDocument(string content)
{
var chunks = new List<string>();
// Simple character-based chunking with overlap
int position = 0;
while (position < content.Length)
{
int chunkLength = Math.Min(ChunkSize, content.Length - position);
string chunk = content.Substring(position, chunkLength);
chunks.Add(chunk);
// Move position forward, accounting for overlap
position += ChunkSize - ChunkOverlap;
// Ensure we don't go past the end
if (position >= content.Length)
{
break;
}
}
return chunks;
}
/// <summary>
/// Retrieves relevant chunks for a given query
/// </summary>
public async Task<List<string>> RetrieveRelevantChunksAsync(
string query,
string collectionName,
int topK = 3)
{
// Search semantic memory for relevant chunks
var searchResults = _memory.SearchAsync(
collection: collectionName,
query: query,
limit: topK,
minRelevanceScore: 0.7
);
var relevantChunks = new List<string>();
await foreach (var result in searchResults)
{
relevantChunks.Add(result.Metadata.Text);
}
return relevantChunks;
}
}
}
This implementation demonstrates the core RAG components. The ChunkDocument method splits documents into overlapping segments. Overlap is crucial because it ensures that concepts spanning chunk boundaries are not lost. The chunk size of 1000 characters represents a balance between context preservation and embedding quality.
The IngestDocumentAsync method processes each chunk and stores it in semantic memory. Semantic Kernel's memory abstraction automatically generates embeddings using the configured embedding service and stores them in the vector database. This abstraction simplifies the implementation by handling the embedding generation and storage details.
The RetrieveRelevantChunksAsync method performs similarity search. It converts the query into an embedding and finds the most similar document chunks. The minRelevanceScore parameter filters out chunks with low similarity, ensuring only truly relevant content is retrieved.
Now let us create the main RAG-enabled chatbot. Create a file called RAGChatbot.cs:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Embeddings;
using Microsoft.SemanticKernel.Memory;
using Microsoft.SemanticKernel.Connectors.Qdrant;
using Microsoft.Extensions.Configuration;
namespace ResearchChatbot
{
/// <summary>
/// Research chatbot with RAG capabilities
/// </summary>
public class RAGChatbot
{
private readonly Kernel _kernel;
private readonly ISemanticTextMemory _memory;
private readonly DocumentIngestionService _ingestionService;
private readonly IChatCompletionService _chatService;
private const string CollectionName = "research_documents";
public RAGChatbot(IConfiguration configuration)
{
// Retrieve configuration
string azureEndpoint = configuration["AzureOpenAI:Endpoint"]
?? throw new InvalidOperationException("Endpoint not configured");
string azureApiKey = configuration["AzureOpenAI:ApiKey"]
?? throw new InvalidOperationException("API key not configured");
string chatDeployment = configuration["AzureOpenAI:DeploymentName"]
?? throw new InvalidOperationException("Deployment not configured");
string embeddingDeployment = configuration["AzureOpenAI:EmbeddingDeployment"]
?? "text-embedding-ada-002";
// Build kernel with both chat and embedding services
var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.AddAzureOpenAIChatCompletion(
deploymentName: chatDeployment,
endpoint: azureEndpoint,
apiKey: azureApiKey
);
kernelBuilder.AddAzureOpenAITextEmbeddingGeneration(
deploymentName: embeddingDeployment,
endpoint: azureEndpoint,
apiKey: azureApiKey
);
_kernel = kernelBuilder.Build();
// Get services from kernel
_chatService = _kernel.GetRequiredService<IChatCompletionService>();
var embeddingService = _kernel.GetRequiredService<ITextEmbeddingGenerationService>();
// Configure Qdrant vector store
var qdrantClient = new QdrantMemoryStore("http://localhost:6333", 1536);
// Create semantic memory with Qdrant backend
_memory = new MemoryBuilder()
.WithMemoryStore(qdrantClient)
.WithTextEmbeddingGeneration(embeddingService)
.Build();
// Initialize document ingestion service
_ingestionService = new DocumentIngestionService(_memory, embeddingService);
}
/// <summary>
/// Ingests a research document into the knowledge base
/// </summary>
public async Task IngestDocumentAsync(string filePath)
{
string content = await File.ReadAllTextAsync(filePath);
string documentId = Path.GetFileNameWithoutExtension(filePath);
await _ingestionService.IngestDocumentAsync(content, documentId, CollectionName);
}
/// <summary>
/// Runs the chatbot with RAG-enhanced responses
/// </summary>
public async Task RunAsync()
{
Console.WriteLine("RAG-Enhanced Research Chatbot initialized.");
Console.WriteLine("Type 'ingest <filepath>' to add a document to the knowledge base.");
Console.WriteLine("Type 'exit' to quit.");
Console.WriteLine();
var chatHistory = new ChatHistory();
chatHistory.AddSystemMessage(
"You are a research assistant. Use the provided context to answer questions accurately. " +
"If the context doesn't contain relevant information, say so clearly."
);
while (true)
{
Console.Write("You: ");
string userInput = Console.ReadLine() ?? string.Empty;
if (userInput.ToLower() == "exit")
{
break;
}
if (userInput.ToLower().StartsWith("ingest "))
{
string filePath = userInput.Substring(7).Trim();
try
{
await IngestDocumentAsync(filePath);
Console.WriteLine("Document ingested successfully.");
}
catch (Exception ex)
{
Console.WriteLine($"Error ingesting document: {ex.Message}");
}
Console.WriteLine();
continue;
}
try
{
// Retrieve relevant context from vector store
var relevantChunks = await _ingestionService.RetrieveRelevantChunksAsync(
userInput,
CollectionName,
topK: 3
);
// Build context from retrieved chunks
string context = string.Empty;
if (relevantChunks.Any())
{
context = "Relevant context from knowledge base:\n\n" +
string.Join("\n\n---\n\n", relevantChunks) +
"\n\nBased on the above context, please answer the following question:\n\n";
}
// Add user message with context
chatHistory.AddUserMessage(context + userInput);
// Get response from LLM
var response = await _chatService.GetChatMessageContentAsync(
chatHistory,
kernel: _kernel
);
chatHistory.AddAssistantMessage(response.Content ?? string.Empty);
Console.WriteLine($"Assistant: {response.Content}");
Console.WriteLine();
}
catch (Exception ex)
{
Console.WriteLine($"Error: {ex.Message}");
Console.WriteLine();
}
}
}
}
}
This RAG chatbot implementation demonstrates the complete retrieval-augmented generation workflow. When a user asks a question, the system first retrieves relevant document chunks from the vector store. These chunks are then prepended to the user's question as context. The LLM receives both the retrieved context and the question, enabling it to provide answers grounded in the specific documents rather than relying solely on its training data.
The system message instructs the LLM to use the provided context and to acknowledge when the context doesn't contain relevant information. This instruction reduces hallucinations and improves response reliability.
To use this RAG chatbot, you would modify your Program.cs:
using Microsoft.Extensions.Configuration;
using ResearchChatbot;
var configuration = new ConfigurationBuilder()
.AddEnvironmentVariables()
.AddUserSecrets<Program>()
.Build();
var chatbot = new RAGChatbot(configuration);
await chatbot.RunAsync();
You also need to configure the embedding deployment in your user secrets:
dotnet user-secrets set "AzureOpenAI:EmbeddingDeployment" "text-embedding-ada-002"
The embedding model converts text into high-dimensional vectors that capture semantic meaning. Azure OpenAI's text-embedding-ada-002 model produces 1536-dimensional vectors. When we search for similar chunks, we compare these vectors using cosine similarity, which measures the angle between vectors in high-dimensional space.
IMPLEMENTING GRAPHRAG
GraphRAG extends traditional RAG by incorporating knowledge graph structures. Instead of treating documents as flat collections of chunks, GraphRAG builds a graph where entities are nodes and relationships are edges. This structure enables more sophisticated retrieval strategies, including multi-hop reasoning and relationship-based queries.
Implementing GraphRAG requires several components. First, we need entity extraction to identify important entities in documents. Second, we need relationship extraction to identify how entities relate to each other. Third, we need a graph database to store this structured knowledge. Fourth, we need graph traversal algorithms to retrieve relevant subgraphs for a given query.
For our implementation, we will use Neo4j as the graph database and implement a simple entity and relationship extraction system using the LLM itself. First, install Neo4j locally or use a cloud instance. You can run Neo4j via Docker:
docker run -p 7474:7474 -p 7687:7687 -e NEO4J_AUTH=neo4j/password neo4j
Add the Neo4j .NET driver package:
dotnet add package Neo4j.Driver
Now, let us create an entity extraction service. Create a file called EntityExtractionService.cs:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using System.Text.Json;
namespace ResearchChatbot
{
/// <summary>
/// Service for extracting entities and relationships from text using LLM
/// </summary>
public class EntityExtractionService
{
private readonly IChatCompletionService _chatService;
private readonly Kernel _kernel;
public EntityExtractionService(Kernel kernel)
{
_kernel = kernel;
_chatService = kernel.GetRequiredService<IChatCompletionService>();
}
/// <summary>
/// Extracts entities and relationships from a text chunk
/// </summary>
public async Task<ExtractionResult> ExtractEntitiesAndRelationshipsAsync(string text)
{
// Create a prompt that instructs the LLM to extract structured information
string prompt = @"
Extract entities and relationships from the following text.
Return the result as JSON with this structure:
{
""entities"": [
{""name"": ""entity name"", ""type"": ""entity type""}
],
""relationships"": [
{""source"": ""entity1"", ""target"": ""entity2"", ""type"": ""relationship type""}
]
}
Focus on extracting:
- People, organizations, locations, concepts, technologies
- Clear relationships between entities
Text:
" + text + @"
Return only valid JSON, no additional text.";
var chatHistory = new ChatHistory();
chatHistory.AddUserMessage(prompt);
try
{
var response = await _chatService.GetChatMessageContentAsync(
chatHistory,
kernel: _kernel
);
string jsonResponse = response.Content ?? string.Empty;
// Clean up the response in case the LLM added markdown formatting
jsonResponse = jsonResponse.Trim();
if (jsonResponse.StartsWith("```json"))
{
jsonResponse = jsonResponse.Substring(7);
}
if (jsonResponse.StartsWith("```"))
{
jsonResponse = jsonResponse.Substring(3);
}
if (jsonResponse.EndsWith("```"))
{
jsonResponse = jsonResponse.Substring(0, jsonResponse.Length - 3);
}
jsonResponse = jsonResponse.Trim();
// Parse the JSON response
var result = JsonSerializer.Deserialize<ExtractionResult>(
jsonResponse,
new JsonSerializerOptions { PropertyNameCaseInsensitive = true }
);
return result ?? new ExtractionResult();
}
catch (Exception ex)
{
Console.WriteLine($"Error extracting entities: {ex.Message}");
return new ExtractionResult();
}
}
}
/// <summary>
/// Represents an extracted entity
/// </summary>
public class Entity
{
public string Name { get; set; } = string.Empty;
public string Type { get; set; } = string.Empty;
}
/// <summary>
/// Represents a relationship between entities
/// </summary>
public class Relationship
{
public string Source { get; set; } = string.Empty;
public string Target { get; set; } = string.Empty;
public string Type { get; set; } = string.Empty;
}
/// <summary>
/// Contains extraction results
/// </summary>
public class ExtractionResult
{
public List<Entity> Entities { get; set; } = new();
public List<Relationship> Relationships { get; set; } = new();
}
}
This service uses the LLM itself to extract structured information from text. The prompt instructs the model to identify entities and relationships and return them in a structured JSON format. This approach leverages the LLM's understanding of language and context to perform extraction without requiring separate NLP models.
The response cleaning logic handles cases where the LLM wraps the JSON in markdown code blocks, which some models do despite instructions to return only JSON. This defensive programming ensures robustness across different model behaviors.
Now, let us create a graph database service for Neo4j. Create a file called GraphDatabaseService.cs:
using Neo4j.Driver;
namespace ResearchChatbot
{
/// <summary>
/// Service for interacting with Neo4j graph database
/// </summary>
public class GraphDatabaseService : IDisposable
{
private readonly IDriver _driver;
public GraphDatabaseService(string uri, string username, string password)
{
_driver = GraphDatabase.Driver(uri, AuthTokens.Basic(username, password));
}
/// <summary>
/// Stores an entity in the graph database
/// </summary>
public async Task StoreEntityAsync(Entity entity, string documentId)
{
await using var session = _driver.AsyncSession();
await session.ExecuteWriteAsync(async tx =>
{
var query = @"
MERGE (e:Entity {name: $name})
SET e.type = $type
MERGE (d:Document {id: $documentId})
MERGE (e)-[:MENTIONED_IN]->(d)";
await tx.RunAsync(query, new
{
name = entity.Name,
type = entity.Type,
documentId = documentId
});
});
}
/// <summary>
/// Stores a relationship in the graph database
/// </summary>
public async Task StoreRelationshipAsync(Relationship relationship, string documentId)
{
await using var session = _driver.AsyncSession();
await session.ExecuteWriteAsync(async tx =>
{
var query = @"
MERGE (source:Entity {name: $source})
MERGE (target:Entity {name: $target})
MERGE (source)-[r:RELATES_TO {type: $relType}]->(target)
SET r.documentId = $documentId";
await tx.RunAsync(query, new
{
source = relationship.Source,
target = relationship.Target,
relType = relationship.Type,
documentId = documentId
});
});
}
/// <summary>
/// Retrieves entities related to a given entity within a certain depth
/// </summary>
public async Task<List<string>> GetRelatedEntitiesAsync(string entityName, int maxDepth = 2)
{
await using var session = _driver.AsyncSession();
var result = await session.ExecuteReadAsync(async tx =>
{
var query = @"
MATCH path = (start:Entity {name: $entityName})-[*1.." + maxDepth + @"]-(related:Entity)
RETURN DISTINCT related.name AS name, related.type AS type";
var cursor = await tx.RunAsync(query, new { entityName });
var entities = new List<string>();
await foreach (var record in cursor)
{
string name = record["name"].As<string>();
string type = record["type"].As<string>();
entities.Add($"{name} ({type})");
}
return entities;
});
return result;
}
/// <summary>
/// Retrieves a subgraph around entities mentioned in a query
/// </summary>
public async Task<string> GetRelevantSubgraphAsync(List<string> queryEntities)
{
await using var session = _driver.AsyncSession();
var result = await session.ExecuteReadAsync(async tx =>
{
var query = @"
MATCH (e:Entity)
WHERE e.name IN $entities
MATCH path = (e)-[r*0..2]-(related:Entity)
RETURN DISTINCT e.name AS entity,
type(r[0]) AS relationship,
related.name AS relatedEntity
LIMIT 50";
var cursor = await tx.RunAsync(query, new { entities = queryEntities });
var subgraphDescription = new List<string>();
await foreach (var record in cursor)
{
string entity = record["entity"].As<string>();
string relationship = record["relationship"]?.As<string>() ?? "connected to";
string relatedEntity = record["relatedEntity"].As<string>();
subgraphDescription.Add($"{entity} {relationship} {relatedEntity}");
}
return string.Join("\n", subgraphDescription);
});
return result;
}
public void Dispose()
{
_driver?.Dispose();
}
}
}
This service provides methods for storing entities and relationships in Neo4j and retrieving relevant subgraphs. The MERGE clauses in Cypher queries ensure that entities are created only if they don't already exist, preventing duplicates. The relationship storage also links entities to the source document, enabling provenance tracking.
The GetRelatedEntitiesAsync method performs graph traversal to find entities connected to a given entity within a specified depth. This enables discovery of indirect relationships that might not be apparent from simple text search.
The GetRelevantSubgraphAsync method retrieves a subgraph containing entities mentioned in a query and their immediate neighbors. This subgraph provides rich context for the LLM, including not just individual facts but also the relationships between entities.
Now, let us create the GraphRAG chatbot that combines these components. Create a file called GraphRAGChatbot.cs:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.Extensions.Configuration;
namespace ResearchChatbot
{
/// <summary>
/// Research chatbot with GraphRAG capabilities
/// </summary>
public class GraphRAGChatbot
{
private readonly Kernel _kernel;
private readonly IChatCompletionService _chatService;
private readonly EntityExtractionService _extractionService;
private readonly GraphDatabaseService _graphService;
public GraphRAGChatbot(IConfiguration configuration)
{
// Retrieve configuration
string azureEndpoint = configuration["AzureOpenAI:Endpoint"]
?? throw new InvalidOperationException("Endpoint not configured");
string azureApiKey = configuration["AzureOpenAI:ApiKey"]
?? throw new InvalidOperationException("API key not configured");
string chatDeployment = configuration["AzureOpenAI:DeploymentName"]
?? throw new InvalidOperationException("Deployment not configured");
// Build kernel
var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.AddAzureOpenAIChatCompletion(
deploymentName: chatDeployment,
endpoint: azureEndpoint,
apiKey: azureApiKey
);
_kernel = kernelBuilder.Build();
_chatService = _kernel.GetRequiredService<IChatCompletionService>();
// Initialize services
_extractionService = new EntityExtractionService(_kernel);
_graphService = new GraphDatabaseService(
"bolt://localhost:7687",
"neo4j",
"password"
);
}
/// <summary>
/// Ingests a document by extracting entities and storing them in the graph
/// </summary>
public async Task IngestDocumentAsync(string filePath)
{
string content = await File.ReadAllTextAsync(filePath);
string documentId = Path.GetFileNameWithoutExtension(filePath);
Console.WriteLine($"Extracting entities from {documentId}...");
// Extract entities and relationships
var extractionResult = await _extractionService.ExtractEntitiesAndRelationshipsAsync(content);
Console.WriteLine($"Found {extractionResult.Entities.Count} entities and {extractionResult.Relationships.Count} relationships.");
// Store entities in graph
foreach (var entity in extractionResult.Entities)
{
await _graphService.StoreEntityAsync(entity, documentId);
}
// Store relationships in graph
foreach (var relationship in extractionResult.Relationships)
{
await _graphService.StoreRelationshipAsync(relationship, documentId);
}
Console.WriteLine($"Document {documentId} ingested into knowledge graph.");
}
/// <summary>
/// Runs the chatbot with GraphRAG-enhanced responses
/// </summary>
public async Task RunAsync()
{
Console.WriteLine("GraphRAG-Enhanced Research Chatbot initialized.");
Console.WriteLine("Type 'ingest <filepath>' to add a document to the knowledge graph.");
Console.WriteLine("Type 'exit' to quit.");
Console.WriteLine();
var chatHistory = new ChatHistory();
chatHistory.AddSystemMessage(
"You are a research assistant with access to a knowledge graph. " +
"Use the provided graph context to answer questions about entities and their relationships. " +
"Explain connections between concepts when relevant."
);
while (true)
{
Console.Write("You: ");
string userInput = Console.ReadLine() ?? string.Empty;
if (userInput.ToLower() == "exit")
{
break;
}
if (userInput.ToLower().StartsWith("ingest "))
{
string filePath = userInput.Substring(7).Trim();
try
{
await IngestDocumentAsync(filePath);
Console.WriteLine("Document ingested successfully.");
}
catch (Exception ex)
{
Console.WriteLine($"Error ingesting document: {ex.Message}");
}
Console.WriteLine();
continue;
}
try
{
// Extract entities from the user's question
var queryExtraction = await _extractionService.ExtractEntitiesAndRelationshipsAsync(userInput);
var queryEntityNames = queryExtraction.Entities.Select(e => e.Name).ToList();
// Retrieve relevant subgraph
string graphContext = string.Empty;
if (queryEntityNames.Any())
{
graphContext = await _graphService.GetRelevantSubgraphAsync(queryEntityNames);
if (!string.IsNullOrEmpty(graphContext))
{
graphContext = "Relevant knowledge graph context:\n\n" +
graphContext +
"\n\nBased on the above relationships, please answer:\n\n";
}
}
// Add user message with graph context
chatHistory.AddUserMessage(graphContext + userInput);
// Get response from LLM
var response = await _chatService.GetChatMessageContentAsync(
chatHistory,
kernel: _kernel
);
chatHistory.AddAssistantMessage(response.Content ?? string.Empty);
Console.WriteLine($"Assistant: {response.Content}");
Console.WriteLine();
}
catch (Exception ex)
{
Console.WriteLine($"Error: {ex.Message}");
Console.WriteLine();
}
}
}
}
}
This GraphRAG implementation demonstrates how knowledge graphs enhance retrieval. When a user asks a question, the system first extracts entities mentioned in the question. It then retrieves a subgraph containing those entities and their relationships. This subgraph provides structured context that helps the LLM understand not just individual facts but also how concepts relate to each other.
GraphRAG excels at answering questions that require understanding relationships, such as "How is entity A connected to entity B?" or "What are all the technologies used by organization X?" The graph structure makes these multi-hop queries natural and efficient.
FINE-TUNING LLMS FOR SPECIALIZED TASKS
Fine-tuning adapts a pre-trained LLM to perform better on specific tasks or domains. While RAG and GraphRAG provide external knowledge, fine-tuning modifies the model's internal parameters to encode domain-specific patterns and behaviors. Fine-tuning is particularly valuable when you need the model to adopt a specific tone, follow domain-specific conventions, or perform specialized tasks consistently.
For .NET developers, several approaches to fine-tuning exist. Azure OpenAI Service provides fine-tuning capabilities for GPT-3.5 Turbo and other models. OpenAI's API also supports fine-tuning. For local models, tools like Axolotl, LLaMA Factory, and Unsloth enable fine-tuning on consumer hardware.
Let us explore how to prepare data and initiate fine-tuning using Azure OpenAI Service. Fine-tuning requires a dataset of input-output pairs in JSONL format. Each line contains a conversation with messages in the chat format.
Create a file called FineTuningDataPreparation.cs:
using System.Text.Json;
namespace ResearchChatbot
{
/// <summary>
/// Service for preparing fine-tuning datasets
/// </summary>
public class FineTuningDataPreparation
{
/// <summary>
/// Represents a single training example
/// </summary>
public class TrainingExample
{
public List<Message> Messages { get; set; } = new();
}
/// <summary>
/// Represents a message in the conversation
/// </summary>
public class Message
{
public string Role { get; set; } = string.Empty;
public string Content { get; set; } = string.Empty;
}
/// <summary>
/// Creates a training dataset for research assistant fine-tuning
/// </summary>
public async Task CreateTrainingDatasetAsync(string outputPath)
{
var examples = new List<TrainingExample>();
// Example 1: Explaining a research concept
examples.Add(new TrainingExample
{
Messages = new List<Message>
{
new Message
{
Role = "system",
Content = "You are a research assistant specializing in computer science. " +
"Provide detailed, academic-quality explanations."
},
new Message
{
Role = "user",
Content = "Explain the concept of attention mechanisms in transformers."
},
new Message
{
Role = "assistant",
Content = "Attention mechanisms in transformer architectures enable the model to " +
"weigh the importance of different input tokens when processing each output token. " +
"The self-attention mechanism computes three vectors for each token: Query, Key, and Value. " +
"The attention score between tokens is calculated by taking the dot product of the Query " +
"vector of one token with the Key vectors of all tokens, then applying softmax to obtain " +
"attention weights. These weights determine how much each token contributes to the " +
"representation of the current token. This mechanism allows transformers to capture " +
"long-range dependencies without the sequential processing limitations of RNNs."
}
}
});
// Example 2: Providing research methodology guidance
examples.Add(new TrainingExample
{
Messages = new List<Message>
{
new Message
{
Role = "system",
Content = "You are a research assistant specializing in computer science. " +
"Provide detailed, academic-quality explanations."
},
new Message
{
Role = "user",
Content = "What is the best approach for evaluating a new machine learning model?"
},
new Message
{
Role = "assistant",
Content = "Evaluating a machine learning model requires a comprehensive approach. " +
"First, split your dataset into training, validation, and test sets to prevent " +
"overfitting. Use cross-validation on the training set to tune hyperparameters. " +
"Select appropriate metrics based on your task: accuracy, precision, recall, and " +
"F1-score for classification; MSE, RMSE, or MAE for regression. Consider the business " +
"context when choosing metrics. Perform error analysis to understand failure modes. " +
"Compare against baseline models and state-of-the-art approaches. Evaluate computational " +
"efficiency and scalability. Finally, test on held-out data that the model has never seen."
}
}
});
// Example 3: Citation and reference formatting
examples.Add(new TrainingExample
{
Messages = new List<Message>
{
new Message
{
Role = "system",
Content = "You are a research assistant specializing in computer science. " +
"Provide detailed, academic-quality explanations."
},
new Message
{
Role = "user",
Content = "How should I cite a paper in my research document?"
},
new Message
{
Role = "assistant",
Content = "Citation format depends on your field's standard. In computer science, " +
"IEEE and ACM formats are common. For IEEE: [1] A. Author, B. Author, \"Title of Paper,\" " +
"Journal Name, vol. X, no. Y, pp. Z-ZZ, Month Year. For ACM: Author, A. and Author, B. Year. " +
"Title of paper. Journal Name X, Y (Month Year), Z-ZZ. Include DOI when available. " +
"For conference papers, include the conference name and location. Use reference management " +
"tools like Zotero, Mendeley, or BibTeX to maintain consistency and automate formatting."
}
}
});
// Write examples to JSONL file
await using var writer = new StreamWriter(outputPath);
foreach (var example in examples)
{
string json = JsonSerializer.Serialize(example, new JsonSerializerOptions
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
});
await writer.WriteLineAsync(json);
}
Console.WriteLine($"Training dataset created with {examples.Count} examples at {outputPath}");
}
/// <summary>
/// Validates a training dataset
/// </summary>
public async Task<bool> ValidateDatasetAsync(string filePath)
{
int lineNumber = 0;
bool isValid = true;
await using var reader = new StreamReader(filePath);
while (!reader.EndOfStream)
{
lineNumber++;
string line = await reader.ReadLineAsync() ?? string.Empty;
try
{
var example = JsonSerializer.Deserialize<TrainingExample>(line, new JsonSerializerOptions
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
});
if (example == null || example.Messages == null || example.Messages.Count < 2)
{
Console.WriteLine($"Line {lineNumber}: Invalid example structure");
isValid = false;
continue;
}
// Check that messages alternate between user and assistant
bool hasSystem = example.Messages.Any(m => m.Role == "system");
bool hasUser = example.Messages.Any(m => m.Role == "user");
bool hasAssistant = example.Messages.Any(m => m.Role == "assistant");
if (!hasUser || !hasAssistant)
{
Console.WriteLine($"Line {lineNumber}: Missing user or assistant message");
isValid = false;
}
}
catch (JsonException ex)
{
Console.WriteLine($"Line {lineNumber}: JSON parsing error - {ex.Message}");
isValid = false;
}
}
Console.WriteLine(isValid
? $"Dataset validation passed. {lineNumber} examples checked."
: "Dataset validation failed. Please fix errors.");
return isValid;
}
}
}
This data preparation service creates training examples in the format required by Azure OpenAI fine-tuning. Each example contains a system message that sets the behavior, a user message with the input, and an assistant message with the desired output. The quality and diversity of training examples directly impact fine-tuning effectiveness.
For production fine-tuning, you would need hundreds or thousands of high-quality examples covering the range of inputs and outputs you expect. The examples should be representative of real usage and demonstrate the specific behaviors you want the model to learn.
Now, let us create a service that initiates fine-tuning using the Azure OpenAI API. Create a file called FineTuningService.cs:
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;
namespace ResearchChatbot
{
/// <summary>
/// Service for managing fine-tuning jobs with Azure OpenAI
/// </summary>
public class FineTuningService
{
private readonly HttpClient _httpClient;
private readonly string _endpoint;
private readonly string _apiVersion;
public FineTuningService(string endpoint, string apiKey)
{
_httpClient = new HttpClient();
_httpClient.DefaultRequestHeaders.Add("api-key", apiKey);
_endpoint = endpoint.TrimEnd('/');
_apiVersion = "2024-02-15-preview";
}
/// <summary>
/// Uploads a training file to Azure OpenAI
/// </summary>
public async Task<string> UploadTrainingFileAsync(string filePath)
{
string url = $"{_endpoint}/openai/files?api-version={_apiVersion}";
using var form = new MultipartFormDataContent();
var fileContent = new ByteArrayContent(await File.ReadAllBytesAsync(filePath));
fileContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
form.Add(fileContent, "file", Path.GetFileName(filePath));
form.Add(new StringContent("fine-tune"), "purpose");
var response = await _httpClient.PostAsync(url, form);
response.EnsureSuccessStatusCode();
string responseContent = await response.Content.ReadAsStringAsync();
var responseObj = JsonSerializer.Deserialize<JsonElement>(responseContent);
string fileId = responseObj.GetProperty("id").GetString()
?? throw new InvalidOperationException("File ID not returned");
Console.WriteLine($"Training file uploaded successfully. File ID: {fileId}");
return fileId;
}
/// <summary>
/// Creates a fine-tuning job
/// </summary>
public async Task<string> CreateFineTuningJobAsync(
string trainingFileId,
string baseModel,
string suffix)
{
string url = $"{_endpoint}/openai/fine_tuning/jobs?api-version={_apiVersion}";
var requestBody = new
{
training_file = trainingFileId,
model = baseModel,
suffix = suffix,
hyperparameters = new
{
n_epochs = 3,
batch_size = 1,
learning_rate_multiplier = 1.0
}
};
string jsonRequest = JsonSerializer.Serialize(requestBody);
var content = new StringContent(jsonRequest, Encoding.UTF8, "application/json");
var response = await _httpClient.PostAsync(url, content);
response.EnsureSuccessStatusCode();
string responseContent = await response.Content.ReadAsStringAsync();
var responseObj = JsonSerializer.Deserialize<JsonElement>(responseContent);
string jobId = responseObj.GetProperty("id").GetString()
?? throw new InvalidOperationException("Job ID not returned");
Console.WriteLine($"Fine-tuning job created successfully. Job ID: {jobId}");
return jobId;
}
/// <summary>
/// Checks the status of a fine-tuning job
/// </summary>
public async Task<string> GetFineTuningJobStatusAsync(string jobId)
{
string url = $"{_endpoint}/openai/fine_tuning/jobs/{jobId}?api-version={_apiVersion}";
var response = await _httpClient.GetAsync(url);
response.EnsureSuccessStatusCode();
string responseContent = await response.Content.ReadAsStringAsync();
var responseObj = JsonSerializer.Deserialize<JsonElement>(responseContent);
string status = responseObj.GetProperty("status").GetString()
?? throw new InvalidOperationException("Status not returned");
return status;
}
/// <summary>
/// Retrieves the fine-tuned model name once job is complete
/// </summary>
public async Task<string> GetFineTunedModelNameAsync(string jobId)
{
string url = $"{_endpoint}/openai/fine_tuning/jobs/{jobId}?api-version={_apiVersion}";
var response = await _httpClient.GetAsync(url);
response.EnsureSuccessStatusCode();
string responseContent = await response.Content.ReadAsStringAsync();
var responseObj = JsonSerializer.Deserialize<JsonElement>(responseContent);
if (responseObj.TryGetProperty("fine_tuned_model", out var modelElement))
{
string modelName = modelElement.GetString()
?? throw new InvalidOperationException("Model name not available");
return modelName;
}
throw new InvalidOperationException("Fine-tuned model not yet available");
}
}
}
This service handles the complete fine-tuning workflow. The UploadTrainingFileAsync method uploads your JSONL training file to Azure OpenAI. The CreateFineTuningJobAsync method initiates the fine-tuning process with specified hyperparameters. The number of epochs determines how many times the model trains on the entire dataset. The batch size affects memory usage and training stability. The learning rate multiplier controls how aggressively the model's weights are updated.
Fine-tuning typically takes several hours depending on dataset size and model complexity. You can monitor progress using the GetFineTuningJobStatusAsync method. Once complete, the fine-tuned model becomes available as a deployment that you can use just like any other model.
For local model fine-tuning, tools like Axolotl provide comprehensive workflows. Axolotl supports various fine-tuning techniques including full fine-tuning, LoRA (Low-Rank Adaptation), and QLoRA (Quantized LoRA). LoRA is particularly popular because it fine-tunes only a small number of additional parameters, making the process much more memory-efficient while maintaining good performance.
To use Axolotl from .NET, you would typically prepare your dataset in the required format, then invoke Axolotl via a Python script or command-line process. Create a file called LocalFineTuningHelper.cs:
using System.Diagnostics;
namespace ResearchChatbot
{
/// <summary>
/// Helper for invoking local fine-tuning tools
/// </summary>
public class LocalFineTuningHelper
{
/// <summary>
/// Invokes Axolotl for local model fine-tuning
/// </summary>
public async Task<bool> RunAxolotlFineTuningAsync(string configPath, string outputDir)
{
// Axolotl requires Python and must be installed in the environment
// This method demonstrates how to invoke it from .NET
var processInfo = new ProcessStartInfo
{
FileName = "python",
Arguments = $"-m axolotl.cli.train {configPath}",
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
CreateNoWindow = true,
WorkingDirectory = outputDir
};
using var process = new Process { StartInfo = processInfo };
process.OutputDataReceived += (sender, args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
Console.WriteLine($"[Axolotl] {args.Data}");
}
};
process.ErrorDataReceived += (sender, args) =>
{
if (!string.IsNullOrEmpty(args.Data))
{
Console.WriteLine($"[Axolotl Error] {args.Data}");
}
};
process.Start();
process.BeginOutputReadLine();
process.BeginErrorReadLine();
await process.WaitForExitAsync();
return process.ExitCode == 0;
}
/// <summary>
/// Creates an Axolotl configuration file for LoRA fine-tuning
/// </summary>
public async Task CreateAxolotlConfigAsync(string outputPath, string modelName, string datasetPath)
{
string config = $@"
base_model: {modelName}
model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer
load_in_8bit: false
load_in_4bit: true
strict: false
datasets:
- path: {datasetPath}
type: alpaca
dataset_prepared_path:
val_set_size: 0.05
output_dir: ./lora-out
adapter: lora
lora_r: 8
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules:
- q_proj
- v_proj
sequence_len: 2048
sample_packing: true
wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:
gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 3
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.0002
train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
warmup_steps: 10
evals_per_epoch: 4
eval_table_size:
saves_per_epoch: 1
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
";
await File.WriteAllTextAsync(outputPath, config);
Console.WriteLine($"Axolotl configuration created at {outputPath}");
}
}
}
This helper demonstrates how to integrate local fine-tuning tools into your .NET workflow. The configuration file specifies all aspects of the fine-tuning process including the base model, dataset location, LoRA parameters, and training hyperparameters. LoRA rank (lora_r) and alpha (lora_alpha) control the capacity of the adapter, with higher values allowing more expressive adaptations but requiring more memory.
The Process class enables .NET applications to invoke Python scripts and other external tools. This pattern allows you to leverage the rich Python ecosystem for AI tasks while maintaining your primary application logic in .NET.
ADVANCED RAG TECHNIQUES
Beyond basic RAG, several advanced techniques can significantly improve retrieval quality and response accuracy. Let us explore some of these enhancements.
Hybrid search combines vector similarity search with traditional keyword search. This approach leverages both semantic understanding and exact term matching. Create a file called HybridSearchService.cs:
using Microsoft.SemanticKernel.Memory;
namespace ResearchChatbot
{
/// <summary>
/// Service implementing hybrid search combining vector and keyword search
/// </summary>
public class HybridSearchService
{
private readonly ISemanticTextMemory _vectorMemory;
private readonly Dictionary<string, List<DocumentChunk>> _keywordIndex;
public HybridSearchService(ISemanticTextMemory vectorMemory)
{
_vectorMemory = vectorMemory;
_keywordIndex = new Dictionary<string, List<DocumentChunk>>();
}
/// <summary>
/// Represents a document chunk with metadata
/// </summary>
public class DocumentChunk
{
public string Id { get; set; } = string.Empty;
public string Content { get; set; } = string.Empty;
public string DocumentId { get; set; } = string.Empty;
public double Score { get; set; }
}
/// <summary>
/// Indexes a document chunk for keyword search
/// </summary>
public void IndexChunkForKeywordSearch(string chunkId, string content, string documentId)
{
// Simple keyword extraction: split on whitespace and punctuation
var words = content.ToLower()
.Split(new[] { ' ', '.', ',', ';', ':', '!', '?', '\n', '\r', '\t' },
StringSplitOptions.RemoveEmptyEntries);
var chunk = new DocumentChunk
{
Id = chunkId,
Content = content,
DocumentId = documentId
};
foreach (var word in words)
{
if (word.Length < 3) continue; // Skip very short words
if (!_keywordIndex.ContainsKey(word))
{
_keywordIndex[word] = new List<DocumentChunk>();
}
if (!_keywordIndex[word].Any(c => c.Id == chunkId))
{
_keywordIndex[word].Add(chunk);
}
}
}
/// <summary>
/// Performs keyword search using inverted index
/// </summary>
private List<DocumentChunk> KeywordSearch(string query, int topK)
{
var queryWords = query.ToLower()
.Split(new[] { ' ', '.', ',', ';', ':', '!', '?', '\n', '\r', '\t' },
StringSplitOptions.RemoveEmptyEntries);
var chunkScores = new Dictionary<string, double>();
foreach (var word in queryWords)
{
if (_keywordIndex.ContainsKey(word))
{
foreach (var chunk in _keywordIndex[word])
{
if (!chunkScores.ContainsKey(chunk.Id))
{
chunkScores[chunk.Id] = 0;
}
chunkScores[chunk.Id] += 1.0; // Simple term frequency
}
}
}
// Get top chunks by score
var topChunks = chunkScores
.OrderByDescending(kvp => kvp.Value)
.Take(topK)
.Select(kvp =>
{
var chunk = _keywordIndex.Values
.SelectMany(list => list)
.First(c => c.Id == kvp.Key);
chunk.Score = kvp.Value;
return chunk;
})
.ToList();
return topChunks;
}
/// <summary>
/// Performs hybrid search combining vector and keyword approaches
/// </summary>
public async Task<List<string>> HybridSearchAsync(
string query,
string collectionName,
int topK = 5,
double vectorWeight = 0.7)
{
// Perform vector search
var vectorResults = new List<DocumentChunk>();
var vectorSearchResults = _vectorMemory.SearchAsync(
collection: collectionName,
query: query,
limit: topK,
minRelevanceScore: 0.5
);
await foreach (var result in vectorSearchResults)
{
vectorResults.Add(new DocumentChunk
{
Id = result.Metadata.Id,
Content = result.Metadata.Text,
Score = result.Relevance
});
}
// Perform keyword search
var keywordResults = KeywordSearch(query, topK);
// Combine results using weighted scoring
var combinedScores = new Dictionary<string, (double score, string content)>();
foreach (var result in vectorResults)
{
combinedScores[result.Id] = (result.Score * vectorWeight, result.Content);
}
double keywordWeight = 1.0 - vectorWeight;
foreach (var result in keywordResults)
{
if (combinedScores.ContainsKey(result.Id))
{
var existing = combinedScores[result.Id];
combinedScores[result.Id] = (
existing.score + result.Score * keywordWeight,
existing.content
);
}
else
{
combinedScores[result.Id] = (result.Score * keywordWeight, result.Content);
}
}
// Return top results
var topResults = combinedScores
.OrderByDescending(kvp => kvp.Value.score)
.Take(topK)
.Select(kvp => kvp.Value.content)
.ToList();
return topResults;
}
}
}
This hybrid search implementation maintains both a vector index (through Semantic Memory) and a keyword index. The keyword index is a simple inverted index mapping words to document chunks. When searching, both approaches run in parallel, and their results are combined using weighted scoring. The vectorWeight parameter controls the balance between semantic and keyword matching.
Hybrid search is particularly effective for queries containing specific technical terms or proper nouns that might not be well-represented in embedding space. It also helps with queries where exact term matching is important.
Another advanced technique is query expansion, where the original query is augmented with related terms or rephrased versions. Create a file called QueryExpansionService.cs:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
namespace ResearchChatbot
{
/// <summary>
/// Service for expanding queries to improve retrieval
/// </summary>
public class QueryExpansionService
{
private readonly IChatCompletionService _chatService;
private readonly Kernel _kernel;
public QueryExpansionService(Kernel kernel)
{
_kernel = kernel;
_chatService = kernel.GetRequiredService<IChatCompletionService>();
}
/// <summary>
/// Generates multiple variations of a query for better retrieval
/// </summary>
public async Task<List<string>> ExpandQueryAsync(string originalQuery)
{
string prompt = $@"
Given the following query, generate 3 alternative phrasings that capture the same intent
but use different words or perspectives. This will help retrieve more relevant documents.
Original query: {originalQuery}
Provide the alternatives as a numbered list, one per line.";
var chatHistory = new ChatHistory();
chatHistory.AddUserMessage(prompt);
var response = await _chatService.GetChatMessageContentAsync(
chatHistory,
kernel: _kernel
);
string content = response.Content ?? string.Empty;
// Parse the response to extract alternatives
var alternatives = new List<string> { originalQuery };
var lines = content.Split('\n', StringSplitOptions.RemoveEmptyEntries);
foreach (var line in lines)
{
// Remove numbering and clean up
string cleaned = line.Trim();
if (cleaned.Length > 2 && char.IsDigit(cleaned[0]))
{
cleaned = cleaned.Substring(cleaned.IndexOf('.') + 1).Trim();
}
if (!string.IsNullOrWhiteSpace(cleaned) && cleaned != originalQuery)
{
alternatives.Add(cleaned);
}
}
return alternatives;
}
/// <summary>
/// Decomposes a complex query into simpler sub-queries
/// </summary>
public async Task<List<string>> DecomposeQueryAsync(string complexQuery)
{
string prompt = $@"
The following query might be asking multiple things or require multiple pieces of information
to answer fully. Break it down into simpler, atomic sub-queries that each focus on one aspect.
Complex query: {complexQuery}
Provide the sub-queries as a numbered list, one per line.";
var chatHistory = new ChatHistory();
chatHistory.AddUserMessage(prompt);
var response = await _chatService.GetChatMessageContentAsync(
chatHistory,
kernel: _kernel
);
string content = response.Content ?? string.Empty;
var subQueries = new List<string>();
var lines = content.Split('\n', StringSplitOptions.RemoveEmptyEntries);
foreach (var line in lines)
{
string cleaned = line.Trim();
if (cleaned.Length > 2 && char.IsDigit(cleaned[0]))
{
cleaned = cleaned.Substring(cleaned.IndexOf('.') + 1).Trim();
}
if (!string.IsNullOrWhiteSpace(cleaned))
{
subQueries.Add(cleaned);
}
}
return subQueries;
}
}
}
Query expansion uses the LLM to generate alternative phrasings of the user's question. Each alternative is then used to retrieve documents, and the results are combined. This technique increases recall by capturing documents that might use different terminology than the original query.
Query decomposition breaks complex questions into simpler sub-questions. Each sub-question is answered separately, and the answers are then synthesized into a comprehensive response. This approach is particularly valuable for multi-faceted research questions.
PRODUCTION CONSIDERATIONS AND BEST PRACTICES
Deploying LLM applications to production requires careful attention to performance, reliability, cost, and security. Let us explore key considerations and best practices.
Caching is critical for reducing costs and improving response times. Implement a caching layer that stores LLM responses for identical or similar queries. Create a file called ResponseCacheService.cs:
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Microsoft.Extensions.Caching.Memory;
namespace ResearchChatbot
{
/// <summary>
/// Service for caching LLM responses to reduce costs and improve performance
/// </summary>
public class ResponseCacheService
{
private readonly IMemoryCache _cache;
private readonly TimeSpan _defaultExpiration;
public ResponseCacheService(IMemoryCache cache, TimeSpan? defaultExpiration = null)
{
_cache = cache;
_defaultExpiration = defaultExpiration ?? TimeSpan.FromHours(24);
}
/// <summary>
/// Generates a cache key from a prompt
/// </summary>
private string GenerateCacheKey(string prompt, string model)
{
string combined = $"{model}:{prompt}";
using var sha256 = SHA256.Create();
byte[] hashBytes = sha256.ComputeHash(Encoding.UTF8.GetBytes(combined));
return Convert.ToBase64String(hashBytes);
}
/// <summary>
/// Attempts to retrieve a cached response
/// </summary>
public bool TryGetCachedResponse(string prompt, string model, out string response)
{
string cacheKey = GenerateCacheKey(prompt, model);
return _cache.TryGetValue(cacheKey, out response);
}
/// <summary>
/// Caches a response
/// </summary>
public void CacheResponse(string prompt, string model, string response, TimeSpan? expiration = null)
{
string cacheKey = GenerateCacheKey(prompt, model);
var cacheOptions = new MemoryCacheEntryOptions
{
AbsoluteExpirationRelativeToNow = expiration ?? _defaultExpiration
};
_cache.Set(cacheKey, response, cacheOptions);
}
/// <summary>
/// Implements semantic caching using embeddings
/// </summary>
public class SemanticCache
{
private readonly List<CachedItem> _items;
private readonly double _similarityThreshold;
public class CachedItem
{
public string Prompt { get; set; } = string.Empty;
public string Response { get; set; } = string.Empty;
public float[] Embedding { get; set; } = Array.Empty<float>();
public DateTime ExpiresAt { get; set; }
}
public SemanticCache(double similarityThreshold = 0.95)
{
_items = new List<CachedItem>();
_similarityThreshold = similarityThreshold;
}
/// <summary>
/// Calculates cosine similarity between two embeddings
/// </summary>
private double CosineSimilarity(float[] a, float[] b)
{
if (a.Length != b.Length)
{
throw new ArgumentException("Embeddings must have same dimension");
}
double dotProduct = 0;
double magnitudeA = 0;
double magnitudeB = 0;
for (int i = 0; i < a.Length; i++)
{
dotProduct += a[i] * b[i];
magnitudeA += a[i] * a[i];
magnitudeB += b[i] * b[i];
}
return dotProduct / (Math.Sqrt(magnitudeA) * Math.Sqrt(magnitudeB));
}
/// <summary>
/// Attempts to find a semantically similar cached response
/// </summary>
public bool TryGetSimilarResponse(float[] queryEmbedding, out string response)
{
response = string.Empty;
// Remove expired items
_items.RemoveAll(item => item.ExpiresAt < DateTime.UtcNow);
double bestSimilarity = 0;
CachedItem bestMatch = null;
foreach (var item in _items)
{
double similarity = CosineSimilarity(queryEmbedding, item.Embedding);
if (similarity > bestSimilarity && similarity >= _similarityThreshold)
{
bestSimilarity = similarity;
bestMatch = item;
}
}
if (bestMatch != null)
{
response = bestMatch.Response;
return true;
}
return false;
}
/// <summary>
/// Adds a new item to the semantic cache
/// </summary>
public void Add(string prompt, string response, float[] embedding, TimeSpan expiration)
{
_items.Add(new CachedItem
{
Prompt = prompt,
Response = response,
Embedding = embedding,
ExpiresAt = DateTime.UtcNow.Add(expiration)
});
}
}
}
}
This caching implementation provides both exact matching and semantic caching. Exact matching uses a hash of the prompt as the cache key. Semantic caching embeds the query and finds cached responses for semantically similar queries, even if the exact wording differs. This approach significantly increases cache hit rates.
Rate limiting protects your application from excessive API costs and ensures fair resource allocation. Implement rate limiting using the token bucket algorithm:
using System.Collections.Concurrent;
namespace ResearchChatbot
{
/// <summary>
/// Token bucket rate limiter for controlling API request rates
/// </summary>
public class RateLimiter
{
private readonly ConcurrentDictionary<string, TokenBucket> _buckets;
private readonly int _maxTokens;
private readonly TimeSpan _refillInterval;
private readonly int _tokensPerRefill;
public class TokenBucket
{
public int Tokens { get; set; }
public DateTime LastRefill { get; set; }
}
public RateLimiter(int maxTokens, TimeSpan refillInterval, int tokensPerRefill)
{
_buckets = new ConcurrentDictionary<string, TokenBucket>();
_maxTokens = maxTokens;
_refillInterval = refillInterval;
_tokensPerRefill = tokensPerRefill;
}
/// <summary>
/// Attempts to consume tokens for a user
/// </summary>
public bool TryConsumeTokens(string userId, int tokensRequired = 1)
{
var bucket = _buckets.GetOrAdd(userId, _ => new TokenBucket
{
Tokens = _maxTokens,
LastRefill = DateTime.UtcNow
});
lock (bucket)
{
// Refill tokens if enough time has passed
var timeSinceRefill = DateTime.UtcNow - bucket.LastRefill;
if (timeSinceRefill >= _refillInterval)
{
int refills = (int)(timeSinceRefill.TotalMilliseconds / _refillInterval.TotalMilliseconds);
bucket.Tokens = Math.Min(_maxTokens, bucket.Tokens + refills * _tokensPerRefill);
bucket.LastRefill = DateTime.UtcNow;
}
// Check if enough tokens available
if (bucket.Tokens >= tokensRequired)
{
bucket.Tokens -= tokensRequired;
return true;
}
return false;
}
}
/// <summary>
/// Gets the time until tokens will be available
/// </summary>
public TimeSpan GetTimeUntilAvailable(string userId, int tokensRequired = 1)
{
if (!_buckets.TryGetValue(userId, out var bucket))
{
return TimeSpan.Zero;
}
lock (bucket)
{
if (bucket.Tokens >= tokensRequired)
{
return TimeSpan.Zero;
}
int tokensNeeded = tokensRequired - bucket.Tokens;
int refillsNeeded = (int)Math.Ceiling((double)tokensNeeded / _tokensPerRefill);
return TimeSpan.FromMilliseconds(_refillInterval.TotalMilliseconds * refillsNeeded);
}
}
}
}
This rate limiter uses the token bucket algorithm, which allows bursts of requests while maintaining an average rate limit. Each user has a bucket that refills at a constant rate. Requests consume tokens, and when the bucket is empty, requests are denied until tokens refill.
Error handling and retry logic are essential for production reliability. Implement exponential backoff with jitter:
namespace ResearchChatbot
{
/// <summary>
/// Retry policy with exponential backoff and jitter
/// </summary>
public class RetryPolicy
{
private readonly int _maxRetries;
private readonly TimeSpan _baseDelay;
private readonly Random _random;
public RetryPolicy(int maxRetries = 3, TimeSpan? baseDelay = null)
{
_maxRetries = maxRetries;
_baseDelay = baseDelay ?? TimeSpan.FromSeconds(1);
_random = new Random();
}
/// <summary>
/// Executes an operation with retry logic
/// </summary>
public async Task<T> ExecuteAsync<T>(Func<Task<T>> operation)
{
int attempt = 0;
Exception lastException = null;
while (attempt < _maxRetries)
{
try
{
return await operation();
}
catch (HttpRequestException ex) when (IsTransientError(ex))
{
lastException = ex;
attempt++;
if (attempt >= _maxRetries)
{
break;
}
// Calculate delay with exponential backoff and jitter
double exponentialDelay = Math.Pow(2, attempt) * _baseDelay.TotalMilliseconds;
double jitter = _random.NextDouble() * 0.3 * exponentialDelay;
int totalDelay = (int)(exponentialDelay + jitter);
Console.WriteLine($"Attempt {attempt} failed. Retrying in {totalDelay}ms...");
await Task.Delay(totalDelay);
}
}
throw new Exception($"Operation failed after {_maxRetries} attempts", lastException);
}
/// <summary>
/// Determines if an error is transient and should be retried
/// </summary>
private bool IsTransientError(HttpRequestException ex)
{
// Retry on network errors or 5xx server errors
if (ex.StatusCode.HasValue)
{
int statusCode = (int)ex.StatusCode.Value;
return statusCode >= 500 || statusCode == 429; // Server errors or rate limiting
}
return true; // Network errors
}
}
}
This retry policy implements exponential backoff, where the delay between retries increases exponentially. Jitter adds randomness to prevent thundering herd problems when multiple clients retry simultaneously. The policy only retries transient errors like network failures or server errors, not client errors like invalid requests.
Monitoring and observability are crucial for production systems. Implement structured logging and metrics collection:
using Microsoft.Extensions.Logging;
using System.Diagnostics;
namespace ResearchChatbot
{
/// <summary>
/// Telemetry service for monitoring LLM application performance
/// </summary>
public class TelemetryService
{
private readonly ILogger<TelemetryService> _logger;
private readonly ConcurrentDictionary<string, Metrics> _metrics;
public class Metrics
{
public long TotalRequests { get; set; }
public long SuccessfulRequests { get; set; }
public long FailedRequests { get; set; }
public long TotalTokensUsed { get; set; }
public double TotalLatencyMs { get; set; }
public long CacheHits { get; set; }
public long CacheMisses { get; set; }
}
public TelemetryService(ILogger<TelemetryService> logger)
{
_logger = logger;
_metrics = new ConcurrentDictionary<string, Metrics>();
}
/// <summary>
/// Tracks an LLM request with timing and token usage
/// </summary>
public async Task<T> TrackRequestAsync<T>(
string operationName,
Func<Task<T>> operation,
Func<T, int> getTokenCount = null)
{
var metrics = _metrics.GetOrAdd(operationName, _ => new Metrics());
var stopwatch = Stopwatch.StartNew();
try
{
Interlocked.Increment(ref metrics.TotalRequests);
T result = await operation();
stopwatch.Stop();
Interlocked.Increment(ref metrics.SuccessfulRequests);
double latency = stopwatch.Elapsed.TotalMilliseconds;
lock (metrics)
{
metrics.TotalLatencyMs += latency;
}
if (getTokenCount != null)
{
int tokens = getTokenCount(result);
Interlocked.Add(ref metrics.TotalTokensUsed, tokens);
}
_logger.LogInformation(
"Operation {OperationName} completed successfully in {Latency}ms",
operationName,
latency
);
return result;
}
catch (Exception ex)
{
stopwatch.Stop();
Interlocked.Increment(ref metrics.FailedRequests);
_logger.LogError(
ex,
"Operation {OperationName} failed after {Latency}ms",
operationName,
stopwatch.Elapsed.TotalMilliseconds
);
throw;
}
}
/// <summary>
/// Records a cache hit or miss
/// </summary>
public void RecordCacheResult(string operationName, bool isHit)
{
var metrics = _metrics.GetOrAdd(operationName, _ => new Metrics());
if (isHit)
{
Interlocked.Increment(ref metrics.CacheHits);
}
else
{
Interlocked.Increment(ref metrics.CacheMisses);
}
}
/// <summary>
/// Gets current metrics for an operation
/// </summary>
public Metrics GetMetrics(string operationName)
{
return _metrics.GetOrAdd(operationName, _ => new Metrics());
}
/// <summary>
/// Logs a summary of all metrics
/// </summary>
public void LogMetricsSummary()
{
foreach (var kvp in _metrics)
{
var m = kvp.Value;
double avgLatency = m.TotalRequests > 0
? m.TotalLatencyMs / m.TotalRequests
: 0;
double successRate = m.TotalRequests > 0
? (double)m.SuccessfulRequests / m.TotalRequests * 100
: 0;
double cacheHitRate = (m.CacheHits + m.CacheMisses) > 0
? (double)m.CacheHits / (m.CacheHits + m.CacheMisses) * 100
: 0;
_logger.LogInformation(
"Metrics for {Operation}: Total={Total}, Success={Success} ({SuccessRate:F2}%), " +
"Failed={Failed}, AvgLatency={AvgLatency:F2}ms, Tokens={Tokens}, " +
"CacheHitRate={CacheHitRate:F2}%",
kvp.Key,
m.TotalRequests,
m.SuccessfulRequests,
successRate,
m.FailedRequests,
avgLatency,
m.TotalTokensUsed,
cacheHitRate
);
}
}
}
}
This telemetry service tracks key metrics including request counts, success rates, latency, token usage, and cache hit rates. These metrics help you understand system performance, identify bottlenecks, and optimize costs. In production, you would integrate with monitoring platforms like Application Insights, Prometheus, or Datadog.
CONCLUSION
Building LLM applications with .NET on Windows provides a powerful combination of enterprise-grade development tools and cutting-edge AI capabilities. Throughout this article, we have explored the complete spectrum of LLM integration, from basic chat completions to sophisticated RAG and GraphRAG implementations.
The .NET ecosystem offers multiple integration pathways, each suited to different scenarios. Semantic Kernel provides a comprehensive orchestration framework ideal for complex applications requiring planning, memory, and function calling. Direct API integration offers maximum control for specialized requirements. The choice depends on your specific needs, team expertise, and architectural constraints.
Retrieval Augmented Generation represents a crucial technique for grounding LLM responses in specific knowledge bases. By combining vector search with document chunking and context injection, RAG dramatically improves accuracy for domain-specific questions. GraphRAG extends this further by incorporating structured knowledge graphs, enabling sophisticated multi-hop reasoning and relationship-based queries.
Fine-tuning provides another dimension of customization, allowing you to adapt models to specific tasks, tones, or domains. Whether using Azure OpenAI's fine-tuning service or local tools like Axolotl, the process involves careful data preparation, hyperparameter tuning, and evaluation. Fine-tuning complements RAG by encoding domain-specific patterns directly into model weights.
Production deployment requires attention to caching, rate limiting, error handling, and monitoring. These operational concerns ensure that your LLM application performs reliably, controls costs, and provides visibility into system behavior. The patterns and code examples provided in this article establish a foundation for building production-ready systems.
As you build LLM applications, remember that the field evolves rapidly. New models, techniques, and tools emerge constantly. The architectural patterns and best practices covered here provide a solid foundation that adapts to these changes. Focus on building maintainable, observable systems that can evolve as the technology landscape shifts.
The research chatbot example demonstrates how these concepts combine into a cohesive application. By integrating RAG for knowledge retrieval, GraphRAG for relationship understanding, and proper production patterns for reliability, you can build sophisticated AI-powered systems that deliver real value to users while maintaining the quality and reliability expected of enterprise software.
