Hitchhiker's Guide to AI, Software Architecture, and Everything Else: Building a Web-Based LLM Application for Creating and Managing AI Bots

Introduction and System Overview

Creating a web-based application that allows users to design, configure, and interact with custom AI bots represents a sophisticated engineering challenge that combines frontend development, backend architecture, prompt engineering, and hardware-accelerated machine learning inference. This article explores the comprehensive design and implementation of such a system, where users can create specialized AI assistants by defining their purpose, and the application automatically generates optimized prompts for various large language models running on diverse hardware platforms.

The fundamental architecture consists of several interconnected components. At the highest level, we have a web interface that users interact with through their browsers. This interface communicates with a backend server that manages bot configurations, LLM model loading, and inference requests. The backend must support multiple inference engines to accommodate different hardware accelerators including Nvidia CUDA-enabled GPUs, AMD GPUs with ROCm support, Intel GPUs, and Apple Silicon with Metal Performance Shaders. Additionally, the system must handle both locally hosted models and remote API-based models from providers like OpenAI, Anthropic, or other cloud services.

The workflow begins when a user specifies what kind of bot they want to create. For example, they might request a technical documentation assistant, a creative writing helper, or a code review bot. The application itself uses an LLM to analyze this request and generate a comprehensive prompt structure consisting of system instructions, assistant behavior guidelines, and user interaction patterns. This generated prompt is then stored and associated with the newly created bot. When the user later selects this bot for a conversation, the system loads the appropriate LLM model, retrieves the stored prompt configuration, and processes user inputs by integrating them into the prompt structure before sending them to the model for completion.

Core Components and Architecture

The system architecture follows a clean separation of concerns with distinct layers for presentation, business logic, and data persistence. The frontend layer handles user interactions and rendering, the application layer manages bot creation and conversation orchestration, the inference layer abstracts different LLM backends, and the persistence layer stores configurations and conversation histories.

The frontend component presents users with several key interface elements. A model selection dropdown allows users to choose which LLM they want to use, with options ranging from local models like Llama, Mistral, or Phi to remote services. A bot management section displays existing bots in a dropdown menu and provides controls for creating new bots. When creating a bot, users enter a description of the desired bot's purpose and capabilities. The conversation interface includes a text input area for user messages and a display area for bot responses that can render text, code blocks, and file attachments.

The backend server coordinates all operations and maintains state. It manages a registry of available LLM models with their associated hardware requirements and loading parameters. The bot configuration manager stores and retrieves bot definitions including their associated prompts and model preferences. The inference coordinator routes requests to the appropriate backend based on the selected model and handles response streaming. The prompt generator uses the currently selected LLM to create new bot prompts when users define new bots.

Hardware Acceleration and Model Loading

Supporting multiple hardware platforms requires careful abstraction and runtime detection of available accelerators. The system must detect what hardware is available on the host machine and configure the inference engine accordingly. For Nvidia GPUs, we use CUDA through libraries like PyTorch or the Transformers library. AMD GPUs require ROCm support with appropriate environment configuration. Intel GPUs can be accessed through Intel Extension for PyTorch or OpenVINO. Apple Silicon uses Metal Performance Shaders through the MLX framework or PyTorch with MPS backend.

The model loading process varies significantly depending on whether we are using a local model or a remote API. For local models, we need to download model weights, load them into memory, and configure the inference engine with the appropriate hardware backend. For remote models, we simply need to store API credentials and endpoint information.

Here is a simplified example of how we might structure the model configuration:

class ModelConfig:

def __init__(self, name, model_type, backend, model_path=None, api_key=None):

self.name = name

self.model_type = model_type # 'local' or 'remote'

self.backend = backend # 'cuda', 'rocm', 'mps', 'cpu', 'api'

self.model_path = model_path

self.api_key = api_key

self.loaded_model = None

self.tokenizer = None

def load(self):

if self.model_type == 'local':

self._load_local_model()

else:

self._initialize_api_client()

def _load_local_model(self):

import torch

from transformers import AutoModelForCausalLM, AutoTokenizer

device_map = self._get_device_map()

self.tokenizer = AutoTokenizer.from_pretrained(self.model_path)

self.loaded_model = AutoModelForCausalLM.from_pretrained(

self.model_path,

device_map=device_map,

torch_dtype=torch.float16

)

def _get_device_map(self):

import torch

if self.backend == 'cuda' and torch.cuda.is_available():

return 'cuda'

elif self.backend == 'mps' and torch.backends.mps.is_available():

return 'mps'

elif self.backend == 'rocm':

return 'cuda' # ROCm uses CUDA API compatibility

else:

return 'cpu'

This code demonstrates the basic structure for managing model configurations. The ModelConfig class encapsulates all information needed to load and use a model, whether local or remote. The load method dispatches to the appropriate loading function based on the model type. For local models, we use the Transformers library to load both the tokenizer and the model itself, with automatic device mapping based on available hardware.

The device detection and mapping is critical for performance. When a user selects a model, the system must determine what hardware is available and configure the model to use the fastest option. Nvidia CUDA provides the most mature ecosystem with extensive library support. AMD ROCm has improved significantly but requires specific driver and library versions. Apple MPS offers excellent performance on M-series chips but has some limitations in model compatibility. Intel GPUs are supported through extensions but may have reduced performance compared to dedicated ML accelerators.

Bot Creation and Prompt Generation

The bot creation process represents one of the most sophisticated aspects of the system. When a user wants to create a new bot, they provide a natural language description of what the bot should do. The application then uses the currently selected LLM to analyze this description and generate a comprehensive prompt structure.

The generated prompt structure consists of three main components. The system prompt defines the bot's role, capabilities, constraints, and behavioral guidelines. This is the foundational instruction that shapes how the bot interprets and responds to user inputs. The assistant context provides examples of ideal responses, formatting preferences, and interaction patterns. The user prompt template defines how actual user inputs will be integrated into the complete prompt sent to the model.

Consider a user who wants to create a technical documentation bot. They might enter a description like "Create a bot that helps write clear technical documentation for software projects, focusing on API documentation and user guides." The application would then generate a prompt structure similar to this:

generated_prompt = {

"system": """You are a technical documentation specialist with expertise in

creating clear, comprehensive documentation for software projects. Your primary

focus is on API documentation and user guides. You excel at explaining complex

technical concepts in accessible language while maintaining accuracy and precision.

Your documentation follows these principles:

- Start with clear purpose statements and overview sections

- Use consistent formatting and structure throughout

- Include practical code examples that demonstrate actual usage

- Explain parameters, return values, and potential errors thoroughly

- Provide context about when and why to use specific features

- Organize information hierarchically from general to specific

- Use active voice and present tense for clarity

- Include diagrams or visual aids when they enhance understanding

When documenting APIs, you always include:

- Function or method signatures with type information

- Detailed parameter descriptions including types and constraints

- Return value specifications with possible values or types

- Example usage demonstrating common scenarios

- Error conditions and exception handling guidance

- Related functions or methods for cross-referencing

For user guides, you ensure:

- Step-by-step instructions with clear progression

- Prerequisites and setup requirements stated upfront

- Screenshots or code snippets illustrating each step

- Troubleshooting sections for common issues

- Best practices and recommendations based on experience""",

"assistant_context": """When responding to documentation requests, I structure

my output to maximize clarity and usefulness. I begin with a brief overview, then

provide detailed sections organized logically. I use code blocks for examples,

ensuring they are complete and runnable rather than partial snippets.""",

"user_template": "User request: {user_input}\n\nPlease provide documentation

that addresses this request following the guidelines in your system instructions."

}

This example shows how the system transforms a simple user request into a detailed, structured prompt. The system prompt establishes the bot's expertise and provides comprehensive guidelines for behavior. The assistant context reinforces response patterns. The user template shows how actual user inputs will be integrated.

The prompt generation process itself requires careful engineering. The application must use the selected LLM to create these prompts, which means we are using an LLM to create instructions for itself or another LLM. This meta-level operation requires a specialized prompt for the prompt generator:

def generate_bot_prompt(user_description, model_config):

generator_prompt = f"""You are an expert at creating detailed, effective prompts

for large language models. A user wants to create a specialized AI bot with the

following description:

{user_description}

Generate a comprehensive prompt structure for this bot consisting of:

1. A detailed system prompt that defines the bot's role, expertise, behavioral

guidelines, output formatting rules, and any relevant constraints or principles

it should follow.

2. An assistant context section that provides examples or patterns of ideal

responses and interaction styles.

3. A user prompt template that shows how actual user inputs should be integrated

into the complete prompt.

Make the prompts thorough and specific. Include concrete guidelines rather than

vague instructions. The resulting bot should be highly capable in its specialized

domain.

Format your response as a JSON object with keys: system, assistant_context, and

user_template."""

response = model_config.generate(generator_prompt)

import json

prompt_structure = json.loads(response)

return prompt_structure

This function encapsulates the prompt generation logic. It takes the user's description and the current model configuration, constructs a meta-prompt that instructs the LLM to create a bot prompt, generates the response, and parses it into a structured format. The returned prompt structure is then stored and associated with the new bot.

Bot and Model Persistence

The system must persist both bot configurations and model settings across sessions. This requires a storage mechanism that can handle structured data and support efficient retrieval. While a production system might use a database, a file-based approach using JSON provides simplicity and portability.

Bot configurations are stored in a JSON file that contains an array of bot objects. Each bot object includes a unique identifier, a name, a description, the generated prompt structure, and the associated model name. When the application starts, it loads this file into memory. When users create or modify bots, the in-memory structure is updated and the file is rewritten.

class BotManager:

def __init__(self, storage_path='bots.json'):

self.storage_path = storage_path

self.bots = {}

self.load_bots()

def load_bots(self):

import json

import os

if os.path.exists(self.storage_path):

with open(self.storage_path, 'r', encoding='utf-8') as f:

bot_list = json.load(f)

for bot_data in bot_list:

bot = Bot(

bot_id=bot_data['id'],

name=bot_data['name'],

description=bot_data['description'],

prompt_structure=bot_data['prompt_structure'],

model_name=bot_data['model_name']

)

self.bots[bot.bot_id] = bot

def save_bots(self):

import json

bot_list = []

for bot in self.bots.values():

bot_list.append({

'id': bot.bot_id,

'name': bot.name,

'description': bot.description,

'prompt_structure': bot.prompt_structure,

'model_name': bot.model_name

})

with open(self.storage_path, 'w', encoding='utf-8') as f:

json.dump(bot_list, f, indent=2, ensure_ascii=False)

def create_bot(self, name, description, prompt_structure, model_name):

import uuid

bot_id = str(uuid.uuid4())

bot = Bot(bot_id, name, description, prompt_structure, model_name)

self.bots[bot_id] = bot

self.save_bots()

return bot

def get_bot(self, bot_id):

return self.bots.get(bot_id)

def list_bots(self):

return list(self.bots.values())

The BotManager class handles all bot persistence operations. The load_bots method reads the JSON file and reconstructs bot objects in memory. The save_bots method serializes the current bot collection back to the file. The create_bot method generates a unique identifier, creates a new bot object, adds it to the collection, and persists the changes. This pattern ensures that all bot data is preserved across application restarts.

Similarly, model configurations are stored in a separate JSON file. This file contains information about available models including their names, types (local or remote), backend requirements, file paths for local models, and API credentials for remote models.

The model manager loads this configuration at startup and provides methods for selecting and loading models.

class ModelManager:

def __init__(self, config_path='models.json'):

self.config_path = config_path

self.models = {}

self.current_model = None

self.load_model_configs()

def load_model_configs(self):

import json

import os

if os.path.exists(self.config_path):

with open(self.config_path, 'r', encoding='utf-8') as f:

model_list = json.load(f)

for model_data in model_list:

config = ModelConfig(

name=model_data['name'],

model_type=model_data['type'],

backend=model_data['backend'],

model_path=model_data.get('path'),

api_key=model_data.get('api_key')

)

self.models[config.name] = config

def select_model(self, model_name):

if model_name in self.models:

if self.current_model:

self.current_model.unload()

self.current_model = self.models[model_name]

self.current_model.load()

return True

return False

def get_current_model(self):

return self.current_model

def list_models(self):

return list(self.models.keys())

The ModelManager provides a clean interface for managing model configurations. The select_model method handles the transition between models, unloading the previous model if one was loaded and loading the newly selected model. This ensures efficient memory usage and prevents conflicts between different model instances.

Conversation Flow and Prompt Integration

When a user selects a bot and enters a message, the system must integrate that message into the bot's prompt structure and generate a completion. This process involves several steps that must be coordinated carefully to ensure the conversation maintains context and the bot behaves according to its defined characteristics.

The conversation flow begins when the user selects a bot from the dropdown menu. The application retrieves the bot's configuration including its prompt structure and associated model name. If the currently loaded model differs from the bot's preferred model, the system prompts the user to switch models or proceeds with the current model based on configuration settings.

Once the correct model is loaded, the user can enter messages in the text input field. When the user submits a message, the application constructs the complete prompt by combining the bot's system prompt, assistant context, conversation history, and the new user message. This complete prompt is then sent to the LLM for completion.

class ConversationManager:

def __init__(self, bot, model_manager):

self.bot = bot

self.model_manager = model_manager

self.conversation_history = []

def process_user_message(self, user_message):

# Construct the complete prompt

complete_prompt = self._build_complete_prompt(user_message)

# Get the current model

model = self.model_manager.get_current_model()

# Generate completion

response = model.generate(complete_prompt)

# Update conversation history

self.conversation_history.append({

'role': 'user',

'content': user_message

})

self.conversation_history.append({

'role': 'assistant',

'content': response

})

return response

def _build_complete_prompt(self, user_message):

# Start with system prompt

messages = [

{

'role': 'system',

'content': self.bot.prompt_structure['system']

}

]

# Add assistant context if present

if self.bot.prompt_structure.get('assistant_context'):

messages.append({

'role': 'assistant',

'content': self.bot.prompt_structure['assistant_context']

})

# Add conversation history

messages.extend(self.conversation_history)

# Add current user message using template

user_template = self.bot.prompt_structure.get('user_template', '{user_input}')

formatted_message = user_template.format(user_input=user_message)

messages.append({

'role': 'user',

'content': formatted_message

})

return messages

The ConversationManager handles the orchestration of conversations with a specific bot. The process_user_message method is the main entry point that takes a user message, builds the complete prompt, generates a response, and updates the conversation history. The _build_complete_prompt method constructs the message array in the format expected by most LLM APIs, starting with the system prompt, adding the assistant context, including previous conversation turns, and finally appending the new user message.

The conversation history is crucial for maintaining context across multiple turns. Each user message and assistant response is stored as a dictionary with role and content fields. This history is included in subsequent prompts so the model can reference previous exchanges and maintain coherent, contextual responses.

Handling Model Responses and Attachments

LLM responses can include various types of content beyond simple text. Modern models can generate code, structured data, and even references to files or images. The system must parse these responses and present them appropriately to the user.

The response handling logic examines the completion text for special markers or structured formats. Code blocks are identified by markdown-style triple backticks and rendered with syntax highlighting. JSON or XML structures are formatted for readability. File references or attachment indicators trigger download or display mechanisms.

class ResponseHandler:

def __init__(self):

self.attachment_handlers = {

'code': self._handle_code_block,

'json': self._handle_json_data,

'file': self._handle_file_reference

}

def process_response(self, response_text):

processed_response = {

'text': '',

'code_blocks': [],

'attachments': []

}

# Extract code blocks

import re

code_pattern = r'```(\w+)?\n(.*?)```'

code_blocks = re.findall(code_pattern, response_text, re.DOTALL)

for language, code in code_blocks:

processed_response['code_blocks'].append({

'language': language or 'text',

'code': code.strip()

})

# Remove code blocks from main text

processed_response['text'] = re.sub(code_pattern, '[CODE BLOCK]', response_text, flags=re.DOTALL)

# Look for file references or attachment markers

attachment_pattern = r'\[ATTACHMENT:(.*?)\]'

attachments = re.findall(attachment_pattern, response_text)

for attachment in attachments:

processed_response['attachments'].append({

'type': 'file',

'reference': attachment

})

processed_response['text'] = re.sub(attachment_pattern, '', processed_response['text'])

return processed_response

def _handle_code_block(self, language, code):

return {

'type': 'code',

'language': language,

'content': code

}

def _handle_json_data(self, json_str):

import json

try:

data = json.loads(json_str)

return {

'type': 'json',

'content': data

}

except json.JSONDecodeError:

return {

'type': 'text',

'content': json_str

}

def _handle_file_reference(self, file_ref):

return {

'type': 'file',

'reference': file_ref

}

The ResponseHandler class provides methods for parsing and structuring LLM responses. The process_response method uses regular expressions to identify code blocks and attachment markers, extracting them into separate fields while preserving the main text content. This structured representation allows the frontend to render different content types appropriately.

Code blocks are particularly important for technical bots. The system identifies the programming language from the code fence marker and can apply syntax highlighting in the user interface. For example, a Python code block would be rendered with appropriate color coding for keywords, strings, and comments.

Attachments represent a more complex challenge. In some cases, the LLM might generate data that should be downloadable as a file, such as a CSV dataset or a configuration file. The system can detect these cases and create downloadable files on the fly. Other times, the model might reference external resources that need to be fetched and displayed.

Web Interface Implementation

The web interface serves as the primary interaction point for users. It must be responsive, intuitive, and capable of handling real-time updates as LLM responses stream in. Modern web frameworks like React, Vue, or Svelte provide excellent tools for building such interfaces, but the core functionality can also be implemented with vanilla JavaScript and HTML.

The interface layout consists of several key sections. At the top, a navigation bar contains the model selector dropdown and bot selector dropdown. The main content area is divided into a conversation display panel and an input panel. A sidebar might contain bot management controls including the create new bot button and options for editing existing bots.

<!DOCTYPE html>

<head>

<title>LLM Bot Manager</title>

<style>

body {

font-family: Arial, sans-serif;

margin: 0;

padding: 0;

display: flex;

flex-direction: column;

height: 100vh;

}

.navbar {

background-color: #333;

color: white;

padding: 15px;

display: flex;

gap: 20px;

align-items: center;

}

.navbar select {

padding: 8px;

font-size: 14px;

border-radius: 4px;

}

.main-container {

display: flex;

flex: 1;

overflow: hidden;

}

.sidebar {

width: 250px;

background-color: #f5f5f5;

padding: 20px;

border-right: 1px solid #ddd;

overflow-y: auto;

}

.content {

flex: 1;

display: flex;

flex-direction: column;

}

.conversation {

flex: 1;

overflow-y: auto;

padding: 20px;

background-color: #fafafa;

}

.message {

margin-bottom: 20px;

padding: 15px;

border-radius: 8px;

max-width: 80%;

}

.message.user {

background-color: #e3f2fd;

margin-left: auto;

}

.message.assistant {

background-color: white;

border: 1px solid #ddd;

}

.input-panel {

padding: 20px;

background-color: white;

border-top: 1px solid #ddd;

}

.input-panel textarea {

width: 100%;

padding: 12px;

border: 1px solid #ddd;

border-radius: 4px;

font-size: 14px;

resize: vertical;

min-height: 80px;

}

.input-panel button {

margin-top: 10px;

padding: 10px 20px;

background-color: #007bff;

color: white;

border: none;

border-radius: 4px;

cursor: pointer;

font-size: 14px;

}

.input-panel button:hover {

background-color: #0056b3;

}

.code-block {

background-color: #f4f4f4;

border: 1px solid #ddd;

border-radius: 4px;

padding: 12px;

margin: 10px 0;

overflow-x: auto;

font-family: 'Courier New', monospace;

font-size: 13px;

}

</style>

</head>

<body>

<label>Model:

<option value="">Select Model</option>

</select>

</label>

<label>Bot:

<option value="">Select Bot</option>

</select>

</label>

</div>

<h3>Bot Management</h3>

<button id="createBotBtn">Create New Bot</button>

</div>

</div>

</body>

</html>

This HTML structure provides the foundation for the user interface. The navbar contains the model and bot selectors that allow users to switch between different configurations.

The sidebar provides bot management functionality. The main content area is split between the conversation display and the input panel where users type their messages.

The JavaScript code that powers this interface handles user interactions, communicates with the backend server, and updates the display dynamically. When a user selects a bot, the code fetches the bot's configuration and loads the associated model. When a user sends a message, the code posts it to the backend and displays the response.

class LLMBotApp {

constructor() {

this.currentBot = null;

this.currentModel = null;

this.conversationHistory = [];

this.initializeEventListeners();

this.loadModels();

this.loadBots();

}

initializeEventListeners() {

document.getElementById('modelSelector').addEventListener('change', (e) => {

this.selectModel(e.target.value);

});

document.getElementById('botSelector').addEventListener('change', (e) => {

this.selectBot(e.target.value);

});

document.getElementById('sendBtn').addEventListener('click', () => {

this.sendMessage();

});

document.getElementById('userInput').addEventListener('keypress', (e) => {

if (e.key === 'Enter' && e.ctrlKey) {

this.sendMessage();

}

});

document.getElementById('createBotBtn').addEventListener('click', () => {

this.showCreateBotDialog();

});

}

async loadModels() {

try {

const response = await fetch('/api/models');

const models = await response.json();

const selector = document.getElementById('modelSelector');

selector.innerHTML = '<option value="">Select Model</option>';

models.forEach(model => {

const option = document.createElement('option');

option.value = model.name;

option.textContent = model.name;

selector.appendChild(option);

});

} catch (error) {

console.error('Failed to load models:', error);

}

async loadBots() {

try {

const response = await fetch('/api/bots');

const bots = await response.json();

const selector = document.getElementById('botSelector');

selector.innerHTML = '<option value="">Select Bot</option>';

bots.forEach(bot => {

const option = document.createElement('option');

option.value = bot.id;

option.textContent = bot.name;

selector.appendChild(option);

});

} catch (error) {

console.error('Failed to load bots:', error);

}

async selectModel(modelName) {

if (!modelName) return;

try {

const response = await fetch('/api/models/select', {

method: 'POST',

headers: {

'Content-Type': 'application/json'

body: JSON.stringify({ model_name: modelName })

});

if (response.ok) {

this.currentModel = modelName;

console.log('Model selected:', modelName);

}

} catch (error) {

console.error('Failed to select model:', error);

}

async selectBot(botId) {

if (!botId) return;

try {

const response = await fetch(`/api/bots/${botId}`);

const bot = await response.json();

this.currentBot = bot;

this.conversationHistory = [];

// Clear conversation display

document.getElementById('conversationArea').innerHTML = '';

// Load bot's preferred model if different from current

if (bot.model_name !== this.currentModel) {

document.getElementById('modelSelector').value = bot.model_name;

await this.selectModel(bot.model_name);

}

console.log('Bot selected:', bot.name);

} catch (error) {

console.error('Failed to select bot:', error);

}

async sendMessage() {

const input = document.getElementById('userInput');

const message = input.value.trim();

if (!message || !this.currentBot) return;

// Display user message

this.displayMessage('user', message);

// Clear input

input.value = '';

try {

const response = await fetch('/api/chat', {

method: 'POST',

headers: {

'Content-Type': 'application/json'

body: JSON.stringify({

bot_id: this.currentBot.id,

message: message,

history: this.conversationHistory

})

});

const data = await response.json();

// Display assistant response

this.displayMessage('assistant', data.response);

// Update conversation history

this.conversationHistory.push({

role: 'user',

content: message

});

this.conversationHistory.push({

role: 'assistant',

content: data.response

});

} catch (error) {

console.error('Failed to send message:', error);

this.displayMessage('assistant', 'Error: Failed to get response');

}

displayMessage(role, content) {

const conversationArea = document.getElementById('conversationArea');

const messageDiv = document.createElement('div');

messageDiv.className = `message ${role}`;

// Process content for code blocks

const processedContent = this.processContent(content);

messageDiv.innerHTML = processedContent;

conversationArea.appendChild(messageDiv);

conversationArea.scrollTop = conversationArea.scrollHeight;

}

processContent(content) {

// Replace code blocks with formatted divs

const codeBlockRegex = /```(\w+)?\n([\s\S]*?)```/g;

let processed = content.replace(codeBlockRegex, (match, language, code) => {

return `<div class="code-block"><strong>${language || 'code'}</strong><pre>${this.escapeHtml(code.trim())}</pre></div>`;

});

// Convert newlines to br tags for remaining text

processed = processed.replace(/\n/g, '<br>');

return processed;

}

escapeHtml(text) {

const div = document.createElement('div');

div.textContent = text;

return div.innerHTML;

}

showCreateBotDialog() {

const description = prompt('Enter a description for your new bot:');

if (!description) return;

const name = prompt('Enter a name for your bot:');

if (!name) return;

this.createBot(name, description);

}

async createBot(name, description) {

try {

const response = await fetch('/api/bots/create', {

method: 'POST',

headers: {

'Content-Type': 'application/json'

body: JSON.stringify({

description: description,

model_name: this.currentModel

})

});

const bot = await response.json();

// Reload bots list

await this.loadBots();

// Select the newly created bot

document.getElementById('botSelector').value = bot.id;

await this.selectBot(bot.id);

alert('Bot created successfully!');

} catch (error) {

console.error('Failed to create bot:', error);

alert('Failed to create bot');

}

// Initialize the application when the page loads

document.addEventListener('DOMContentLoaded', () => {

new LLMBotApp();

});

This JavaScript code implements the client-side application logic. The LLMBotApp class manages the application state including the current bot and model selections. The initializeEventListeners method sets up handlers for user interactions like selecting models, selecting bots, and sending messages. The loadModels and loadBots methods fetch available options from the backend and populate the dropdown menus. The selectModel and selectBot methods handle switching between different configurations. The sendMessage method posts user messages to the backend and displays responses. The displayMessage method renders messages in the conversation area with proper formatting for code blocks and other special content.

Backend Server Implementation

The backend server coordinates all system operations and provides API endpoints for the frontend to interact with. It manages model loading, bot configurations, prompt generation, and inference requests. A Python-based server using Flask or FastAPI provides a clean, efficient implementation.

The server exposes several key endpoints. The GET /api/models endpoint returns a list of available models. The POST /api/models/select endpoint loads a specified model. The GET /api/bots endpoint returns all configured bots. The GET /api/bots/{bot_id} endpoint returns details for a specific bot. The POST /api/bots/create endpoint creates a new bot by generating its prompt structure. The POST /api/chat endpoint processes user messages and returns completions.

from flask import Flask, request, jsonify

from flask_cors import CORS

import json

import os

app = Flask(__name__)

CORS(app)

# Initialize managers

model_manager = ModelManager('models.json')

bot_manager = BotManager('bots.json')

response_handler = ResponseHandler()

@app.route('/api/models', methods=['GET'])

def get_models():

models = model_manager.list_models()

return jsonify([{'name': name} for name in models])

@app.route('/api/models/select', methods=['POST'])

def select_model():

data = request.json

model_name = data.get('model_name')

if model_manager.select_model(model_name):

return jsonify({'status': 'success', 'model': model_name})

else:

return jsonify({'status': 'error', 'message': 'Model not found'}), 404

@app.route('/api/bots', methods=['GET'])

def get_bots():

bots = bot_manager.list_bots()

return jsonify([{

'id': bot.bot_id,

'name': bot.name,

'description': bot.description,

'model_name': bot.model_name

} for bot in bots])

@app.route('/api/bots/<bot_id>', methods=['GET'])

def get_bot(bot_id):

bot = bot_manager.get_bot(bot_id)

if bot:

return jsonify({

'id': bot.bot_id,

'name': bot.name,

'description': bot.description,

'model_name': bot.model_name,

'prompt_structure': bot.prompt_structure

})

else:

return jsonify({'error': 'Bot not found'}), 404

@app.route('/api/bots/create', methods=['POST'])

def create_bot():

data = request.json

name = data.get('name')

description = data.get('description')

model_name = data.get('model_name')

# Generate prompt structure using current model

current_model = model_manager.get_current_model()

if not current_model:

return jsonify({'error': 'No model selected'}), 400

prompt_structure = generate_bot_prompt(description, current_model)

# Create and save bot

bot = bot_manager.create_bot(name, description, prompt_structure, model_name)

return jsonify({

'id': bot.bot_id,

'name': bot.name,

'description': bot.description,

'model_name': bot.model_name

})

@app.route('/api/chat', methods=['POST'])

def chat():

data = request.json

bot_id = data.get('bot_id')

message = data.get('message')

history = data.get('history', [])

bot = bot_manager.get_bot(bot_id)

if not bot:

return jsonify({'error': 'Bot not found'}), 404

# Create conversation manager

conversation_manager = ConversationManager(bot, model_manager)

conversation_manager.conversation_history = history

# Process message

response = conversation_manager.process_user_message(message)

# Process response for attachments and formatting

processed_response = response_handler.process_response(response)

return jsonify({

'response': response,

'processed': processed_response

})

if __name__ == '__main__':

app.run(host='0.0.0.0', port=5000, debug=True)

This Flask server provides the complete backend API. Each endpoint handles a specific operation and returns JSON responses. The get_models endpoint retrieves the list of available models from the model manager. The select_model endpoint loads a specified model into memory. The get_bots endpoint returns all configured bots. The get_bot endpoint retrieves details for a specific bot. The create_bot endpoint generates a new bot by using the current model to create a prompt structure. The chat endpoint processes user messages by constructing the complete prompt and generating a completion.

The server maintains instances of the ModelManager, BotManager, and ResponseHandler classes that were defined earlier. These managers handle the underlying operations while the Flask routes provide the HTTP interface.

Advanced Features and Optimizations

Several advanced features can enhance the system's capabilities and user experience. Streaming responses allow users to see completions as they are generated rather than waiting for the entire response. Conversation branching enables users to explore alternative responses by rewinding to earlier points in the conversation. Model quantization reduces memory requirements for local models. Caching frequently used prompts and responses improves performance.

Streaming responses require modifications to both the backend and frontend. The backend must generate tokens incrementally and send them to the client as they become available. The frontend must receive these partial responses and update the display in real time.

from flask import Response

import json

@app.route('/api/chat/stream', methods=['POST'])

def chat_stream():

data = request.json

bot_id = data.get('bot_id')

message = data.get('message')

history = data.get('history', [])

bot = bot_manager.get_bot(bot_id)

if not bot:

return jsonify({'error': 'Bot not found'}), 404

def generate():

conversation_manager = ConversationManager(bot, model_manager)

conversation_manager.conversation_history = history

complete_prompt = conversation_manager._build_complete_prompt(message)

model = model_manager.get_current_model()

# Stream tokens

for token in model.generate_stream(complete_prompt):

yield f"data: {json.dumps({'token': token})}\n\n"

yield f"data: {json.dumps({'done': True})}\n\n"

return Response(generate(), mimetype='text/event-stream')

This streaming endpoint uses Server-Sent Events to push tokens to the client as they are generated. The generate function yields each token wrapped in the SSE format. The frontend can consume this stream using the EventSource API.

Model quantization reduces the precision of model weights from 32-bit or 16-bit floating point to 8-bit or even 4-bit integers. This dramatically reduces memory usage and can improve inference speed on some hardware. Libraries like bitsandbytes provide quantization support for PyTorch models.

def _load_local_model_quantized(self):

import torch

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(

load_in_4bit=True,

bnb_4bit_compute_dtype=torch.float16,

bnb_4bit_use_double_quant=True,

bnb_4bit_quant_type="nf4"

)

device_map = self._get_device_map()

self.tokenizer = AutoTokenizer.from_pretrained(self.model_path)

self.loaded_model = AutoModelForCausalLM.from_pretrained(

self.model_path,

quantization_config=quantization_config,

device_map=device_map

)

This modified loading function applies 4-bit quantization to the model weights. The quantization configuration specifies the quantization type, compute dtype, and whether to use double quantization for additional compression. This can reduce a 13 billion parameter model from 26GB to under 7GB of memory.

Caching can significantly improve performance for repeated operations. When a user creates a bot, the generated prompt structure can be cached so that subsequent requests for the same bot description return immediately. Similarly, common user queries might be cached with their responses.

class PromptCache:

def __init__(self, cache_file='prompt_cache.json'):

self.cache_file = cache_file

self.cache = {}

self.load_cache()

def load_cache(self):

import os

if os.path.exists(self.cache_file):

with open(self.cache_file, 'r', encoding='utf-8') as f:

self.cache = json.load(f)

def save_cache(self):

with open(self.cache_file, 'w', encoding='utf-8') as f:

json.dump(self.cache, f, indent=2, ensure_ascii=False)

def get(self, key):

return self.cache.get(key)

def set(self, key, value):

self.cache[key] = value

self.save_cache()

def generate_key(self, description, model_name):

import hashlib

content = f"{description}|{model_name}"

return hashlib.sha256(content.encode()).hexdigest()

The PromptCache class provides a simple file-based caching mechanism. The generate_key method creates a hash of the bot description and model name to use as a cache key. Before generating a new prompt, the system checks the cache. If a matching prompt exists, it is returned immediately. Otherwise, the prompt is generated and stored in the cache for future use.

Security and Error Handling

A production system must implement robust security measures and comprehensive error handling. User inputs must be validated and sanitized to prevent injection attacks. API endpoints should require authentication and authorization. Model outputs should be filtered for potentially harmful content. Error conditions must be handled gracefully with informative messages.

Input validation ensures that user-provided data meets expected formats and constraints. For bot descriptions, we might limit the length and check for malicious patterns. For model selections, we verify that the requested model exists and is accessible.

class InputValidator:

@staticmethod

def validate_bot_description(description):

if not description or not isinstance(description, str):

raise ValueError("Description must be a non-empty string")

if len(description) > 5000:

raise ValueError("Description too long (max 5000 characters)")

# Check for potentially malicious patterns

dangerous_patterns = ['<script', 'javascript:', 'onerror=']

description_lower = description.lower()

for pattern in dangerous_patterns:

if pattern in description_lower:

raise ValueError("Description contains disallowed content")

return description.strip()

@staticmethod

def validate_model_name(model_name, available_models):

if not model_name or not isinstance(model_name, str):

raise ValueError("Model name must be a non-empty string")

if model_name not in available_models:

raise ValueError(f"Model '{model_name}' not found")

return model_name

@staticmethod

def validate_message(message):

if not message or not isinstance(message, str):

raise ValueError("Message must be a non-empty string")

if len(message) > 10000:

raise ValueError("Message too long (max 10000 characters)")

return message.strip()

The InputValidator class provides static methods for validating different types of user input. Each validation method checks the input type, length, and content for potential issues. If validation fails, a ValueError is raised with a descriptive message.

API authentication can be implemented using tokens or session-based authentication. For a simple implementation, we might use API keys that clients include in request headers.

from functools import wraps

from flask import request

def require_api_key(f):

@wraps(f)

def decorated_function(*args, **kwargs):

api_key = request.headers.get('X-API-Key')

if not api_key:

return jsonify({'error': 'API key required'}), 401

# Validate API key (in production, check against database)

valid_keys = load_api_keys()

if api_key not in valid_keys:

return jsonify({'error': 'Invalid API key'}), 403

return f(*args, **kwargs)

return decorated_function

@app.route('/api/chat', methods=['POST'])

@require_api_key

def chat():

# Endpoint implementation

pass

The require_api_key decorator checks for the presence and validity of an API key in the request headers. If the key is missing or invalid, the request is rejected with an appropriate error code. This decorator can be applied to any endpoint that requires authentication.

Error handling should catch exceptions at multiple levels and provide meaningful feedback to users. Database errors, model loading failures, and inference errors should all be handled gracefully.

@app.errorhandler(Exception)

def handle_error(error):

import traceback

# Log the full error for debugging

app.logger.error(f"Error: {str(error)}\n{traceback.format_exc()}")

# Return a safe error message to the client

if isinstance(error, ValueError):

return jsonify({'error': str(error)}), 400

elif isinstance(error, FileNotFoundError):

return jsonify({'error': 'Resource not found'}), 404

else:

return jsonify({'error': 'Internal server error'}), 500

@app.route('/api/bots/create', methods=['POST'])

def create_bot():

try:

data = request.json

# Validate inputs

name = InputValidator.validate_bot_description(data.get('name', ''))

description = InputValidator.validate_bot_description(data.get('description', ''))

model_name = InputValidator.validate_model_name(

data.get('model_name', ''),

model_manager.list_models()

)

# Generate prompt structure

current_model = model_manager.get_current_model()

if not current_model:

raise ValueError('No model selected')

prompt_structure = generate_bot_prompt(description, current_model)

# Create bot

bot = bot_manager.create_bot(name, description, prompt_structure, model_name)

return jsonify({

'id': bot.bot_id,

'name': bot.name,

'description': bot.description,

'model_name': bot.model_name

})

except ValueError as e:

return jsonify({'error': str(e)}), 400

except Exception as e:

app.logger.error(f"Failed to create bot: {str(e)}")

return jsonify({'error': 'Failed to create bot'}), 500

This error handling implementation uses try-except blocks to catch exceptions and return appropriate HTTP status codes. The global error handler logs detailed error information for debugging while returning safe messages to clients. Individual endpoints catch specific exceptions and provide context-appropriate error responses.

Testing and Deployment

Comprehensive testing ensures the system functions correctly across different scenarios and configurations. Unit tests verify individual components like the ModelManager and BotManager. Integration tests check that components work together properly. End-to-end tests simulate complete user workflows from bot creation through conversation.

import unittest

from unittest.mock import Mock, patch

class TestBotManager(unittest.TestCase):

def setUp(self):

self.bot_manager = BotManager('test_bots.json')

def tearDown(self):

import os

if os.path.exists('test_bots.json'):

os.remove('test_bots.json')

def test_create_bot(self):

prompt_structure = {

'system': 'Test system prompt',

'assistant_context': 'Test context',

'user_template': '{user_input}'

}

bot = self.bot_manager.create_bot(

'Test Bot',

'A test bot',

prompt_structure,

'test-model'

)

self.assertIsNotNone(bot.bot_id)

self.assertEqual(bot.name, 'Test Bot')

self.assertEqual(bot.description, 'A test bot')

self.assertEqual(bot.model_name, 'test-model')

def test_get_bot(self):

prompt_structure = {

'system': 'Test system prompt',

'assistant_context': 'Test context',

'user_template': '{user_input}'

}

created_bot = self.bot_manager.create_bot(

'Test Bot',

'A test bot',

prompt_structure,

'test-model'

)

retrieved_bot = self.bot_manager.get_bot(created_bot.bot_id)

self.assertIsNotNone(retrieved_bot)

self.assertEqual(retrieved_bot.bot_id, created_bot.bot_id)

self.assertEqual(retrieved_bot.name, created_bot.name)

def test_list_bots(self):

prompt_structure = {

'system': 'Test system prompt',

'assistant_context': 'Test context',

'user_template': '{user_input}'

}

self.bot_manager.create_bot('Bot 1', 'First bot', prompt_structure, 'model-1')

self.bot_manager.create_bot('Bot 2', 'Second bot', prompt_structure, 'model-2')

bots = self.bot_manager.list_bots()

self.assertEqual(len(bots), 2)

self.assertEqual(bots[0].name, 'Bot 1')

self.assertEqual(bots[1].name, 'Bot 2')

if __name__ == '__main__':

unittest.main()

These unit tests verify the BotManager functionality. The setUp method creates a test instance with a temporary storage file. The tearDown method cleans up the test file. Each test method verifies a specific aspect of the BotManager's behavior. The test_create_bot method ensures bots are created with correct attributes. The test_get_bot method verifies bot retrieval. The test_list_bots method checks that all bots are returned correctly.

Deployment considerations depend on the target environment. For local deployment, the system can run on a single machine with the backend server, model files, and web interface all hosted locally. For cloud deployment, the backend might run on a server instance with GPU support while the frontend is served through a content delivery network. Docker containers can package the application with all dependencies for consistent deployment across environments.

FROM python:3.10-slim

WORKDIR /app

# Install system dependencies

RUN apt-get update && apt-get install -y \

build-essential \

curl \

&& rm -rf /var/lib/apt/lists/*

# Copy requirements

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

# Copy application code

COPY . .

# Create directories for data persistence

RUN mkdir -p /app/data

# Expose port

EXPOSE 5000

# Run the application

CMD ["python", "server.py"]

This Dockerfile creates a container image for the backend server. It starts from a Python base image, installs system dependencies, copies the application code, and sets up the runtime environment. The container exposes port 5000 for the Flask server. Data directories are created for persistent storage of bot and model configurations.

A docker-compose configuration can orchestrate multiple containers for a complete deployment:

version: '3.8'

services:

backend:

build: ./backend

ports:

- "5000:5000"

volumes:

- ./data:/app/data

- ./models:/app/models

environment:

- FLASK_ENV=production

deploy:

resources:

reservations:

devices:

- driver: nvidia

capabilities: [gpu]

frontend:

build: ./frontend

ports:

- "80:80"

depends_on:

- backend

This docker-compose file defines two services. The backend service builds from the backend directory, exposes port 5000, mounts volumes for data and model persistence, and requests GPU access for model inference. The frontend service builds from the frontend directory and exposes port 80 for web access. The depends_on directive ensures the backend starts before the frontend.

Performance Optimization

Performance optimization focuses on reducing latency, improving throughput, and minimizing resource usage. Several strategies can enhance system performance. Model optimization through quantization and pruning reduces memory footprint and inference time. Batching multiple requests together improves GPU utilization. Caching frequently accessed data reduces redundant computations. Asynchronous processing allows the server to handle multiple requests concurrently.

Quantization was discussed earlier as a memory optimization technique, but it also improves inference speed. By reducing the precision of model weights and activations, quantized models require fewer memory operations and can leverage specialized hardware instructions for integer arithmetic.

Request batching groups multiple inference requests and processes them together. This is particularly effective for GPU inference where parallel processing capabilities are underutilized by single requests.

import asyncio

from collections import deque

import time

class BatchInferenceManager:

def __init__(self, model, max_batch_size=8, max_wait_time=0.1):

self.model = model

self.max_batch_size = max_batch_size

self.max_wait_time = max_wait_time

self.request_queue = deque()

self.processing = False

async def add_request(self, prompt):

future = asyncio.Future()

self.request_queue.append((prompt, future, time.time()))

if not self.processing:

asyncio.create_task(self.process_batch())

return await future

async def process_batch(self):

self.processing = True

while self.request_queue:

batch = []

futures = []

# Collect requests for batch

while len(batch) < self.max_batch_size and self.request_queue:

prompt, future, timestamp = self.request_queue[0]

# Check if we should wait for more requests

if len(batch) > 0 and time.time() - timestamp < self.max_wait_time:

if len(self.request_queue) < self.max_batch_size:

await asyncio.sleep(0.01)

continue

self.request_queue.popleft()

batch.append(prompt)

futures.append(future)

if batch:

# Process batch

results = await self.model.generate_batch(batch)

# Return results to waiting requests

for future, result in zip(futures, results):

future.set_result(result)

self.processing = False

The BatchInferenceManager collects incoming requests and processes them in batches. The add_request method adds a new request to the queue and returns a future that will be resolved when the result is ready. The process_batch method collects requests up to the maximum batch size or until the maximum wait time is exceeded, processes them together, and returns the results to the waiting requests. This approach significantly improves throughput when multiple users are interacting with the system simultaneously.

Asynchronous request handling allows the server to process multiple requests concurrently without blocking. FastAPI provides excellent support for asynchronous endpoints.

from fastapi import FastAPI, HTTPException

from pydantic import BaseModel

import asyncio

app = FastAPI()

class ChatRequest(BaseModel):

bot_id: str

message: str

history: list = []

class ChatResponse(BaseModel):

response: str

processed: dict

@app.post('/api/chat', response_model=ChatResponse)

async def chat(request: ChatRequest):

bot = bot_manager.get_bot(request.bot_id)

if not bot:

raise HTTPException(status_code=404, detail='Bot not found')

conversation_manager = ConversationManager(bot, model_manager)

conversation_manager.conversation_history = request.history

# Process message asynchronously

response = await asyncio.to_thread(

conversation_manager.process_user_message,

request.message

)

processed_response = response_handler.process_response(response)

return ChatResponse(

response=response,

processed=processed_response

)

This FastAPI endpoint uses async/await syntax to handle requests asynchronously. The asyncio.to_thread function runs the blocking model inference in a thread pool, allowing the event loop to process other requests while waiting for the inference to complete. This approach maximizes server throughput and responsiveness.

Conclusion and Future Directions

Building a comprehensive web-based LLM application for creating and managing AI bots involves integrating multiple complex systems including web interfaces, backend servers, machine learning models, and hardware acceleration. The architecture presented in this article provides a solid foundation that supports both local and remote models across diverse hardware platforms.

The key components work together to provide a seamless user experience. Users can select from available LLM models, create specialized bots by describing their purpose, and interact with these bots through a clean web interface. The system automatically generates optimized prompts for each bot, stores configurations persistently, and handles the complexity of model loading and inference.

Future enhancements could include fine-tuning capabilities that allow users to train custom models on their own data, multi-modal support for processing images and audio alongside text, collaborative features that enable teams to share and improve bots together, and advanced analytics that track bot performance and user satisfaction. The modular architecture makes it straightforward to add these features incrementally while maintaining system stability and performance.

The implementation demonstrates how modern web technologies, machine learning frameworks, and hardware acceleration can be combined to create powerful, flexible AI applications. By abstracting the complexity of different LLM backends and providing intuitive interfaces for bot creation and management, the system makes advanced AI capabilities accessible to users without requiring deep technical expertise.

Full Running Example

The following complete implementation provides a production-ready system for creating and managing LLM bots. This code includes all necessary components from model management through web interface, with no mocks or simulations.

# File: models.py

import torch

from transformers import AutoModelForCausalLM, AutoTokenizer

import json

import os

from typing import Optional, List, Dict, Any

import requests

class ModelConfig:

"""Configuration and management for a single LLM model"""

def __init__(self, name: str, model_type: str, backend: str,

model_path: Optional[str] = None, api_key: Optional[str] = None,

api_endpoint: Optional[str] = None):

self.name = name

self.model_type = model_type

self.backend = backend

self.model_path = model_path

self.api_key = api_key

self.api_endpoint = api_endpoint

self.loaded_model = None

self.tokenizer = None

self.device = None

def load(self):

"""Load the model based on its type"""

if self.model_type == 'local':

self._load_local_model()

else:

self._initialize_api_client()

def unload(self):

"""Unload the model to free memory"""

if self.loaded_model is not None:

del self.loaded_model

self.loaded_model = None

if self.tokenizer is not None:

del self.tokenizer

self.tokenizer = None

if torch.cuda.is_available():

torch.cuda.empty_cache()

def _load_local_model(self):

"""Load a local model with appropriate hardware acceleration"""

self.device = self._get_device()

print(f"Loading model {self.name} on device {self.device}")

self.tokenizer = AutoTokenizer.from_pretrained(

self.model_path,

trust_remote_code=True

)

if self.tokenizer.pad_token is None:

self.tokenizer.pad_token = self.tokenizer.eos_token

self.loaded_model = AutoModelForCausalLM.from_pretrained(

self.model_path,

device_map=self.device,

torch_dtype=torch.float16 if self.device != 'cpu' else torch.float32,

trust_remote_code=True,

low_cpu_mem_usage=True

)

self.loaded_model.eval()

def _get_device(self) -> str:

"""Determine the best available device for this backend"""

if self.backend == 'cuda' and torch.cuda.is_available():

return 'cuda'

elif self.backend == 'mps' and torch.backends.mps.is_available():

return 'mps'

elif self.backend == 'rocm' and torch.cuda.is_available():

return 'cuda'

else:

return 'cpu'

def _initialize_api_client(self):

"""Initialize API client for remote models"""

if not self.api_key or not self.api_endpoint:

raise ValueError(f"API key and endpoint required for remote model {self.name}")

print(f"Initialized API client for {self.name}")

def generate(self, messages: List[Dict[str, str]], max_tokens: int = 1000,

temperature: float = 0.7) -> str:

"""Generate a completion for the given messages"""

if self.model_type == 'local':

return self._generate_local(messages, max_tokens, temperature)

else:

return self._generate_remote(messages, max_tokens, temperature)

def _generate_local(self, messages: List[Dict[str, str]], max_tokens: int,

temperature: float) -> str:

"""Generate completion using local model"""

if self.loaded_model is None:

raise RuntimeError(f"Model {self.name} not loaded")

# Format messages into prompt

prompt = self._format_messages(messages)

# Tokenize

inputs = self.tokenizer(prompt, return_tensors="pt", padding=True)

inputs = {k: v.to(self.device) for k, v in inputs.items()}

# Generate

with torch.no_grad():

outputs = self.loaded_model.generate(

**inputs,

max_new_tokens=max_tokens,

temperature=temperature,

do_sample=temperature > 0,

pad_token_id=self.tokenizer.pad_token_id,

eos_token_id=self.tokenizer.eos_token_id

)

# Decode

generated_text = self.tokenizer.decode(outputs[0], skip_special_tokens=True)

# Extract only the new generated text

response = generated_text[len(prompt):].strip()

return response

def _generate_remote(self, messages: List[Dict[str, str]], max_tokens: int,

temperature: float) -> str:

"""Generate completion using remote API"""

headers = {

'Authorization': f'Bearer {self.api_key}',

'Content-Type': 'application/json'

}

data = {

'messages': messages,

'max_tokens': max_tokens,

'temperature': temperature

}

response = requests.post(self.api_endpoint, headers=headers, json=data)

response.raise_for_status()

result = response.json()

return result['choices'][0]['message']['content']

def _format_messages(self, messages: List[Dict[str, str]]) -> str:

"""Format messages into a prompt string"""

formatted = ""

for msg in messages:

role = msg['role']

content = msg['content']

if role == 'system':

formatted += f"System: {content}\n\n"

elif role == 'user':

formatted += f"User: {content}\n\n"

elif role == 'assistant':

formatted += f"Assistant: {content}\n\n"

formatted += "Assistant: "

return formatted

class ModelManager:

"""Manages multiple LLM models and handles model selection"""

def __init__(self, config_path: str = 'models.json'):

self.config_path = config_path

self.models: Dict[str, ModelConfig] = {}

self.current_model: Optional[ModelConfig] = None

self.load_model_configs()

def load_model_configs(self):

"""Load model configurations from JSON file"""

if os.path.exists(self.config_path):

with open(self.config_path, 'r', encoding='utf-8') as f:

model_list = json.load(f)

for model_data in model_list:

config = ModelConfig(

name=model_data['name'],

model_type=model_data['type'],

backend=model_data['backend'],

model_path=model_data.get('path'),

api_key=model_data.get('api_key'),

api_endpoint=model_data.get('api_endpoint')

)

self.models[config.name] = config

else:

# Create default configuration

self._create_default_config()

def _create_default_config(self):

"""Create a default model configuration file"""

default_models = [

{

'name': 'gpt2-local',

'type': 'local',

'backend': 'cuda',

'path': 'gpt2'

}

]

with open(self.config_path, 'w', encoding='utf-8') as f:

json.dump(default_models, f, indent=2)

self.load_model_configs()

def select_model(self, model_name: str) -> bool:

"""Select and load a model"""

if model_name not in self.models:

return False

if self.current_model:

self.current_model.unload()

self.current_model = self.models[model_name]

self.current_model.load()

return True

def get_current_model(self) -> Optional[ModelConfig]:

"""Get the currently selected model"""

return self.current_model

def list_models(self) -> List[str]:

"""List all available model names"""

return list(self.models.keys())

# File: bots.py

import json

import os

import uuid

from typing import Dict, List, Optional, Any

class Bot:

"""Represents a configured AI bot with its prompt structure"""

def __init__(self, bot_id: str, name: str, description: str,

prompt_structure: Dict[str, str], model_name: str):

self.bot_id = bot_id

self.name = name

self.description = description

self.prompt_structure = prompt_structure

self.model_name = model_name

class BotManager:

"""Manages bot configurations and persistence"""

def __init__(self, storage_path: str = 'bots.json'):

self.storage_path = storage_path

self.bots: Dict[str, Bot] = {}

self.load_bots()

def load_bots(self):

"""Load bots from JSON file"""

if os.path.exists(self.storage_path):

with open(self.storage_path, 'r', encoding='utf-8') as f:

bot_list = json.load(f)

for bot_data in bot_list:

bot = Bot(

bot_id=bot_data['id'],

name=bot_data['name'],

description=bot_data['description'],

prompt_structure=bot_data['prompt_structure'],

model_name=bot_data['model_name']

)

self.bots[bot.bot_id] = bot

else:

# Create empty bots file

self.save_bots()

def save_bots(self):

"""Save bots to JSON file"""

bot_list = []

for bot in self.bots.values():

bot_list.append({

'id': bot.bot_id,

'name': bot.name,

'description': bot.description,

'prompt_structure': bot.prompt_structure,

'model_name': bot.model_name

})

with open(self.storage_path, 'w', encoding='utf-8') as f:

json.dump(bot_list, f, indent=2, ensure_ascii=False)

def create_bot(self, name: str, description: str,

prompt_structure: Dict[str, str], model_name: str) -> Bot:

"""Create a new bot"""

bot_id = str(uuid.uuid4())

bot = Bot(bot_id, name, description, prompt_structure, model_name)

self.bots[bot_id] = bot

self.save_bots()

return bot

def get_bot(self, bot_id: str) -> Optional[Bot]:

"""Get a bot by ID"""

return self.bots.get(bot_id)

def list_bots(self) -> List[Bot]:

"""List all bots"""

return list(self.bots.values())

def delete_bot(self, bot_id: str) -> bool:

"""Delete a bot"""

if bot_id in self.bots:

del self.bots[bot_id]

self.save_bots()

return True

return False

# File: conversation.py

from typing import List, Dict, Any

from bots import Bot

from models import ModelManager

class ConversationManager:

"""Manages conversations with a specific bot"""

def __init__(self, bot: Bot, model_manager: ModelManager):

self.bot = bot

self.model_manager = model_manager

self.conversation_history: List[Dict[str, str]] = []

def process_user_message(self, user_message: str,

max_tokens: int = 1000,

temperature: float = 0.7) -> str:

"""Process a user message and generate a response"""

complete_prompt = self._build_complete_prompt(user_message)

model = self.model_manager.get_current_model()

if not model:

raise RuntimeError("No model selected")

response = model.generate(complete_prompt, max_tokens, temperature)

self.conversation_history.append({

'role': 'user',

'content': user_message

})

self.conversation_history.append({

'role': 'assistant',

'content': response

})

return response

def _build_complete_prompt(self, user_message: str) -> List[Dict[str, str]]:

"""Build the complete prompt including system, context, history, and user message"""

messages = []

# Add system prompt

if self.bot.prompt_structure.get('system'):

messages.append({

'role': 'system',

'content': self.bot.prompt_structure['system']

})

# Add assistant context

if self.bot.prompt_structure.get('assistant_context'):

messages.append({

'role': 'assistant',

'content': self.bot.prompt_structure['assistant_context']

})

# Add conversation history

messages.extend(self.conversation_history)

# Add current user message

user_template = self.bot.prompt_structure.get('user_template', '{user_input}')

formatted_message = user_template.format(user_input=user_message)

messages.append({

'role': 'user',

'content': formatted_message

})

return messages

def clear_history(self):

"""Clear the conversation history"""

self.conversation_history = []

def get_history(self) -> List[Dict[str, str]]:

"""Get the conversation history"""

return self.conversation_history.copy()

# File: prompt_generator.py

from models import ModelConfig

from typing import Dict

import json

def generate_bot_prompt(user_description: str, model_config: ModelConfig) -> Dict[str, str]:

"""Generate a comprehensive prompt structure for a new bot"""

generator_prompt = f"""You are an expert at creating detailed, effective prompts for large language models. A user wants to create a specialized AI bot with the following description:

{user_description}

Generate a comprehensive prompt structure for this bot consisting of:

1. A detailed system prompt that defines the bot's role, expertise, behavioral guidelines, output formatting rules, and any relevant constraints or principles it should follow.

2. An assistant context section that provides examples or patterns of ideal responses and interaction styles.

3. A user prompt template that shows how actual user inputs should be integrated into the complete prompt. Use the placeholder {{user_input}} where the user's message should be inserted.

Make the prompts thorough and specific. Include concrete guidelines rather than vague instructions. The resulting bot should be highly capable in its specialized domain.

Format your response as a JSON object with keys: system, assistant_context, and user_template. Ensure the JSON is valid and properly escaped.

Example format:

{{

"system": "You are a...",

"assistant_context": "When responding...",

"user_template": "User request: {{user_input}}\\n\\nPlease..."

}}

Now generate the prompt structure:"""

messages = [{'role': 'user', 'content': generator_prompt}]

response = model_config.generate(messages, max_tokens=2000, temperature=0.7)

# Extract JSON from response

try:

# Try to find JSON in the response

start_idx = response.find('{')

end_idx = response.rfind('}') + 1

if start_idx != -1 and end_idx > start_idx:

json_str = response[start_idx:end_idx]

prompt_structure = json.loads(json_str)

# Validate required keys

required_keys = ['system', 'assistant_context', 'user_template']

if all(key in prompt_structure for key in required_keys):

return prompt_structure

except json.JSONDecodeError:

pass

# Fallback to default structure if parsing fails

return {

'system': f"You are a specialized AI assistant. {user_description}",

'assistant_context': "I provide helpful, accurate, and detailed responses.",

'user_template': "{user_input}"

}

# File: response_handler.py

import re

from typing import Dict, List, Any

class ResponseHandler:

"""Handles processing and formatting of LLM responses"""

def process_response(self, response_text: str) -> Dict[str, Any]:

"""Process response text and extract structured content"""

processed_response = {

'text': response_text,

'code_blocks': [],

'attachments': []

}

# Extract code blocks

code_pattern = r'```(\w+)?\n(.*?)```'

code_blocks = re.findall(code_pattern, response_text, re.DOTALL)

for language, code in code_blocks:

processed_response['code_blocks'].append({

'language': language or 'text',

'code': code.strip()

})

# Extract attachment markers

attachment_pattern = r'\[ATTACHMENT:(.*?)\]'

attachments = re.findall(attachment_pattern, response_text)

for attachment in attachments:

processed_response['attachments'].append({

'type': 'file',

'reference': attachment

})

return processed_response

# File: server.py

from flask import Flask, request, jsonify, send_from_directory

from flask_cors import CORS

import os

import sys

from models import ModelManager

from bots import BotManager

from conversation import ConversationManager

from prompt_generator import generate_bot_prompt

from response_handler import ResponseHandler

app = Flask(__name__, static_folder='static')

CORS(app)

# Initialize managers

model_manager = ModelManager('models.json')

bot_manager = BotManager('bots.json')

response_handler = ResponseHandler()

# Store active conversations

active_conversations = {}

@app.route('/')

def index():

"""Serve the main page"""

return send_from_directory('static', 'index.html')

@app.route('/api/models', methods=['GET'])

def get_models():

"""Get list of available models"""

models = model_manager.list_models()

return jsonify([{'name': name} for name in models])

@app.route('/api/models/select', methods=['POST'])

def select_model():

"""Select a model to use"""

data = request.json

model_name = data.get('model_name')

if not model_name:

return jsonify({'error': 'Model name required'}), 400

if model_manager.select_model(model_name):

return jsonify({'status': 'success', 'model': model_name})

else:

return jsonify({'error': 'Model not found'}), 404

@app.route('/api/bots', methods=['GET'])

def get_bots():

"""Get list of all bots"""

bots = bot_manager.list_bots()

return jsonify([{

'id': bot.bot_id,

'name': bot.name,

'description': bot.description,

'model_name': bot.model_name

} for bot in bots])

@app.route('/api/bots/<bot_id>', methods=['GET'])

def get_bot(bot_id):

"""Get details for a specific bot"""

bot = bot_manager.get_bot(bot_id)

if bot:

return jsonify({

'id': bot.bot_id,

'name': bot.name,

'description': bot.description,

'model_name': bot.model_name,

'prompt_structure': bot.prompt_structure

})

else:

return jsonify({'error': 'Bot not found'}), 404

@app.route('/api/bots/create', methods=['POST'])

def create_bot():

"""Create a new bot"""

try:

data = request.json

name = data.get('name', '').strip()

description = data.get('description', '').strip()

model_name = data.get('model_name', '').strip()

if not name or not description:

return jsonify({'error': 'Name and description required'}), 400

current_model = model_manager.get_current_model()

if not current_model:

return jsonify({'error': 'No model selected'}), 400

# Generate prompt structure

prompt_structure = generate_bot_prompt(description, current_model)

# Use current model if not specified

if not model_name:

model_name = current_model.name

# Create bot

bot = bot_manager.create_bot(name, description, prompt_structure, model_name)

return jsonify({

'id': bot.bot_id,

'name': bot.name,

'description': bot.description,

'model_name': bot.model_name

})

except Exception as e:

app.logger.error(f"Failed to create bot: {str(e)}")

return jsonify({'error': str(e)}), 500

@app.route('/api/bots/<bot_id>', methods=['DELETE'])

def delete_bot(bot_id):

"""Delete a bot"""

if bot_manager.delete_bot(bot_id):

# Remove any active conversations for this bot

if bot_id in active_conversations:

del active_conversations[bot_id]

return jsonify({'status': 'success'})

else:

return jsonify({'error': 'Bot not found'}), 404

@app.route('/api/chat', methods=['POST'])

def chat():

"""Process a chat message"""

try:

data = request.json

bot_id = data.get('bot_id')

message = data.get('message', '').strip()

if not bot_id or not message:

return jsonify({'error': 'Bot ID and message required'}), 400

bot = bot_manager.get_bot(bot_id)

if not bot:

return jsonify({'error': 'Bot not found'}), 404

# Get or create conversation manager

if bot_id not in active_conversations:

active_conversations[bot_id] = ConversationManager(bot, model_manager)

conversation_manager = active_conversations[bot_id]

# Process message

response = conversation_manager.process_user_message(message)

# Process response for special content

processed = response_handler.process_response(response)

return jsonify({

'response': response,

'processed': processed,

'history': conversation_manager.get_history()

})

except Exception as e:

app.logger.error(f"Chat error: {str(e)}")

return jsonify({'error': str(e)}), 500

@app.route('/api/chat/clear/<bot_id>', methods=['POST'])

def clear_chat(bot_id):

"""Clear conversation history for a bot"""

if bot_id in active_conversations:

active_conversations[bot_id].clear_history()

return jsonify({'status': 'success'})

else:

return jsonify({'error': 'No active conversation'}), 404

if __name__ == '__main__':

# Ensure static directory exists

os.makedirs('static', exist_ok=True)

# Run server

app.run(host='0.0.0.0', port=5000, debug=True)

# File: static/index.html

<!DOCTYPE html>

<head>

<title>LLM Bot Manager</title>

<style>

* {

margin: 0;

padding: 0;

box-sizing: border-box;

}

body {

font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;

height: 100vh;

display: flex;

flex-direction: column;

background-color: #f5f5f5;

}

.navbar {

background-color: #2c3e50;

color: white;

padding: 15px 20px;

display: flex;

gap: 20px;

align-items: center;

box-shadow: 0 2px 4px rgba(0,0,0,0.1);

}

.navbar h1 {

font-size: 20px;

margin-right: auto;

}

.navbar label {

display: flex;

align-items: center;

gap: 8px;

font-size: 14px;

}

.navbar select {

padding: 6px 10px;

font-size: 14px;

border: none;

border-radius: 4px;

background-color: white;

cursor: pointer;

}

.main-container {

display: flex;

flex: 1;

overflow: hidden;

}

.sidebar {

width: 280px;

background-color: white;

padding: 20px;

border-right: 1px solid #e0e0e0;

overflow-y: auto;

display: flex;

flex-direction: column;

gap: 15px;

}

.sidebar h3 {

font-size: 16px;

color: #2c3e50;

margin-bottom: 10px;

}

.sidebar button {

padding: 10px 15px;

background-color: #3498db;

color: white;

border: none;

border-radius: 4px;

cursor: pointer;

font-size: 14px;

transition: background-color 0.2s;

}

.sidebar button:hover {

background-color: #2980b9;

}

.bot-item {

padding: 12px;

background-color: #f8f9fa;

border-radius: 4px;

cursor: pointer;

transition: background-color 0.2s;

border: 2px solid transparent;

}

.bot-item:hover {

background-color: #e9ecef;

}

.bot-item.active {

background-color: #e3f2fd;

border-color: #2196f3;

}

.bot-item h4 {

font-size: 14px;

color: #2c3e50;

margin-bottom: 4px;

}

.bot-item p {

font-size: 12px;

color: #7f8c8d;

}

.content {

flex: 1;

display: flex;

flex-direction: column;

background-color: white;

}

.conversation {

flex: 1;

overflow-y: auto;

padding: 20px;

background-color: #fafafa;

}

.message {

margin-bottom: 16px;

padding: 12px 16px;

border-radius: 8px;

max-width: 80%;

word-wrap: break-word;

}

.message.user {

background-color: #e3f2fd;

margin-left: auto;

border-bottom-right-radius: 4px;

}

.message.assistant {

background-color: white;

border: 1px solid #e0e0e0;

border-bottom-left-radius: 4px;

}

.message-role {

font-size: 11px;

font-weight: 600;

text-transform: uppercase;

color: #7f8c8d;

margin-bottom: 6px;

}

.message-content {

font-size: 14px;

line-height: 1.5;

color: #2c3e50;

}

.code-block {

background-color: #282c34;

color: #abb2bf;

border-radius: 4px;

padding: 12px;

margin: 10px 0;

overflow-x: auto;

font-family: 'Courier New', Consolas, Monaco, monospace;

font-size: 13px;

}

.code-block-header {

color: #61afef;

font-size: 11px;

margin-bottom: 8px;

text-transform: uppercase;

}

.input-panel {

padding: 20px;

background-color: white;

border-top: 1px solid #e0e0e0;

}

.input-controls {

display: flex;

gap: 10px;

align-items: flex-end;

}

.input-panel textarea {

flex: 1;

padding: 12px;

border: 1px solid #e0e0e0;

border-radius: 4px;

font-size: 14px;

font-family: inherit;

resize: vertical;

min-height: 60px;

max-height: 200px;

}

.input-panel textarea:focus {

outline: none;

border-color: #3498db;

}

.input-panel button {

padding: 12px 24px;

background-color: #3498db;

color: white;

border: none;

border-radius: 4px;

cursor: pointer;

font-size: 14px;

font-weight: 500;

transition: background-color 0.2s;

}

.input-panel button:hover {

background-color: #2980b9;

}

.input-panel button:disabled {

background-color: #bdc3c7;

cursor: not-allowed;

}

.empty-state {

display: flex;

flex-direction: column;

align-items: center;

justify-content: center;

height: 100%;

color: #7f8c8d;

text-align: center;

padding: 40px;

}

.empty-state h2 {

font-size: 24px;

margin-bottom: 10px;

}

.empty-state p {

font-size: 14px;

}

.loading {

text-align: center;

padding: 20px;

color: #7f8c8d;

}

.error {

background-color: #ffebee;

color: #c62828;

padding: 12px;

border-radius: 4px;

margin: 10px 0;

font-size: 14px;

}

.modal {

display: none;

position: fixed;

top: 0;

left: 0;

width: 100%;

height: 100%;

background-color: rgba(0,0,0,0.5);

z-index: 1000;

align-items: center;

justify-content: center;

}

.modal.active {

display: flex;

}

.modal-content {

background-color: white;

padding: 30px;

border-radius: 8px;

max-width: 500px;

width: 90%;

}

.modal-content h2 {

font-size: 20px;

margin-bottom: 20px;

color: #2c3e50;

}

.modal-content label {

display: block;

margin-bottom: 8px;

font-size: 14px;

font-weight: 500;

color: #2c3e50;

}

.modal-content input,

.modal-content textarea {

width: 100%;

padding: 10px;

border: 1px solid #e0e0e0;

border-radius: 4px;

font-size: 14px;

font-family: inherit;

margin-bottom: 15px;

}

.modal-content textarea {

min-height: 100px;

resize: vertical;

}

.modal-buttons {

display: flex;

gap: 10px;

justify-content: flex-end;

}

.modal-buttons button {

padding: 10px 20px;

border: none;

border-radius: 4px;

cursor: pointer;

font-size: 14px;

transition: background-color 0.2s;

}

.modal-buttons .btn-primary {

background-color: #3498db;

color: white;

}

.modal-buttons .btn-primary:hover {

background-color: #2980b9;

}

.modal-buttons .btn-secondary {

background-color: #ecf0f1;

color: #2c3e50;

}

.modal-buttons .btn-secondary:hover {

background-color: #bdc3c7;

}

</style>

</head>

<body>

<h1>LLM Bot Manager</h1>

<label>

Model:

<option value="">Select Model</option>

</select>

</label>

</div>

<button id="createBotBtn">Create New Bot</button>

</div>

<h2>Welcome to LLM Bot Manager</h2>

<p>Select a bot from the sidebar or create a new one to get started</p>

</div>

</div>

<h2>Create New Bot</h2>

<label for="botDescription">Bot Description</label>

<button class="btn-secondary" id="cancelCreateBtn">Cancel</button>

<button class="btn-primary" id="confirmCreateBtn">Create Bot</button>

</div>

class LLMBotApp {

constructor() {

this.currentBot = null;

this.currentModel = null;

this.apiBase = '';

this.initializeEventListeners();

this.loadModels();

this.loadBots();

}

initializeEventListeners() {

document.getElementById('modelSelector').addEventListener('change', (e) => {

this.selectModel(e.target.value);

});

document.getElementById('sendBtn').addEventListener('click', () => {

this.sendMessage();

});

document.getElementById('userInput').addEventListener('keypress', (e) => {

if (e.key === 'Enter' && e.ctrlKey) {

this.sendMessage();

}

});

document.getElementById('createBotBtn').addEventListener('click', () => {

this.showCreateBotModal();

});

document.getElementById('cancelCreateBtn').addEventListener('click', () => {

this.hideCreateBotModal();

});

document.getElementById('confirmCreateBtn').addEventListener('click', () => {

this.createBot();

});

}

async loadModels() {

try {

const response = await fetch(`${this.apiBase}/api/models`);

const models = await response.json();

const selector = document.getElementById('modelSelector');

selector.innerHTML = '<option value="">Select Model</option>';

models.forEach(model => {

const option = document.createElement('option');

option.value = model.name;

option.textContent = model.name;

selector.appendChild(option);

});

} catch (error) {

console.error('Failed to load models:', error);

this.showError('Failed to load models');

}

async loadBots() {

try {

const response = await fetch(`${this.apiBase}/api/bots`);

const bots = await response.json();

const botList = document.getElementById('botList');

botList.innerHTML = '';

if (bots.length === 0) {

botList.innerHTML = '<p style="color: #7f8c8d; font-size: 14px;">No bots yet</p>';

return;

}

bots.forEach(bot => {

const botItem = document.createElement('div');

botItem.className = 'bot-item';

botItem.innerHTML = `

<h4>${this.escapeHtml(bot.name)}</h4>

<p>${this.escapeHtml(bot.description.substring(0, 60))}${bot.description.length > 60 ? '...' : ''}</p>

botItem.addEventListener('click', () => {

this.selectBot(bot.id);

});

botList.appendChild(botItem);

});

} catch (error) {

console.error('Failed to load bots:', error);

this.showError('Failed to load bots');

}

async selectModel(modelName) {

if (!modelName) return;

try {

const response = await fetch(`${this.apiBase}/api/models/select`, {

method: 'POST',

headers: {

'Content-Type': 'application/json'

body: JSON.stringify({ model_name: modelName })

});

if (response.ok) {

this.currentModel = modelName;

console.log('Model selected:', modelName);

} else {

throw new Error('Failed to select model');

}

} catch (error) {

console.error('Failed to select model:', error);

this.showError('Failed to select model');

}

async selectBot(botId) {

try {

const response = await fetch(`${this.apiBase}/api/bots/${botId}`);

const bot = await response.json();

this.currentBot = bot;

// Update UI

document.querySelectorAll('.bot-item').forEach(item => {

item.classList.remove('active');

});

event.currentTarget.classList.add('active');

// Clear conversation

this.clearConversation();

// Enable input

document.getElementById('sendBtn').disabled = false;

// Load bot's model if different

if (bot.model_name !== this.currentModel) {

document.getElementById('modelSelector').value = bot.model_name;

await this.selectModel(bot.model_name);

}

console.log('Bot selected:', bot.name);

} catch (error) {

console.error('Failed to select bot:', error);

this.showError('Failed to select bot');

}

async sendMessage() {

const input = document.getElementById('userInput');

const message = input.value.trim();

if (!message || !this.currentBot) return;

// Disable input while processing

const sendBtn = document.getElementById('sendBtn');

sendBtn.disabled = true;

input.disabled = true;

// Display user message

this.displayMessage('user', message);

// Clear input

input.value = '';

try {

const response = await fetch(`${this.apiBase}/api/chat`, {

method: 'POST',

headers: {

'Content-Type': 'application/json'

body: JSON.stringify({

bot_id: this.currentBot.id,

message: message

})

});

if (!response.ok) {

throw new Error('Failed to get response');

}

const data = await response.json();

// Display assistant response

this.displayMessage('assistant', data.response);

} catch (error) {

console.error('Failed to send message:', error);

this.displayMessage('assistant', 'Error: Failed to get response from the bot');

} finally {

// Re-enable input

sendBtn.disabled = false;

input.disabled = false;

input.focus();

}

displayMessage(role, content) {

const conversationArea = document.getElementById('conversationArea');

// Remove empty state if present

const emptyState = conversationArea.querySelector('.empty-state');

if (emptyState) {

emptyState.remove();

}

const messageDiv = document.createElement('div');

messageDiv.className = `message ${role}`;

const roleDiv = document.createElement('div');

roleDiv.className = 'message-role';

roleDiv.textContent = role;

const contentDiv = document.createElement('div');

contentDiv.className = 'message-content';

contentDiv.innerHTML = this.processContent(content);

messageDiv.appendChild(roleDiv);

messageDiv.appendChild(contentDiv);

conversationArea.appendChild(messageDiv);

conversationArea.scrollTop = conversationArea.scrollHeight;

}

processContent(content) {

// Escape HTML

let processed = this.escapeHtml(content);

// Process code blocks

const codeBlockRegex = /```(\w+)?\n([\s\S]*?)```/g;

processed = processed.replace(codeBlockRegex, (match, language, code) => {

return `<div class="code-block"><div class="code-block-header">${language || 'code'}</div><pre>${code.trim()}</pre></div>`;

});

// Convert newlines to br tags

processed = processed.replace(/\n/g, '<br>');

return processed;

}

clearConversation() {

const conversationArea = document.getElementById('conversationArea');

conversationArea.innerHTML = '';

}

showCreateBotModal() {

if (!this.currentModel) {

alert('Please select a model first');

return;

}

document.getElementById('createBotModal').classList.add('active');

document.getElementById('botName').value = '';

document.getElementById('botDescription').value = '';

document.getElementById('botName').focus();

}

hideCreateBotModal() {

document.getElementById('createBotModal').classList.remove('active');

}

async createBot() {

const name = document.getElementById('botName').value.trim();

const description = document.getElementById('botDescription').value.trim();

if (!name || !description) {

alert('Please enter both name and description');

return;

}

try {

const response = await fetch(`${this.apiBase}/api/bots/create`, {

method: 'POST',

headers: {

'Content-Type': 'application/json'

body: JSON.stringify({

description: description,

model_name: this.currentModel

})

});

if (!response.ok) {

throw new Error('Failed to create bot');

}

const bot = await response.json();

// Hide modal

this.hideCreateBotModal();

// Reload bots

await this.loadBots();

// Select the new bot

await this.selectBot(bot.id);

} catch (error) {

console.error('Failed to create bot:', error);

alert('Failed to create bot. Please try again.');

}

showError(message) {

const conversationArea = document.getElementById('conversationArea');

const errorDiv = document.createElement('div');

errorDiv.className = 'error';

errorDiv.textContent = message;

conversationArea.appendChild(errorDiv);

}

escapeHtml(text) {

const div = document.createElement('div');

div.textContent = text;

return div.innerHTML;

}

// Initialize app when DOM is ready

document.addEventListener('DOMContentLoaded', () => {

new LLMBotApp();

});

</script>

</body>

</html>

This complete implementation provides a fully functional web-based LLM bot management system. The code includes all components from model management through the web interface with no mocks or simulations. The system supports both local and remote LLMs across different hardware platforms, allows users to create custom bots with automatically generated prompts, and provides a clean interface for managing and interacting with these bots. All bot and model configurations are persisted to files, and the conversation system maintains context across multiple turns. The implementation follows clean code principles with proper separation of concerns, comprehensive error handling, and a responsive user interface.

Hitchhiker's Guide to AI, Software Architecture, and Everything Else

Monday, September 07, 2026

Building a Web-Based LLM Application for Creating and Managing AI Bots

Introduction and System Overview

Core Components and Architecture

Hardware Acceleration and Model Loading

Bot Creation and Prompt Generation

Bot and Model Persistence

Conversation Flow and Prompt Integration

Handling Model Responses and Attachments

Web Interface Implementation

Backend Server Implementation

Advanced Features and Optimizations

Testing and Deployment

Performance Optimization

Conclusion and Future Directions

Full Running Example

No comments:

About Me