Motivation
I asked Claude Sonnet 4 to generate a simulator of the Transformer architecture. After a couple of follow-up prompts due to errors and inconsistencies of the generated code, it came up with a simulator on a single Web page. The page includes animations (CSS styles, JavaScript, and HTML with grid layout).
User Guide
🚀 Quick Start Guide: Transformer Architecture Simulator
What is this?
An interactive visualization that shows exactly how transformer AI models (like ChatGPT) process text step-by-step, from input to output. Perfect for understanding AI without technical background!
🎮 Basic Controls
Getting Started
- Enter Text: Type in the input box (default: “AI is”)
- Click “Start”: Begin the simulation
- Watch the Magic: See the text flow through 15 processing steps
Control Buttons
- ▶️ Start: Begin processing your text
- ⏹️ Stop: Halt the simulation
- ⏭️ Step: Manually advance one step at a time
- 🔄 Auto Step: Automatic progression through all steps
- 🔄 Reset: Clear everything and start fresh
- Speed Slider: Adjust animation speed (slow to fast)
📊 What You’ll See
The 15 Processing Steps
- 🔤 Tokenizer: Splits text into pieces
- 📊 Embeddings: Converts words to numbers
- 🎯 Encoder Attention: Understands word relationships
- ➕ Add & Norm: Stabilizes processing
- 🧠Feed Forward: Complex transformations
- ✅ Encoder Output: Input fully understood
- 🔒 Masked Attention: Prevents cheating
- ➕ Add & Norm: More stabilization
- 🔗 Cross-Attention: Connects input to output
- ➕ Add & Norm: Further processing
- 🧠Decoder Feed Forward: Final transformations
- ✅ Decoder Output: Ready for prediction
- 📊 Output Layer: Calculate word probabilities
- 🎲 Token Selection: Pick the next word
- ✨ Completion: Final result: “AI is exciting”
Visual Elements
- Glowing Boxes: Show current processing step
- Animated Arrows: Data flowing between steps
- Math Formulas: Core calculations (simplified)
- Probability Bars: Show word likelihood
- Attention Heads: Multiple processing streams
💡 Tips for Best Experience
Recommended Usage
- Start with Default Text: Use “AI is” for your first run
- Use Auto Step: Click “Auto Step” for smooth animation
- Adjust Speed: Use slider to find comfortable pace
- Watch Info Panel: Read explanations as you go
- Try Step Mode: Use “Step” button to go at your own pace
Understanding the Flow
- Left to Right: Data flows horizontally within rows
- Top to Bottom: Complex transitions between processing layers
- Cyan Arrows: Show exactly where data is moving
- Active Highlighting: Current step glows and scales up
Reset & Retry
- Reset Clears Everything: All text, displays, and states
- Try Different Inputs: Change the input text and restart
- Mobile Friendly: Works perfectly on phones and tablets
🎯 Learning Goals
After using this simulator, you’ll understand:
- How AI breaks down and processes text
- Why transformers are so powerful
- The role of attention in AI
- How probability drives text generation
- Why AI can generate human-like text
🔧 Troubleshooting
Simulator seems stuck? → Click Reset and try again
Text too small? → Zoom in your browser
Animation too fast/slow? → Adjust the speed slider
Want to restart? → Always click Reset for a clean slate
🌟 Advanced Features
- Math Displays: Simplified formulas show core operations
- Multiple Attention Heads: See parallel processing
- Probability Visualization: Understand AI decision making
- Complete Data Flow: Every step is animated and explained
Ready to explore AI? Click Start and watch the magic happen! ✨
The full code
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Transformer Architecture Simulator</title>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
```
body {
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
background: linear-gradient(135deg, #0a0a0a 0%, #1a1a2e 50%, #16213e 100%);
color: #ffffff;
overflow-x: auto;
min-height: 100vh;
}
.container {
display: flex;
flex-direction: column;
min-height: 100vh;
}
.header {
background: rgba(255, 255, 255, 0.1);
backdrop-filter: blur(10px);
padding: 20px;
border-bottom: 1px solid rgba(255, 255, 255, 0.2);
}
.title {
text-align: center;
font-size: 2.5em;
font-weight: bold;
background: linear-gradient(45deg, #ff6b6b, #4ecdc4, #45b7d1, #96ceb4);
background-size: 400% 400%;
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
animation: gradientShift 3s ease-in-out infinite;
}
@keyframes gradientShift {
0%, 100% { background-position: 0% 50%; }
50% { background-position: 100% 50%; }
}
.controls {
display: flex;
justify-content: center;
gap: 15px;
margin-top: 15px;
flex-wrap: wrap;
}
.control-btn {
padding: 12px 24px;
border: none;
border-radius: 25px;
font-size: 14px;
font-weight: bold;
cursor: pointer;
transition: all 0.3s ease;
background: linear-gradient(45deg, #667eea 0%, #764ba2 100%);
color: white;
box-shadow: 0 4px 15px rgba(0, 0, 0, 0.3);
}
.control-btn:hover {
transform: translateY(-2px);
box-shadow: 0 8px 25px rgba(0, 0, 0, 0.4);
}
.control-btn:active {
transform: translateY(0);
}
.control-btn.start { background: linear-gradient(45deg, #11998e, #38ef7d); }
.control-btn.stop { background: linear-gradient(45deg, #fc466b, #3f5efb); }
.control-btn.step { background: linear-gradient(45deg, #ffecd2, #fcb69f); color: #333; }
.control-btn.reset { background: linear-gradient(45deg, #a8edea, #fed6e3); color: #333; }
.speed-control {
display: flex;
align-items: center;
gap: 10px;
}
.speed-slider {
width: 120px;
height: 5px;
border-radius: 5px;
background: #ddd;
outline: none;
opacity: 0.7;
transition: opacity 0.2s;
}
.speed-slider:hover {
opacity: 1;
}
.main-content {
display: flex;
flex: 1;
gap: 20px;
padding: 20px;
}
.visualization-area {
flex: 3;
background: rgba(255, 255, 255, 0.05);
border-radius: 15px;
padding: 20px;
border: 1px solid rgba(255, 255, 255, 0.1);
position: relative;
overflow: hidden;
}
.info-panel {
flex: 1;
background: rgba(255, 255, 255, 0.05);
border-radius: 15px;
padding: 20px;
border: 1px solid rgba(255, 255, 255, 0.1);
max-height: 80vh;
overflow-y: auto;
}
.step-indicator {
text-align: center;
margin-bottom: 20px;
font-size: 1.2em;
font-weight: bold;
}
.current-step {
background: linear-gradient(45deg, #ff9a9e, #fecfef);
padding: 10px 20px;
border-radius: 20px;
display: inline-block;
color: #333;
}
.transformer-container {
display: grid;
grid-template-columns: repeat(5, 1fr);
grid-template-rows: repeat(3, 1fr);
gap: 15px;
height: 60vh;
max-height: 500px;
position: relative;
padding: 10px;
}
.layer-container {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
opacity: 0.3;
transition: all 0.5s ease;
position: relative;
}
.layer-container.active {
opacity: 1;
transform: scale(1.05);
}
.layer-container.processing {
animation: pulse 1s infinite;
}
@keyframes pulse {
0%, 100% { transform: scale(1.05); }
50% { transform: scale(1.1); }
}
.layer-box {
background: linear-gradient(135deg, rgba(255, 255, 255, 0.1) 0%, rgba(255, 255, 255, 0.05) 100%);
border: 2px solid rgba(255, 255, 255, 0.2);
border-radius: 10px;
padding: 12px;
text-align: center;
width: 100%;
height: 100%;
position: relative;
backdrop-filter: blur(10px);
display: flex;
flex-direction: column;
justify-content: space-between;
}
.layer-title {
font-weight: bold;
font-size: 0.8em;
margin-bottom: 5px;
}
.layer-subtitle {
font-size: 0.7em;
opacity: 0.8;
margin-bottom: 8px;
}
.data-flow {
position: absolute;
width: 4px;
background: linear-gradient(to bottom, #4ecdc4, #44a08d);
z-index: -1;
opacity: 0;
transition: opacity 0.5s ease;
}
.data-flow.active {
opacity: 1;
animation: flowAnimation 2s infinite;
}
@keyframes flowAnimation {
0% { background-position: 0% 0%; }
100% { background-position: 0% 100%; }
}
.token-container {
display: flex;
gap: 10px;
flex-wrap: wrap;
justify-content: center;
}
.token {
background: linear-gradient(45deg, #667eea, #764ba2);
padding: 5px 12px;
border-radius: 15px;
font-size: 0.9em;
transition: all 0.3s ease;
border: 1px solid rgba(255, 255, 255, 0.3);
}
.token.active {
background: linear-gradient(45deg, #ff6b6b, #ee5a24);
transform: scale(1.1);
box-shadow: 0 0 20px rgba(255, 107, 107, 0.5);
}
.math-display {
background: rgba(0, 0, 0, 0.3);
border-radius: 10px;
padding: 15px;
margin: 10px 0;
font-family: 'Courier New', monospace;
border-left: 4px solid #4ecdc4;
}
.attention-matrix {
display: grid;
grid-template-columns: repeat(4, 1fr);
gap: 5px;
margin: 10px 0;
}
.attention-cell {
width: 30px;
height: 30px;
border-radius: 5px;
display: flex;
align-items: center;
justify-content: center;
font-size: 0.8em;
transition: all 0.3s ease;
border: 1px solid rgba(255, 255, 255, 0.2);
}
.multi-head-attention {
display: flex;
gap: 3px;
justify-content: center;
flex-wrap: wrap;
}
.attention-head {
background: linear-gradient(45deg, #667eea, #764ba2);
padding: 3px 6px;
border-radius: 5px;
text-align: center;
font-size: 0.6em;
min-width: 35px;
transition: all 0.3s ease;
}
.attention-head.active {
background: linear-gradient(45deg, #ff6b6b, #ee5a24);
transform: scale(1.1);
}
.feed-forward {
display: flex;
flex-direction: column;
align-items: center;
gap: 3px;
}
.ff-layer {
background: linear-gradient(45deg, #ffecd2, #fcb69f);
color: #333;
padding: 3px 8px;
border-radius: 5px;
font-size: 0.7em;
transition: all 0.3s ease;
}
.ff-layer.active {
transform: scale(1.1);
box-shadow: 0 0 10px rgba(255, 204, 102, 0.5);
}
.probability-display {
display: flex;
flex-direction: column;
gap: 3px;
max-width: 100%;
}
.prob-bar {
display: flex;
align-items: center;
gap: 5px;
}
.prob-label {
min-width: 30px;
font-size: 0.7em;
}
.prob-value {
flex: 1;
height: 12px;
background: rgba(255, 255, 255, 0.1);
border-radius: 6px;
overflow: hidden;
position: relative;
}
.prob-fill {
height: 100%;
background: linear-gradient(45deg, #4ecdc4, #44a08d);
border-radius: 6px;
transition: width 0.5s ease;
}
.input-area {
margin-bottom: 20px;
}
.input-text {
width: 100%;
padding: 15px;
border-radius: 10px;
border: 1px solid rgba(255, 255, 255, 0.3);
background: rgba(255, 255, 255, 0.1);
color: white;
font-size: 1em;
resize: vertical;
min-height: 60px;
}
.input-text::placeholder {
color: rgba(255, 255, 255, 0.6);
}
.glow {
animation: glowEffect 2s infinite;
}
@keyframes glowEffect {
0%, 100% { box-shadow: 0 0 5px rgba(78, 205, 196, 0.5); }
50% { box-shadow: 0 0 25px rgba(78, 205, 196, 0.8); }
}
.hidden {
display: none !important;
}
@media (max-width: 768px) {
.main-content {
flex-direction: column;
}
.controls {
flex-direction: column;
align-items: center;
}
.title {
font-size: 1.8em;
}
}
</style>
```
</head>
<body>
<div class="container">
<div class="header">
<h1 class="title">🤖 Interactive Transformer Architecture Simulator</h1>
<div class="controls">
<button class="control-btn start" id="startBtn">▶️ Start</button>
<button class="control-btn stop" id="stopBtn">⏹️ Stop</button>
<button class="control-btn step" id="stepBtn">⏭️ Step</button>
<button class="control-btn" id="autoStepBtn">🔄 Auto Step</button>
<button class="control-btn reset" id="resetBtn">🔄 Reset</button>
<div class="speed-control">
<span>Speed:</span>
<input type="range" class="speed-slider" id="speedSlider" min="100" max="3000" value="1000">
<span id="speedValue">1.0s</span>
</div>
</div>
</div>
```
<div class="main-content">
<div class="visualization-area">
<div class="input-area">
<textarea class="input-text" id="inputText" placeholder="Text">AI is</textarea>
</div>
<div class="step-indicator">
<div class="current-step" id="currentStep">Ready to Start</div>
</div>
<div class="transformer-container" id="transformerContainer">
<!-- Row 1: Input Processing -->
<div class="layer-container" id="tokenizer" style="grid-column: 1; grid-row: 1;">
<div class="layer-box">
<div class="layer-title">🔤 Tokenizer</div>
<div class="layer-subtitle">Text to tokens</div>
<div class="token-container" id="tokenDisplay"></div>
</div>
</div>
<div class="layer-container" id="embeddings" style="grid-column: 2; grid-row: 1;">
<div class="layer-box">
<div class="layer-title">📊 Embeddings</div>
<div class="layer-subtitle">Token + Position</div>
<div class="math-display" id="embeddingMath" style="font-size: 0.7em; padding: 5px;"></div>
</div>
</div>
<!-- Encoder Stack -->
<div class="layer-container" id="encoderAttention" style="grid-column: 3; grid-row: 1;">
<div class="layer-box">
<div class="layer-title">🎯 Encoder Attention</div>
<div class="layer-subtitle">Self-Attention</div>
<div class="multi-head-attention" id="encoderHeads"></div>
<div class="math-display" id="encoderAttentionMath" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<div class="layer-container" id="encoderNorm1" style="grid-column: 4; grid-row: 1;">
<div class="layer-box">
<div class="layer-title">➕ Add & Norm</div>
<div class="layer-subtitle">Residual + Norm</div>
<div class="math-display" id="encoderNorm1Math" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<div class="layer-container" id="encoderFF" style="grid-column: 5; grid-row: 1;">
<div class="layer-box">
<div class="layer-title">🧠Feed Forward</div>
<div class="layer-subtitle">Linear + ReLU</div>
<div class="feed-forward">
<div class="ff-layer" style="padding: 3px 8px; font-size: 0.7em;">Linear</div>
<div class="ff-layer" style="padding: 3px 8px; font-size: 0.7em;">ReLU</div>
<div class="ff-layer" style="padding: 3px 8px; font-size: 0.7em;">Linear</div>
</div>
<div class="math-display" id="encoderFFMath" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<!-- Row 2: Decoder Stack -->
<div class="layer-container" id="encoderNorm2" style="grid-column: 1; grid-row: 2;">
<div class="layer-box">
<div class="layer-title">✅ Encoder Out</div>
<div class="layer-subtitle">Final encoding</div>
<div class="math-display" id="encoderNorm2Math" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<div class="layer-container" id="decoderSelfAttention" style="grid-column: 2; grid-row: 2;">
<div class="layer-box">
<div class="layer-title">🔒 Masked Attention</div>
<div class="layer-subtitle">Decoder Self-Attn</div>
<div class="multi-head-attention" id="decoderSelfHeads"></div>
<div class="math-display" id="decoderSelfAttentionMath" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<div class="layer-container" id="decoderNorm1" style="grid-column: 3; grid-row: 2;">
<div class="layer-box">
<div class="layer-title">➕ Add & Norm</div>
<div class="layer-subtitle">After mask attn</div>
<div class="math-display" id="decoderNorm1Math" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<div class="layer-container" id="decoderCrossAttention" style="grid-column: 4; grid-row: 2;">
<div class="layer-box">
<div class="layer-title">🔗 Cross-Attention</div>
<div class="layer-subtitle">Dec ↔ Enc</div>
<div class="multi-head-attention" id="decoderCrossHeads"></div>
<div class="math-display" id="decoderCrossAttentionMath" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<div class="layer-container" id="decoderNorm2" style="grid-column: 5; grid-row: 2;">
<div class="layer-box">
<div class="layer-title">➕ Add & Norm</div>
<div class="layer-subtitle">After cross attn</div>
<div class="math-display" id="decoderNorm2Math" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<!-- Row 3: Output Processing -->
<div class="layer-container" id="decoderFF" style="grid-column: 1; grid-row: 3;">
<div class="layer-box">
<div class="layer-title">🧠Decoder FF</div>
<div class="layer-subtitle">Final transform</div>
<div class="feed-forward">
<div class="ff-layer" style="padding: 3px 8px; font-size: 0.7em;">Linear</div>
<div class="ff-layer" style="padding: 3px 8px; font-size: 0.7em;">ReLU</div>
<div class="ff-layer" style="padding: 3px 8px; font-size: 0.7em;">Linear</div>
</div>
<div class="math-display" id="decoderFFMath" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<div class="layer-container" id="decoderNorm3" style="grid-column: 2; grid-row: 3;">
<div class="layer-box">
<div class="layer-title">✅ Decoder Out</div>
<div class="layer-subtitle">Ready for output</div>
<div class="math-display" id="decoderNorm3Math" style="font-size: 0.6em; padding: 3px;"></div>
</div>
</div>
<div class="layer-container" id="outputLayer" style="grid-column: 3; grid-row: 3;">
<div class="layer-box">
<div class="layer-title">📊 Output Layer</div>
<div class="layer-subtitle">Linear + Softmax</div>
<div class="math-display" id="outputMath" style="font-size: 0.6em; padding: 3px;"></div>
<div class="probability-display" id="probabilities" style="margin-top: 5px;"></div>
</div>
</div>
<div class="layer-container" id="tokenSelection" style="grid-column: 4; grid-row: 3;">
<div class="layer-box">
<div class="layer-title">🎲 Token Select</div>
<div class="layer-subtitle">Sampling</div>
<div class="math-display" id="selectionMath" style="font-size: 0.6em; padding: 3px;"></div>
<div class="token" id="selectedToken" style="margin-top: 5px;"></div>
</div>
</div>
<div class="layer-container" id="completion" style="grid-column: 5; grid-row: 3;">
<div class="layer-box">
<div class="layer-title">✨ Done</div>
<div class="layer-subtitle">Output</div>
<div class="completion-text" id="completionText" style="font-size: 0.25em; margin-top: 0; word-wrap: break-word; overflow: hidden; line-height: 0.8;"></div>
</div>
</div>
</div>
</div>
<div class="info-panel">
<h3>📖 Current Process</h3>
<div id="processInfo">
<p>Welcome to the Interactive Transformer Simulator!</p>
<p>This visualization shows how transformer models process text step by step.</p>
<br>
<p><strong>How to use:</strong></p>
<ul>
<li>Enter text in the input area</li>
<li>Click "Start" to begin the simulation</li>
<li>Use "Step" for manual progression</li>
<li>Use "Auto Step" for automatic progression</li>
<li>Adjust speed with the slider</li>
</ul>
</div>
<h3 style="margin-top: 30px;">🧮 Mathematical Details</h3>
<div id="mathDetails">
<p>Mathematical operations will appear here during the simulation.</p>
</div>
<h3 style="margin-top: 30px;">📊 Layer Information</h3>
<div id="layerInfo">
<p>Detailed layer information will be shown here as you progress through the simulation.</p>
</div>
</div>
</div>
</div>
<script>
class TransformerSimulator {
constructor() {
this.currentStep = 0;
this.isRunning = false;
this.isAutoStep = false;
this.speed = 1000;
this.tokens = [];
this.completion = "";
this.steps = [
'tokenizer',
'embeddings',
'encoderAttention',
'encoderNorm1',
'encoderFF',
'encoderNorm2',
'decoderSelfAttention',
'decoderNorm1',
'decoderCrossAttention',
'decoderNorm2',
'decoderFF',
'decoderNorm3',
'outputLayer',
'tokenSelection',
'completion'
];
this.stepNames = {
'tokenizer': 'Tokenization',
'embeddings': 'Input Embeddings',
'encoderAttention': 'Encoder Self-Attention',
'encoderNorm1': 'Encoder Add & Norm 1',
'encoderFF': 'Encoder Feed Forward',
'encoderNorm2': 'Encoder Add & Norm 2',
'decoderSelfAttention': 'Decoder Masked Self-Attention',
'decoderNorm1': 'Decoder Add & Norm 1',
'decoderCrossAttention': 'Decoder Cross-Attention',
'decoderNorm2': 'Decoder Add & Norm 2',
'decoderFF': 'Decoder Feed Forward',
'decoderNorm3': 'Decoder Add & Norm 3',
'outputLayer': 'Output Projection & Softmax',
'tokenSelection': 'Token Selection',
'completion': 'Text Generation Complete'
};
this.vocabulary = ['exciting', 'bright', 'promising', 'uncertain', 'revolutionary', 'transformative', 'amazing', 'incredible', 'limitless', 'boundless', 'the', 'future', 'of', 'AI', 'is'];
this.initializeEventListeners();
this.createDataFlowArrows();
this.reset();
}
initializeEventListeners() {
document.getElementById('startBtn').addEventListener('click', () => this.start());
document.getElementById('stopBtn').addEventListener('click', () => this.stop());
document.getElementById('stepBtn').addEventListener('click', () => this.step());
document.getElementById('autoStepBtn').addEventListener('click', () => this.toggleAutoStep());
document.getElementById('resetBtn').addEventListener('click', () => this.reset());
const speedSlider = document.getElementById('speedSlider');
speedSlider.addEventListener('input', (e) => {
this.speed = parseInt(e.target.value);
document.getElementById('speedValue').textContent = (this.speed / 1000).toFixed(1) + 's';
});
}
start() {
if (!this.isRunning) {
this.isRunning = true;
this.tokenize();
this.step();
}
}
stop() {
this.isRunning = false;
this.isAutoStep = false;
document.getElementById('autoStepBtn').textContent = '🔄 Auto Step';
}
step() {
if (this.currentStep < this.steps.length) {
this.executeStep(this.steps[this.currentStep]);
this.currentStep++;
if (this.isAutoStep && this.currentStep < this.steps.length) {
setTimeout(() => this.step(), this.speed);
} else if (this.currentStep >= this.steps.length) {
// All steps completed
this.isRunning = false;
this.isAutoStep = false;
document.getElementById('autoStepBtn').textContent = '🔄 Auto Step';
}
}
}
toggleAutoStep() {
this.isAutoStep = !this.isAutoStep;
const btn = document.getElementById('autoStepBtn');
if (this.isAutoStep) {
btn.textContent = '⏸️ Pause Auto';
if (this.isRunning && this.currentStep < this.steps.length) {
setTimeout(() => this.step(), this.speed);
}
} else {
btn.textContent = '🔄 Auto Step';
}
}
reset() {
this.currentStep = 0;
this.isRunning = false;
this.isAutoStep = false;
this.tokens = [];
this.completion = "";
document.getElementById('currentStep').textContent = 'Ready to Start';
document.getElementById('autoStepBtn').textContent = '🔄 Auto Step';
// Clear ALL text displays and inputs - COMPLETELY EMPTY
document.getElementById('inputText').value = '';
document.getElementById('completionText').textContent = '';
document.getElementById('tokenDisplay').innerHTML = '';
document.getElementById('embeddingMath').innerHTML = '';
document.getElementById('encoderAttentionMath').innerHTML = '';
document.getElementById('encoderNorm1Math').innerHTML = '';
document.getElementById('encoderFFMath').innerHTML = '';
document.getElementById('encoderNorm2Math').innerHTML = '';
document.getElementById('decoderSelfAttentionMath').innerHTML = '';
document.getElementById('decoderNorm1Math').innerHTML = '';
document.getElementById('decoderCrossAttentionMath').innerHTML = '';
document.getElementById('decoderNorm2Math').innerHTML = '';
document.getElementById('decoderFFMath').innerHTML = '';
document.getElementById('decoderNorm3Math').innerHTML = '';
document.getElementById('outputMath').innerHTML = '';
document.getElementById('selectionMath').innerHTML = '';
document.getElementById('selectedToken').textContent = '';
document.getElementById('probabilities').innerHTML = '';
document.getElementById('encoderHeads').innerHTML = '';
document.getElementById('decoderSelfHeads').innerHTML = '';
document.getElementById('decoderCrossHeads').innerHTML = '';
// Reset all layer states including completion
document.querySelectorAll('.layer-container').forEach(layer => {
layer.classList.remove('active', 'processing', 'glow');
layer.style.opacity = '0.3';
layer.style.background = '';
layer.style.border = '';
layer.style.transform = '';
});
// Hide all arrows
document.querySelectorAll('.data-flow-arrow').forEach(arrow => {
arrow.classList.remove('active');
arrow.style.opacity = '0';
});
this.updateProcessInfo('Ready to start! Enter text and click Start to see the transformer process it step by step.');
this.updateMathDetails('');
this.updateLayerInfo('');
}
createDataFlowArrows() {
const container = document.getElementById('transformerContainer');
// Define step order based on actual processing flow including completion
const stepOrder = [
'tokenizer', 'embeddings', 'encoderAttention', 'encoderNorm1', 'encoderFF',
'encoderNorm2', 'decoderSelfAttention', 'decoderNorm1', 'decoderCrossAttention', 'decoderNorm2',
'decoderFF', 'decoderNorm3', 'outputLayer', 'tokenSelection', 'completion'
];
// Create arrows between consecutive steps
for (let i = 0; i < stepOrder.length - 1; i++) {
const arrow = this.createArrowForSteps(stepOrder[i], stepOrder[i + 1], i);
container.appendChild(arrow);
}
}
createArrowForSteps(fromStepId, toStepId, connectionId) {
const arrow = document.createElement('div');
arrow.className = 'data-flow-arrow';
arrow.id = `arrow-${connectionId}`;
// Get grid positions for each step
const gridPositions = {
'tokenizer': {col: 0, row: 0},
'embeddings': {col: 1, row: 0},
'encoderAttention': {col: 2, row: 0},
'encoderNorm1': {col: 3, row: 0},
'encoderFF': {col: 4, row: 0},
'encoderNorm2': {col: 0, row: 1},
'decoderSelfAttention': {col: 1, row: 1},
'decoderNorm1': {col: 2, row: 1},
'decoderCrossAttention': {col: 3, row: 1},
'decoderNorm2': {col: 4, row: 1},
'decoderFF': {col: 0, row: 2},
'decoderNorm3': {col: 1, row: 2},
'outputLayer': {col: 2, row: 2},
'tokenSelection': {col: 3, row: 2},
'completion': {col: 4, row: 2}
};
const from = gridPositions[fromStepId];
const to = gridPositions[toStepId];
if (!from || !to) return arrow;
const cellWidth = 20; // 100% / 5 columns
const cellHeight = 33.33; // 100% / 3 rows
// Calculate arrow path
let startX, startY, endX, endY;
if (from.row === to.row && Math.abs(from.col - to.col) === 1) {
// Horizontal arrow (adjacent in same row)
startX = (from.col + 0.9) * cellWidth;
startY = (from.row + 0.5) * cellHeight;
endX = (to.col + 0.1) * cellWidth;
endY = (to.row + 0.5) * cellHeight;
} else if (fromStepId === 'encoderFF' && toStepId === 'encoderNorm2') {
// Special case: End of row 1 to start of row 2
startX = (from.col + 0.1) * cellWidth;
startY = (from.row + 0.9) * cellHeight;
endX = (to.col + 0.5) * cellWidth;
endY = (to.row + 0.1) * cellHeight;
} else if (fromStepId === 'decoderNorm2' && toStepId === 'decoderFF') {
// Special case: End of row 2 to start of row 3
startX = (from.col + 0.1) * cellWidth;
startY = (from.row + 0.9) * cellHeight;
endX = (to.col + 0.5) * cellWidth;
endY = (to.row + 0.1) * cellHeight;
} else {
// Default positioning for other cases
startX = (from.col + 0.9) * cellWidth;
startY = (from.row + 0.5) * cellHeight;
endX = (to.col + 0.1) * cellWidth;
endY = (to.row + 0.5) * cellHeight;
}
arrow.style.position = 'absolute';
arrow.style.left = '0';
arrow.style.top = '0';
arrow.style.width = '100%';
arrow.style.height = '100%';
arrow.style.pointerEvents = 'none';
arrow.style.zIndex = '5';
arrow.style.opacity = '0';
arrow.innerHTML = `
<svg width="100%" height="100%" style="position: absolute;">
<defs>
<marker id="arrowhead-${connectionId}" markerWidth="5" markerHeight="3"
refX="4" refY="1.5" orient="auto" fill="#4ecdc4">
<polygon points="0 0, 5 1.5, 0 3" />
</marker>
<linearGradient id="gradient-${connectionId}">
<stop offset="0%" stop-color="#4ecdc4" stop-opacity="0.8"/>
<stop offset="100%" stop-color="#44a08d" stop-opacity="1"/>
</linearGradient>
</defs>
<line x1="${startX}%" y1="${startY}%" x2="${endX}%" y2="${endY}%"
stroke="url(#gradient-${connectionId})" stroke-width="1.5"
marker-end="url(#arrowhead-${connectionId})" />
</svg>
`;
return arrow;
}
tokenize() {
const input = document.getElementById('inputText').value.trim();
if (!input) {
document.getElementById('inputText').value = "AI is";
}
const textToTokenize = document.getElementById('inputText').value.trim();
this.tokens = textToTokenize.split(/\s+/).slice(0, 3); // Limit to 3 tokens for very compact display
const tokenDisplay = document.getElementById('tokenDisplay');
tokenDisplay.innerHTML = '';
this.tokens.forEach((token, i) => {
const tokenElement = document.createElement('div');
tokenElement.className = 'token';
tokenElement.textContent = token;
tokenElement.id = `token-${i}`;
tokenDisplay.appendChild(tokenElement);
});
}
executeStep(stepId) {
// Clear previous active states
document.querySelectorAll('.layer-container').forEach(layer => {
layer.classList.remove('active', 'processing');
});
// Hide all arrows
document.querySelectorAll('.data-flow-arrow').forEach(arrow => {
arrow.classList.remove('active');
});
// Activate current step
const currentLayer = document.getElementById(stepId);
currentLayer.classList.add('active', 'processing');
currentLayer.style.opacity = '1';
// Activate arrow leading to current step
if (this.currentStep > 0) {
const arrowId = `arrow-${this.currentStep - 1}`;
const arrow = document.getElementById(arrowId);
if (arrow) {
arrow.classList.add('active');
}
}
// Update step indicator
document.getElementById('currentStep').textContent = this.stepNames[stepId];
// Execute step-specific logic
switch(stepId) {
case 'tokenizer':
this.executeTokenizer();
break;
case 'embeddings':
this.executeEmbeddings();
break;
case 'encoderAttention':
this.executeEncoderAttention();
break;
case 'encoderNorm1':
this.executeEncoderNorm1();
break;
case 'encoderFF':
this.executeEncoderFF();
break;
case 'encoderNorm2':
this.executeEncoderNorm2();
break;
case 'decoderSelfAttention':
this.executeDecoderSelfAttention();
break;
case 'decoderNorm1':
this.executeDecoderNorm1();
break;
case 'decoderCrossAttention':
this.executeDecoderCrossAttention();
break;
case 'decoderNorm2':
this.executeDecoderNorm2();
break;
case 'decoderFF':
this.executeDecoderFF();
break;
case 'decoderNorm3':
this.executeDecoderNorm3();
break;
case 'outputLayer':
this.executeOutputLayer();
break;
case 'tokenSelection':
this.executeTokenSelection();
break;
case 'completion':
this.executeCompletion();
break;
}
// Add glow effect
setTimeout(() => {
currentLayer.classList.remove('processing');
currentLayer.classList.add('glow');
setTimeout(() => currentLayer.classList.remove('glow'), 1000);
}, 500);
}
executeTokenizer() {
this.updateProcessInfo('Tokenizing input text into discrete tokens that the model can understand.');
this.updateMathDetails('Tokenization: Split text → [token₁, token₂, ..., tokenâ‚™]');
this.updateLayerInfo('The tokenizer converts raw text into a sequence of tokens. Each token represents a word or subword unit.');
// Animate tokens
this.tokens.forEach((token, i) => {
setTimeout(() => {
document.getElementById(`token-${i}`).classList.add('active');
setTimeout(() => document.getElementById(`token-${i}`).classList.remove('active'), 500);
}, i * 200);
});
}
executeEmbeddings() {
this.updateProcessInfo('Converting tokens to dense vector representations and adding positional information.');
const mathContent = `E = Token + Position`;
document.getElementById('embeddingMath').innerHTML = mathContent;
this.updateMathDetails('Token Embedding + Positional Encoding creates input vectors');
this.updateLayerInfo('Embeddings convert discrete tokens into continuous vectors with position information.');
}
executeEncoderAttention() {
this.createAttentionHeads('encoderHeads');
this.updateProcessInfo('Computing self-attention to understand relationships between all input tokens.');
const mathContent = `Attn = softmax(QK^T/√d)V`;
document.getElementById('encoderAttentionMath').innerHTML = mathContent;
this.updateMathDetails('Multi-head attention: Q, K, V projections with scaled dot-product attention');
this.updateLayerInfo('Multi-head attention allows the model to attend to different aspects simultaneously.');
}
executeEncoderNorm1() {
this.updateProcessInfo('Applying residual connection and layer normalization for training stability.');
const mathContent = `out = LN(X + Attn(X))`;
document.getElementById('encoderNorm1Math').innerHTML = mathContent;
this.updateMathDetails('Residual connection + Layer normalization for gradient flow');
this.updateLayerInfo('Residual connections help with gradient flow during training.');
}
executeEncoderFF() {
this.animateFeedForward('encoderFF');
this.updateProcessInfo('Processing through feed-forward neural network for non-linear transformations.');
const mathContent = `FFN = ReLU(XW₁)W₂`;
document.getElementById('encoderFFMath').innerHTML = mathContent;
this.updateMathDetails('Feed-forward network: Linear → ReLU → Linear');
this.updateLayerInfo('Feed-forward networks provide non-linear transformations.');
}
executeEncoderNorm2() {
this.updateProcessInfo('Final residual connection and normalization for the encoder layer.');
const mathContent = `out = LN(X + FFN(X))`;
document.getElementById('encoderNorm2Math').innerHTML = mathContent;
this.updateMathDetails('Final encoder layer normalization');
this.updateLayerInfo('Encoder output contains rich contextual representations.');
}
executeDecoderSelfAttention() {
this.createAttentionHeads('decoderSelfHeads');
this.updateProcessInfo('Masked self-attention in decoder to prevent looking at future tokens.');
const mathContent = `MaskedAttn(Q,K,V)`;
document.getElementById('decoderSelfAttentionMath').innerHTML = mathContent;
this.updateMathDetails('Masked attention prevents attending to future tokens');
this.updateLayerInfo('Masked attention ensures causal generation.');
}
executeDecoderNorm1() {
this.updateProcessInfo('Residual connection and normalization after masked self-attention.');
const mathContent = `out = LN(X + MaskedAttn)`;
document.getElementById('decoderNorm1Math').innerHTML = mathContent;
this.updateMathDetails('Normalization after masked attention');
this.updateLayerInfo('Preparing for cross-attention with encoder.');
}
executeDecoderCrossAttention() {
this.createAttentionHeads('decoderCrossHeads');
this.updateProcessInfo('Cross-attention between decoder and encoder outputs.');
const mathContent = `CrossAttn(Q_dec, K_enc, V_enc)`;
document.getElementById('decoderCrossAttentionMath').innerHTML = mathContent;
this.updateMathDetails('Cross-attention: Decoder queries attend to encoder keys/values');
this.updateLayerInfo('Cross-attention connects decoder to input context.');
}
executeDecoderNorm2() {
this.updateProcessInfo('Residual connection and normalization after cross-attention.');
const mathContent = `out = LN(X + CrossAttn)`;
document.getElementById('decoderNorm2Math').innerHTML = mathContent;
this.updateMathDetails('Combining decoder and encoder information');
this.updateLayerInfo('Integrating encoder context into decoder flow.');
}
executeDecoderFF() {
this.animateFeedForward('decoderFF');
this.updateProcessInfo('Decoder feed-forward network for final token representation.');
const mathContent = `FFN = ReLU(XW₁)W₂`;
document.getElementById('decoderFFMath').innerHTML = mathContent;
this.updateMathDetails('Final feed-forward transformation in decoder');
this.updateLayerInfo('Final non-linear transformation in decoder stack.');
}
executeDecoderNorm3() {
this.updateProcessInfo('Final decoder layer normalization.');
const mathContent = `out = LN(X + FFN(X))`;
document.getElementById('decoderNorm3Math').innerHTML = mathContent;
this.updateMathDetails('Final decoder output ready for projection');
this.updateLayerInfo('Decoder output ready for vocabulary projection.');
}
executeOutputLayer() {
this.updateProcessInfo('Projecting decoder output to vocabulary space and computing probabilities.');
this.createProbabilityDisplay();
const mathContent = `P = softmax(XW_vocab)`;
document.getElementById('outputMath').innerHTML = mathContent;
this.updateMathDetails('Linear projection + softmax for token probabilities');
this.updateLayerInfo('Converting hidden states to vocabulary probabilities.');
}
executeTokenSelection() {
this.updateProcessInfo('Selecting next token based on probability distribution.');
// Always select "exciting" as the completion word
const selectedWord = 'exciting';
document.getElementById('selectedToken').textContent = selectedWord;
document.getElementById('selectedToken').classList.add('active');
const mathContent = `Selected: "${selectedWord}"`;
document.getElementById('selectionMath').innerHTML = mathContent;
this.updateMathDetails('Probabilistic sampling determines next token');
this.updateLayerInfo('Token selection completes the generation step.');
this.completion = selectedWord;
}
createAttentionHeads(containerId) {
const container = document.getElementById(containerId);
container.innerHTML = '';
for (let i = 1; i <= 3; i++) { // Reduced to 3 heads for smaller space
const head = document.createElement('div');
head.className = 'attention-head';
head.textContent = `H${i}`;
container.appendChild(head);
setTimeout(() => {
head.classList.add('active');
setTimeout(() => head.classList.remove('active'), 300);
}, i * 100);
}
}
animateFeedForward(layerId) {
const ffLayers = document.querySelectorAll(`#${layerId} .ff-layer`);
ffLayers.forEach((layer, i) => {
setTimeout(() => {
layer.classList.add('active');
setTimeout(() => layer.classList.remove('active'), 300);
}, i * 150);
});
}
createProbabilityDisplay() {
const container = document.getElementById('probabilities');
container.innerHTML = '';
// Show "exciting" as highest probability, followed by other words
const sampleWords = ['exciting', 'bright', 'amazing'];
const probabilities = [0.6, 0.25, 0.15]; // Higher probability for "exciting"
sampleWords.forEach((word, i) => {
const probBar = document.createElement('div');
probBar.className = 'prob-bar';
const label = document.createElement('div');
label.className = 'prob-label';
label.textContent = word.slice(0, 3); // Truncate to 3 chars
const valueContainer = document.createElement('div');
valueContainer.className = 'prob-value';
const fill = document.createElement('div');
fill.className = 'prob-fill';
fill.style.width = '0%';
valueContainer.appendChild(fill);
probBar.appendChild(label);
probBar.appendChild(valueContainer);
container.appendChild(probBar);
setTimeout(() => {
fill.style.width = (probabilities[i] * 100) + '%';
}, i * 100);
});
}
executeCompletion() {
this.updateProcessInfo('Text generation complete! The transformer generated "exciting" as the final word.');
// Display the final generated text with "exciting" as the completion
const finalText = document.getElementById('inputText').value + ' exciting';
document.getElementById('completionText').textContent = finalText;
// Highlight the completion box with special animation
const completionLayer = document.getElementById('completion');
completionLayer.classList.add('active');
completionLayer.style.background = 'linear-gradient(45deg, rgba(78, 205, 196, 0.4), rgba(68, 160, 141, 0.4))';
completionLayer.style.border = '2px solid #4ecdc4';
completionLayer.style.transform = 'scale(1.1)';
// Add special glow effect for completion
setTimeout(() => {
completionLayer.classList.add('glow');
}, 200);
this.updateMathDetails('Final output: "' + finalText + '" - Generation complete with "exciting"');
this.updateLayerInfo('FINAL STEP: The transformer has successfully generated "exciting" as the completion word!');
// Set completion to "exciting" for consistency
this.completion = "exciting";
// Stop the simulation
this.isRunning = false;
this.isAutoStep = false;
document.getElementById('autoStepBtn').textContent = '🔄 Auto Step';
}
updateProcessInfo(text) {
document.getElementById('processInfo').innerHTML = `<p>${text}</p>`;
}
updateMathDetails(html) {
document.getElementById('mathDetails').innerHTML = html;
}
updateLayerInfo(text) {
document.getElementById('layerInfo').innerHTML = `<p>${text}</p>`;
}
}
// Initialize the simulator when the page loads
window.addEventListener('DOMContentLoaded', () => {
const simulator = new TransformerSimulator();
});
</script>
```
</body>
</html>
No comments:
Post a Comment