Introduction
Musical Instrument Digital Interface, more commonly known as MIDI, is a compact and enduring protocol that allows electronic musical instruments, computers, and controllers to exchange performance information. Rather than transmitting audio, MIDI communicates what notes to play, when to start and stop them, how loud to play them, and which articulations or controls to move. This separation of performance data from sound synthesis makes MIDI uniquely flexible. A single performance can drive a piano, a string ensemble, a virtual synth, or a sampler, and the same data can be edited and transformed algorithmically without the destructive problems typical of audio processing. This article takes you through the foundation of the MIDI standard and the SMF file format, builds intuition for channels, messages, timing, and controllers, and then demonstrates small, correct examples including raw hexadecimal messages and a minimal Standard MIDI File. It concludes with concrete strategies for applying large language models to MIDI analysis and generation in ways that are reliable, reproducible, and aligned with the standard.
What MIDI Is and What It Is Not
MIDI is a serial message protocol operating at 31,250 bits per second in its original DIN configuration, subsequently encapsulated in USB, Bluetooth, and network transport layers. The messages describe musical gestures and device configuration, not sound. A note-on message says that a note should start, identified by a number from 0 to 127 where middle C is 60 by convention. A velocity field indicates how hard the note is struck. A note-off message ends it. Controllers change continuous parameters, a program change selects an instrument preset, and pitch bend nudges the pitch in fine steps. None of these messages contains a waveform; they are instructions. Because of that, the same performance data can be reinterpreted by any sound engine that understands MIDI. The protocol is stateless in concept but depends in practice on persistent device state, which is why initialization messages for program selection, bank selection, controller resets, and channel tuning are part of robust MIDI workflows.
Channels, Devices, and Messages
A MIDI cable and port carry up to sixteen logical channels, numbered from one to sixteen for humans and encoded as zero to fifteen in the low four bits of the status byte. Each channel can be treated as an independent instrument part. A channel voice message affects only its channel, while a system message affects the entire device regardless of channel.
A note-on message has a status byte of 0x9n, where n is the channel number from zero to fifteen, followed by two data bytes for key number and velocity. A note-off message uses 0x8n with the same data fields. Many devices also interpret a note-on with velocity zero as note-off to reduce data bandwidth and running status overhead. A control change has status 0xBn, with two data bytes for controller number and value. A program change has status 0xCn and a single data byte for the program number from zero to 127. A pitch bend message uses status 0xEn with two data bytes that combine into a fourteen-bit value, centered at 8192 representing no bend; the least significant seven bits are sent first, then the most significant seven bits, so the neutral pitch bend value is sent as 0x00 followed by 0x40.
System common and real-time messages support transport and synchronization. A timing clock at 0xF8 ticks twenty-four times per quarter note to drive arpeggiators and sequencers. Start, continue, and stop are sent as 0xFA, 0xFB, and 0xFC. Active sensing at 0xFE helps detect cable disconnects. System exclusive messages start at 0xF0, carry a manufacturer ID and arbitrary vendor-defined payload, and terminate at 0xF7. They are used for instrument dumps, parameter edits beyond the standard controllers, and universal non-real-time messages such as master volume and device inquiry.
Timing, Delta Times, and the Standard MIDI File
While live MIDI streams are timing-sensitive at the serial level, files encapsulate temporal structure using delta times measured in ticks, or pulses per quarter note. The Standard MIDI File (SMF) organizes data into a header chunk and one or more track chunks. The header chunk, identified by the ASCII string MThd, contains the format type, the number of tracks, and the division field. The division is usually a ticks-per-quarter-note integer known as PPQ, such as 480 or 960, which determines the grid. Each track chunk, labeled MTrk, contains a sequence of events, each prefixed by a variable-length quantity that encodes the time since the previous event. The variable-length format uses seven bits per byte for the number and a continuation bit in the top bit; the final byte has the top bit clear. Event bytes can be channel messages or meta events. Meta events begin with 0xFF and add structural information such as tempo, time signature, track name, and end-of-track.
Tempo is specified as a meta event with type 0x51, a data length of three, and a big-endian integer for microseconds per quarter note. A commonly used tempo of 120 beats per minute corresponds to 500,000 microseconds per quarter note. Time signature uses type 0x58 and encodes numerator, the power-of-two denominator as a binary exponent, MIDI clocks per metronome click, and thirty-second notes per MIDI quarter note. End-of-track uses type 0x2F with zero-length data and is mandatory to delimit a track.
A Correct Minimal Standard MIDI File by Bytes
A tiny but correct single-track SMF that plays middle C for half a second at 120 BPM can be written entirely as bytes. The file starts with the header chunk. The ASCII characters MThd are 0x4D, 0x54, 0x68, 0x64. The header length is 6 bytes, given as 0x00, 0x00, 0x00, 0x06. The format type 0 for a single-track file is 0x00, 0x00. The number of tracks is one, given as 0x00, 0x01. The division is ticks per quarter note; a value of 480 is 0x01, 0xE0. The track chunk begins with MTrk as 0x4D, 0x54, 0x72, 0x6B, followed by a four-byte length that must equal the number of bytes in the track event data. Inside, the first event sets tempo at delta time 0, then a note-on at delta time 0, then a delta time for note duration, then note-off, and finally end-of-track at delta time 0.
If we choose 120 BPM, a quarter note equals 500,000 microseconds. With a division of 480 ticks per quarter note, a half-second equals one quarter note. If we want a note lasting exactly a quarter note, the delta time between note-on and note-off is 480 ticks. The variable-length encoding of 480 is 0x83, 0x60, because 480 in binary is 0b1 1110 0000. The lower seven bits are 0x60, and the upper bits produce 0x03 with the continuation bit set to form 0x83.
The following sequence of bytes is a correct complete MIDI file:
MThd header:
4D 54 68 64 00 00 00 06 00 00 00 01 01 E0
MTrk header and contents:
4D 54 72 6B 00 00 00 1B
00 FF 51 03 07 A1 20
00 90 3C 64
83 60 80 3C 40
00 FF 2F 00
This breaks down as follows. The header declares a format 0 file with one track and a division of 480 ticks per quarter note. The track length of 0x0000001B equals twenty-seven bytes and must match the number of bytes after the length field up to but not including the next chunk. The first event at delta time 0 is the set-tempo meta event with three data bytes of 07 A1 20, representing 500,000 microseconds per quarter. The second event at delta time 0 is a note-on on channel 1 with status 0x90, key 0x3C which is 60 for middle C, and velocity 0x64 which is 100. The third event after a delta time of 480 ticks, encoded as 83 60, is a note-off on channel 1 with status 0x80, the same key 0x3C, and a velocity 0x40 which is 64 and perfectly valid. The final event at delta time 0 is the end-of-track meta event 0xFF 0x2F with a zero-length data byte 0x00.
If you place those bytes in a file named example.mid, any compliant sequencer or DAW will play one middle C with a default piano sound for a quarter note at 120 BPM. The example is small and complete. The variable-length value and track length are correct, and the meta events are properly terminated.
Channel Voice Messages in Hex with Correct Interpretation
A short live stream example can clarify channel messages. Consider three messages in order: 0x90, 0x3C, 0x64 to start middle C with velocity 100, 0xB0, 0x07, 0x50 to set channel volume controller 7 to value 80, and 0x80, 0x3C, 0x00 to end the note with velocity zero. The first byte in each case has the top bit set which identifies it as a status byte. The channel is encoded as the low four bits of the status, which in these examples is zero representing channel one. The data bytes always have the top bit clear and thus fall in the range zero to 127. A receiver updates its channel volume state and scales synthesis output accordingly. Using note-off velocity zero is legal and commonly used, although some instruments ignore release velocity and simply treat any note-off as a gate close.
Running Status and Efficiency
To reduce bandwidth, MIDI allows running status where consecutive messages of the same type and channel can omit repeating the status byte. A stream of note-on events on the same channel can be written as 0x90, 0x3C, 0x64, 0x40, 0x64, 0x43, 0x64, where 0x90 applies to the subsequent data pairs until a new status byte appears. A Standard MIDI File uses running status too, but only within track event streams; it cannot cross meta events, and any meta or system message terminates running status. If you are generating files programmatically, you should either deliberately maintain running status with care or, more simply, always include status bytes which is larger but reduces error risk. Most libraries will handle this detail for you.
General MIDI, Bank Select, and Program Mapping
While core MIDI does not define instruments by number, General MIDI (GM) provides a cross-device mapping from program numbers to instrument families. Program 0 is Acoustic Grand Piano and program 40 is Violin, continuing through 127. Drum kits are special in GM: channel 10, which is channel nine zero-based, is used for percussive instruments where each note number triggers a different drum. Bank select uses two controllers to extend program space. Controller 0 is Bank Select MSB and controller 32 is Bank Select LSB. To select a banked program reliably, you send both controller 0 and controller 32 with desired values, then send the program change. Devices may ignore one or both bank messages, but ordering them before the program change is the interoperable practice.
Pitch Bend Range and Semantics
Pitch bend uses a fourteen-bit center at 8192. The musical range in semitones is device-defined and often adjustable via registered parameter number, or RPN, messages. To set a pitch bend range of two semitones, you send an RPN selection of 0, 0 using controller 101 and 100 for MSB and LSB respectively, followed by data entry controller 6 for the MSB value 2 and controller 38 for LSB usually set to zero. The correct sequence for channel one is 0xB0, 0x65, 0x00, 0xB0, 0x64, 0x00, 0xB0, 0x06, 0x02, 0xB0, 0x26, 0x00, and optionally a reset of RPN by setting 0x65 and 0x64 to 0x7F, 0x7F. After configuring the pitch bend range, a bend upward by one semitone corresponds to half the range and thus a bend value of 8192 plus half of 8192 equals 12288, which splits into LSB 0x00 and MSB 0x60 since 12288 equals 0x3000, and the message is 0xE0, 0x00, 0x60.
Meta Events That Structure Musical Context
Meta events do not transmit over cables but exist within files to provide context. A track name at delta time zero written as 0x00, 0xFF, 0x03, length, characters, helps DAWs label parts. A time signature of four four at the beginning looks like 0x00, 0xFF, 0x58, 0x04, 0x04, 0x02, 0x18, 0x08, because the numerator is 4, the denominator encoded as 2 since 2 to the power 2 is 4, the metronome click occurs every twenty-four MIDI clocks, and there are eight thirty-second notes per quarter. A key signature meta event helps notation software interpret accidentals and is encoded as sharps or flats count and a minor or major flag.
A More Expressive SMF Example with Controller and Program Change
A slightly richer example helps connect these ideas. Suppose you want a file that sets a tempo of 100 BPM, names a track, selects a violin preset, gradually increases modulation with controller 1 to add vibrato, performs a short phrase on channel one, and ends. The division remains 480 ticks per quarter, and the tempo in microseconds per quarter is 600,000, which is 0x09, 0x27, 0xC0. The track data begins with delta time zero for both the tempo and the name, then issues a program change to program number 40 for violin assuming GM mapping, then a controller sweep over half a measure.
A correct event sequence for the track could be described precisely. You start with 0x00, 0xFF, 0x51, 0x03, 0x09, 0x27, 0xC0 to set tempo. You then write the name as 0x00, 0xFF, 0x03, length, ASCII bytes, for example 0x00, 0xFF, 0x03, 0x06, 0x56, 0x69, 0x6F, 0x6C, 0x69, 0x6E for the name Violin. You then issue a bank select neutralization and a program change. If you are not banking, you can omit bank select and send 0x00, 0xC0, 0x28, where 0x28 is forty in hexadecimal. You then at delta time zero send a modulation value, for example 0x00, 0xB0, 0x01, 0x00, and over 240 ticks ramp to 0x40 by inserting intermediate controller messages at small delta times. Next, you perform a note-on with 0x90, a key such as 0x3E for D4, and a velocity such as 0x50 which is eighty. After a duration of 360 ticks, you send the note-off with 0x80 and a modest release velocity. Finally, you mark the end with 0x00, 0xFF, 0x2F, 0x00. If you compute the track length by counting all bytes between the MTrk length and the end-of-track, a DAW will parse it. If you are not comfortable hand-counting the track length, using a MIDI library to construct the file will compute lengths and variable-length quantities accurately.
MIDI 2.0 and Property Exchange in Brief
MIDI 2.0 introduces higher resolution for controllers through universal MIDI packets and a registration and discovery layer that allows devices to negotiate profiles and property exchange. It preserves backward compatibility with MIDI 1.0 but expands the range of expressive control from seven-bit values to thirty-two-bit precision where supported. It defines profiles such as a Drawbar Organ profile so controls behave uniformly across vendors. It also adds jitter reduction timestamps for tighter timing over USB and network paths. While adoption is still growing, it is useful to design systems with an abstraction layer so that when MIDI 2.0 is available, your code can map enhanced resolution and properties to familiar musical gestures.
Common Pitfalls and How to Avoid Them
Timing errors often stem from confusion between tempo and division. The division in ticks per quarter note is a static resolution, while tempo maps musical time to real time. If you change tempo mid-track, all subsequent delta times still count ticks, but the wall-clock duration per tick changes. Note-off mismatches occur when a generator sends a note-on and forgets to send the corresponding note-off, leading to hanging notes, which you can prevent by tracking active notes per channel and per key and always closing them, even on interruptions. Running status confusion can corrupt a stream when a meta event appears and the generator continues to omit status bytes. An easy fix is to always restart status after any meta or system event in file generation. Pitch bend misuse happens when developers assume the value is seven-bit rather than fourteen-bit, which yields coarse or wrong bends; always pack LSB then MSB and center at 0x00 0x40. Program change off-by-one errors occur when developers treat GM program numbers as one-based; the standard is zero-based, so Acoustic Grand is zero not one.
Using LLMs for MIDI Analysis
Language models can parse and reason about MIDI structure if you provide them with the right abstractions. One effective method is to convert event streams into a compact, textual representation that preserves timing, channel, and message semantics, then prompt the model to summarize or analyze patterns. A representation like t=0: meta tempo 120bpm; t=0: program ch1=40; t=0: note_on ch1 60 vel 100; t=480: note_off ch1 60 vel 64 gives the model enough structure to infer duration, density, and harmonic content. You can ask the model to identify scales used by mapping pitch classes modulo twelve and tallying counts, then to infer likely key centers by comparing against major and minor templates. You can also instruct the model to detect polyrhythms by inspecting inter-onset intervals and their ratios relative to the division. When you need precise metrics like note durations in milliseconds under variable tempo, you should compute the timing numerically in your code and present the results to the model rather than asking it to infer microseconds per tick. A robust approach is to extract per-track event lists, convert all delta times to absolute tick positions, map tempo changes to cumulative time segments, and compute wall-clock times once. After that, you can ask the model qualitative questions about phrasing, articulation, and structure, backed by exact timing.
Another valuable analysis task for LLMs is controller intent inference. If you present the model with controller curves labeled by musical meaning, such as modulation or expression over time, it can infer whether the performer intended crescendos, vibrato intensity changes, or tremolo depth, and it can suggest smoothing or quantization strategies. You can also have the model classify articulation by looking at note lengths relative to grid positions and neighbors. For example, a staccato articulation can be inferred when note durations are consistently short relative to inter-onset intervals, whereas legato overlaps indicate slurred phrasing.
Using LLMs for MIDI Generation
Language models excel at symbolic pattern generation when constrained by clear schemas. If you design a mini-language that maps to legal MIDI, you can have the model generate sequences that you translate deterministically to proper files. A safe schema will contain an explicit PPQ, a tempo, a list of tracks, and for each track an ordered list of events with absolute tick times, note numbers, velocities, and durations. You avoid ambiguity by prohibiting negative times, by constraining velocities to zero through 127, and by requiring that every note has an end. The generation pipeline then becomes a validation step that checks ranges and overlaps, a translator that converts absolute times to sorted events with delta times and variable-length quantities, and a file writer that computes chunk lengths. The model’s role is to produce the musical plan; your code’s role is to guarantee correctness.
When aiming for stylistic coherence like a twelve-bar blues or a four-on-the-floor pattern, you can prompt the model with structural scaffolding that includes chord changes by bar, scale degrees, and rhythmic templates. The model can output phrases relative to the current chord, such as scale degree two up to major third down to root with syncopation at offbeats, which you then map to MIDI notes via the chord’s root and mode. This reduces hallucination risk because the mapping logic for harmony and rhythm resides in deterministic code.
A Tiny, Correct Generation Example in Python
The following Python snippet constructs a small SMF without relying on external libraries. It carefully encodes variable-length quantities, status bytes, and correct chunk lengths. You can copy this into a local file and run it to produce a valid .mid file.
def vlq(n: int) -> bytes:
if n < 0:
raise ValueError("Negative delta time not allowed")
stack = [n & 0x7F]
n >>= 7
while n:
stack.append((n & 0x7F) | 0x80)
n >>= 7
return bytes(reversed(stack))
def mthd(format_type=0, ntrks=1, division=480) -> bytes:
return b"MThd" + (6).to_bytes(4, "big") + format_type.to_bytes(2, "big") + ntrks.to_bytes(2, "big") + division.to_bytes(2, "big")
def mtrk(events: list[bytes]) -> bytes:
body = b"".join(events)
return b"MTrk" + len(body).to_bytes(4, "big") + body
def meta(delta, meta_type, data: bytes) -> bytes:
return vlq(delta) + bytes([0xFF, meta_type, len(data)]) + data
def note_on(delta, ch, key, vel) -> bytes:
return vlq(delta) + bytes([0x90 | (ch & 0x0F), key & 0x7F, vel & 0x7F])
def note_off(delta, ch, key, vel) -> bytes:
return vlq(delta) + bytes([0x80 | (ch & 0x0F), key & 0x7F, vel & 0x7F])
def program_change(delta, ch, program) -> bytes:
return vlq(delta) + bytes([0xC0 | (ch & 0x0F), program & 0x7F])
def control_change(delta, ch, ctrl, val) -> bytes:
return vlq(delta) + bytes([0xB0 | (ch & 0x0F), ctrl & 0x7F, val & 0x7F])
def set_tempo(delta, bpm) -> bytes:
mpqn = int(round(60_000_000 / bpm))
return meta(delta, 0x51, mpqn.to_bytes(3, "big"))
def end_of_track() -> bytes:
return meta(0, 0x2F, b"")
# Build a simple track: 120 BPM, program 0 (piano), C4-E4-G4 arpeggio
PPQ = 480
events = []
events.append(set_tempo(0, 120))
events.append(program_change(0, 0, 0))
events.append(control_change(0, 0, 7, 100)) # volume
events.append(note_on(0, 0, 60, 96))
events.append(note_off(PPQ // 2, 0, 60, 64))
events.append(note_on(0, 0, 64, 96))
events.append(note_off(PPQ // 2, 0, 64, 64))
events.append(note_on(0, 0, 67, 96))
events.append(note_off(PPQ // 2, 0, 67, 64))
events.append(end_of_track())
data = mthd() + mtrk(events)
with open("arp.mid", "wb") as f:
f.write(data)
print("Written arp.mid")
This code produces a correct file. The variable-length quantities are used for delta times. The tempo is encoded as a meta event with microseconds per quarter note derived from BPM. The program change is zero-based and selects Acoustic Grand Piano in GM. The note durations are half a quarter note each at the selected PPQ, and the final meta end-of-track delimits the track.
LLM-Assisted Generation with a Safe Mini-Language
An intermediate schema lets a model write structured performance data that you can validate strictly before conversion. A compact JSON plan could include fields for ppq, bpm, and a list of note objects with absolute tick starts and lengths. A sample plan might look like this:
{
"ppq": 480,
"bpm": 110,
"program": 0,
"notes": [
{"ch": 0, "key": 60, "vel": 96, "start": 0, "length": 480},
{"ch": 0, "key": 64, "vel": 96, "start": 480, "length": 480},
{"ch": 0, "key": 67, "vel": 96, "start": 960, "length": 480}
]
}
If you provide the model with this schema and several correct examples, it will tend to produce valid numbers and structures. Your converter sorts by start time, inserts note-on events at each start, inserts note-off events at start plus length, computes delta times by differences, and writes the file. If a note has a negative start or a velocity over 127, you can reject and ask the model to regenerate with the violations cited, which typically causes immediate correction.
LLM-Assisted Analysis with Deterministic Timing
A robust analysis pipeline first canonicalizes the file. You parse each track into absolute tick times, apply all tempo events to derive a piecewise mapping from ticks to microseconds, and then compute absolute times for each note-on and note-off. You then derive durations in milliseconds, inter-onset intervals, pitch-class histograms, and controller envelopes. When you present these derived features to an LLM, it can explain phrasing choices, detect swing feel by comparing offbeat timing against a triplet grid, and identify expressive intent by correlating velocity, duration, and controller changes. The key is that the model reasons about the musical meaning while your code provides numerically exact measurements.
Best Practices for Cross-Device Reliability
Device initialization avoids surprises. If you rely on a particular sound, send bank select MSB and LSB followed by program change at the start of a track. If you rely on pitch bend in semitone units, set the bend range via RPN. If you rely on modulation depth to drive vibrato on a given synth, test the mapping and send an initial controller value to a known state. If you stream MIDI, consider sending an all-notes-off command such as controller 123 with value 0 on each channel during transport stop to prevent stuck notes. If you want tempo-synced effects, ensure that your device listens to MIDI clock and that you start the transport with a 0xFA before clocking to give devices time to latch sync.
Conclusion
MIDI remains the universal language of electronic musical performance because it is compact, expressive, and device-agnostic. A solid understanding of channels, status bytes, controllers, meta events, and timing unlocks precise control of instruments and workflows. Small, correct examples like a hand-assembled SMF and carefully constructed hex messages provide anchoring intuition. Large language models fit naturally into this world when they are guided by deterministic encoders and validators. They can analyze structure, infer style, and propose musical plans while code guarantees the low-level correctness of bytes, delta times, and ranges. With this partnership, you can compose and transform rich musical performances that are both creative and technically impeccable.
LLM Integration: End-to-end generation and analysis loop
Overview
This chapter extends the toolkit with a complete, runnable LLM integration loop. It supports two workflows. In the generation workflow, you describe the music in natural language, the LLM responds with a strict JSON plan in the toolkit’s schema, the program validates and writes a correct MIDI file, and then reports what it created. In the analysis workflow, you point the program at an existing MIDI file, it computes precise musical features, the program summarizes them into a compact prompt, and the LLM returns an interpretive analysis or transformation suggestions. The integration is transport-agnostic and uses a simple “LLM client” abstraction so you can plug in your preferred provider. For security and reproducibility, the LLM is never allowed to write bytes directly; it only emits structured JSON that our validator checks.
Below is a single, self-contained script that includes everything from the previous chapter plus the LLM integration. You can paste it into a file named midi_llm_toolkit_llm.py and run it. To connect to your LLM, implement one small method that calls your provider’s SDK or HTTP API and returns text. Place your API key in an environment variable and keep it out of your source control.
# midi_llm_toolkit_llm.py
# Complete analyzer/generator with LLM integration.
# - Deterministic MIDI parse/analyze/generate core (SMF Format 1 with PPQ)
# - LLM generation: text prompt -> strict JSON plan -> MIDI file
# - LLM analysis: MIDI file -> features JSON -> LLM narrative analysis
#
# Quick start:
# python midi_llm_toolkit_llm.py --generate "chill piano arpeggio at 90 bpm, 2 bars in C major" out.mid
# python midi_llm_toolkit_llm.py --analyze input.mid
#
# Configure your LLM by setting: export LLM_API_KEY=... and editing LLMClient.call_model()
from __future__ import annotations
from typing import List, Tuple, Dict, Any, Optional
import json, os, sys, argparse, time
########################
# Low-level MIDI encode
########################
def vlq(n: int) -> bytes:
if n < 0:
raise ValueError("Negative delta time not allowed")
stack = [n & 0x7F]
n >>= 7
while n:
stack.append((n & 0x7F) | 0x80)
n >>= 7
return bytes(reversed(stack))
def mthd(format_type=1, ntrks=1, division=480) -> bytes:
return b"MThd" + (6).to_bytes(4, "big") + format_type.to_bytes(2, "big") + ntrks.to_bytes(2, "big") + division.to_bytes(2, "big")
def mtrk(events: List[bytes]) -> bytes:
body = b"".join(events)
return b"MTrk" + len(body).to_bytes(4, "big") + body
def meta(delta, meta_type, data: bytes) -> bytes:
return vlq(delta) + bytes([0xFF, meta_type]) + vlq(len(data)) + data
def note_on(delta, ch, key, vel) -> bytes:
return vlq(delta) + bytes([0x90 | (ch & 0x0F), key & 0x7F, vel & 0x7F])
def note_off(delta, ch, key, vel) -> bytes:
return vlq(delta) + bytes([0x80 | (ch & 0x0F), key & 0x7F, vel & 0x7F])
def program_change(delta, ch, program) -> bytes:
return vlq(delta) + bytes([0xC0 | (ch & 0x0F), program & 0x7F])
def control_change(delta, ch, ctrl, val) -> bytes:
return vlq(delta) + bytes([0xB0 | (ch & 0x0F), ctrl & 0x7F, val & 0x7F])
def set_tempo(delta, bpm) -> bytes:
mpqn = int(round(60_000_000 / float(bpm)))
return meta(delta, 0x51, mpqn.to_bytes(3, "big"))
def track_name(delta, name: str) -> bytes:
data = name.encode("utf-8")
return meta(delta, 0x03, data)
def end_of_track() -> bytes:
return meta(0, 0x2F, b"")
########################
# Minimal SMF reader
########################
class MidiParseError(Exception):
pass
class ByteStream:
def __init__(self, data: bytes):
self.data = data
self.i = 0
def read(self, n: int) -> bytes:
if self.i + n > len(self.data):
raise MidiParseError("Unexpected EOF")
b = self.data[self.i:self.i+n]
self.i += n
return b
def read_u8(self) -> int:
return self.read(1)[0]
def read_u16(self) -> int:
return int.from_bytes(self.read(2), "big")
def read_u32(self) -> int:
return int.from_bytes(self.read(4), "big")
def read_vlq(self) -> int:
val = 0
while True:
b = self.read_u8()
val = (val << 7) | (b & 0x7F)
if (b & 0x80) == 0:
break
return val
def parse_midi(data: bytes) -> Dict[str, Any]:
s = ByteStream(data)
if s.read(4) != b"MThd":
raise MidiParseError("Missing MThd")
header_len = s.read_u32()
if header_len != 6:
header = s.read(header_len)
if len(header) < 6:
raise MidiParseError("Invalid MThd length")
format_type = int.from_bytes(header[0:2], "big")
ntrks = int.from_bytes(header[2:4], "big")
division = int.from_bytes(header[4:6], "big")
else:
format_type = s.read_u16()
ntrks = s.read_u16()
division = s.read_u16()
if division & 0x8000:
raise MidiParseError("SMPTE timecode division not supported")
tracks = []
for _ in range(ntrks):
if s.read(4) != b"MTrk":
raise MidiParseError("Missing MTrk")
length = s.read_u32()
track_data = ByteStream(s.read(length))
events = []
abs_ticks = 0
running_status = None
while track_data.i < len(track_data.data):
delta = track_data.read_vlq()
abs_ticks += delta
status = track_data.read_u8()
if status < 0x80:
track_data.i -= 1
if running_status is None:
raise MidiParseError("Running status without prior status")
status_byte = running_status
else:
status_byte = status
running_status = status if status < 0xF0 else None
if status_byte == 0xFF:
meta_type = track_data.read_u8()
length = track_data.read_vlq()
meta_data = track_data.read(length)
events.append(("meta", abs_ticks, meta_type, meta_data))
running_status = None
elif status_byte in (0xF0, 0xF7):
length = track_data.read_vlq()
sysx_data = track_data.read(length)
events.append(("sysex", abs_ticks, status_byte, sysx_data))
running_status = None
else:
msg_type = status_byte & 0xF0
ch = status_byte & 0x0F
if msg_type in (0x80, 0x90, 0xA0, 0xB0, 0xE0):
d1 = track_data.read_u8()
d2 = track_data.read_u8()
events.append(("chan", abs_ticks, msg_type, ch, d1, d2))
elif msg_type in (0xC0, 0xD0):
d1 = track_data.read_u8()
events.append(("chan", abs_ticks, msg_type, ch, d1, None))
else:
raise MidiParseError(f"Unknown status byte {status_byte:#x}")
tracks.append(events)
return {"format": format_type, "division": division, "tracks": tracks}
########################
# Tempo map and timing
########################
def build_tempo_map(tracks: List[List[Tuple]]) -> List[Tuple[int, int]]:
tempo_events = []
for events in tracks:
for e in events:
if e[0] == "meta" and e[2] == 0x51 and len(e[3]) == 3:
mpqn = int.from_bytes(e[3], "big")
tempo_events.append((e[1], mpqn))
if not tempo_events:
tempo_events = [(0, 500_000)]
tempo_events.sort(key=lambda x: x[0])
dedup = []
for t, mpqn in tempo_events:
if dedup and dedup[-1][0] == t:
dedup[-1] = (t, mpqn)
else:
dedup.append((t, mpqn))
return dedup
def ticks_to_ms(abs_tick: int, tempo_map: List[Tuple[int, int]], ppq: int) -> float:
ms = 0.0
last_tick = 0
for t_tick, mpqn in tempo_map:
if abs_tick <= t_tick:
break
seg_ticks = t_tick - last_tick
ms += seg_ticks * (mpqn / 1000.0) / ppq
last_tick = t_tick
current_mpqn = None
for t_tick, mpqn in tempo_map:
if t_tick <= abs_tick:
current_mpqn = mpqn
else:
break
if current_mpqn is None:
current_mpqn = 500_000
ms += (abs_tick - last_tick) * (current_mpqn / 1000.0) / ppq
return ms
########################
# Note extraction and analysis
########################
def extract_notes(parsed: Dict[str, Any]) -> List[Dict[str, Any]]:
ppq = parsed["division"]
tempo_map = build_tempo_map(parsed["tracks"])
active = {}
notes = []
for events in parsed["tracks"]:
for e in events:
if e[0] != "chan":
continue
_, abs_tick, msg_type, ch, d1, d2 = e
if msg_type == 0x90 and d2 != 0:
active.setdefault((ch, d1), []).append((abs_tick, d2))
elif msg_type == 0x80 or (msg_type == 0x90 and d2 == 0):
stack = active.get((ch, d1), [])
if stack:
on_tick, vel = stack.pop(0)
start_ms = ticks_to_ms(on_tick, tempo_map, ppq)
end_ms = ticks_to_ms(abs_tick, tempo_map, ppq)
notes.append({
"ch": ch, "key": d1, "vel": vel,
"start_tick": on_tick, "end_tick": abs_tick,
"start_ms": start_ms, "end_ms": end_ms
})
notes.sort(key=lambda n: (n["start_tick"], n["ch"], n["key"]))
return notes
def analyze_midi_file(path: str) -> Dict[str, Any]:
with open(path, "rb") as f:
data = f.read()
parsed = parse_midi(data)
ppq = parsed["division"]
tempo_map = build_tempo_map(parsed["tracks"])
notes = extract_notes(parsed)
if notes:
total_ms = max(n["end_ms"] for n in notes) - min(n["start_ms"] for n in notes)
else:
total_ms = 0.0
pitch_classes = [n["key"] % 12 for n in notes]
pc_hist = {pc: pitch_classes.count(pc) for pc in range(12)}
durations_ms = [n["end_ms"] - n["start_ms"] for n in notes]
avg_dur = sum(durations_ms) / len(durations_ms) if durations_ms else 0.0
onsets_by_ch = {}
for n in notes:
onsets_by_ch.setdefault(n["ch"], []).append(n["start_ms"])
iois = []
for _, onsets in onsets_by_ch.items():
onsets.sort()
for i in range(1, len(onsets)):
iois.append(onsets[i] - onsets[i-1])
avg_ioi = sum(iois) / len(iois) if iois else 0.0
staccato_ratio = 0.0
if iois:
short = 0
count = 0
by_ch = {}
for n in notes:
by_ch.setdefault(n["ch"], []).append(n)
for ch, ns in by_ch.items():
ns.sort(key=lambda x: x["start_ms"])
for i in range(len(ns)-1):
dur = ns[i]["end_ms"] - ns[i]["start_ms"]
gap = ns[i+1]["start_ms"] - ns[i]["start_ms"]
if gap > 0:
if dur < 0.6 * gap:
short += 1
count += 1
staccato_ratio = short / count if count else 0.0
return {
"ppq": ppq,
"tempo_map": tempo_map,
"note_count": len(notes),
"duration_ms": total_ms,
"avg_note_duration_ms": avg_dur,
"avg_ioi_ms": avg_ioi,
"staccato_ratio": staccato_ratio,
"pitch_class_histogram": pc_hist,
"notes_preview": notes[:10],
}
########################
# Plan validation and generation
########################
def validate_plan(plan: Dict[str, Any]) -> Dict[str, Any]:
if "ppq" not in plan or int(plan["ppq"]) <= 0:
raise ValueError("Plan must include positive integer 'ppq'")
ppq = int(plan["ppq"])
if "bpm" not in plan or float(plan["bpm"]) <= 0:
raise ValueError("Plan must include positive 'bpm'")
bpm = float(plan["bpm"])
top_program = int(plan.get("program", 0))
if not (0 <= top_program <= 127):
raise ValueError("Top-level 'program' must be 0..127")
tracks = plan.get("tracks")
if tracks is None:
notes = plan.get("notes", [])
# Accept "ch" at note-level for backward compatibility
tracks = [{"name": "Track 1", "channel": 0, "program": top_program, "notes": notes}]
validated_tracks = []
for idx, tr in enumerate(tracks):
name = tr.get("name", f"Track {idx+1}")
ch = int(tr.get("channel", 0))
if not (0 <= ch <= 15):
raise ValueError("Channel must be 0..15")
program = int(tr.get("program", top_program))
if not (0 <= program <= 127):
raise ValueError("Program must be 0..127")
vnotes = []
for n in tr.get("notes", []):
key = int(n["key"])
vel = int(n["vel"])
start = int(n["start"])
length = int(n["length"])
if not (0 <= key <= 127):
raise ValueError("key out of range 0..127")
if not (1 <= vel <= 127):
raise ValueError("vel out of range 1..127")
if start < 0 or length <= 0:
raise ValueError("start must be >=0 and length > 0")
vnotes.append({"key": key, "vel": vel, "start": start, "length": length})
vnotes.sort(key=lambda x: (x["start"], x["key"]))
validated_tracks.append({"name": name, "channel": ch, "program": program, "notes": vnotes})
return {"ppq": ppq, "bpm": bpm, "tracks": validated_tracks}
def write_midi_from_plan(plan: Dict[str, Any], out_path: str) -> None:
vp = validate_plan(plan)
ppq = vp["ppq"]
bpm = vp["bpm"]
trks = []
for tr in vp["tracks"]:
events = []
if not trks:
events.append(set_tempo(0, bpm))
if tr["name"]:
events.append(track_name(0, tr["name"]))
events.append(program_change(0, tr["channel"], tr["program"]))
pending = []
for note in tr["notes"]:
start = note["start"]
end = note["start"] + note["length"]
key = note["key"]
vel = note["vel"]
ch = tr["channel"]
pending.append((start, "on", ch, key, vel))
pending.append((end, "off", ch, key, 64))
pending.sort(key=lambda x: (x[0], 0 if x[1]=="off" else 1))
last_tick = 0
for tick, kind, ch, key, val in pending:
delta = tick - last_tick
if kind == "on":
events.append(note_on(delta, ch, key, val))
else:
events.append(note_off(delta, ch, key, val))
last_tick = tick
events.append(end_of_track())
trks.append(mtrk(events))
data = mthd(1, len(trks), ppq) + b"".join(trks)
with open(out_path, "wb") as f:
f.write(data)
########################
# LLM integration
########################
LLM_PLAN_SYSTEM_PROMPT = """You are a MIDI composition assistant. Produce ONLY a JSON object that strictly matches this schema:
{
"ppq": <positive int, e.g., 480>,
"bpm": <positive number>,
"program": <0..127, default 0>,
"tracks": [
{
"name": <short string>,
"channel": <0..15>,
"program": <0..127>,
"notes": [
{"key": <0..127>, "vel": <1..127>, "start": <>=0 int>, "length": <>>0 int>
]
}
]
}
Constraints: times are absolute ticks; notes must be sorted non-decreasing by start; no overlapping note-ons without matching offs for the same key+channel; keep values in range; keep it concise. Do not include comments, trailing commas, extra fields, or prose. Output only JSON.
"""
LLM_ANALYSIS_SYSTEM_PROMPT = """You are a MIDI analyst. You will receive JSON with precise features from a parsed MIDI file. Provide a concise, insightful prose interpretation that addresses tempo, density, articulation, pitch-class tendencies, and any stylistic hints. Suggest one or two concrete edits or generative transformations. Keep it under 200 words. Output only prose without JSON.
"""
def json_sanitize(s: str) -> str:
# Extract the first well-formed JSON object from the string
# This tolerates accidental prose by the model; we enforce JSON-only in prompt, but we guard anyway.
start = s.find("{")
end = s.rfind("}")
if start == -1 or end == -1 or end < start:
raise ValueError("No JSON object found in LLM output")
frag = s[start:end+1]
return frag
class LLMClient:
def __init__(self, model: str = "your-model-name"):
self.model = model
self.api_key = os.getenv("LLM_API_KEY", "")
def call_model(self, system_prompt: str, user_prompt: str, temperature: float = 0.2) -> str:
# Replace this stub with your provider call. Examples include OpenAI, Anthropic, Azure, etc.
# The function must return the assistant text string.
# This stub raises unless you intentionally return a canned result for testing.
if not self.api_key:
# For offline testing, return a minimal valid JSON plan to avoid hard dependency on a provider.
if "Produce ONLY a JSON object" in system_prompt:
return json.dumps({
"ppq": 480,
"bpm": 90,
"program": 0,
"tracks": [
{
"name": "Piano",
"channel": 0,
"program": 0,
"notes": [
{"key": 60, "vel": 96, "start": 0, "length": 480},
{"key": 64, "vel": 92, "start": 480, "length": 480},
{"key": 67, "vel": 88, "start": 960, "length": 960}
]
}
]
})
else:
return "The piece maintains a steady tempo with moderate density and mostly diatonic pitch classes around C major. Articulation leans toward detached eighths with occasional longer tones. To develop it, add a simple left-hand pattern outlining I–V–vi–IV and apply a gentle modulation wheel swell into each bar."
# Example: OpenAI Chat Completions pattern (pseudo):
# import openai
# openai.api_key = self.api_key
# resp = openai.chat.completions.create(
# model=self.model,
# messages=[{"role":"system","content":system_prompt},
# {"role":"user","content":user_prompt}],
# temperature=temperature,
# )
# return resp.choices[0].message.content
raise RuntimeError("LLM provider call not implemented. Set LLM_API_KEY or implement call_model().")
def llm_generate_plan_text(description: str, ppq_default: int = 480) -> Dict[str, Any]:
client = LLMClient()
user_prompt = (
f"Compose a short piece as described:\n"
f"Description: {description}\n"
f"Use ppq {ppq_default} unless musical reasons justify otherwise."
)
text = client.call_model(LLM_PLAN_SYSTEM_PROMPT, user_prompt, temperature=0.3)
json_text = json_sanitize(text)
try:
plan = json.loads(json_text)
except Exception as e:
raise ValueError(f"LLM did not return valid JSON: {e}\nRaw: {text[:500]}")
return plan
def llm_analyze_features(features: Dict[str, Any]) -> str:
client = LLMClient()
# Keep the JSON compact to fit provider limits while retaining key info
compact = {
"ppq": features["ppq"],
"tempo_map": features["tempo_map"][:6],
"note_count": features["note_count"],
"duration_ms": round(features["duration_ms"], 2),
"avg_note_duration_ms": round(features["avg_note_duration_ms"], 2),
"avg_ioi_ms": round(features["avg_ioi_ms"], 2),
"staccato_ratio": round(features["staccato_ratio"], 3),
"pitch_class_histogram": features["pitch_class_histogram"],
"notes_preview": features["notes_preview"],
}
user_prompt = "Analyze this MIDI feature JSON and respond with prose:\n" + json.dumps(compact)
return client.call_model(LLM_ANALYSIS_SYSTEM_PROMPT, user_prompt, temperature=0.2)
########################
# CLI glue
########################
def analyze_cli(path: str) -> None:
feats = analyze_midi_file(path)
print(json.dumps(feats, indent=2))
narrative = llm_analyze_features(feats)
print("\nLLM analysis:\n" + narrative)
def generate_cli(description: str, out_path: str) -> None:
print(f"Requesting plan from LLM for: {description}")
plan = llm_generate_plan_text(description)
print("Plan from LLM:")
print(json.dumps(plan, indent=2))
try:
write_midi_from_plan(plan, out_path)
except Exception as e:
# If validation fails, relay feedback to the user for a quick retry
print(f"Plan validation or writing failed: {e}")
raise
print(f"Wrote MIDI to {out_path}")
def main():
parser = argparse.ArgumentParser(description="LLM-powered MIDI analyzer/generator")
sub = parser.add_subparsers(dest="cmd")
ga = sub.add_parser("--generate", help="Generate MIDI from text")
ga.add_argument("description", type=str, help="Text description of the music")
ga.add_argument("out_path", type=str, help="Output .mid file path")
aa = sub.add_parser("--analyze", help="Analyze an existing MIDI file")
aa.add_argument("path", type=str, help="Path to .mid file")
args = parser.parse_args()
if args.cmd == "--generate":
generate_cli(args.description, args.out_path)
elif args.cmd == "--analyze":
analyze_cli(args.path)
else:
parser.print_help()
if __name__ == "__main__":
main()
How to plug in your LLM
You only need to implement LLMClient.call_model. Choose your provider’s Python SDK or HTTP endpoint, pass a system message and a user message, and return the assistant’s text. Keep temperature low for plan generation to minimize out-of-range values. For safety, the program extracts the first JSON object if the model adds extra tokens, then parses and validates it. If your provider supports response format constraints such as JSON tools or function calling, you can tighten correctness further by declaring the plan schema and letting the model return a structured object natively.
Prompting tips that reduce errors
You get the most consistent plans when you specify PPQ, BPM, a target program number, and the number of bars with an explicit meter. If you also supply a pitch range such as C3 to C5 and a density like two notes per beat, the model produces cleaner onsets. For multitrack plans, mention roles by channel and instrument, for example a bass on channel one with program 32 and a piano on channel two with program zero. If the validator rejects the plan for a range error, reflect the exact message back to the model and request a corrected plan in the same schema, which typically converges in one regeneration.
End-to-end examples
If you run the generation command without an API key, the script uses a deterministic fallback that returns a valid, minimal plan for quick testing. When you set your API key and implement the provider call, you can request richer prompts such as a four-bar swing in F major at 140 BPM with ride cymbal on channel ten and walking bass on channel one, and the loop will write a correct SMF that opens cleanly in your DAW. For analysis, point it at any SMF and the program prints exact numeric features and a short LLM narrative that interprets tempo, density, articulation, and pitch use, along with concrete next steps like adding a counterline or changing modulation depth.
No comments:
Post a Comment