MOTIVATION
Resistance is futile. That sentence, once confined to the mythology of science fiction, has taken on an unexpected second life in the offices of enterprise software architects. Agentic AI systems have crossed from research curiosity to operational backbone. They schedule meetings, draft legal briefs, coordinate supply chains, and debug production systems, often without a human ever touching a keyboard. The question is no longer whether your organisation will deploy autonomous AI agents. The question is whether you will deploy them well.
This article argues that the answer lies in architecture, specifically in an architecture drawn from one of the most enduring metaphors in popular science fiction: the Borg Collective. A hive mind of perfectly coordinated agents, each specialised yet interchangeable, governed by a single point of intelligence but resilient enough to function without it. Sound familiar? It should, because that is precisely what a well-designed agentic AI platform looks like.
The Borg Collective Architecture presented here is not a theoretical exercise. It is a concrete, production-ready design implemented in Rust, built on hexagonal layering, event-driven communication, and a careful taxonomy of reliability patterns. Every design decision has a reason, and this article will give you that reason alongside the code.
CHAPTER 1: THE DAWN OF AGENTIC AI
Cast your mind back to early 2024. ChatGPT had existed for barely fifteen months. The most sophisticated AI systems most organisations had were chat assistants that answered questions and occasionally hallucinated citations. By mid-2025, foundation models had improved to the point where single-shot reasoning was reliable enough for production use. Tool use, function calling, and retrieval-augmented generation had matured. And then something shifted.
The shift was agency. Not a single model answering a question, but multiple models, each with specialised capabilities, collaborating autonomously to complete tasks that previously required teams of humans. A legal-tech startup deployed an agent that ingested a 400-page contract, identified ambiguous clauses, proposed rewrites consistent with the firm's historical precedents, and submitted the revised document for human review, all within forty minutes. A logistics company built an agent network that rerouted twelve thousand shipments around a port closure, negotiating with carrier APIs in real time.
These systems did not arrive without growing pains. The first generation of agentic platforms were fragile, expensive, and opaque. They ran on Python orchestration frameworks designed for single-machine execution. They had no meaningful security boundaries between agents. They leaked credentials, spun runaway LLM loops that consumed thousands of dollars of API credit overnight, and provided no audit trail when something went wrong.
By September 2026, the industry understands what went wrong. Agentic AI is not merely LLM-in-a-loop. It is a distributed system, and it demands the engineering discipline that distributed systems have required for decades: strong typing, isolation boundaries, structured communication, observability, and principled fault tolerance. The Borg Collective Architecture addresses each of these concerns head on. It is not the only valid approach, but it is a coherent one, and coherence is the rarest commodity in a field moving this fast.
CHAPTER 2: WHY RUST IS THE LANGUAGE OF THE COLLECTIVE
Every architecture begins with a language choice, and every language choice is a statement of values. Choosing Rust for an enterprise agentic AI platform is a statement that says: we value correctness over convenience, performance over flexibility, and explicit resource management over garbage-collected simplicity.
That might sound austere, but consider the operational context. A production agentic platform runs continuously, handles thousands of concurrent agent executions, streams audio and video through adapter plugins, and must never crash because an agent mis-allocated memory. Python, the lingua franca of the AI world, is excellent for experimentation but ill-suited to these demands. Go is fast and concurrent but lacks the zero-cost abstractions and ownership model that prevent entire categories of bugs at compile time. C and C++ provide the necessary performance but require heroic discipline to use safely at scale.
Rust threads this needle. Its ownership and borrow-checker system eliminates use-after-free, double-free, and data-race bugs, not at runtime through a garbage collector that pauses at inconvenient moments, but at compile time, before a single byte of code ever runs. In a long-running agentic system where individual agents can outlive their originating request by minutes or hours, the difference between a garbage-collected language and Rust can be the difference between a platform that degrades under load and one that maintains sub-millisecond tail latencies indefinitely.
Rust's second great advantage for this platform is its first-class support for WebAssembly. The Borg Collective's security architecture compiles every agent's capability plugin to WebAssembly bytecode and executes it inside a Wasmtime 26.x runtime sandbox. This is not merely a theoretical security boundary; it is a hardware-enforced isolation mechanism that confines what a plugin can read, write, and call, regardless of what the underlying AI model tries to do. Writing both the host runtime and the plugin interface in Rust means the boundary contracts are expressed as Rust types, and the compiler ensures they are honoured on both sides.
The Actix actor framework, built on top of Tokio 1.42's asynchronous runtime, provides the concurrency model. Each agent in the Collective runs as an Actix 0.13 actor, receiving typed messages and responding asynchronously without shared mutable state. The actor model maps naturally to the agent concept: each actor has its own inbox, its own isolated state, and communicates only by passing messages. Scaling from ten agents to ten thousand requires no architectural changes; the Tokio scheduler simply distributes the work across available CPU cores.
Consider how this plays out in the domain model. The very first type a new contributor encounters in the Borg Collective codebase is AgentId, a strongly-typed value object that wraps a UUID:
use std::fmt;
use uuid::Uuid;
/// AgentId is a strongly-typed value object wrapping a UUID.
/// The newtype pattern prevents accidentally passing a raw Uuid
/// where an AgentId is expected, catching entire classes of
/// misidentification bugs at compile time, long before they
/// reach a production environment.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct AgentId(Uuid);
impl AgentId {
/// Creates a new AgentId backed by a cryptographically random
/// UUID. The randomness comes from the system's entropy source,
/// making collisions astronomically unlikely even at scale.
pub fn new() -> Self {
AgentId(Uuid::new_v4())
}
/// Exposes the inner UUID for serialisation or persistence
/// layers without leaking the representation into domain logic.
pub fn as_uuid(&self) -> &Uuid {
&self.0
}
}
impl fmt::Display for AgentId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
// The "agent:" prefix makes IDs visually distinct in log
// streams and audit trails without requiring a separate
// type-discriminator field.
write!(f, "agent:{}", self.0)
}
}
This tiny type is the foundation on which all agent identity rests in the system. It illustrates a principle that runs throughout the entire codebase: the domain model uses the type system to encode business invariants. An AgentId cannot be accidentally confused with a CubeId, a TenantId, or a raw string, because they are distinct types. A function that expects an AgentId will simply not compile if you hand it anything else. This is the zero-cost abstraction Rust is famous for: you pay nothing at runtime for the guarantee you receive at compile time.
The payoff for this discipline becomes visible when the codebase grows to hundreds of thousands of lines and dozens of contributors. Refactoring a field that changes from a raw string to an AgentId causes a cascade of compiler errors that point precisely to every place that needs updating, with no runtime test required. The compiler is the first line of defence, and in a codebase of this complexity, it is by far the most valuable one.
CHAPTER 3: THE ARCHITECTURE PHILOSOPHY -- HEXAGONAL LAYERS AND EVENTS
Every large software system eventually faces the same crisis. The initial architecture, whatever it was, begins to show its seams. Business logic becomes entangled with database access patterns. The HTTP layer leaks into the domain model. Tests become slow and brittle because they require real databases and real external services. Changing one provider means touching dozens of files.
The Borg Collective Architecture is designed to resist this decay. It draws on two established patterns: Hexagonal Architecture (also known as Ports and Adapters, coined by Alistair Cockburn in his 2005 paper) and Event-Driven Architecture. Together, they produce a system where the domain model is pure, the integration points are explicit, and the communication fabric is decoupled.
In hexagonal thinking, the application sits at the centre of a hexagon. On one side are the primary ports, the entry points through which the world drives the application. On the other side are the secondary ports, the interfaces through which the application reaches out to the world. Concrete adapters implement those interfaces. The critical rule is that the domain model depends on nothing outside itself. It does not import database drivers, HTTP clients, or AI provider SDKs. It knows only abstract interfaces, expressed as Rust traits.
The primary port for LLM access in the Borg Collective is a trait called ILlmPort. Every piece of agent logic that needs to call a language model does so through this interface, and nowhere else:
use async_trait::async_trait;
/// LlmRequest encapsulates everything needed to call any language
/// model, regardless of whether it runs on a remote API or locally
/// through an Ollama instance. The domain model constructs one of
/// these and passes it to the port without ever knowing which model
/// will ultimately respond.
#[derive(Debug, Clone)]
pub struct LlmRequest {
/// The model identifier, e.g. "gpt-5" or "claude-4-opus".
pub model: String,
/// The ordered conversation turns to submit to the model.
pub messages: Vec<ChatMessage>,
/// Sampling temperature, where 0.0 is deterministic and 1.0
/// is highly creative.
pub temperature: f32,
/// Hard token ceiling for the generated response.
pub max_tokens: u32,
}
/// ILlmPort is the hexagonal port through which all domain logic
/// reaches the language model tier. No provider details, API keys,
/// or HTTP machinery cross this boundary. Swapping GPT-5 for
/// Claude 4 Opus requires only a new adapter implementation -- the
/// domain code never needs to change.
#[async_trait]
pub trait ILlmPort: Send + Sync {
/// Submits a request to the backing model and returns the
/// generated completion text, or a typed error that the caller
/// can pattern-match to distinguish transient from permanent
/// failures.
async fn complete(
&self,
req: LlmRequest,
) -> Result<String, LlmError>;
/// Returns the canonical model identifier for observability
/// purposes, such as "gpt-5" or "claude-4-opus".
fn model_name(&self) -> &str;
}
The ILlmPort trait is the boundary between the domain and the outside world. Behind it, four concrete adapters exist in the Borg Collective codebase: one for OpenAI's GPT-5 and GPT-4o, one for Anthropic's Claude 4 family, one for Google's Gemini 2.5 Pro, and one for locally-hosted models served through Ollama -- which supports Llama 4, Mistral Large 3, and dozens of community-maintained models. The domain model never sees any of these. It sees only ILlmPort.
This matters for testability as much as for flexibility. The test suite for domain logic uses a MockLlmPort that returns canned responses in microseconds, with no network dependency whatsoever. The same tests that run in CI in two seconds would take thirty seconds with real network calls. More importantly, mock adapters enable property-based testing: you can flood the mock with ten thousand diverse inputs and verify that domain invariants hold, something impractical with a metered production API where each call costs money and time.
Event-driven communication complements the hexagonal model. Within a single deployment, the domain model communicates with other subsystems by publishing events to NATS JetStream subjects rather than calling other services directly, which means two services can evolve independently as long as the event schema remains compatible. Schema evolution is managed through Protobuf-defined message types with explicit field deprecation rules, giving you the contract safety of a strongly-typed system without locking the entire organisation into a single serialisation format forever.
CHAPTER 4: THE COLLECTIVE HIERARCHY -- QUEEN, CUBES, AND DRONES
With the philosophical foundation established, we can examine the structural heart of the Borg Collective Architecture: the three-tier actor hierarchy. The names are chosen deliberately. The Star Trek Borg is one of science fiction's most enduring images of a distributed intelligence: a hive of specialised units coordinated by a central consciousness, each unit expendable, the whole far greater than the sum of its parts. The architecture mirrors this image with precision.
The SuperQueen is the top of the hierarchy. There is exactly one SuperQueen per deployment. It is an Actix actor that bootstraps the entire system, maintains the registry of active Cubes, enforces global policies, and serves as the entry point for all external work requests. The SuperQueen does not execute agent logic itself. Its sole responsibility is coordination: deciding which Cube should handle a given workload, spawning new Cubes when capacity demands, and draining Cubes gracefully during planned maintenance windows.
A Cube is an isolation boundary. Each Cube is an Actix actor that owns a bounded pool of Drone agents. Cubes are the enforcement point for resource quotas: a Cube configured to run at most eight Drones will never run nine, regardless of how much work is queued. Cubes also own the lifecycle of their Drones, restarting failed Drones according to a supervisor policy, evacuating Drones gracefully when the Cube itself is being shut down, and reporting health metrics to the SuperQueen on a configurable heartbeat interval.
The Drone is where the work actually happens. Each Drone is a specialised agent that executes a specific task pattern, such as research, summarisation, code generation, or multi-step planning. Drones are designed to be stateless with respect to inter-task state: any data that needs to persist across invocations is written to the LLM Wiki memory tier, not stored in the actor's local fields. This statelessness is what makes Drones interchangeable and allows the Cube to restart a crashed Drone without losing work in progress.
The message that connects these tiers is the SpawnDrone command, delivered from the SuperQueen to a Cube. The handler illustrates the actor pattern in its cleanest form:
use actix::prelude::*;
use std::collections::HashMap;
/// SpawnDrone instructs a CubeActor to instantiate a new Drone capable
/// of fulfilling the requested capability. The result is the freshly
/// assigned AgentId for the newly spawned Drone.
#[derive(Message)]
#[rtype(result = "Result<AgentId, SpawnError>")]
pub struct SpawnDrone {
/// The logical capability name, such as "research" or "summarise".
pub capability: String,
/// The initial task context that the new Drone will begin executing
/// immediately upon assimilation into the Collective.
pub task_context: TaskContext,
}
/// QueenActor is the top-level supervisor for the entire Collective.
/// It maintains a live map of Cube addresses and dispatches work into
/// the Cube best positioned to accept it based on load and capability.
pub struct QueenActor {
/// Maps each active CubeId to its Actix mailbox address, allowing
/// the Queen to send messages to any Cube without holding a
/// direct reference to the Cube's internal state.
cubes: HashMap<CubeId, Addr<CubeActor>>,
/// Monotonically increasing logical clock, incremented on every
/// message handled, used for causal ordering of audit events even
/// when wall-clock timestamps lack sufficient resolution.
logical_clock: u64,
}
impl Actor for QueenActor {
type Context = Context<Self>;
fn started(&mut self, _ctx: &mut Context<Self>) {
tracing::info!("SuperQueen online -- the Collective awakens");
}
}
impl Handler<SpawnDrone> for QueenActor {
type Result = ResponseFuture<Result<AgentId, SpawnError>>;
fn handle(
&mut self,
msg: SpawnDrone,
_ctx: &mut Context<Self>,
) -> Self::Result {
// Increment the logical clock before every outbound delegation
// so that downstream audit entries can be causally ordered
// even across concurrent message arrivals.
self.logical_clock += 1;
let clock = self.logical_clock;
// Select the Cube with the lowest current load that advertises
// the requested capability, or spawn a fresh Cube if none
// currently qualify.
let cube_addr = self
.select_cube_for(&msg.capability)
.expect("Cube selection must not fail in normal operation");
Box::pin(async move {
// Forward the spawn command into the selected Cube and
// await the AgentId it assigns to the new Drone. A
// MailboxError indicates the Cube shut down unexpectedly,
// which is converted into a typed SpawnError.
cube_addr
.send(msg)
.await
.map_err(|_| SpawnError::MailboxClosed(clock))?
})
}
}
The actor model gives the Collective an important emergent property: backpressure. If the QueenActor's mailbox fills up because all Cubes are saturated, the system naturally slows down rather than spawning unbounded work that would exhaust available memory. This is the kind of emergent reliability that comes from choosing the right concurrency model at the foundation, rather than bolting rate-limiting onto an architecture that was never designed for it.
The three-tier hierarchy also provides a natural administrative boundary. Operators manage Cubes as units: they can drain a Cube for maintenance, adjust its Drone pool size without touching the SuperQueen, and monitor its resource usage independently of other Cubes. When different tenants in a multi-tenant deployment need strict resource isolation, each tenant gets one or more dedicated Cubes, with the SuperQueen enforcing the routing rules that keep tenant work segregated at the Drone level.
CHAPTER 5: THE NEURAL SPINE -- NATS JETSTREAM AND THE MESSAGE FABRIC
If the actor hierarchy is the skeleton of the Borg Collective, NATS JetStream is its nervous system. Every significant event in the system, from a task request arriving from an external API to a Drone completing its work, flows through a structured message envelope on a NATS subject. The design has three main benefits: decoupling, durability, and observability.
NATS JetStream, at version 2.11 in September 2026, is a durable, at-least-once delivery messaging system built on the NATS messaging server. It adds persistence to NATS's famously low-latency pub/sub model through configurable streams and consumer groups. In the Borg Collective, three stream families carry most of the traffic: the task stream, the event stream, and the audit stream.
The task stream carries inbound work from the REST API, the WebSocket UI, and the borgctl command-line tool. Messages on the task stream are ordered within a subject, persistent until acknowledged, and subject to a configurable retention policy based on both age and count. If the entire Collective restarts due to a planned upgrade or an unexpected failure, unacknowledged task messages are re-delivered to the first available consumer, ensuring that no work is silently lost during the restart window.
The event stream carries agent-to-agent communication. When a research Drone finishes gathering information and needs a summarisation Drone to process it, it publishes an event to the event stream rather than calling the summarisation Drone directly. This indirection has two practical benefits. First, the research Drone does not need to know which specific summarisation Drone will handle the work, or even whether one is currently available; the message simply waits until a consumer is ready. Second, the event stream is observable: operators can inspect the subject hierarchy in real time to understand what work is in flight and how the Collective's internal traffic is distributed.
Every message in the Collective travels inside a MessageEnvelope:
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use uuid::Uuid;
/// MessageEnvelope wraps every payload that crosses a NATS subject
/// boundary. Separating routing metadata from the payload allows
/// routers, filters, and audit components to inspect and act on
/// messages without deserialising the inner payload, which can be
/// encoded in any format the producer and consumer agree on.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MessageEnvelope {
/// Unique message identifier for deduplication at the consumer
/// end, enabling exactly-once processing semantics on top of the
/// at-least-once delivery guarantee NATS provides natively.
pub message_id: Uuid,
/// AgentId of the sender in string form for cross-process
/// portability across Cube and process boundaries.
pub source_agent: String,
/// Optional target AgentId. None indicates a broadcast to all
/// consumers on the subject; Some targets a specific Drone.
pub destination_agent: Option<String>,
/// Lamport logical clock value at the moment of publication,
/// enabling causal ordering across distributed agents that may
/// run on different machines with drifting clocks.
pub logical_clock: u64,
/// Priority tier from 0 (critical system messages) through 9
/// (low-priority background tasks). The consumer layer uses
/// this to implement priority scheduling at the application level.
pub priority: u8,
/// The raw serialised payload, encoded as MessagePack for
/// compactness in high-throughput paths or JSON for human-readable
/// debugging sessions when operators need to inspect traffic.
pub payload: Vec<u8>,
/// Key-value metadata for routing rules, W3C Trace Context
/// propagation, and content-based filtering at the NATS
/// subject level.
pub headers: HashMap<String, String>,
}
impl MessageEnvelope {
/// Constructs a new envelope with a fresh message ID, leaving
/// the destination unset for the caller to specify if needed.
/// This constructor is the only way to create an envelope,
/// ensuring message_id is always a freshly minted value.
pub fn new(
source_agent: impl Into<String>,
logical_clock: u64,
priority: u8,
payload: Vec<u8>,
) -> Self {
MessageEnvelope {
message_id: Uuid::new_v4(),
source_agent: source_agent.into(),
destination_agent: None,
logical_clock,
priority,
payload,
headers: HashMap::new(),
}
}
}
The priority field deserves special attention. NATS JetStream itself does not implement priority queuing, so the Collective implements it in the consumer layer. Each consumer maintains ten internal queues, one per priority tier. When a message is dequeued for processing, the consumer always drains higher-priority tiers first, with a weighted-random fallback that prevents lower-priority messages from starving indefinitely under sustained high load. The algorithm is simple enough to fit in a single function, but its effect is profound: a critical security alert from the guardrail pipeline will always be processed before a batch summarisation task, regardless of which one arrived first on the wire.
The dead-letter queue mechanism provides the safety net. Any message that fails processing after a configurable number of retries is moved to the dead-letter subject, where an operator-facing inspector can examine it, re-queue it with a corrected payload, or discard it permanently. Without a dead-letter queue, a malformed message that consistently causes a Drone to crash would loop forever, consuming retry budget and generating alert noise. With a dead-letter queue, it is quarantined and visible, turning a previously silent failure mode into an actionable alert in the operator's dashboard.
CHAPTER 6: SECURITY FROM THE GROUND UP -- THE TEN-STEP GUARDRAIL PIPELINE
Enterprise AI faces a problem that consumer AI largely ignores: the consequences of failure are asymmetric. When a consumer chatbot produces a problematic response, the damage is local and recoverable. When an enterprise agent with write access to financial systems, customer databases, or infrastructure control planes produces a problematic response, the damage can be catastrophic and irreversible.
The Borg Collective's security architecture acknowledges this asymmetry and responds with a pipeline of ten independent guardrail checks that every agent action must pass before it is executed. The pipeline is implemented as a chain of responsibility, each link implementing a single Guardrail trait. If any check fails, the pipeline short-circuits and returns a typed error, which the calling Drone logs, emits as an audit event, and optionally escalates to a human supervisor through a configurable escalation channel.
The ten stages run in this order. Authentication verifies the agent's cryptographic identity against a Vault-managed key. Authorisation checks the requested action against the agent's capability set, which is declared in a signed YAML manifest. Rate limiting enforces per-agent and per-tenant request budgets using a token-bucket algorithm backed by Redis 8.x. Input validation detects injection attacks and prompt-injection patterns using a combination of regex rules and a lightweight classification model. Content filtering scans input for prohibited content categories defined by the deployment's compliance policy. Output validation verifies that the model's response does not contain sensitive data leakage, such as PII or credentials that appeared in the input context. The WASM capability check confirms that the requested operation is within the plugin's declared permissions set. The resource quota check ensures the operation will not exceed compute or cost budgets. The audit log write records the action in the tamper-evident ledger before it occurs, not after. And finally the execution gate is the single point where the action is allowed to proceed.
The rate-limit stage is illustrative of the entire pipeline's design philosophy:
use async_trait::async_trait;
use std::sync::Arc;
/// RateLimitGuardrail enforces per-agent request budgets using a
/// token-bucket algorithm backed by Redis 8.x. Failing here spares
/// all downstream GPU quota and protects the Collective from runaway
/// agent loops -- one of the most common and most expensive failure
/// modes in the first generation of agentic AI platforms.
pub struct RateLimitGuardrail {
/// Shared Redis connection, multiplexed across all concurrent
/// guardrail checks to avoid exhausting the connection pool
/// during traffic spikes.
redis: Arc<redis::aio::MultiplexedConnection>,
/// Maximum requests allowed per agent within one window period.
agent_limit: u32,
/// Duration of the rate-limit window in seconds.
window_secs: u64,
}
#[async_trait]
impl Guardrail for RateLimitGuardrail {
async fn check(
&self,
ctx: &SecurityContext,
) -> Result<(), GuardrailError> {
// The Redis key encodes both the agent identity and the
// current time-window bucket so that keys expire naturally
// without requiring a background TTL maintenance job.
let key = format!(
"rl:{}:{}",
ctx.agent_id,
current_window(self.window_secs)
);
let count: u32 = redis::cmd("INCR")
.arg(&key)
.query_async(&mut self.redis.clone())
.await
.map_err(GuardrailError::Redis)?;
// On the very first request in a new window, set an expiry
// so that the key is automatically cleaned up after the
// window closes, preventing unbounded key accumulation.
if count == 1 {
redis::cmd("EXPIRE")
.arg(&key)
.arg(self.window_secs)
.query_async(&mut self.redis.clone())
.await
.map_err(GuardrailError::Redis)?;
}
if count > self.agent_limit {
return Err(GuardrailError::RateLimitExceeded {
agent_id: ctx.agent_id.clone(),
limit: self.agent_limit,
window_secs: self.window_secs,
});
}
Ok(())
}
}
Notice that the guardrail is purely reactive: it either passes the request through or returns an error. It does not modify the request, it does not retry, and it does not log. Those concerns belong to the pipeline coordinator, which wraps every guardrail in structured error handling and ensures the audit trail is consistent regardless of which stage fails. This separation of concerns is what makes the pipeline testable: each guardrail can be tested in isolation with a fabricated SecurityContext, without spinning up the full pipeline.
The WASM capability check stage deserves particular mention. Every agent plugin in the Borg Collective is compiled to a WebAssembly module and loaded through Wasmtime 26.x. The permissions for each module are declared in a capabilities manifest: a signed YAML file that lists exactly which system calls, external services, and data categories the plugin may access. The WASM capability check guardrail loads this manifest, verifies its cryptographic signature against a Vault 1.18-managed key, and confirms that the requested action is within the declared permission set. If an AI model, through some form of prompt injection, attempts to instruct an agent to access a resource outside its declared capabilities, the Wasmtime sandbox prevents execution at the host level. The model cannot escape the box regardless of how cleverly it phrases the request, because the sandbox operates beneath the abstraction layer that the model can influence.
CHAPTER 7: MEMORY AS INTELLIGENCE -- THE FIVE-TIER LLM WIKI
A single-shot language model has no memory. Each inference is stateless, beginning afresh with only the tokens in its context window. For a chat assistant, this is mildly inconvenient. For an autonomous agent that operates over hours or days, accumulating knowledge as it works, it is fatal. The Borg Collective's memory architecture, called the LLM Wiki, addresses this with a five-tier hierarchy inspired by how human memory actually works: fast and volatile at the top, slow and durable at the bottom.
The first tier is the working context. This is the agent's current context window, holding the last few turns of interaction, the current task state, and immediate intermediate results. It lives entirely in RAM and is reconstructed on each inference call from the tiers below. It is the fastest tier but the smallest and the most volatile; a Drone restart wipes it entirely, but the tiers below can reconstitute it quickly.
The second tier is the episodic cache, backed by Redis 8.x. When an agent completes a reasoning step, it serialises a summary of that step and writes it to Redis under a key that encodes the agent ID and session ID. Redis is an appropriate choice here because the episodic cache is read far more often than it is written, access patterns are highly predictable, and the data has a natural time-to-live after which it is either promoted to the third tier or discarded. Redis 8.x, with its native vector-similarity search module, can also answer the question "which of my recent reasoning steps is most relevant to this new subtask?" without requiring a round trip to the semantic search tier, making in-session recall essentially free.
The third tier is the semantic memory store, implemented on Qdrant 1.13. Every significant piece of knowledge that an agent acquires during its operation -- research findings, tool call results, synthesised conclusions -- is embedded using a local embedding model and stored as a dense vector in Qdrant alongside its source text. When a Drone needs context for a new task, it queries Qdrant with a vector similarity search, retrieving the top-k most semantically relevant prior experiences. This retrieval-augmented memory is what allows an agent to pick up a task it has not touched in a week and immediately recall the relevant prior context, as if it had never stopped working.
The fourth tier is the relational knowledge graph, implemented on Neo4j 5.x. Semantic similarity is powerful but it is not the only dimension of relevance. A knowledge graph tracks explicit relationships: this document was produced by that agent, this decision was based on those facts, this entity is associated with those attributes. When an agent needs to understand not just what is relevant but how things relate to each other, it queries Neo4j. Graph traversal answers questions like "give me all documents produced by agents in the finance team that reference the Q3 earnings report," which vector search alone cannot answer reliably because similarity does not encode directional relationships.
The fifth and deepest tier is long-term archival storage in PostgreSQL 17. All memory that has passed through the upper tiers and been deemed worth preserving is eventually written to PostgreSQL in a structured schema that supports arbitrary querying, full-text search through pg_trgm, and long-term retention policies enforced by row-level security. PostgreSQL is also the ground truth for the audit ledger, agent configuration, and tenant metadata. It is the source of record that remains readable and queryable even if all other tiers are cleared, restarted, or migrated.
The tiered architecture produces a system where fast reads are cheap, expensive operations are avoided until strictly necessary, and no knowledge is permanently lost until a human operator explicitly decides to delete it. The Borg never forgets, because it has been designed from the ground up not to.
CHAPTER 8: ROUTING THE MIND -- THE LLM ROUTER AND QUOTA MANAGEMENT
A production agentic platform does not call a single language model. It calls many, choosing between them based on cost, latency, capability, and availability. A research task might warrant GPT-5's broad world knowledge, while a code-review task is better served by a model fine-tuned on software engineering. A low-priority batch job might use a smaller, cheaper model to preserve the daily budget for time-sensitive interactive tasks. And when OpenAI's API returns a 503, the system must seamlessly failover to an alternative without human intervention.
The LLM Router in the Borg Collective handles all of this. It exposes the same ILlmPort interface that every individual adapter implements, making it invisible to the domain model: from the Drone's perspective, it calls ILlmPort.complete() and receives a response. Internally, the router maintains a registry of available adapters, a priority ordering per capability category, and a circuit breaker for each provider.
Model selection follows a decision cascade. First, the router consults the task's required capability tag, which the Drone embeds in the LlmRequest. If the task is tagged "code-generation", only models in the code-generation capability group are considered. Second, within that group, models are ordered by their configured priority score, which operators can tune to reflect cost preferences and quality requirements for each task type. Third, the circuit breaker state for each candidate model is checked: a model whose circuit breaker is open is skipped entirely and does not receive a request. Fourth, the current quota usage for each surviving candidate is checked against the configured daily and per-minute budgets stored in Redis.
The first model that passes all four filters receives the request. If no model passes all filters, the router falls back to a lower-capability tier and tries the cascade again, eventually falling all the way to the local Ollama deployment if every cloud provider is unavailable or over budget. This graceful degradation is critical for enterprise deployments where SLA guarantees cannot depend on the continued availability of any single external API. Operators define the fallback chains in the routing configuration, which is hot-reloadable without restarting the Collective.
Quota management adds a financial dimension to routing decisions. The quota system tracks spending in real time across three granularities: per-agent, per-tenant, and per-deployment-wide. Alerts fire when any budget crosses 80 percent, giving operators time to adjust before hard limits are reached. When a hard limit is reached, the router immediately routes all affected requests to cheaper fallback models rather than failing them outright, ensuring continuity of service at the cost of some degradation in output quality.
CHAPTER 9: THIRTEEN PATTERNS OF AGENT BEHAVIOUR
The Borg Collective does not define a single agent behaviour. It defines thirteen, divided into six patterns for individual agents and seven for agent collaborations. Each pattern is a named, reusable template that operators can compose to build complex behaviours from simple building blocks, without touching the underlying actor infrastructure.
For individual agents, the six patterns cover the full spectrum of autonomous reasoning approaches. The ReAct pattern, Reason plus Act, is the oldest and most widely understood: the agent alternates between reasoning about what to do next and executing a tool call, using the tool's output to inform the next reasoning step. The PlanAndExecute pattern separates planning from execution entirely, generating a complete plan before touching any tool, which produces more coherent multi-step behaviour at the cost of less adaptive mid-course correction. The ReflectionAgent pattern adds a self-evaluation step after each reasoning cycle, asking the model to critique its own output and identify potential errors before proceeding to the next step. The ToolCallAgent pattern is the foundation for MCP 2.0 integration, treating every external capability as a typed tool call with a validated input and output schema. The CritiqueAndRevise pattern pairs a generator agent with a critic agent in a single actor, using the critic's structured feedback to iteratively improve the output before it is delivered to the requester. The MonteCarloTreeSearch pattern applies tree-search techniques to agent reasoning, exploring multiple branching reasoning paths and selecting the most promising one based on a value function, at the cost of significant compute that makes it suitable only for high-stakes decisions.
For collaboration patterns, the seven templates cover the range of ways multiple Drones can work together productively. The MapReduce pattern distributes a large task across multiple Drones in parallel and aggregates their results, making it ideal for research tasks that can be partitioned across independent sources or topics. The Consensus pattern runs the same task through multiple independent Drones and accepts the result only when a configurable majority agree, providing robustness against individual model hallucination that is particularly valuable in high-stakes domains. The Pipeline pattern passes results from one Drone to the next in sequence, each adding value in turn, and is the right choice when the output of each stage is the natural input to the next. The Debate pattern assigns opposing positions to two Drones and has them argue before a judge Drone decides the outcome, producing well-reasoned outputs on contested questions where a single perspective would miss important counterarguments. The Supervisor pattern places one Drone in an oversight role over a group of worker Drones, intervening when subordinates diverge from the assigned task or produce outputs that fail a quality threshold. The Swarm pattern distributes work across a dynamic pool of identical Drones without central coordination, using shared LLM Wiki state as stigmergy rather than explicit messaging. The HiveMind pattern is the most advanced: a shared reasoning scratchpad in which multiple Drones contribute to and read from a common Neo4j graph node, building a collective reasoning process that no single Drone could sustain alone.
These patterns are not implemented as separate code paths. Each is a configuration of actors, message flows, and prompt templates that the Cube instantiates from a YAML behaviour descriptor. Adding a new pattern means adding a new descriptor and its associated prompt templates, not modifying the core actor infrastructure. This extensibility is what keeps the Collective's behaviour repertoire growing without the codebase growing proportionally.
CHAPTER 10: MCP 2.0 -- THE PROTOCOL THAT UNIFIED AI TOOLS
The Model Context Protocol, originally proposed by Anthropic and adopted as an open standard in late 2024, reached version 2.0 by mid-2026. Where the original MCP defined a JSON-RPC protocol for exposing tools to language models, MCP 2.0 added streaming result delivery, bidirectional capability negotiation, typed input and output schema validation with OpenAPI 3.1 compatibility, and cryptographic tool provenance records that satisfy regulatory audit requirements.
The Borg Collective implements both sides of the MCP 2.0 protocol: a client that allows Drones to call external tools exposed by any MCP server anywhere on the network, and a server that exposes the Collective's own capabilities to external agents and platforms. This dual role makes the Collective a full participant in the emerging ecosystem of federated AI tools: it can consume services from other platforms and provide services to them, without either party needing to know the other's internal implementation.
On the client side, the MCP adapter discovers available tools by querying the server's capability manifest, which lists each tool's name, description, parameter schema, and response schema in a standardised format. The Drone selects tools by fuzzy-matching the task description against tool names and descriptions. This matching process uses a lightweight embedding model running locally to find semantically similar matches even when the exact tool name is unknown, so a Drone that needs to "look up stock prices" can discover that the connected MCP server exposes a "fetch_equity_quote" tool without any hardcoded knowledge of that tool's name.
On the server side, the Borg Collective exposes every registered agent capability as an MCP tool. An external system can discover that the Collective has a "web-research" tool, call it with a query string and a maximum source count, and receive a structured research report complete with citations. The external caller does not know or care that the tool is implemented by a swarm of research Drones coordinated by a MapReduce pattern; it simply makes a typed tool call and receives a typed response.
MCP 2.0's tool provenance feature is particularly valuable in regulated industries. Every tool response includes a signed provenance record that identifies which version of which agent capability produced the result, at what wall-clock time, with what input hash, and via which model routing path. Regulated firms can include these provenance records in their audit trails, satisfying requirements that every AI-generated output must be traceable to its exact producing system and model version. The Collective signs all provenance records using an asymmetric key managed by Vault 1.18, allowing external auditors to verify provenance without access to the Collective's internal systems.
CHAPTER 11: THE ADAPTER ECOSYSTEM -- WASM PLUGINS AND COMMUNICATION CHANNELS
The Borg Collective reaches the outside world through a structured adapter ecosystem. Every external integration -- whether a Telegram bot, an email gateway, a voice interface, or a legacy REST API -- is implemented as a WASM plugin loaded by the adapter service. This architecture provides three guarantees that are simply not achievable with dynamically-linked native libraries.
Isolation ensures that a buggy or malicious adapter cannot corrupt the host process; the Wasmtime sandbox confines the plugin to its allocated linear memory and prevents it from making arbitrary system calls. Portability means that any adapter compiled to WASM can run on any Borg deployment without recompilation, regardless of the host operating system or CPU architecture, which matters enormously for organisations that run a mix of Linux x86-64, Linux ARM64, and macOS development environments. Versioning allows multiple versions of the same adapter to coexist in the system, enabling blue-green deployments of individual plugins without restarting the entire Collective.
The adapter service maintains a hot-reload mechanism. When a new version of an adapter WASM module is pushed to the configured object storage bucket, the adapter service detects the change via a file system watch, validates the new module's cryptographic signature against the Vault-managed signing key, and hot-swaps the active instance. Existing in-flight calls complete against the old version; new calls are routed to the new version. The entire swap takes approximately 80 milliseconds, an interval that is invisible to end users.
Communication adapters in the default distribution include Telegram with full bot API support including inline keyboards and document handling, Email through SMTP with DKIM signing and IMAP idle-mode polling for inbound messages, Slack with Events API support including slash commands and modal views, HTTP webhook for arbitrary REST-API callbacks from external systems, Discord with both bot token and OAuth2 application support, Voice through WebRTC-based real-time audio processing that feeds into the speech-to-text and text-to-speech adapter pipeline, and RSS/Atom for research agents that need to monitor external content sources continuously.
Each adapter exposes a uniform interface to the Drone layer. A Drone that wants to send a Telegram message, an email, and a Slack notification for the same event calls the same abstract IChannelPort trait with three different channel identifiers. The adapter service routes each call to the correct plugin without the Drone knowing which communication technology is involved. Adding a new communication channel means implementing a WASM plugin that satisfies the IChannelPort interface and registering it in the adapter configuration; no changes to the Drone codebase, the Cube actor, or the SuperQueen are required.
CHAPTER 12: THE INTER-BORG BRIDGE -- FEDERATED AI ACROSS THE ENTERPRISE
A single Borg Collective deployment is powerful. A network of Borg Collective deployments, each with its own specialisation and data residency, coordinated through a secure federation layer, is transformative. The Inter-Borg Bridge (IBB) makes this federation possible by connecting multiple Collective deployments into a single logical fabric while preserving each deployment's autonomy and security boundary.
The IBB uses gRPC with Tonic 0.13 over mutual TLS for all inter-collective communication. Every Collective in the federation presents a certificate signed by a shared enterprise Certificate Authority managed by HashiCorp Vault 1.18. Vault rotates these certificates automatically on a configurable schedule, typically every 24 hours, eliminating the operational burden of manual certificate management and the associated risk of certificates expiring unnoticed. Mutual TLS ensures that both ends of every connection are authenticated: a rogue service cannot impersonate a legitimate Collective by merely knowing its address, because the TLS handshake requires a valid certificate from the trusted CA on both sides.
The IBB protocol defines three message types that cover the full range of inter-collective interactions. Task delegation routes a task to a remote Collective that has a specialised capability not available locally, delivering the work envelope through the mTLS gRPC channel and awaiting the result. Knowledge sharing propagates newly acquired knowledge to peer Collectives that have opted in to a shared knowledge domain, allowing the entire federation to benefit from what any member learns. Health federation shares capacity and load metrics with the federation registry so that the routing layer can make informed delegation decisions based on the current state of the entire network, not just the local deployment.
Task delegation through the IBB follows the same pattern as local task routing, which is no accident. The requesting Collective's SuperQueen publishes a task envelope with a remote destination hint. The IBB adapter picks it up, establishes or reuses an mTLS gRPC connection to the target Collective's IBB endpoint, and delivers the envelope. The remote Collective processes the task using its own actor hierarchy and returns the result through the same channel. From the Drone's perspective, a remote task delegation is indistinguishable from a local one: it publishes an event and eventually receives a result.
The federation registry is a Qdrant collection shared across the federation, with each member holding a read replica. The registry maps capability tags to available Collectives and their current load scores, updated by each member's health federation messages on a sub-second cadence. When a Collective needs to delegate a task and its local router finds no suitable model, the IBB queries the registry and selects the most capable, least loaded remote Collective. The entire selection and delegation process adds roughly 15 milliseconds of latency beyond the local processing path -- an acceptable overhead for the cross-collective task distribution that only the IBB can provide.
CHAPTER 13: SEEING INSIDE THE HIVE -- PROMETHEUS, OTEL TRACING, AND THE AUDIT LEDGER
You cannot manage what you cannot measure. This aphorism is old, but it gains new urgency in an agentic AI system. When an autonomous agent makes a decision that costs the business ten thousand dollars, you need to know exactly which model made that decision, with which inputs, at what time, following which reasoning steps, and in response to which original task. Vague logging is not enough. You need a structured, tamper-evident audit trail and a rich metrics layer that can surface problems before they escalate into incidents.
The observability architecture of the Borg Collective has three components that work in concert: metrics, distributed traces, and the audit ledger. Metrics answer the question "is the system healthy right now?" Distributed traces answer the question "what happened during this specific request?" The audit ledger answers the question "what did every agent do, and can we prove it to a regulator?"
Metrics are exposed through Prometheus using the prometheus_client 0.23 format. The core metric families include borg_llm_calls_total, a counter of LLM API calls labelled by model name and call outcome; borg_llm_latency_ms, a histogram of response times per model with buckets tuned to the observed latency distribution; borg_agent_tasks_total, task completions labelled by agent type and result; borg_guardrail_rejections_total, security rejections labelled by the guardrail stage that fired; borg_nats_messages_published_total, total messages published per subject prefix; borg_cube_active_drones, a gauge of live Drones per Cube; and borg_memory_tier_hits_total, cache hit counts per LLM Wiki tier.
The ObservabilityService provides the single facade through which all domain components record metrics, keeping the domain layer independent of the concrete Prometheus implementation:
use prometheus_client::metrics::counter::Counter;
use prometheus_client::metrics::family::Family;
use prometheus_client::metrics::histogram::Histogram;
/// ObservabilityService is the metrics and tracing facade for the
/// Collective's domain layer. Domain components receive this service
/// through dependency injection, which insulates them from the
/// concrete Prometheus and OpenTelemetry implementations and enables
/// the entire metrics layer to be replaced or mocked in tests.
pub struct ObservabilityService {
/// Total LLM calls, labelled by model name and call outcome.
/// Using a Family allows per-label cardinality without declaring
/// every combination of label values at construction time.
llm_calls_total: Family<Vec<(String, String)>, Counter>,
/// LLM response latency histogram, labelled by model name.
/// Prometheus will compute quantiles from bucket observations.
llm_latency_ms: Family<Vec<(String, String)>, Histogram>,
}
impl ObservabilityService {
/// Records one completed LLM call with its latency and outcome.
/// This method is called from every LLM adapter implementation
/// so that metric coverage is guaranteed regardless of which
/// model is active at runtime.
pub fn record_llm_call(
&self,
model: &str,
latency_ms: f64,
outcome: &str,
) {
// Increment the calls counter with structured label values
// so that Prometheus can aggregate by model, by outcome,
// or by any combination thereof.
self.llm_calls_total
.get_or_create(&vec![
("model".to_owned(), model.to_owned()),
("outcome".to_owned(), outcome.to_owned()),
])
.inc();
// Observe the latency in the histogram. Prometheus computes
// p50, p90, p99, and other quantiles automatically from the
// accumulated bucket observations.
self.llm_latency_ms
.get_or_create(&vec![
("model".to_owned(), model.to_owned()),
])
.observe(latency_ms);
}
}
Distributed traces propagate through the Collective using OpenTelemetry 0.28 with the W3C Trace Context standard. Every MessageEnvelope carries a W3C traceparent header in its headers map, which the processing Drone uses to continue the trace as a child span. The result is a single distributed trace that spans the entire request lifecycle: from the initial REST API arrival, through the SuperQueen's routing decision, into the Cube, through the guardrail pipeline, across the LLM API call, into the LLM Wiki write, and back to the API response. An operator experiencing a latency issue can pull up any specific trace in Grafana and see exactly where the time was spent, without digging through raw logs.
The audit ledger is the third observability component, and in many ways the most important. Every action in the Collective that touches external state or crosses a security boundary is recorded in an append-only PostgreSQL table. Each entry is hashed using SHA-3-256 and chained to the previous entry's hash, making the ledger tamper-evident without requiring a blockchain consensus mechanism:
use sha3::{Digest, Sha3_256};
use std::time::SystemTime;
/// AuditEntry is one immutable record in the tamper-evident audit
/// ledger. Each entry's hash incorporates the preceding entry's
/// hash, binding every record to its exact position in the chain.
/// Inserting, deleting, or modifying any entry invalidates all
/// subsequent hashes, making tampering immediately detectable by
/// any party that holds the genesis hash.
#[derive(Debug, Clone)]
pub struct AuditEntry {
/// Monotonically increasing sequence number within the ledger,
/// used to detect gaps caused by deletion attempts.
pub sequence: u64,
/// Wall-clock timestamp at the moment the entry was created,
/// stored with nanosecond precision.
pub timestamp: SystemTime,
/// String identity of the agent that performed the action.
pub agent_id: String,
/// Name of the action, such as "tool_call" or "model_inference".
pub action: String,
/// The resource the action targeted, such as a file path or URL.
pub resource: String,
/// The outcome: "success", "failure", or "rejected".
pub outcome: String,
/// The SHA-3-256 hash of the immediately preceding entry in the
/// chain. The genesis entry uses a zero-filled value by
/// convention so that the chain can be verified from the start.
pub prev_hash: [u8; 32],
}
impl AuditEntry {
/// Computes the SHA-3-256 hash of this entry in canonical field
/// order. Any change to any single field produces a completely
/// different hash, making individual record tampering detectable
/// by any auditor who recomputes the chain.
pub fn compute_hash(&self) -> [u8; 32] {
let mut hasher = Sha3_256::new();
// Fields are hashed in fixed declaration order to ensure
// determinism across different versions of the software
// and different serialisation libraries.
hasher.update(self.sequence.to_le_bytes());
hasher.update(
self.timestamp
.duration_since(SystemTime::UNIX_EPOCH)
.unwrap_or_default()
.as_nanos()
.to_le_bytes(),
);
hasher.update(self.agent_id.as_bytes());
hasher.update(self.action.as_bytes());
hasher.update(self.resource.as_bytes());
hasher.update(self.outcome.as_bytes());
// Chaining the previous hash binds this entry irrevocably
// to its position in the sequence.
hasher.update(&self.prev_hash);
hasher.finalize().into()
}
}
The audit ledger's chain structure is deliberately simple. It does not require a consensus mechanism or a distributed ledger technology. The single-writer constraint -- only the Collective's dedicated audit service appends to the ledger -- makes consensus unnecessary. What the chain provides is tamper evidence: any modification of a historical record breaks the chain at that point, and the break is detectable by recomputing hashes sequentially from the genesis entry forward. This is sufficient for the audit requirements of most regulated industries, including financial services under MiFID III and healthcare under HIPAA, without the operational complexity of a full blockchain deployment.
CHAPTER 14: WHEN THINGS GO WRONG -- RETRY, CIRCUIT BREAKER, SAGA, AND BULKHEAD
Every distributed system fails, sooner or later. The question is not whether the Borg Collective will encounter network partitions, LLM API timeouts, database connection pool exhaustion, or downstream service degradation. The question is whether it will fail gracefully, recovering automatically from transient faults, isolating persistent faults, and providing operators with the information they need to remediate the non-recoverable ones.
The Collective implements four reliability patterns that compose naturally. They are not mutually exclusive: a single LLM call may be governed by a retry policy, monitored by a circuit breaker, executed inside a saga step, and running within a bulkhead-isolated thread pool, all simultaneously.
The first pattern is the Retry Policy. Transient failures -- network blips, temporary API rate limits, brief database connection hiccups -- account for the majority of failures in any distributed system, and they are recoverable by simply trying again after a short delay. The retry policy governs how long to wait between attempts:
use std::time::Duration;
/// RetryPolicy captures the full configuration for exponential
/// back-off retries. All parameters are drawn from operator-
/// supplied YAML configuration, allowing per-service tuning without
/// a recompile cycle. Sensible defaults are provided for deployments
/// that do not need custom tuning.
#[derive(Debug, Clone)]
pub struct RetryPolicy {
/// Maximum total attempts, including the initial try. Setting
/// this to 1 effectively disables retrying.
pub max_attempts: u32,
/// Delay applied before the very first retry attempt.
pub base_delay: Duration,
/// Multiplicative factor applied to the delay on each successive
/// retry. A value of 2.0 doubles the wait time with each
/// attempt, producing classic exponential back-off.
pub multiplier: f64,
/// Hard ceiling that caps the computed delay regardless of how
/// many retries have elapsed, preventing multi-hour waits from
/// accumulating in very long retry sequences.
pub max_delay: Duration,
/// Fraction of the computed delay added or subtracted as random
/// jitter, expressed as a value between 0.0 and 1.0. Spreading
/// retries across a natural window prevents retry storms when
/// many agents fail simultaneously due to a shared dependency.
pub jitter_factor: f64,
}
impl RetryPolicy {
/// Computes the back-off duration before the given attempt number
/// (zero-indexed, where 0 is the delay before the second try).
/// The result lies within the jitter band around the capped
/// exponential value and is always non-negative.
pub fn delay_for(&self, attempt: u32) -> Duration {
let base_ms = self.base_delay.as_millis() as f64;
// Compute the raw exponential delay for this attempt number.
let raw = base_ms * self.multiplier.powi(attempt as i32);
// Cap at the configured maximum to prevent runaway waits.
let capped = raw.min(self.max_delay.as_millis() as f64);
// Apply symmetric jitter: the actual delay is drawn from
// [capped * (1 - j), capped * (1 + j)] where j = jitter_factor.
// The random value spans the full [-1.0, 1.0) range so the
// jitter is equally likely to shorten or lengthen the delay.
let jitter_offset = capped
* self.jitter_factor
* (rand::random::<f64>() * 2.0 - 1.0);
let ms = (capped + jitter_offset).max(0.0) as u64;
Duration::from_millis(ms)
}
}
The jitter factor is a small but critical detail. Without it, all agents that hit the same transient failure at the same moment will retry in synchrony, producing a thundering-herd effect that amplifies the original failure. With jitter, their retries spread across a natural window, and the downstream service sees a gradual ramp of load rather than a sudden spike.
The second pattern is the Circuit Breaker. Where the retry policy handles individual transient faults, the circuit breaker handles a service that is persistently degraded. When the failure rate within a sliding time window exceeds a configured threshold, the circuit opens, and all subsequent requests are rejected immediately without attempting the call:
use std::time::Instant;
/// CircuitState models the three canonical states of the circuit
/// breaker pattern. Transitions are driven entirely by observed
/// failure behaviour -- never by operator commands -- which keeps the
/// reliability layer fully self-managing and free from manual
/// intervention during an incident.
#[derive(Debug, Clone)]
pub enum CircuitState {
/// Requests flow normally; the downstream service is healthy and
/// meeting its latency and success-rate targets.
Closed,
/// The failure threshold was exceeded. All requests are rejected
/// without attempting the downstream call, protecting the service
/// from additional load while it recovers and protecting the
/// calling Drones from accumulating latency against a dead service.
Open {
/// Records when the circuit opened so the reset timer can be
/// evaluated on each subsequent incoming request without
/// requiring a background task.
opened_at: Instant,
},
/// A single probe request is allowed through to test whether the
/// downstream service has recovered. A successful probe closes
/// the circuit; a failed probe re-opens it and resets the reset
/// timer, restarting the cool-down period from scratch.
HalfOpen,
}
impl CircuitState {
/// Returns true if the current state permits a request to proceed.
/// An Open circuit transitions to HalfOpen automatically when the
/// reset timeout has elapsed, requiring no external action.
pub fn allows_request(
&self,
reset_timeout: Duration,
) -> bool {
match self {
CircuitState::Closed => true,
CircuitState::HalfOpen => true,
CircuitState::Open { opened_at } => {
// Transition from Open to HalfOpen by permitting a
// single request once the cool-down period expires.
opened_at.elapsed() >= reset_timeout
}
}
}
}
The third pattern is the Saga. Some operations in the Collective span multiple services in sequence: reserve a compute budget, call the LLM, write the result to the LLM Wiki, notify the requesting system, and release the compute budget reservation. A traditional database transaction cannot span all five of these steps because they involve different systems with different transaction semantics. The Saga pattern resolves this by pairing every forward step with a compensation action that undoes it, then running compensation in reverse order if any step fails:
use std::future::Future;
use std::pin::Pin;
/// StepToken is an opaque value returned by a Saga step's forward
/// action. It carries whatever information the compensation action
/// needs to undo the step's effects, such as a reservation ID or a
/// newly created resource's identifier.
pub type StepToken = Box<dyn std::any::Any + Send>;
/// SagaOrchestrator drives a sequence of steps to completion,
/// running compensation in reverse for any steps that succeeded
/// before the first failure, so that no partial-success state
/// persists in the system after an error.
pub struct SagaOrchestrator {
steps: Vec<SagaStep>,
}
impl SagaOrchestrator {
/// Runs all steps in order, collecting the token each successful
/// step returns. On the first failure, compensates all
/// previously completed steps in reverse order using their
/// collected tokens, then returns the original error.
pub async fn run(&self) -> Result<(), SagaError> {
let mut completed: Vec<(usize, StepToken)> = Vec::new();
for (i, step) in self.steps.iter().enumerate() {
match (step.execute)().await {
Ok(token) => {
tracing::info!(
step = step.name,
"saga step completed successfully"
);
completed.push((i, token));
}
Err(e) => {
tracing::warn!(
step = step.name,
error = %e,
"saga step failed -- starting compensation"
);
// Compensate in reverse order so that
// dependencies are unwound correctly: the last
// completed step is compensated first.
for (idx, token) in
completed.into_iter().rev()
{
let comp = &self.steps[idx];
if let Err(ce) =
(comp.compensate)(token).await
{
tracing::error!(
step = comp.name,
error = %ce,
"compensation failed -- \
manual intervention required"
);
}
}
return Err(e);
}
}
}
Ok(())
}
}
The Saga pattern does not guarantee atomicity in the strict database sense. Between the moment a forward step completes and the moment its compensation would undo it, there is a window during which the partial state is visible. This is an inherent property of distributed transactions, not a flaw in the implementation. The Collective's design acknowledges this window explicitly in the documentation and designs its Saga steps to make the intermediate states observable but clearly flagged as transient, so downstream consumers can choose to wait for finality before acting on them.
The fourth pattern is the Bulkhead. Named after the watertight compartments in a ship's hull that prevent a single breach from sinking the vessel, the bulkhead pattern isolates Drone pools so that a surge of expensive operations in one pool cannot starve other pools of execution threads. The Collective configures dedicated Tokio thread pools for the LLM call tier, the NATS consumer tier, the WASM execution tier, and the storage tier. A burst of slow LLM calls fills only the LLM pool; the NATS consumer pool continues processing messages at full speed, and storage writes remain unaffected. The pool sizes are tunable per tier in the operations configuration, allowing operators to match the allocation to the actual observed workload mix of their deployment.
CHAPTER 15: THE OPERATOR EXPERIENCE -- BORGSETUP, BORGCTL, AND THE WEB UI
All of the architecture described so far serves one ultimate purpose: to give operators, developers, and business users a system they can actually use day to day. The best-designed architecture in the world is worthless if it requires a PhD in distributed systems to operate. The Borg Collective addresses this with three operator-facing tools that together cover initial setup, ongoing administration, and real-time visibility.
The first tool is borgsetup, the zero-to-running installation assistant. borgsetup is a compiled Rust binary that walks the operator through a guided setup sequence, making decisions that would otherwise require deep platform knowledge. It detects the host environment, distinguishing between a Kubernetes cluster, a Docker Compose stack, and bare metal, and tailors its recommendations accordingly. It prompts for the required API keys and credentials, storing them directly into Vault rather than writing them to files that might be accidentally committed to version control. It generates the full YAML configuration suite -- brain.yaml, capabilities.yaml, permissions.yaml, adapters.yaml, config.yaml, and operations.yaml -- from the operator's answers to a structured questionnaire. It validates each configuration section against its JSON Schema before writing anything to disk, so typographical errors are caught before they become runtime crashes. For small deployments, it optionally bootstraps an embedded NATS server and a PostgreSQL instance rather than requiring the operator to provision external infrastructure. A fresh installation on a prepared Kubernetes cluster takes approximately twelve minutes from running borgsetup to processing the first task, including TLS certificate provisioning through Vault.
The second tool is borgctl, the day-to-day command-line interface. borgctl communicates with the running Collective through the gRPC management API, using the same mTLS authentication as the Inter-Borg Bridge, and provides a comprehensive suite of commands covering every operational domain. Agent management commands allow operators to list all active Drones with their current task assignment and resource consumption, inspect the full state of a specific agent including its LLM Wiki context, and forcibly terminate runaway agents. Task management commands cover submitting new tasks with structured input, polling task status, retrieving results in both human-readable and machine-readable formats, and re-queuing tasks that ended up in the dead-letter queue. Configuration management commands allow viewing the current effective configuration (merged from all YAML layers) and pushing incremental updates that take effect without a service restart. Memory management commands provide direct access to the LLM Wiki: operators can query any tier by agent ID or semantic similarity, evict stale entries, and trigger compaction jobs that consolidate fragmented episodic cache entries into the semantic store. Observability commands fetch live health summaries, pull the most recent audit log entries with hash verification status, and export raw trace data for import into Grafana.
The third component is the React-based Web UI, served through the Actix-web 5.x server embedded in the Collective. The Web UI provides a real-time dashboard showing the Cube and Drone hierarchy as a live graph with animated edges representing in-flight messages, giving operators instant intuition about where work is concentrating. Task submissions from the UI support rich structured input including file attachments processed by the document adapter, making the Collective accessible to non-technical business users who should not need to know that borgctl exists. The audit log viewer presents the tamper-evident ledger entries with inline hash verification status, so compliance teams can spot-check the chain's integrity without writing a verification script. A configuration editor with JSON Schema validation catches malformed configuration before it is committed. The model routing visualiser shows which models are currently active, which circuit breakers are open, and how the request load is distributed across providers in real time, making capacity planning conversations significantly more grounded in actual data.
CHAPTER 16: THE LIVING COLLECTIVE -- COMPOSITION ROOT AND FUTURE DIRECTIONS
Every component described in the preceding fifteen chapters connects through the composition root: the single function that wires all dependencies together and starts the system. In hexagonal architecture, the composition root is the only place where concrete implementations are instantiated and injected into the abstract interfaces they implement. Everywhere else in the codebase, code depends on traits. Only in the composition root does it depend on concrete types.
The composition root for the Borg Collective reads the configuration files in order of precedence, constructs the Redis 8.x and PostgreSQL 17 connection pools, instantiates the NATS JetStream client and verifies stream existence, builds the LLM adapter instances for each configured provider (OpenAI for GPT-5 and GPT-4o, Anthropic for Claude 4 Sonnet and Opus, Google for Gemini 2.5 Pro, and Ollama for Llama 4 and Mistral Large 3 in local deployments), constructs the ObservabilityService and injects it into the adapter registry, builds the guardrail pipeline with all ten configured stages, starts the Actix system and spawns the SuperQueen actor, registers the WASM plugin adapters in the adapter service, connects the IBB if federation is enabled and the peer list is non-empty, and finally starts the REST API and WebSocket servers to begin accepting external work.
This wiring happens once, at startup, in a single file that is designed to be read top-to-bottom as a narrative description of the system's structure. Any developer who wants to understand how the Collective is assembled starts here. The function reads like a bill of materials: here is the database connection, here is the messaging layer, here is the LLM adapter, here is the security pipeline, here is where all of them meet.
This composition-root discipline pays dividends that compound over time. Because every dependency is injected, any component can be replaced in a test by a mock. Because the injection happens in one place, the structure of the entire system is visible in one file rather than scattered across hundreds of import statements. Because the domain model depends only on traits, the entire domain layer can be tested without starting NATS, PostgreSQL, Redis, or any LLM provider. Domain-logic tests complete in milliseconds; integration tests complete in seconds; only end-to-end tests require the full stack and the real external services.
As for future directions: the roadmap for the Borg Collective extends across ten phases. Phase zero, which is complete, established the workspace layout and foundational types. Phases one through four built the core actors, messaging fabric, security pipeline, and memory tiers respectively. Phases five and six added the LLM router and the thirteen behaviour patterns. The platform in September 2026 is at phase seven, which introduced the IBB federation layer and MCP 2.0 support.
Phase eight will introduce formal agent learning: the ability for the LLM Wiki to identify recurring patterns across agent experiences and generate updated prompt templates that improve performance on those task types without requiring manual prompt engineering. This closes the loop between observability and improvement: the metrics that reveal underperforming task types will feed directly into the learning pipeline that addresses them.
Phase nine will add multi-modal reasoning, enabling Drones to reason about images, audio, and video in addition to text, using the growing ecosystem of multi-modal foundation models from OpenAI, Google, and Anthropic. The adapter layer is already designed for this: adding a multi-modal LLM means adding a new adapter behind ILlmPort that accepts image and audio tokens alongside text tokens. The domain model needs no changes.
Phase ten will close the loop with supervised agent self-modification: the ability for sufficiently privileged agents to propose changes to their own behaviour pattern descriptors, subject to human review through the Web UI and the full guardrail pipeline, progressively blurring the boundary between operator and tool. This is the point at which the Collective begins, in a very limited and carefully controlled sense, to assimilate itself.
These future phases are architectural extensions, not replacements. Every design decision made in the current architecture -- hexagonal layering, actor isolation, event-driven decoupling, typed interfaces, pipeline-based security -- was made with this trajectory in mind. Adding multi-modal reasoning means adding new adapter implementations behind existing ports. Adding agent learning means adding a new memory tier and a new behaviour pattern descriptor. The architecture's shape, its hexagonal shell and event-driven nervous system, remains stable as the capabilities inside it grow.
This is what it means to design for the long term in a field that moves as fast as agentic AI: not to predict every future requirement in advance, which is impossible, but to build an architecture flexible enough that future requirements can be accommodated without dismantling what already exists and works. The Borg Collective Architecture is not finished. It is designed to grow.
Resistance, as they say, is futile. But with an architecture this solid, you might not want to resist at all.
ABOUT THIS ARTICLE
The Borg Collective Architecture is an open design inspired by enterprise Rust development practices, the hexagonal architecture pattern as articulated by Alistair Cockburn, and the operational realities of running autonomous AI agents in production in 2026.
The Rust code throughout this article targets Rust 1.87 and uses actix 0.13 with the actix-web 5.x HTTP layer, tokio 1.42 as the async runtime, opentelemetry 0.28 for distributed tracing, prometheus_client 0.23 for metrics, and tonic 0.13 for gRPC. The NATS JetStream client is async-nats 0.39. The storage dependencies are sqlx 0.8 for PostgreSQL 17, the redis crate 0.27 for Redis 8.x, the qdrant-client crate for Qdrant 1.13, and neo4rs for Neo4j 5.x. Security infrastructure uses the vault-client crate against HashiCorp Vault 1.18 and wasmtime 26.x for WASM sandboxing.
The LLM models referenced areolder models, that are provided by Cloud providers for far less cost. Cloud providers include OpenAI (GPT-5 and GPT-4o), Anthropic (Claude 4 Sonnet and Claude 4 Opus), and Google (Gemini 2.5 Pro). Local deployments use Ollama with Llama 4 and Mistral Large 3.
No comments:
Post a Comment