A Harness Engineer's Field Guide
A large language model is a brilliant, fast, and just unreliable enough that you would never hand one the keys to the building unmonitored. This is a field guide to the software that does the monitoring — the harness — built from first principles in C++26, against the stateless July 2026 revision of the Model Context Protocol.
Contents
- Why C++, and What a Harness Actually Is
- The Protocol Underneath, and the Stateless Revolution of July 2026
- Surveying the Ground, Honestly
- The Blueprint: Five Layers and One Principle
- Three New Tools from the Standard
- The Foundation: Shared Types and the HTTP Seam
- Resilience as a First-Class Citizen
- Talking to Models Without Caring Where They Live
- Speaking MCP Correctly, the Stateless Way
- Memory and Budget: The Conversation's Keeper
- The Guardrails: Where the Harness Says No
- Concurrency Without Chaos
- The Heart of the Machine: The Orchestrator Loop
- Wiring It All Together
- Proving It Works Before Trusting It
- From Zero to a Running Platform
- What This Platform Teaches, and Where It Goes Next
Chapter One — Why C++, and What a Harness Actually Is
Picture a large language model as a wildly talented but slightly feral consultant. It reasons well. It writes fluent prose. It can plan a multi-step task in seconds — and then, with total confidence, hallucinate a file path that doesn't exist, invent an argument a tool never asked for, or cheerfully propose deleting a production database because that looked like the fastest route to the goal. Nobody hires that consultant and hands over the keys to the building. What you do instead is surround them with a competent operations team: someone who hands them exactly the information the task needs, someone who reviews every proposed action before it happens, someone with a hard rule that certain doors simply don't open without a second signature, and someone keeping a meticulous log of every decision, so that when something does go wrong, the postmortem takes an hour instead of a week. Translated into software, that operations team is what practitioners call the harness — and building it well is a far more interesting, far more demanding engineering problem than calling an API and printing the response.
This article builds that harness from first principles, in C++26, against the Model Context Protocol as it stands after its sweeping July 2026 revision — and it does so for a deliberate reason. The properties that make a harness trustworthy are predictable timeouts, memory that can't silently leak, concurrency that can't silently race, and interfaces whose contracts are checked rather than merely hoped for. Those are exactly the properties C++ has spent decades getting right, and C++26 adds three new capabilities — static reflection, contracts, and the sender/receiver model — that map almost eerily well onto a harness's everyday problems. So this is not a Python agent framework ported line by line into a systems language. It's what an agent framework looks like when the language itself was built for demanding, safety-critical control code.
Two words will recur constantly, so let's fix their meaning now before anything later feels vague. Agentic means a program running a loop: the model chooses an action from a fixed menu of tools, the program executes the chosen action, the outcome is reported back to the model, and the cycle repeats until a final answer emerges or the harness itself calls time. Harness engineering is the craft of writing everything around that loop — the system prompt, the tool manifest, the budget that keeps a conversation from silently outgrowing its context window, the policy that decides which actions may proceed unattended and which require a human's signature, the retry logic that absorbs a flaky network without bothering anyone, and the audit trail that makes the whole system explainable after the fact. The model is one component in that system — and, as this article will insist from beginning to end, it should be the most replaceable one.
Chapter Two — The Protocol Underneath, and the Stateless Revolution of July 2026
Every tool an agent calls has to be described to it somehow, and hand-rolling a bespoke integration for every external system a project might ever need isn't an engineering strategy — it's a treadmill. The Model Context Protocol exists to get everyone off that treadmill: a shared, standardized way for models and the programs around them to describe and invoke tools. In the summer of 2026 the protocol went through the largest change of its short life. The maintainers describe the 2026-07-28 revision as delivering a stateless core that scales on ordinary HTTP infrastructure, extensions that include server-rendered UIs through MCP Apps and long-running work through the Tasks extension, authorization aligned more closely with OAuth and OpenID Connect deployments, and a formal deprecation policy so the protocol can evolve without breaking what you've built. The revision removes the initialization handshake and protocol-level sessions, and shifts version and capability data into every request — the kind of change that reads like a changelog technicality and ends up reshaping every client that has to speak the protocol.
The story behind the change is one every systems engineer will recognize instantly. MCP started life on personal laptops, where a client opened a session with a server and both sides simply remembered each other for the life of that one process. Then the protocol succeeded, servers moved into the cloud, and a design built around one client remembering one server across a long-lived connection became a design where any request might land on any instance behind a load balancer — and the session the client thought it had didn't exist on the machine that actually answered. The maintainers themselves have been candid that this was one of the hard lessons learned over the protocol's first two years of production use. The fix is the same one distributed systems engineers have reached for since long before language models existed: make every request carry everything it needs to be understood on its own, and stop pretending the network remembers anything for you.
For a harness, that decision cascades into two very concrete obligations. The first is discovery. Where an older client would open a session with an initialize call and learn the server's capabilities once, a client speaking the 2026-07-28 revision calls a method named `server/discover` to learn versions, capabilities, and identity — and it does this without ever opening anything that could be called a session. A modern server must implement `server/discover` to advertise this information, and every request the client sends afterward carries `Mcp-Method` and `Mcp-Name` headers describing itself, because those headers — not a session id — are what Streamable HTTP POST requests now require in order to be routed and understood. The second obligation concerns anything that used to live implicitly inside a session: a cursor into a long search, a handle to a partially read file, a token representing where a paginated call left off. None of that state disappears; the revision insists it be represented by server-minted handles passed as ordinary tool arguments, which means the harness's own conversation state becomes the one and only place such handles are remembered between calls. That turns out to be a genuinely good discipline for an agent harness independent of MCP, because it means the harness — not some invisible transport-level session — is always the single source of truth for what has happened in a conversation. And that is precisely the property you want when you're also trying to build a reliable audit trail.
One more piece of the protocol's vocabulary matters enormously to the safety story this article tells later: tool annotations. A server can mark a tool with hints such as `destructiveHint` or `readOnlyHint`, part of a wider set of tool annotations the protocol introduced to let a client reason about risk without having to understand what a tool actually does internally. The harness built in this article treats those hints as advisory rather than authoritative. The absence of `destructiveHint` is read as "the server made no claim," never as "this action is safe" — because a hint is an author's assertion, not a guarantee, and a deny-by-default policy layer has to behave accordingly.
Chapter Three — Surveying the Ground, Honestly
Before writing a line of client code, it's worth asking a question a lot of technical writing skips past too quickly: does a mature C++ implementation of this protocol already exist — and if so, shouldn't this article simply be about using it? The honest answer, at the time of writing, is that nothing occupies the position the official TypeScript, Python, or C# SDKs occupy for their languages. The C# SDK, for comparison, is maintained in collaboration with Microsoft and ships as three coherent packages: a core client and server layer, a hosting and dependency-injection layer, and an ASP.NET Core HTTP layer, each building cleanly on the last. Nothing in the C++ world currently has that shape. What exists instead is a small constellation of community projects at very different levels of maturity: one built against C++14 that focuses on exposing a stable C API so the core protocol logic can be shared across bindings for half a dozen other languages, and another, more experimental effort still targeting C++17 that implements only the stdio transport alongside the basic lifecycle and tool-call flow. Neither targets C++26, and neither — as far as the sources available while writing this could establish — has yet adopted the stateless 2026-07-28 revision this harness is built against.
That gap isn't a criticism of those projects. Protocols evolve faster than volunteer-maintained SDKs can always track, and that's a completely ordinary state of affairs for an ecosystem this young. It is, however, a real engineering decision point, and the decision this article makes is to build a small, purpose-built MCP client rather than adopt an immature dependency for a layer this central to the system's correctness. The justification isn't pride of authorship. The vocabulary of MCP methods a harness actually needs is genuinely small — discovery, listing tools, calling a tool, and, once server-to-client patterns are needed, listening to a subscription stream — and a project that owns this thin layer itself is never blocked waiting for someone else's roadmap to catch up with a protocol revision that only just shipped. Chapter Nine builds exactly that client, and it is deliberately, defensibly small.
Chapter Four — The Blueprint: Five Layers and One Principle
Every interesting piece of engineering in this harness follows from a single architectural decision, stated already in Chapter One but worth restating as a design principle rather than a slogan: the model is a replaceable dependency behind an interface, and the control loop is the stable, tested, versioned core of the system. Everything else in the architecture is a consequence of taking that principle seriously enough to actually build interfaces around it, rather than merely asserting it in a design document nobody revisits.
The platform is organized into five layers, each behind its own abstract interface, and none of them knows the concrete details of the others. A model provider interface hides whether inference happens on the same machine through a llama.cpp server or across the internet against a hosted Anthropic endpoint — the orchestrator that calls it never needs to know which. A tool gateway interface hides the entire JSON-RPC and header discipline the 2026-07-28 revision demands behind a small, synchronous, exception-based call, so that nothing above it ever touches raw wire formats. A context manager owns the conversation's history and is the sole authority on what gets kept, what gets summarized, and what must never be touched no matter how tight the budget gets. A policy layer, built to say no by default, decides independently of the model's own judgment which actions may proceed unattended and which must stop and wait for a human. A tool scheduler executes approved calls concurrently, under a real transport-level deadline, so that one slow tool can never quietly stall an entire turn. The orchestrator sits above all five, and it is, deliberately, the only piece of the system complicated enough to need real narrative explanation — because everything beneath it exists specifically to keep it simple.
This layering is not decoration. It's what allows the smoke test built in Chapter Fifteen to exercise the entire control loop — tool approval, denial, dispatch, and final-answer generation — without a single real network call, a single running model, or a single live MCP server anywhere in sight. A harness you can't test without three live services running is a harness you won't test often enough. And a harness you don't test often enough is a harness whose bugs you discover in production instead of in continuous integration.
Chapter Five — Three New Tools from the Standard
C++26 was finalized at the ISO committee's March 2026 meeting in London. The vote to ship it was 114 in favor, 12 opposed, and 3 abstaining — a margin the committee's own reporting describes as showing that virtually every expert in the room had, by then, formed a firm opinion, for or against, about the standard's most contested addition. That addition was contracts. Its presence in the vote tally is a useful reminder, before any of these features are used in anger, that "finalized in the standard" and "battle-tested in every compiler" are two very different claims. This chapter is careful to keep them separate.
Static reflection is the first of the three, and arguably the most quietly transformative. It introduces the reflection operator, written `^^`, which turns an entity — a type, a member, an enumerator — into a compile-time value of type `std::meta::info`, together with a splice syntax, written `[: :]`, that turns such a value back into ordinary code. The committee's own description of the feature is that it enables compile-time introspection on types and behavior in a way that previously required macros or external code generators. For a harness, the very first place this pays off is schema generation: rather than maintaining a tool's JSON schema by hand in a separate file — one that can quietly drift out of sync with the C++ struct the tool handler actually parses into — reflection lets that schema be derived directly from the struct's own declaration, so the two can never disagree. Chapter Eight shows both versions of this idea side by side: the fully buildable, traits-based one that any current compiler accepts, and the reflection-based one, kept honestly labeled as a preview of where the technique is headed once toolchain support catches up.
Contracts are the second feature: language-level preconditions and postconditions that turn an invariant from a comment or a scattered `if` statement into something the function signature itself states and can enforce. A harness's control loop is full of exactly this kind of invariant — a turn should never begin once a turn ceiling has already been exceeded; a response returned to the caller should never be silently empty. Chapter Thirteen's orchestrator states both invariants as contracts using the syntax C++26 defines, and, because contract support is still settling across compilers at the time of writing, pairs each one with a plain `assert` behind a feature-test guard, so the invariant holds regardless of which compiler happens to be building the project on a given day.
The sender/receiver model, exposed through `std::execution`, is the third feature — and the one that most rewards a careful, honest treatment rather than an enthusiastic one. It gives C++ a genuinely standardized vocabulary for asynchronous and parallel work, and it carries a real safety property: structured, lifetime-nested concurrency written against this model tends to be data-race-free by construction, which is a serious claim and a valuable one for code that fans out several tool calls at once. Herb Sutter, who has tracked this feature through years of committee work, is candid in his own account that it is currently harder to adopt than most C++ features — it lacks great documentation and some of the small helper libraries a comfortable ecosystem needs — and that a team adopting it today should expect to spend real time learning it and to write some adapters of its own. This article takes that caution seriously. Chapter Twelve's buildable concurrency layer is a small, ordinary thread pool with a transport-level timeout, something any current compiler builds today. The sender/receiver version of the same idea is preserved as a clearly labeled sketch, written using only vocabulary the standard genuinely defines, with the one piece that is unavoidably implementation-supplied — a timed scheduler — marked as exactly that rather than invented.
Chapter Six — The Foundation: Shared Types and the HTTP Seam
Every layer above this one depends on a small set of types shared across the entire codebase, and getting these types right the first time is what prevents an entire category of bug that otherwise surfaces weeks later as mysteriously corrupted conversation state. The most important decision baked into these types is a subtle one: every field on `ChatMessage` is set by name at every construction site in this project, never by positional aggregate initialization — because a positional list silently shifts every field into the wrong slot the moment a new field is added anywhere upstream. That mistake is invisible at the point where it happens and only shows up much later, as data that is quietly, plausibly wrong.
// file: include/agent/types.hpp
#pragma once
#include <optional>
#include <string>
#include <vector>
#include <nlohmann/json.hpp>
namespace agent {
using json = nlohmann::json;
// The four roles a message in the conversation can take. This mirrors
// the vocabulary both the OpenAI-compatible llama.cpp endpoint and the
// Anthropic Messages API use, even though the two backends serialize
// "system" and "tool" differently, which is why the mapping logic
// lives in each provider rather than here.
enum class Role {
System,
User,
Assistant,
Tool,
};
// A single request, as the model sees it, to invoke one tool with a
// specific set of arguments. id, name, and arguments come from the
// model (or from the backend acting on its behalf) and must be
// echoed back verbatim in the matching ToolResult, because that is
// how the model correlates a result with the call that produced it.
// idempotency_key is different: it is harness-generated metadata,
// attached only after the policy layer has approved the call, and a
// model provider never sets it itself.
struct ToolCall {
std::string id;
std::string name;
json arguments;
std::optional<std::string> idempotency_key;
};
// The outcome of executing one ToolCall. content is kept as a json
// value rather than a plain string because MCP tool results can be
// structured; providers that need a plain string serialize it with
// content.dump() when they build their own wire format. is_error is
// what lets a policy denial and a genuine tool failure both be
// reported back to the model without either one being silently
// dropped on the way into the conversation history.
struct ToolResult {
std::string tool_call_id;
std::string name;
json content;
bool is_error = false;
};
// One entry in the conversation. Exactly one group of the optional
// fields is meaningful at a time, depending on role: tool_calls is
// only populated on an Assistant message that requested tools, and
// tool_call_id / tool_name / is_error are only meaningful on a Tool
// message reporting the outcome of one of those calls.
struct ChatMessage {
Role role = Role::User;
std::string content;
std::vector<ToolCall> tool_calls;
std::optional<std::string> tool_call_id;
std::optional<std::string> tool_name;
bool is_error = false;
};
// The MCP tool manifest, translated into a shape both providers can
// consume directly. destructive reflects the server's destructiveHint
// annotation; its absence is treated as false at the parsing site,
// but the policy layer must never treat false as a safety guarantee,
// only as "the server did not claim this action is destructive".
struct ToolDescriptor {
std::string name;
std::string description;
json input_schema;
bool destructive = false;
};
struct ModelRequest {
std::vector<ChatMessage> messages;
std::vector<ToolDescriptor> tools;
int max_tokens = 1024;
};
struct ModelReply {
std::string text;
std::vector<ToolCall> tool_calls;
};
struct AgentResponse {
std::string text;
};
} // namespace agent
With the vocabulary fixed, the next question is how anything in this project ever talks to the outside world. The answer is that it doesn't — not directly. Every outbound call, whether to a local model, a remote model, or an MCP server, passes through one small abstract interface, and this is precisely what lets the entire orchestration logic be tested without a socket ever opening. The timeout parameter on this interface is not decorative, either; it is the mechanism — elaborated fully in Chapter Twelve — by which a slow tool call is genuinely bounded rather than merely abandoned while its underlying connection quietly lingers.
// file: include/agent/http_client.hpp
#pragma once
#include <chrono>
#include <string>
#include <utility>
#include <vector>
namespace agent {
using HttpHeaders = std::vector<std::pair<std::string, std::string>>;
struct HttpResponse {
long status_code = 0;
std::string body;
};
// A tiny seam over whatever HTTP implementation the project links
// against. An implementation throws only for a transport-level
// failure: DNS resolution, connection refused, a TLS handshake
// problem, or the timeout itself firing. A request that completes
// with any HTTP status, including 4xx and 5xx, returns normally as an
// HttpResponse, so callers can decide for themselves whether that
// status is worth retrying; see agent::post_with_retry and
// agent::ensure_success in retry.hpp for the shared policy this
// project applies to that decision. Everything above this interface,
// model providers and the MCP tool gateway alike, depends only on
// this abstract type, which is what makes both of them testable with
// an in-memory fake.
class HttpClient {
public:
virtual ~HttpClient() = default;
virtual HttpResponse post(const std::string& url,
const std::string& body,
const HttpHeaders& headers,
std::chrono::milliseconds timeout) = 0;
};
} // namespace agent
The one concrete implementation of that seam wraps libcurl, chosen because it is the most thoroughly documented, most widely available production-grade HTTP and TLS client the C++ ecosystem has, present in every major package manager. Two details here matter far more than their brief appearance suggests. Disabling libcurl's signal-based timeout mechanism is not stylistic caution: the tool scheduler built in Chapter Twelve runs several of these calls concurrently from a thread pool, and without this one line, a timeout firing on one thread can deliver a signal to a completely different thread and crash the whole process. And returning an ordinary, non-throwing response for any completed HTTP status — success or failure alike — rather than throwing on anything but 200 is what makes it possible for the retry layer in Chapter Seven to actually inspect a failure and decide whether it's worth another attempt. That decision becomes structurally impossible once an exception has already unwound the stack.
// file: include/agent/curl_http_client.hpp
#pragma once
#include "agent/http_client.hpp"
namespace agent {
class CurlHttpClient final : public HttpClient {
public:
CurlHttpClient() = default;
HttpResponse post(const std::string& url,
const std::string& body,
const HttpHeaders& headers,
std::chrono::milliseconds timeout) override;
};
} // namespace agent
// file: src/curl_http_client.cpp
#include "agent/curl_http_client.hpp"
#include <curl/curl.h>
#include <stdexcept>
namespace agent {
namespace {
// curl_global_init/cleanup must run exactly once per process. A
// function-local static with a destructor is a simple, correct way to
// arrange that without introducing an explicit startup step that
// every executable linking this library would otherwise have to
// remember to call.
struct CurlGlobalGuard {
CurlGlobalGuard() { curl_global_init(CURL_GLOBAL_DEFAULT); }
~CurlGlobalGuard() { curl_global_cleanup(); }
};
const CurlGlobalGuard curl_global_guard;
std::size_t write_into_string(char* data, std::size_t size,
std::size_t count, void* user_pointer) {
auto* out = static_cast<std::string*>(user_pointer);
out->append(data, size * count);
return size * count;
}
} // namespace
HttpResponse CurlHttpClient::post(const std::string& url,
const std::string& body,
const HttpHeaders& headers,
std::chrono::milliseconds timeout) {
CURL* handle = curl_easy_init();
if (handle == nullptr) {
throw std::runtime_error("curl_easy_init failed");
}
std::string response_body;
curl_slist* header_list = nullptr;
for (const auto& [key, value] : headers) {
const std::string line = key + ": " + value;
header_list = curl_slist_append(header_list, line.c_str());
}
curl_easy_setopt(handle, CURLOPT_URL, url.c_str());
curl_easy_setopt(handle, CURLOPT_POST, 1L);
curl_easy_setopt(handle, CURLOPT_POSTFIELDS, body.c_str());
curl_easy_setopt(handle, CURLOPT_POSTFIELDSIZE,
static_cast<long>(body.size()));
curl_easy_setopt(handle, CURLOPT_HTTPHEADER, header_list);
curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, write_into_string);
curl_easy_setopt(handle, CURLOPT_WRITEDATA, &response_body);
curl_easy_setopt(handle, CURLOPT_TIMEOUT_MS,
static_cast<long>(timeout.count()));
curl_easy_setopt(handle, CURLOPT_FOLLOWLOCATION, 1L);
// Required alongside a timeout in a multithreaded program: without
// this, libcurl's default signal-based timeout implementation can
// deliver a SIGALRM to an arbitrary thread and crash the process.
// The tool scheduler in this project runs several calls through
// this same client concurrently from a thread pool, so this is
// not an optional hardening detail here, it is load-bearing.
curl_easy_setopt(handle, CURLOPT_NOSIGNAL, 1L);
const CURLcode result = curl_easy_perform(handle);
long status_code = 0;
curl_easy_getinfo(handle, CURLINFO_RESPONSE_CODE, &status_code);
curl_slist_free_all(header_list);
curl_easy_cleanup(handle);
if (result != CURLE_OK) {
throw std::runtime_error(
std::string("HTTP transport error for ") + url + ": " +
curl_easy_strerror(result));
}
// A non-2xx status is deliberately NOT turned into an exception
// here: it is returned to the caller as an ordinary HttpResponse
// so that post_with_retry (see retry.hpp) can inspect the status
// code and decide whether the failure is transient and worth
// retrying, or permanent and worth failing on immediately. A
// caller that just wants "throw on anything but success" can pass
// the result straight through ensure_success from the same header.
return HttpResponse{status_code, std::move(response_body)};
}
} // namespace agent
Chapter Seven — Resilience as a First-Class Citizen
A harness that only works when every network call succeeds on the first try is a demo, not a platform. The difference between the two is almost entirely a matter of what happens in the unglamorous moments when something fails. This chapter builds three small, shared facilities, each one addressing a distinct failure mode, and each one deliberately kept independent of any particular model provider or tool server — so the same resilience logic protects every outbound call in the system.
The first facility distinguishes between a failure worth retrying and one that isn't. A request that times out, a connection that's refused, a server that briefly returns a 500 while it's overloaded — all of these are transient, and a short, backing-off retry usually resolves them without anyone noticing. A request rejected with an authentication error or a malformed body is a different animal entirely; retrying it wastes time and can even make monitoring noisier without ever succeeding. The helper below makes that distinction explicit, rather than leaving it to whatever a caller happens to remember to check.
// file: include/agent/retry.hpp
#pragma once
#include <chrono>
#include <stdexcept>
#include <string>
#include <thread>
#include "agent/http_client.hpp"
namespace agent {
// A small, general-purpose retry helper shared by both model
// providers and the MCP tool gateway. It retries a POST up to
// max_attempts times, but only when the failure looks transient: a
// transport-level exception (DNS failure, connection refused, TLS
// error, timeout) or an HTTP status of 429 (rate limited) or in the
// 500-599 range (server error). Any other status is returned
// immediately without a retry, since retrying a 400 or a 401 cannot
// succeed by waiting. Successive attempts wait with a simple
// exponential backoff. Marked inline because this header is included
// from three separate translation units that are linked into the
// same library target; without inline, that would be an ODR
// violation and a duplicate-symbol error at link time.
inline HttpResponse post_with_retry(HttpClient& http, const std::string& url,
const std::string& body,
const HttpHeaders& headers,
std::chrono::milliseconds timeout,
int max_attempts = 3) {
std::chrono::milliseconds backoff{500};
std::string last_error;
for (int attempt = 1; attempt <= max_attempts; ++attempt) {
try {
HttpResponse response = http.post(url, body, headers, timeout);
const bool retryable_status =
response.status_code == 429 ||
(response.status_code >= 500 && response.status_code < 600);
if (!retryable_status || attempt == max_attempts) {
return response;
}
last_error = "HTTP " + std::to_string(response.status_code);
} catch (const std::exception& error) {
last_error = error.what();
if (attempt == max_attempts) {
throw;
}
}
std::this_thread::sleep_for(backoff);
backoff *= 2;
}
// Unreachable in practice: every iteration above either returns
// or throws on the final attempt. This keeps the compiler
// satisfied that every path out of the function is accounted for.
throw std::runtime_error("post_with_retry exhausted attempts: " + last_error);
}
// Throws a clear, diagnosable error if response does not carry a
// success status, folding the HTTP status and the response body into
// the exception message so a caller never has to guess what a remote
// service actually said. context is a short description of what was
// being attempted, used purely to make the resulting message
// readable in a log.
inline void ensure_success(const HttpResponse& response, const std::string& context) {
if (response.status_code < 200 || response.status_code >= 300) {
throw std::runtime_error(context + " failed with HTTP " +
std::to_string(response.status_code) + ": " +
response.body);
}
}
} // namespace agent
Retries handle brief turbulence, but a dependency that's genuinely down for an extended period needs a different response: stop calling it. Repeatedly retrying a service that's been unreachable for the last ten minutes accomplishes nothing except adding load to an already struggling system and delaying the moment the harness admits defeat and reports a clear error. A circuit breaker solves exactly this: it tracks recent failures and, once they cross a threshold, refuses calls outright for a cooldown period before cautiously allowing a single trial call through to see whether the dependency has recovered.
// file: include/agent/circuit_breaker.hpp
#pragma once
#include <chrono>
#include <mutex>
namespace agent {
// A minimal circuit breaker: after failure_threshold consecutive
// failures, the breaker opens and refuses calls for open_duration,
// after which it allows a trial call through (half-open). A
// successful trial call closes the breaker and resets the failure
// count; a failed trial call reopens it and restarts the timer. This
// is the complement to the per-call retry-with-backoff in retry.hpp:
// retries absorb brief transient blips, the breaker protects a
// sustained outage from being hammered with retries that cannot
// possibly succeed. One known simplification is worth stating
// plainly: under concurrent access from multiple threads, more than
// one trial call could observe half-open at once, since the check in
// allow_call and the eventual record_success or record_failure are
// separate critical sections rather than one atomic reservation. This
// project never calls a single CircuitBreaker instance concurrently
// (only tool calls run concurrently, never model calls), so the race
// is latent rather than exercised here, but a deployment that does
// call generate from multiple threads should add a dedicated
// "trial in flight" flag before relying on the half-open behavior.
class CircuitBreaker {
public:
explicit CircuitBreaker(int failure_threshold = 5,
std::chrono::milliseconds open_duration =
std::chrono::seconds(30))
: failure_threshold_(failure_threshold), open_duration_(open_duration) {}
bool allow_call() {
std::lock_guard<std::mutex> lock(mutex_);
if (!open_) return true;
const auto now = std::chrono::steady_clock::now();
if (now - opened_at_ >= open_duration_) {
half_open_ = true;
return true;
}
return false;
}
void record_success() {
std::lock_guard<std::mutex> lock(mutex_);
consecutive_failures_ = 0;
open_ = false;
half_open_ = false;
}
void record_failure() {
std::lock_guard<std::mutex> lock(mutex_);
if (half_open_) {
half_open_ = false;
open_ = true;
opened_at_ = std::chrono::steady_clock::now();
return;
}
++consecutive_failures_;
if (consecutive_failures_ >= failure_threshold_) {
open_ = true;
opened_at_ = std::chrono::steady_clock::now();
}
}
private:
std::mutex mutex_;
int failure_threshold_;
std::chrono::milliseconds open_duration_;
int consecutive_failures_ = 0;
bool open_ = false;
bool half_open_ = false;
std::chrono::steady_clock::time_point opened_at_{};
};
} // namespace agent
The third facility addresses a subtler danger: what happens when a retry succeeds twice. If a mutating tool call — sending an email, charging a card, deleting a record — times out after the server already received it and started acting on it, a naive retry performs the same action a second time. The fix is an idempotency key: a random token the harness itself generates and attaches to the call before it's dispatched, so a well-behaved server can recognize a duplicate delivery of the same logical request and simply return the original result instead of acting twice. The same random generator, reused as a correlation identifier, is also what ties every log line for a single turn together — which matters enormously in Chapter Thirteen.
// file: include/agent/random_id.hpp
#pragma once
#include <cstdint>
#include <random>
#include <sstream>
#include <string>
namespace agent {
// Generates an opaque random identifier, used both as the
// idempotency key the harness attaches to an approved tool call
// before dispatch and as the correlation identifier that ties every
// log line for one turn together. A thread_local generator means
// concurrent tool dispatch never contends on a shared generator's
// internal state. Marked inline for the same reason as the functions
// in retry.hpp: this header is included from more than one
// translation unit in the same library target.
inline std::string generate_random_id() {
static thread_local std::mt19937_64 generator{std::random_device{}()};
std::uniform_int_distribution<std::uint64_t> distribution;
std::ostringstream out;
out << std::hex << distribution(generator);
return out.str();
}
} // namespace agent
// file: include/agent/logging.hpp
#pragma once
#include <iostream>
#include <string>
#include <nlohmann/json.hpp>
namespace agent {
// Emits one structured, single-line JSON log record to stderr. A real
// deployment would route this to whatever logging or tracing pipeline
// it already operates, but the shape, one self-contained JSON object
// per line, is what most log aggregators expect, and it is what lets
// an operator grep for a single correlation_id and see every model
// call, tool call, and outcome that happened within one turn, in
// order.
inline void log_event(const std::string& correlation_id,
const std::string& event,
const nlohmann::json& details = nlohmann::json::object()) {
nlohmann::json record;
record["correlation_id"] = correlation_id;
record["event"] = event;
record["details"] = details;
std::cerr << record.dump() << '\n';
}
} // namespace agent
Chapter Eight — Talking to Models Without Caring Where They Live
Here is where the architectural principle from Chapter Four earns its keep in visible, concrete form. The orchestrator that drives every conversation on this platform is going to call a single method, `generate`, and it is never going to know — or need to know — whether the reply it gets back came from a model running in the same rack or a model running across the ocean. Everything that makes that possible funnels through one interface.
// file: include/agent/model_provider.hpp
#pragma once
#include "agent/types.hpp"
namespace agent {
class ModelProvider {
public:
virtual ~ModelProvider() = default;
virtual ModelReply generate(const ModelRequest& request) = 0;
};
} // namespace agent
The local implementation targets llama.cpp's server binary running in its OpenAI-compatible HTTP mode on the same machine — by far the simplest way to get a capable open-weight model running privately, with full control over what data ever leaves the host. Building the request body is mostly straightforward JSON assembly. Parsing the reply defensively is where the real engineering lives, because a smaller, locally hosted model is considerably more likely than a large hosted one to emit a slightly malformed tool call — and a harness that lets one bad parse take down an entire turn is not something you can put in front of a real user. Every defensive check below exists because of a specific way a real backend has been observed to misbehave: a `content` field that arrives as JSON null rather than an absent key, a tool call missing its `id` entirely, arguments that arrive as an already-parsed object instead of the embedded string the format nominally specifies.
// file: include/agent/local_llama_provider.hpp
#pragma once
#include <chrono>
#include <string>
#include "agent/http_client.hpp"
#include "agent/model_provider.hpp"
namespace agent {
class LocalLlamaProvider final : public ModelProvider {
public:
LocalLlamaProvider(HttpClient& http, std::string endpoint,
std::chrono::milliseconds timeout =
std::chrono::seconds(120))
: http_(http), endpoint_(std::move(endpoint)), timeout_(timeout) {}
ModelReply generate(const ModelRequest& request) override;
private:
HttpClient& http_;
std::string endpoint_;
std::chrono::milliseconds timeout_;
};
} // namespace agent
// file: src/local_llama_provider.cpp
#include "agent/local_llama_provider.hpp"
#include <iostream>
#include "agent/retry.hpp"
namespace agent {
namespace {
std::string role_to_openai_string(Role role) {
switch (role) {
case Role::System: return "system";
case Role::User: return "user";
case Role::Assistant: return "assistant";
case Role::Tool: return "tool";
}
return "user";
}
// Reads a string field defensively. nlohmann::json::value(key,
// default) still throws if the key is present but holds a JSON null,
// which is exactly what many OpenAI-compatible backends send for
// "content" on an assistant message that only carries tool calls.
// This helper treats a present-but-null field the same as an absent
// one, which was the source of a real crash in an earlier version of
// this file.
std::string safe_string(const json& object, const char* key) {
if (!object.contains(key) || object.at(key).is_null()) return "";
return object.at(key).get<std::string>();
}
json build_messages_array(const std::vector<ChatMessage>& history) {
json messages = json::array();
for (const auto& message : history) {
json entry;
entry["role"] = role_to_openai_string(message.role);
const bool assistant_with_tool_calls =
message.role == Role::Assistant && !message.tool_calls.empty();
if (assistant_with_tool_calls && message.content.empty()) {
// Several OpenAI-compatible backends expect "content" to
// be null, not an empty string, on a tool-calling
// assistant turn; null is accepted universally, while an
// empty string is rejected by some stricter servers.
entry["content"] = nullptr;
} else {
entry["content"] = message.content;
}
if (assistant_with_tool_calls) {
json tool_calls = json::array();
for (const auto& call : message.tool_calls) {
tool_calls.push_back({
{"id", call.id},
{"type", "function"},
{"function", {
{"name", call.name},
{"arguments", call.arguments.dump()},
}},
});
}
entry["tool_calls"] = tool_calls;
}
if (message.role == Role::Tool) {
entry["tool_call_id"] = message.tool_call_id.value_or("");
entry["name"] = message.tool_name.value_or("");
}
messages.push_back(std::move(entry));
}
return messages;
}
json build_tools_array(const std::vector<ToolDescriptor>& tools) {
json array = json::array();
for (const auto& tool : tools) {
array.push_back({
{"type", "function"},
{"function", {
{"name", tool.name},
{"description", tool.description},
{"parameters", tool.input_schema},
}},
});
}
return array;
}
} // namespace
ModelReply LocalLlamaProvider::generate(const ModelRequest& request) {
json payload;
payload["model"] = "local";
payload["messages"] = build_messages_array(request.messages);
payload["max_tokens"] = request.max_tokens;
if (!request.tools.empty()) {
payload["tools"] = build_tools_array(request.tools);
}
const HttpHeaders headers{{"Content-Type", "application/json"}};
const HttpResponse raw =
post_with_retry(http_, endpoint_ + "/v1/chat/completions",
payload.dump(), headers, timeout_);
ensure_success(raw, "local model request");
const json parsed = json::parse(raw.body);
const json& message = parsed.at("choices").at(0).at("message");
ModelReply reply;
reply.text = safe_string(message, "content");
if (message.contains("tool_calls") && !message.at("tool_calls").is_null()) {
std::size_t index = 0;
for (const auto& raw_call : message.at("tool_calls")) {
ToolCall call;
call.id = safe_string(raw_call, "id");
if (call.id.empty()) {
// Some backends omit tool call ids entirely; a
// stable, unique fallback keeps parallel tool calls
// within the same turn distinguishable from another.
call.id = "call_" + std::to_string(index);
}
const json& function = raw_call.at("function");
call.name = safe_string(function, "name");
const json arguments_field =
function.contains("arguments") ? function.at("arguments")
: json::object();
if (arguments_field.is_string()) {
try {
call.arguments = json::parse(arguments_field.get<std::string>());
} catch (const json::parse_error& error) {
std::cerr << "warning: model emitted malformed tool "
"arguments for '"
<< call.name << "': " << error.what() << '\n';
call.arguments = json::object();
}
} else if (arguments_field.is_object()) {
// A handful of backends already return a parsed
// object rather than an embedded JSON string; accept
// that shape directly instead of failing on it.
call.arguments = arguments_field;
} else {
call.arguments = json::object();
}
reply.tool_calls.push_back(std::move(call));
++index;
}
}
return reply;
}
} // namespace agent
The remote implementation targets Anthropic's Messages API, and it has to do genuine translation work, because that API's wire format disagrees with the OpenAI-style format in three specific ways. The system prompt is its own top-level field rather than a message in the array. A tool result is represented as a user message containing a `tool_result` content block rather than as a message with a dedicated tool role. And an assistant turn that requested tools represents those requests as `tool_use` blocks inside the assistant's own content array rather than as a separate `tool_calls` list. None of that complexity ever reaches the orchestrator; it's fully absorbed here, in the translation layer, exactly where it belongs. Note also that the `is_error` flag on a tool-result message is threaded all the way through into the `tool_result` block's own `is_error` field — precisely the piece of information an earlier draft of this platform silently lost on its way from a `ToolResult` into the conversation.
// file: include/agent/remote_anthropic_provider.hpp
#pragma once
#include <chrono>
#include <string>
#include "agent/circuit_breaker.hpp"
#include "agent/http_client.hpp"
#include "agent/model_provider.hpp"
namespace agent {
class RemoteAnthropicProvider final : public ModelProvider {
public:
RemoteAnthropicProvider(HttpClient& http, std::string api_key,
std::string model,
std::chrono::milliseconds timeout =
std::chrono::seconds(120))
: http_(http),
api_key_(std::move(api_key)),
model_(std::move(model)),
timeout_(timeout) {}
ModelReply generate(const ModelRequest& request) override;
private:
HttpClient& http_;
std::string api_key_;
std::string model_;
std::chrono::milliseconds timeout_;
CircuitBreaker circuit_breaker_;
};
} // namespace agent
// file: src/remote_anthropic_provider.cpp
#include "agent/remote_anthropic_provider.hpp"
#include <stdexcept>
#include "agent/retry.hpp"
namespace agent {
namespace {
// Anthropic's API keeps the system prompt out of the messages array
// entirely, so it is extracted here and concatenated if more than one
// system message is present, which the ChatMessage history permits
// even though a single turn will usually contain only one, seeded
// once by main() at the start of the conversation.
std::string extract_system_prompt(const std::vector<ChatMessage>& history) {
std::string system;
for (const auto& message : history) {
if (message.role == Role::System) {
if (!system.empty()) system += "\n";
system += message.content;
}
}
return system;
}
json build_messages_array(const std::vector<ChatMessage>& history) {
json messages = json::array();
for (const auto& message : history) {
if (message.role == Role::System) continue; // handled separately
if (message.role == Role::User) {
messages.push_back({{"role", "user"}, {"content", message.content}});
continue;
}
if (message.role == Role::Assistant) {
json content = json::array();
if (!message.content.empty()) {
content.push_back({{"type", "text"}, {"text", message.content}});
}
for (const auto& call : message.tool_calls) {
content.push_back({
{"type", "tool_use"},
{"id", call.id},
{"name", call.name},
{"input", call.arguments},
});
}
messages.push_back({{"role", "assistant"}, {"content", content}});
continue;
}
if (message.role == Role::Tool) {
// is_error is carried through here so the model can tell
// a failed tool call apart from a successful one; an
// earlier version of this project discarded that flag
// between ToolResult and ChatMessage and lost it entirely.
json block = {
{"type", "tool_result"},
{"tool_use_id", message.tool_call_id.value_or("")},
{"content", message.content},
};
if (message.is_error) {
block["is_error"] = true;
}
json content = json::array();
content.push_back(block);
messages.push_back({{"role", "user"}, {"content", content}});
}
}
return messages;
}
json build_tools_array(const std::vector<ToolDescriptor>& tools) {
json array = json::array();
for (const auto& tool : tools) {
array.push_back({
{"name", tool.name},
{"description", tool.description},
{"input_schema", tool.input_schema},
});
}
return array;
}
} // namespace
ModelReply RemoteAnthropicProvider::generate(const ModelRequest& request) {
if (!circuit_breaker_.allow_call()) {
throw std::runtime_error(
"circuit breaker open for the remote model backend; refusing "
"to call it until it has had time to recover");
}
json payload;
payload["model"] = model_;
payload["max_tokens"] = request.max_tokens;
payload["system"] = extract_system_prompt(request.messages);
payload["messages"] = build_messages_array(request.messages);
if (!request.tools.empty()) {
payload["tools"] = build_tools_array(request.tools);
}
const HttpHeaders headers{
{"x-api-key", api_key_},
{"anthropic-version", "2023-06-01"},
{"content-type", "application/json"},
};
try {
const HttpResponse raw =
post_with_retry(http_, "https://api.anthropic.com/v1/messages",
payload.dump(), headers, timeout_);
ensure_success(raw, "remote model request");
circuit_breaker_.record_success();
const json parsed = json::parse(raw.body);
ModelReply reply;
for (const auto& block : parsed.at("content")) {
const std::string type = block.value("type", "");
if (type == "text") {
reply.text += block.value("text", "");
} else if (type == "tool_use") {
ToolCall call;
call.id = block.value("id", "");
call.name = block.value("name", "");
call.arguments = block.value("input", json::object());
reply.tool_calls.push_back(std::move(call));
}
}
return reply;
} catch (...) {
circuit_breaker_.record_failure();
throw;
}
}
} // namespace agent
A platform isn't only a consumer of tools; sooner or later it's also a provider of them, exposing its own capabilities to other agents through the same protocol. This is where the reflection idea from Chapter Five earns its place, and it's presented here as two files rather than one: a fully buildable, traits-based schema generator that any current compiler accepts, and, immediately beside it, the reflection-based version of the same idea — honestly labeled as a preview that cannot yet be compiled.
// file: include/agent/schema_generator.hpp
//
// This header is used from the SERVER side of MCP, not the client
// side: it is what a C++ program implementing its own MCP tools would
// use to generate the JSON Schema it advertises for each tool, kept
// permanently in sync with the C++ struct the tool handler actually
// parses into. The ToolGateway built in Chapter Nine is a CLIENT: it
// receives inputSchema already generated by whichever server it
// calls, and never needs to build one itself, which is why nothing in
// the buildable executable includes this header. It is kept here
// because a harness is frequently both a consumer of someone else's
// tools and a provider of its own, and the second role needs exactly
// this technique.
#pragma once
#include <string>
#include <vector>
#include <nlohmann/json.hpp>
namespace agent {
using json = nlohmann::json;
struct FieldInfo {
std::string name;
std::string json_type; // "string", "integer", "number", "boolean"
bool required = true;
};
// Every tool argument struct must specialize this function. There is
// deliberately no generic definition: an attempt to use
// make_input_schema on a type nobody has described yet is a compile
// error, not a silently wrong schema.
template <typename T>
std::vector<FieldInfo> describe_fields();
template <typename T>
json make_input_schema() {
json properties = json::object();
json required = json::array();
for (const auto& field : describe_fields<T>()) {
properties[field.name] = {{"type", field.json_type}};
if (field.required) {
required.push_back(field.name);
}
}
json schema;
schema["type"] = "object";
schema["properties"] = properties;
if (!required.empty()) {
schema["required"] = required;
}
return schema;
}
// Example argument type and its hand-written description. A real
// project has one such pair per tool; the pair is small and, unlike a
// schema kept as a separate JSON file, cannot drift out of sync with
// the struct the handler actually parses into, because both live in
// the same translation unit and are reviewed together.
struct SearchDocsArgs {
std::string query;
int max_results = 10;
};
template <>
inline std::vector<FieldInfo> describe_fields<SearchDocsArgs>() {
return {
{"query", "string", true},
{"max_results", "integer", false},
};
}
} // namespace agent
// file: include/agent/schema_generator_reflection.hpp
//
// NOT part of the buildable project (see CMakeLists.txt in Chapter
// Fourteen, which does not reference this header). This is the
// reflection-based replacement for describe_fields<T>() above,
// written against the reflection syntax finalized for C++26: the
// reflection operator ^^, which turns an entity into a
// std::meta::info value, and the splice syntax [: ... :], which
// turns such a value back into code. It will not compile until a
// toolchain ships a conforming implementation of this reflection
// design; keep the traits-based version above as the supported path
// until then. json_type_name_of is included here, fully defined,
// specifically so this file is internally consistent on its own
// terms even though it cannot yet be built, rather than referencing
// a helper that exists nowhere.
#pragma once
#include <string>
#include <vector>
#include "agent/schema_generator.hpp"
namespace agent {
template <typename T>
consteval const char* json_type_name_of();
template <>
consteval const char* json_type_name_of<std::string>() { return "string"; }
template <>
consteval const char* json_type_name_of<int>() { return "integer"; }
template <>
consteval const char* json_type_name_of<bool>() { return "boolean"; }
template <>
consteval const char* json_type_name_of<double>() { return "number"; }
template <typename T>
consteval std::vector<FieldInfo> describe_fields_via_reflection() {
std::vector<FieldInfo> fields;
template for (constexpr auto member : std::meta::nonstatic_data_members_of(^^T)) {
FieldInfo field;
field.name = std::string(std::meta::identifier_of(member));
field.json_type = json_type_name_of<[:type_of(member):]>();
field.required = true;
fields.push_back(field);
}
return fields;
}
} // namespace agent
Chapter Nine — Speaking MCP Correctly, the Stateless Way
This chapter is where the protocol theory from Chapter Two turns into working code, and every line of it exists to satisfy a specific requirement of the 2026-07-28 revision. The interface is deliberately small: discover once, list the available tools, and call one of them — optionally carrying a continuation handle for a paginated or long-running operation and an idempotency key for a mutating one.
// file: include/agent/tool_gateway.hpp
#pragma once
#include <chrono>
#include <optional>
#include <string>
#include <vector>
#include "agent/types.hpp"
namespace agent {
class ToolGateway {
public:
virtual ~ToolGateway() = default;
virtual void discover() = 0;
virtual std::vector<ToolDescriptor> list_tools() = 0;
virtual const std::vector<ToolDescriptor>& tool_manifest() const = 0;
// continuation_handle carries server-minted state for a paginated
// or long-running call across successive invocations, as the
// stateless 2026-07-28 revision requires (see McpToolGateway).
// idempotency_key, when present, is attached to the outgoing call
// so a well-behaved server can recognize and safely ignore a
// duplicate delivery of the same logical call, for example after
// a retry triggered by a timeout.
virtual ToolResult call(
const std::string& tool_name, const json& arguments,
std::chrono::milliseconds timeout,
const std::optional<std::string>& continuation_handle = std::nullopt,
const std::optional<std::string>& idempotency_key = std::nullopt) = 0;
};
} // namespace agent
The concrete client below never opens anything resembling a session. Every single request carries the `Mcp-Method` and `Mcp-Name` headers the revision requires. Discovery happens exactly once, at startup, purely to learn the server's protocol version and capabilities. And both the tool listing and every tool call are checked for the `resultType` field the revision now mandates on results and list responses alike — a check an earlier draft of this platform applied inconsistently, and which is now applied everywhere that field is actually required.
// file: include/agent/mcp_tool_gateway.hpp
#pragma once
#include <atomic>
#include <string>
#include "agent/http_client.hpp"
#include "agent/tool_gateway.hpp"
namespace agent {
class McpToolGateway final : public ToolGateway {
public:
McpToolGateway(HttpClient& http, std::string server_url)
: http_(http), server_url_(std::move(server_url)) {}
void discover() override;
std::vector<ToolDescriptor> list_tools() override;
const std::vector<ToolDescriptor>& tool_manifest() const override {
return manifest_;
}
ToolResult call(const std::string& tool_name, const json& arguments,
std::chrono::milliseconds timeout,
const std::optional<std::string>& continuation_handle,
const std::optional<std::string>& idempotency_key) override;
private:
json post_jsonrpc(const std::string& method, const std::string& mcp_name,
const json& params, std::chrono::milliseconds timeout);
HttpClient& http_;
std::string server_url_;
std::string protocol_version_;
std::vector<ToolDescriptor> manifest_;
std::atomic<int> next_request_id_{1};
};
} // namespace agent
// file: src/mcp_tool_gateway.cpp
#include "agent/mcp_tool_gateway.hpp"
#include <stdexcept>
#include "agent/retry.hpp"
namespace agent {
json McpToolGateway::post_jsonrpc(const std::string& method,
const std::string& mcp_name,
const json& params,
std::chrono::milliseconds timeout) {
json envelope;
envelope["jsonrpc"] = "2.0";
envelope["id"] = next_request_id_.fetch_add(1);
envelope["method"] = method;
envelope["params"] = params;
// Every outgoing request is self-describing, as the stateless
// 2026-07-28 revision requires: there is no session id anywhere
// in this call, only headers naming the method and the target,
// and a JSON-RPC body that stands entirely on its own. The exact
// value a given server expects in Mcp-Name for a method with no
// single named target, such as tools/list or server/discover, is
// not something the sources available while writing this pin
// down precisely; using the method name itself is the most
// defensible reading, and it should be confirmed against the
// specific server this gateway is pointed at before relying on it.
const HttpHeaders headers{
{"Mcp-Method", method},
{"Mcp-Name", mcp_name},
{"Content-Type", "application/json"},
};
const HttpResponse raw =
post_with_retry(http_, server_url_, envelope.dump(), headers, timeout);
ensure_success(raw, "MCP " + method + " request");
const json response = json::parse(raw.body);
if (response.contains("error")) {
throw std::runtime_error("MCP error calling " + method + ": " +
response.at("error").dump());
}
return response.at("result");
}
void McpToolGateway::discover() {
// server/discover's own response shape (protocol version,
// capabilities, server identity) is distinct from the
// resultType-carrying shape the revision introduced for results
// and list responses, so no resultType check is applied here;
// see list_tools and call below for where that check does apply.
const json result =
post_jsonrpc("server/discover", "server/discover", json::object(),
std::chrono::seconds(10));
protocol_version_ = result.value("protocolVersion", "unknown");
}
std::vector<ToolDescriptor> McpToolGateway::list_tools() {
const json result = post_jsonrpc("tools/list", "tools/list", json::object(),
std::chrono::seconds(10));
if (!result.contains("resultType")) {
throw std::runtime_error(
"MCP server response for 'tools/list' is missing the "
"resultType field required since 2026-07-28");
}
manifest_.clear();
for (const auto& raw_tool : result.at("tools")) {
ToolDescriptor descriptor;
descriptor.name = raw_tool.value("name", "");
descriptor.description = raw_tool.value("description", "");
descriptor.input_schema = raw_tool.value("inputSchema", json::object());
// destructiveHint is an annotation the server MAY set; its
// absence must never be read as "this tool is safe", only as
// "this server did not make a claim either way".
if (raw_tool.contains("annotations")) {
descriptor.destructive =
raw_tool.at("annotations").value("destructiveHint", false);
}
manifest_.push_back(std::move(descriptor));
}
return manifest_;
}
ToolResult McpToolGateway::call(const std::string& tool_name,
const json& arguments,
std::chrono::milliseconds timeout,
const std::optional<std::string>& continuation_handle,
const std::optional<std::string>& idempotency_key) {
json call_arguments = arguments;
if (continuation_handle.has_value()) {
// Convention only: confirm the actual field name a given
// server expects for continuation state, since the protocol
// mandates that such state travel as an ordinary argument
// without mandating its field name.
call_arguments["_continuation"] = *continuation_handle;
}
if (idempotency_key.has_value()) {
call_arguments["_idempotency_key"] = *idempotency_key;
}
json params;
params["name"] = tool_name;
params["arguments"] = call_arguments;
const json result = post_jsonrpc("tools/call", tool_name, params, timeout);
if (!result.contains("resultType")) {
throw std::runtime_error(
"MCP server response for '" + tool_name +
"' is missing the resultType field required since 2026-07-28");
}
ToolResult tool_result;
tool_result.name = tool_name;
tool_result.is_error = result.value("isError", false);
tool_result.content = result.value("content", json::object());
return tool_result;
}
} // namespace agent
Chapter Ten — Memory and Budget: The Conversation's Keeper
A model has a finite context window, and a conversation that runs long enough will eventually threaten to overflow it. But which parts of a conversation are safe to compress and which parts must never be touched are two very different questions — and conflating them is a subtle mistake that only shows up much later, when an agent that used to behave safely starts making decisions that no longer reflect the instructions it was actually given. `AgentState` is the plain, deliberately unintelligent record of everything that has happened. It holds no policy of its own; it simply remembers. And every method on it builds a `ChatMessage` through named field assignment rather than a positional list, precisely so that a future field addition can never silently reorder an existing one the way an earlier, more fragile version of this class once risked.
// file: include/agent/agent_state.hpp
#pragma once
#include <vector>
#include "agent/types.hpp"
namespace agent {
class AgentState {
public:
// Seeded once at the start of a conversation (see Chapter
// Fourteen's main.cpp) and preserved verbatim by ContextManager
// regardless of how much the rest of the history is compacted;
// this is what carries the harness's own instructions and its
// tool-review policy to the model on every single turn, even a
// very late one.
void append_system_message(const std::string& text) {
ChatMessage message;
message.role = Role::System;
message.content = text;
history_.push_back(std::move(message));
}
void append_user_message(const std::string& text) {
ChatMessage message;
message.role = Role::User;
message.content = text;
history_.push_back(std::move(message));
}
void append_assistant_final(const std::string& text) {
ChatMessage message;
message.role = Role::Assistant;
message.content = text;
history_.push_back(std::move(message));
}
// Records the assistant's own request to call tools. This message
// must be appended BEFORE the corresponding tool results, because
// every chat-oriented model format requires the "why are these
// tool results here" message to precede the results themselves.
void append_assistant_tool_calls(const std::vector<ToolCall>& calls) {
ChatMessage message;
message.role = Role::Assistant;
message.tool_calls = calls;
history_.push_back(std::move(message));
}
// Every field here is set by name rather than by aggregate
// initializer position. An earlier version of this class built
// ChatMessage with a five-argument positional initializer, which
// meant adding the is_error field below would have silently
// shifted every other field into the wrong slot at every existing
// call site; naming each field removes that entire class of bug.
void append_tool_result(const ToolResult& result) {
ChatMessage message;
message.role = Role::Tool;
message.content = result.content.dump();
message.tool_call_id = result.tool_call_id;
message.tool_name = result.name;
message.is_error = result.is_error;
history_.push_back(std::move(message));
}
void append_tool_denied(const ToolCall& call, const std::string& reason) {
ChatMessage message;
message.role = Role::Tool;
message.content = json{{"error", reason}}.dump();
message.tool_call_id = call.id;
message.tool_name = call.name;
message.is_error = true;
history_.push_back(std::move(message));
}
const std::vector<ChatMessage>& history() const { return history_; }
// Not consumed by anything in this reference implementation, but
// kept available for a cross-call, session-level quota that a
// real deployment would enforce alongside the per-call max_turns
// ceiling checked by Orchestrator::run.
int turn_count() const { return turn_count_; }
void increment_turn() { ++turn_count_; }
private:
std::vector<ChatMessage> history_;
int turn_count_ = 0;
};
} // namespace agent
`ContextManager` is where the policy of what to keep actually lives, and it's the piece of this platform that most directly puts C++26's saturating arithmetic to work. Token counting here is a deliberately rough estimate — roughly one token for every four characters — since an accurate count depends on the specific tokenizer of whichever model happens to be loaded. But the running total accumulated across a long history is exactly the kind of value a bug elsewhere in the system could push past its representable range, and `std::add_sat` clamps at that range's maximum instead of silently wrapping around to a small or even negative number that would quietly defeat the very budget check it exists to enforce. The more important design decision in this file, though, has nothing to do with arithmetic. System messages — the harness's own standing instructions — are split out from the rest of the history before the sliding token-budget window is ever applied, and they are always kept in full. An earlier version of this class had no such split, and would have been perfectly happy to summarize away the agent's own operating instructions the moment a long conversation put enough pressure on the token budget — silently changing what the model believed it was allowed to do.
// file: include/agent/context_manager.hpp
#pragma once
#include <cstdint>
#include <functional>
#include <vector>
#include "agent/agent_state.hpp"
#include "agent/types.hpp"
namespace agent {
class ContextManager {
public:
using Summarizer = std::function<std::string(const std::vector<ChatMessage>&)>;
explicit ContextManager(Summarizer summarizer = nullptr)
: summarizer_(std::move(summarizer)) {}
std::vector<ChatMessage> bounded_view(const AgentState& state,
std::uint32_t token_budget) const;
private:
std::uint32_t estimate_tokens(const ChatMessage& message) const;
Summarizer summarizer_;
};
} // namespace agent
// file: src/context_manager.cpp
#include "agent/context_manager.hpp"
#include <algorithm>
#include <numeric>
namespace agent {
std::uint32_t ContextManager::estimate_tokens(const ChatMessage& message) const {
std::size_t char_count = message.content.size();
for (const auto& call : message.tool_calls) {
char_count += call.name.size() + call.arguments.dump().size();
}
// Roughly four characters per token is a common, deliberately
// rough estimate; it is conservative enough for budgeting even
// though it is not a substitute for the target model's own
// tokenizer.
return static_cast<std::uint32_t>(char_count / 4);
}
std::vector<ChatMessage> ContextManager::bounded_view(
const AgentState& state, std::uint32_t token_budget) const {
const auto& full_history = state.history();
// Harness-authored system messages, typically the standing system
// prompt appended once at conversation start, must never be
// silently dropped by context compaction: losing one would
// silently change the agent's instructions and safety framing
// mid-conversation. They are therefore always retained in full,
// and only the non-system portion of the history is subject to
// the sliding token-budget window below.
std::vector<ChatMessage> system_messages;
std::vector<ChatMessage> other_messages;
for (const auto& message : full_history) {
if (message.role == Role::System) {
system_messages.push_back(message);
} else {
other_messages.push_back(message);
}
}
std::uint32_t system_cost = 0;
for (const auto& message : system_messages) {
system_cost = std::add_sat(system_cost, estimate_tokens(message));
}
// std::sub_sat clamps at zero instead of wrapping to a huge
// unsigned value if the system messages alone already exceed the
// configured budget, which keeps the sliding window below
// well-defined even in that degenerate case.
const std::uint32_t remaining_budget = std::sub_sat(token_budget, system_cost);
std::vector<ChatMessage> kept;
std::uint32_t running_total = 0;
for (auto it = other_messages.rbegin(); it != other_messages.rend(); ++it) {
const std::uint32_t cost = estimate_tokens(*it);
const std::uint32_t candidate_total = std::add_sat(running_total, cost);
if (candidate_total > remaining_budget && !kept.empty()) {
break;
}
running_total = candidate_total;
kept.push_back(*it);
}
std::reverse(kept.begin(), kept.end());
const bool truncated = kept.size() < other_messages.size();
std::vector<ChatMessage> result;
result.reserve(system_messages.size() + kept.size() + (truncated ? 1 : 0));
result.insert(result.end(), system_messages.begin(), system_messages.end());
if (truncated) {
std::vector<ChatMessage> omitted(other_messages.begin(),
other_messages.end() - kept.size());
ChatMessage summary;
summary.role = Role::System;
summary.content = summarizer_
? summarizer_(omitted)
: "[" + std::to_string(omitted.size()) +
" earlier messages omitted to fit the context window]";
result.push_back(std::move(summary));
}
result.insert(result.end(), kept.begin(), kept.end());
return result;
}
} // namespace agent
Chapter Eleven — The Guardrails: Where the Harness Says No
Everything built up to this point exists in service of a single moment: the instant a model, mid-conversation, asks to run a tool, and the harness has to decide — on its own authority, independent of how persuasive the model's reasoning sounded — whether that's actually going to happen. The policy layer below is deliberately unforgiving. A tool that isn't on the configured allow list is refused, full stop. There is no argument a model can construct inside its own output that changes that outcome, because the check never even looks at what the model said — only at whether the name matches something an operator explicitly approved in advance.
// file: include/agent/policy.hpp
#pragma once
#include <algorithm>
#include <string>
#include <vector>
#include "agent/tool_gateway.hpp"
#include "agent/types.hpp"
namespace agent {
class PolicyLayer {
public:
PolicyLayer(std::vector<std::string> allow_list, ToolGateway& gateway)
: allow_list_(std::move(allow_list)), gateway_(gateway) {}
bool is_allow_listed(const std::string& tool_name) const {
return std::find(allow_list_.begin(), allow_list_.end(), tool_name) !=
allow_list_.end();
}
// A call may proceed without confirmation only if it is on the
// allow list AND the server has not flagged it as destructive.
bool allows(const ToolCall& call) const {
if (!is_allow_listed(call.name)) return false;
const ToolDescriptor* descriptor = find_descriptor(call.name);
return descriptor != nullptr && !descriptor->destructive;
}
// A call needs a human's explicit confirmation if it is on the
// allow list but IS flagged destructive.
bool requires_confirmation(const ToolCall& call) const {
if (!is_allow_listed(call.name)) return false;
const ToolDescriptor* descriptor = find_descriptor(call.name);
return descriptor != nullptr && descriptor->destructive;
}
private:
const ToolDescriptor* find_descriptor(const std::string& name) const {
for (const auto& descriptor : gateway_.tool_manifest()) {
if (descriptor.name == name) return &descriptor;
}
return nullptr;
}
std::vector<std::string> allow_list_;
ToolGateway& gateway_;
};
} // namespace agent
A call that's allow-listed but flagged destructive by the server doesn't get an automatic pass either; it gets routed to a human. The interface below is intentionally the thinnest possible seam, because the actual approval channel — a console prompt in this reference implementation, but just as easily a chat message, a ticketing system, or an approval queue in a real deployment — should be swappable without the orchestrator or the policy layer ever noticing the difference.
// file: include/agent/confirmation_service.hpp
#pragma once
#include <iostream>
#include <string>
#include "agent/types.hpp"
namespace agent {
class ConfirmationService {
public:
virtual ~ConfirmationService() = default;
virtual bool ask(const ToolCall& call) = 0;
};
class ConsoleConfirmationService final : public ConfirmationService {
public:
bool ask(const ToolCall& call) override {
std::cout << "The model wants to run '" << call.name
<< "' with arguments " << call.arguments.dump()
<< ". This tool is flagged destructive. Allow it? [y/N] ";
std::string answer;
std::getline(std::cin, answer);
return !answer.empty() && (answer[0] == 'y' || answer[0] == 'Y');
}
};
} // namespace agent
Chapter Twelve — Concurrency Without Chaos
A single turn can ask for several tool calls at once, and running them one after another when they're independent of each other wastes exactly the kind of latency users notice immediately. Running them concurrently is the obvious answer — but concurrency introduces its own failure modes, and the two facilities in this chapter exist specifically to keep those failure modes from ever reaching production. The thread pool is a small, ordinary producer-consumer implementation, kept deliberately unglamorous, with one detail that matters more than its brevity suggests: a request for zero worker threads is silently clamped to one. An earlier version of this class accepted a zero thread count without complaint — and would then deadlock forever the first time anything was submitted to it, with nothing left alive to ever drain the queue.
// file: include/agent/thread_pool.hpp
#pragma once
#include <condition_variable>
#include <functional>
#include <future>
#include <memory>
#include <mutex>
#include <queue>
#include <thread>
#include <type_traits>
#include <vector>
namespace agent {
class ThreadPool {
public:
explicit ThreadPool(std::size_t thread_count) {
// A request for zero worker threads used to be accepted
// silently and would then deadlock forever on the first
// submitted task, since nothing would ever drain the queue.
// Clamping to at least one thread turns a misconfiguration
// into a slow pool instead of a silent hang.
if (thread_count == 0) thread_count = 1;
for (std::size_t i = 0; i < thread_count; ++i) {
workers_.emplace_back([this] { worker_loop(); });
}
}
~ThreadPool() {
{
std::lock_guard<std::mutex> lock(mutex_);
stopping_ = true;
}
condition_.notify_all();
for (auto& worker : workers_) {
if (worker.joinable()) worker.join();
}
}
ThreadPool(const ThreadPool&) = delete;
ThreadPool& operator=(const ThreadPool&) = delete;
template <typename Fn>
auto submit(Fn task) -> std::future<std::invoke_result_t<Fn>> {
using ReturnType = std::invoke_result_t<Fn>;
auto packaged =
std::make_shared<std::packaged_task<ReturnType()>>(std::move(task));
std::future<ReturnType> future = packaged->get_future();
{
std::lock_guard<std::mutex> lock(mutex_);
tasks_.emplace([packaged] { (*packaged)(); });
}
condition_.notify_one();
return future;
}
private:
void worker_loop() {
for (;;) {
std::function<void()> task;
{
std::unique_lock<std::mutex> lock(mutex_);
condition_.wait(lock,
[this] { return stopping_ || !tasks_.empty(); });
if (stopping_ && tasks_.empty()) return;
task = std::move(tasks_.front());
tasks_.pop();
}
task();
}
}
std::vector<std::thread> workers_;
std::queue<std::function<void()>> tasks_;
std::mutex mutex_;
std::condition_variable condition_;
bool stopping_ = false;
};
} // namespace agent
The scheduler built on top of that pool is where the timeout story from Chapter Six pays off completely. Rather than trying to cancel a running call at the application level — something a plain `std::future` simply cannot do — the deadline is enforced exactly where it can genuinely take effect: inside `CurlHttpClient`'s own timeout, threaded all the way down from here. By the time this loop calls `get` on a future, the underlying call has either already finished or curl has already aborted it at the wire level. There is no straggler thread anywhere quietly holding a socket open after the turn has already moved on.
// file: include/agent/tool_scheduler.hpp
#pragma once
#include <chrono>
#include <vector>
#include "agent/thread_pool.hpp"
#include "agent/tool_gateway.hpp"
#include "agent/types.hpp"
namespace agent {
class ToolScheduler {
public:
virtual ~ToolScheduler() = default;
virtual std::vector<ToolResult> run(ToolGateway& gateway,
const std::vector<ToolCall>& calls,
std::chrono::milliseconds per_call_timeout) = 0;
};
class ThreadPoolToolScheduler final : public ToolScheduler {
public:
explicit ThreadPoolToolScheduler(std::size_t thread_count = 4)
: pool_(thread_count) {}
std::vector<ToolResult> run(ToolGateway& gateway,
const std::vector<ToolCall>& calls,
std::chrono::milliseconds per_call_timeout) override;
private:
ThreadPool pool_;
};
} // namespace agent
// file: src/tool_scheduler.cpp
#include "agent/tool_scheduler.hpp"
namespace agent {
std::vector<ToolResult> ThreadPoolToolScheduler::run(
ToolGateway& gateway, const std::vector<ToolCall>& calls,
std::chrono::milliseconds per_call_timeout) {
std::vector<std::future<ToolResult>> futures;
futures.reserve(calls.size());
for (const auto& call : calls) {
futures.push_back(pool_.submit([&gateway, call, per_call_timeout] {
try {
return gateway.call(call.name, call.arguments, per_call_timeout,
/*continuation_handle=*/std::nullopt,
call.idempotency_key);
} catch (const std::exception& error) {
ToolResult failure;
failure.tool_call_id = call.id;
failure.name = call.name;
failure.is_error = true;
failure.content = json{{"error", error.what()}};
return failure;
}
}));
}
std::vector<ToolResult> results;
results.reserve(futures.size());
for (std::size_t i = 0; i < futures.size(); ++i) {
ToolResult result = futures[i].get();
result.tool_call_id = calls[i].id; // the gateway does not know the id
results.push_back(std::move(result));
}
return results;
}
} // namespace agent
The `std::execution` version of this same idea, promised in Chapter Five as the forward-looking alternative, is preserved here in its own file for exactly the reason described there: it uses only vocabulary the standard genuinely defines — `schedule`, `then`, `when_all` — and it marks, honestly and explicitly, the one piece that is not standard vocabulary at all: a timed scheduler capable of producing a sender that completes after a delay, which any real project supplies from whatever execution-context library it links against.
// file: include/agent/tool_scheduler_execution_sketch.hpp
//
// NOT part of the buildable project. This sketch shows the intended
// shape of concurrent tool dispatch once std::execution
// implementations mature, using only sender/receiver vocabulary the
// standard actually defines (schedule, then, when_all). The one piece
// that is NOT standard vocabulary is timer.schedule_after(...),
// marked below: a scheduler that can produce a sender completing
// after a delay is supplied by whatever execution context library the
// project links against, not by the language standard, which
// specifies the scheduler concept abstractly rather than any concrete
// timed scheduler type. The buildable equivalent of this idea is
// ThreadPoolToolScheduler above, which enforces its deadline at the
// transport layer, inside CurlHttpClient, rather than through a
// std::execution timeout combinator.
#pragma once
#include <chrono>
#include <execution>
#include <stdexcept>
namespace agent {
template <typename Scheduler, typename TimedScheduler>
auto run_tool_with_deadline(Scheduler scheduler, TimedScheduler timer,
auto tool_sender, std::chrono::milliseconds deadline) {
auto work = std::execution::on(scheduler, std::move(tool_sender));
// timer.schedule_after is NOT standard vocabulary; it stands in
// for whichever timed-scheduling facility your execution context
// library provides.
auto timeout = std::execution::then(timer.schedule_after(deadline), [] {
throw std::runtime_error("tool call exceeded its deadline");
});
return std::execution::when_all(std::move(work), std::move(timeout));
}
} // namespace agent
Chapter Thirteen — The Heart of the Machine: The Orchestrator Loop
Every chapter so far has been building toward this one function, because this is where a collection of well-designed pieces either does or doesn't add up to a trustworthy system. The interface declares two invariants using C++26's own contract syntax, guarded behind a feature-test macro so the same guarantees hold on a compiler that doesn't yet accept that syntax — enforced there by an ordinary `assert` inside the implementation.
// file: include/agent/orchestrator.hpp
#pragma once
#include <chrono>
#include "agent/agent_state.hpp"
#include "agent/confirmation_service.hpp"
#include "agent/context_manager.hpp"
#include "agent/model_provider.hpp"
#include "agent/policy.hpp"
#include "agent/tool_gateway.hpp"
#include "agent/tool_scheduler.hpp"
#include "agent/types.hpp"
// AGENT_HAVE_CPP26_CONTRACTS is left undefined by default because, at
// the time of writing, contract support is still settling across
// compilers. Define it on the compiler command line only once your
// toolchain's documentation confirms it accepts the pre/post syntax
// used below; until then, the plain assert() calls inside run()
// enforce the same invariants at runtime regardless.
#if defined(AGENT_HAVE_CPP26_CONTRACTS)
#define AGENT_PRE(cond) pre(cond)
#define AGENT_POST(binding, cond) post(binding: cond)
#else
#define AGENT_PRE(cond)
#define AGENT_POST(binding, cond)
#endif
namespace agent {
class Orchestrator {
public:
Orchestrator(ModelProvider& model, ToolGateway& gateway,
ContextManager& context, PolicyLayer& policy,
ConfirmationService& confirmation, ToolScheduler& scheduler,
AgentState& state, int max_turns, std::uint32_t token_budget,
std::chrono::milliseconds tool_timeout)
: model_(model),
gateway_(gateway),
context_(context),
policy_(policy),
confirmation_(confirmation),
scheduler_(scheduler),
state_(state),
max_turns_(max_turns),
token_budget_(token_budget),
tool_timeout_(tool_timeout) {}
AgentResponse run(const std::string& user_message)
AGENT_PRE(max_turns_ > 0)
AGENT_POST(response, !response.text.empty());
private:
ModelProvider& model_;
ToolGateway& gateway_;
ContextManager& context_;
PolicyLayer& policy_;
ConfirmationService& confirmation_;
ToolScheduler& scheduler_;
AgentState& state_;
int max_turns_;
std::uint32_t token_budget_;
std::chrono::milliseconds tool_timeout_;
};
} // namespace agent
Read the implementation below as a story rather than a listing — that's genuinely the easiest way to follow it. A correlation identifier is minted the moment a turn begins, and it will tag every single event this turn produces. The user's message joins the conversation. Then, in a loop bounded by the turn ceiling the contract above already guards, the context manager hands over a properly bounded view of the history, the model is asked what it wants to do, and one of exactly two things happens. Either it gives a final answer, in which case the loop records it and returns. Or it asks for tools — in which case its request is recorded first, exactly as it must be, before a single one of those tools is allowed to run. Every requested call is then judged individually: allow-listed and safe; allow-listed but destructive and now pending a human's word; or refused outright because it is neither. Only calls that survive that judgment receive a freshly minted idempotency key and are handed to the scheduler. And every result, successful or not, is written back into the conversation before the loop turns again.
// file: src/orchestrator.cpp
#include "agent/orchestrator.hpp"
#include <cassert>
#include "agent/logging.hpp"
#include "agent/random_id.hpp"
namespace agent {
AgentResponse Orchestrator::run(const std::string& user_message) {
assert(max_turns_ > 0 && "orchestrator misconfigured with max_turns <= 0");
const std::string correlation_id = generate_random_id();
log_event(correlation_id, "turn_loop_start", {{"max_turns", max_turns_}});
state_.append_user_message(user_message);
for (int turn = 0; turn < max_turns_; ++turn) {
const std::vector<ChatMessage> bounded_history =
context_.bounded_view(state_, token_budget_);
ModelRequest request;
request.messages = bounded_history;
request.tools = gateway_.tool_manifest();
log_event(correlation_id, "model_call",
{{"turn", turn}, {"history_size", bounded_history.size()}});
const ModelReply reply = model_.generate(request);
if (reply.tool_calls.empty()) {
state_.append_assistant_final(reply.text);
state_.increment_turn();
log_event(correlation_id, "final_answer", {{"turn", turn}});
AgentResponse response{reply.text};
assert(!response.text.empty());
return response;
}
// Record the assistant's own request BEFORE the results, so
// the next call to generate() sees a history that correctly
// explains why tool results are about to appear.
state_.append_assistant_tool_calls(reply.tool_calls);
log_event(correlation_id, "tool_calls_requested",
{{"count", reply.tool_calls.size()}});
std::vector<ToolCall> approved;
for (const auto& call : reply.tool_calls) {
if (policy_.allows(call)) {
ToolCall approved_call = call;
approved_call.idempotency_key = generate_random_id();
log_event(correlation_id, "tool_call_approved", {{"name", call.name}});
approved.push_back(std::move(approved_call));
} else if (policy_.requires_confirmation(call)) {
if (confirmation_.ask(call)) {
ToolCall approved_call = call;
approved_call.idempotency_key = generate_random_id();
log_event(correlation_id, "tool_call_approved_after_confirmation",
{{"name", call.name}});
approved.push_back(std::move(approved_call));
} else {
state_.append_tool_denied(call, "denied by human reviewer");
log_event(correlation_id, "tool_call_denied",
{{"name", call.name}, {"reason", "human reviewer declined"}});
}
} else {
state_.append_tool_denied(
call,
"tool call denied by policy: not on the allow list or "
"not present in the server's current tool manifest");
log_event(correlation_id, "tool_call_denied",
{{"name", call.name}, {"reason", "not allow-listed or unknown"}});
}
}
if (!approved.empty()) {
const std::vector<ToolResult> results =
scheduler_.run(gateway_, approved, tool_timeout_);
for (const auto& result : results) {
state_.append_tool_result(result);
log_event(correlation_id, "tool_result",
{{"name", result.name}, {"is_error", result.is_error}});
}
}
state_.increment_turn();
}
log_event(correlation_id, "turn_limit_reached", {{"max_turns", max_turns_}});
AgentResponse response{"Turn limit reached without a final answer."};
state_.append_assistant_final(response.text);
assert(!response.text.empty());
return response;
}
} // namespace agent
Chapter Fourteen — Wiring It All Together
Every layer this article has built exists in isolation until something assembles it into a running program, and `main.cpp` is that assembly point — chosen deliberately to remain thin. It selects a model backend from an environment variable, connects to the configured MCP server, loads the policy file that decides which tools are even eligible for consideration, seeds the system prompt that will travel with the conversation from its very first turn to its last, and then runs a simple interactive loop. Two details here matter more than the file's modest length suggests. Discovery against the MCP server happens eagerly, before the program will accept a single user message, and any failure there is caught and reported with a specific, actionable diagnostic rather than allowed to propagate out of `main` as an unhandled exception — because the single most likely moment for something to be misconfigured is exactly the first time someone follows the installation steps in Chapter Sixteen. And the policy file loader never throws: a missing or malformed file produces a clear warning and an empty allow list, which, in a system designed to say no by default, is exactly the safe direction to fail toward.
// file: src/main.cpp
#include <cstdlib>
#include <fstream>
#include <iostream>
#include <memory>
#include <string>
#include "agent/agent_state.hpp"
#include "agent/confirmation_service.hpp"
#include "agent/context_manager.hpp"
#include "agent/curl_http_client.hpp"
#include "agent/local_llama_provider.hpp"
#include "agent/mcp_tool_gateway.hpp"
#include "agent/orchestrator.hpp"
#include "agent/policy.hpp"
#include "agent/remote_anthropic_provider.hpp"
#include "agent/tool_scheduler.hpp"
using namespace agent;
namespace {
std::string env_or(const char* name, const std::string& fallback) {
const char* value = std::getenv(name);
return value != nullptr ? std::string(value) : fallback;
}
// Never throws: a missing or malformed policy file is reported as a
// clear warning and treated as an empty allow list, which is the
// safe direction to fail in a deny-by-default design, rather than
// crashing the whole program over a configuration typo.
std::vector<std::string> load_allow_list(const std::string& path) {
std::ifstream file(path);
if (!file) {
std::cerr << "warning: could not open policy file '" << path
<< "', starting with an empty allow list\n";
return {};
}
try {
json data;
file >> data;
return data.at("allowed_tools").get<std::vector<std::string>>();
} catch (const std::exception& error) {
std::cerr << "warning: could not parse policy file '" << path
<< "': " << error.what() << "; starting with an empty "
<< "allow list\n";
return {};
}
}
} // namespace
int main() {
CurlHttpClient http;
const std::string backend = env_or("AGENT_BACKEND", "local");
std::unique_ptr<ModelProvider> model;
if (backend == "remote") {
const std::string api_key = env_or("ANTHROPIC_API_KEY", "");
const std::string model_name =
env_or("ANTHROPIC_MODEL", "claude-sonnet-4-6");
if (api_key.empty()) {
std::cerr << "error: ANTHROPIC_API_KEY is not set\n";
return 1;
}
model = std::make_unique<RemoteAnthropicProvider>(http, api_key, model_name);
} else {
const std::string endpoint =
env_or("LLAMA_SERVER_URL", "http://127.0.0.1:8080");
model = std::make_unique<LocalLlamaProvider>(http, endpoint);
}
const std::string mcp_server_url =
env_or("MCP_SERVER_URL", "http://127.0.0.1:9000");
McpToolGateway gateway(http, mcp_server_url);
// Discovery happens once, eagerly, before the harness will accept
// a single user message: if the tool server cannot be reached at
// all, failing loudly right now with a clear diagnostic is far
// more useful than starting a harness that only discovers the
// outage on the user's first tool-using request.
try {
gateway.discover();
gateway.list_tools();
} catch (const std::exception& error) {
std::cerr << "error: could not reach the MCP server at "
<< mcp_server_url << ": " << error.what() << '\n';
std::cerr << "make sure a server implementing the MCP 2026-07-28 "
"revision is running and reachable at that address.\n";
return 1;
}
const std::vector<std::string> allow_list =
load_allow_list(env_or("AGENT_POLICY_FILE", "config/tool_policy.json"));
ContextManager context;
PolicyLayer policy(allow_list, gateway);
ConsoleConfirmationService confirmation;
ThreadPoolToolScheduler scheduler(4);
AgentState state;
// Seeded once, at the very start of the conversation, and then
// preserved verbatim by ContextManager for as long as the
// conversation runs: this is what keeps the model aware of the
// review-and-confirmation policy on every turn, even after
// earlier parts of a long conversation have been summarized away
// to fit the token budget.
state.append_system_message(
"You are an autonomous assistant. You may request tools from the "
"list provided to you, but every tool call is subject to review "
"by a policy layer before it runs, and destructive tools "
"additionally require explicit human confirmation. Explain your "
"reasoning briefly before requesting a tool, and give a direct "
"final answer once you have everything you need.");
Orchestrator orchestrator(*model, gateway, context, policy, confirmation,
scheduler, state, /*max_turns=*/8,
/*token_budget=*/4000,
std::chrono::seconds(30));
std::cout << "Agentic harness ready (backend: " << backend
<< "). Type a message and press enter.\n";
std::string line;
while (std::getline(std::cin, line)) {
if (line.empty()) continue;
try {
const AgentResponse response = orchestrator.run(line);
std::cout << response.text << '\n';
} catch (const std::exception& error) {
std::cerr << "error: " << error.what() << '\n';
} catch (...) {
std::cerr << "error: an unrecognized exception type was thrown "
"while processing that message\n";
}
}
return 0;
}
The policy file this program reads at startup is deliberately narrow in scope. It holds exactly one decision — which tools are eligible to be considered at all — and nothing about which of those additionally require human confirmation, because that second decision comes from the server's own `destructiveHint` annotation at runtime. Duplicating it here would only create two sources of truth that could quietly disagree with each other.
JSON:
{
"allowed_tools": [
"search_docs",
"read_file",
"send_email"
]
}
The build description ties every source file above into two targets — a shared core library and the executable that links against it — plus a test binary that links against the exact same library rather than a separate copy of the logic.
CMAKE
# file: CMakeLists.txt
#
# Top level build description for the agentic harness. It requires a
# C++26 capable compiler, libcurl for outbound HTTPS, and the
# header-only nlohmann/json library for all JSON handling. Both third
# party dependencies are located with find_package, so they must be
# installed on the build machine before configuring (see Chapter
# Sixteen for the installation commands).
cmake_minimum_required(VERSION 3.28)
project(agentic_harness LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 26)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
find_package(CURL REQUIRED)
find_package(nlohmann_json REQUIRED)
# The core library holds every piece of orchestration logic and has no
# notion of "main". Keeping it separate from the executable is what
# lets the smoke test link against the exact same code the real
# program runs, rather than against a copy of it.
add_library(agent_core
src/curl_http_client.cpp
src/local_llama_provider.cpp
src/remote_anthropic_provider.cpp
src/mcp_tool_gateway.cpp
src/context_manager.cpp
src/tool_scheduler.cpp
src/orchestrator.cpp
)
target_include_directories(agent_core PUBLIC include)
target_link_libraries(agent_core PUBLIC CURL::libcurl nlohmann_json::nlohmann_json)
target_compile_features(agent_core PUBLIC cxx_std_26)
add_executable(agentic_harness src/main.cpp)
target_link_libraries(agentic_harness PRIVATE agent_core)
enable_testing()
add_executable(smoke_test tests/smoke_test.cpp)
target_link_libraries(smoke_test PRIVATE agent_core)
add_test(NAME smoke_test COMMAND smoke_test)
Chapter Fifteen — Proving It Works Before Trusting It
The layered architecture from Chapter Four pays its final dividend here: every collaborator the orchestrator depends on is an interface, which means the entire control loop — tool approval, tool denial, dispatch, and final-answer generation — can be exercised end to end without a real model, a real network, or a real MCP server anywhere in the picture. The fake model below scripts exactly the sequence a real conversation takes, asking for a tool on its first call and giving a final answer on its second, and the fake gateway simply echoes back whatever arguments it was given. That's more than enough to prove that the whole pipeline holds together: from a tool request, through policy approval, through concurrent dispatch, to the tool result finding its way back into the model's next turn.
// file: tests/smoke_test.cpp
#include <iostream>
#include "agent/agent_state.hpp"
#include "agent/confirmation_service.hpp"
#include "agent/context_manager.hpp"
#include "agent/orchestrator.hpp"
#include "agent/policy.hpp"
#include "agent/tool_scheduler.hpp"
using namespace agent;
namespace {
class FakeModelProvider final : public ModelProvider {
public:
ModelReply generate(const ModelRequest&) override {
if (call_count_++ == 0) {
ModelReply reply;
ToolCall call{"call-1", "echo", json{{"text", "hello"}}, std::nullopt};
reply.tool_calls.push_back(call);
return reply;
}
ModelReply final_reply;
final_reply.text = "hello";
return final_reply;
}
private:
int call_count_ = 0;
};
class FakeToolGateway final : public ToolGateway {
public:
void discover() override {}
std::vector<ToolDescriptor> list_tools() override {
manifest_ = {ToolDescriptor{"echo", "echoes text back", json::object(), false}};
return manifest_;
}
const std::vector<ToolDescriptor>& tool_manifest() const override {
return manifest_;
}
// Signature must match ToolGateway::call exactly for this class
// to be non-abstract; the two trailing parameters were added when
// idempotency keys were introduced, and this override was updated
// to match at the same time so the smoke test keeps compiling.
ToolResult call(const std::string& name, const json& arguments,
std::chrono::milliseconds, const std::optional<std::string>&,
const std::optional<std::string>&) override {
ToolResult result;
result.name = name;
result.content = arguments;
return result;
}
private:
std::vector<ToolDescriptor> manifest_;
};
class AutoApproveConfirmationService final : public ConfirmationService {
public:
bool ask(const ToolCall&) override { return true; }
};
} // namespace
int main() {
FakeModelProvider model;
FakeToolGateway gateway;
gateway.list_tools();
ContextManager context;
PolicyLayer policy({"echo"}, gateway);
AutoApproveConfirmationService confirmation;
ThreadPoolToolScheduler scheduler(2);
AgentState state;
Orchestrator orchestrator(model, gateway, context, policy, confirmation,
scheduler, state, /*max_turns=*/4,
/*token_budget=*/4000, std::chrono::seconds(5));
const AgentResponse response = orchestrator.run("please echo hello");
if (response.text != "hello") {
std::cerr << "smoke test failed: expected 'hello', got '"
<< response.text << "'\n";
return 1;
}
std::cout << "smoke test passed\n";
return 0;
}
Chapter Sixteen — From Zero to a Running Platform
Everything described across the previous fifteen chapters becomes real with a short, concrete sequence of commands, and it's worth walking through them in the order a first-time reader would actually need them: install the ordinary dependencies, obtain a C++26-capable compiler, stand up a local model server, build the harness itself, run its automated test — and only then run the program for real, against a chosen backend and a chosen MCP server.
bash
# Widely available dependencies, installed the ordinary way.
sudo apt-get update
sudo apt-get install -y build-essential cmake ninja-build git \
libcurl4-openssl-dev nlohmann-json3-dev
# A C++26-capable compiler is the one piece worth building yourself
# rather than trusting the distribution's package; consult your
# chosen compiler's own release notes for the exact configure flags
# and for which C++26 features its current snapshot actually
# implements, since the standard itself was only finalized at the
# March 2026 plenary meeting and support is still settling.
# Fetch, build, and run llama.cpp's OpenAI-compatible server with a
# local GGUF model file of your choosing, listening on localhost.
git clone https://github.com/ggml-org/llama.cpp
cmake -S llama.cpp -B llama.cpp/build -DLLAMA_CURL=OFF
cmake --build llama.cpp/build --config Release -j
./llama.cpp/build/bin/llama-server -m /path/to/your-model.gguf \
--host 127.0.0.1 --port 8080
# In a separate terminal, configure and build the harness itself.
cmake -S . -B build -G Ninja
cmake --build build
# Run the automated smoke test before running the program for real.
ctest --test-dir build --output-on-failure
# Point the harness at your MCP tool server (one implementing the
# 2026-07-28 revision) and choose a model backend.
export MCP_SERVER_URL="http://127.0.0.1:9000"
export AGENT_BACKEND="local" # or "remote"
export LLAMA_SERVER_URL="http://127.0.0.1:8080"
# export ANTHROPIC_API_KEY="..." # only needed for "remote"
# export ANTHROPIC_MODEL="..." # only needed for "remote"
./build/agentic_harness
Switching backends afterward requires no rebuild and no code change whatsoever — only a different value in `AGENT_BACKEND`. That is the entire architectural point made concrete: set it to `remote`, supply an API key, and the same orchestrator, the same policy layer, the same scheduler, and the same conversation state now drive a hosted model instead of a local one. Every one of those pieces was written against `ModelProvider`, never against llama.cpp or Anthropic specifically.
Chapter Seventeen — What This Platform Teaches, and Where It Goes Next
Step back from the individual files and a shape emerges that's worth naming explicitly, because it's the real lesson this platform has to offer beyond its specific choice of protocol and language. Every hard problem an agentic system faces — a model that occasionally hallucinates its own tool arguments, a network that occasionally fails, a conversation that occasionally outgrows its budget, a request that occasionally deserves a human's veto — turns out to have a completely ordinary, well-understood answer in the systems-programming tradition: defensive parsing, retries with backoff, circuit breakers, sliding windows with protected invariants, and deny-by-default authorization. None of these ideas is new or exotic. What's new is building them in a language whose next standard, C++26, offers reflection to keep a schema honest, contracts to keep an invariant visible at a function's own boundary, and a structured concurrency model built for exactly the kind of fan-out this platform's tool scheduler performs. That alignment between an old discipline and a new language generation is what makes this combination worth taking seriously rather than treating as a curiosity.
Three extensions sit naturally on top of everything built here, and naming them honestly as open — rather than folding them in as though they were already solved — is itself part of the discipline this article has tried to model throughout. A persistent, long-term memory store, a `MemoryStore` interface sitting alongside `ModelProvider` and `ToolGateway` in the same layered architecture, would let the harness recall facts across separate conversations rather than only within one. Deterministic transcript replay — recording every model reply and tool result a real conversation produced, then feeding them back through fakes structured exactly like the smoke test's — would turn today's production incidents into tomorrow's regression tests. And chaos testing, deliberately injecting slow responses and dropped connections into the fake `HttpClient` a test harness controls, would validate that the retry, circuit-breaker, and timeout logic of Chapters Seven and Twelve behave exactly as described under conditions considerably less forgiving than a quiet afternoon in a development environment. None of these requires touching a single interface already defined in this article. They require only new implementations behind interfaces that were built, from the very first chapter, to expect exactly this kind of extension.
No comments:
Post a Comment