Matthew McBride← Perspectives

P004 · Technical series · Building AI Agent Experiences, part 3 of 3 · 14 September 2026 · 7 min read

The Runtime Beneath the Experience

Monitoring, drift, and safety for AI agents in production

Resolution, outcome contracts, drift, and the consequence machinery that keeps a governed experience honest after it ships

Applymother applyResolveresolveIntent()ExecuteUI / agentMeasureemitTelemetry()Detect driftonBreach()Repairrepair --autoThe contractstays trueOR THE RUNTIME SAYS SO

Part 3 of a three-part technical series on experience orchestration. Part 1: One Intent, Four Contracts. Part 2: Designing Agent Conduct.

Specifications describe the experience you intended. A runtime tells you which one you shipped.

The first two parts of this series covered the contracts: an intent that resolves to agent conduct, an interface pattern, an outcome contract, and governance rules, plus the design guidelines for the agent's conduct. None of that survives contact with production on its own. Interfaces drift. Agent behavior drifts every time a prompt or a model changes. Outcomes quietly breach their targets. A contract nobody measures is a suggestion.

This article covers the runtime in two generations. The first is the runtime the original protocol specified in February 2026: resolution, outcome telemetry, drift detection, and repair. It was more fully designed than enforced, and I'll say where that matters. The second is a later OX framework I built for operations with serious consequences — physical and financial — implemented as small, strict state machines: a consequence score, a commitment ceremony, and a recovery mode.

The loop

The runtime is a loop, not a pipeline. Every stage feeds the next, and the last stage feeds the first.

  • Apply writes the resolved bindings into a project — mother apply produces per-intent binding files and can inject the runtime into an app's entry point.
  • Resolve turns an intent and its context into the four contracts, in-process or over HTTP.
  • Execute is where the interface renders or the agent behaves, according to the bindings.
  • Measure emits telemetry against the OX contract.
  • Detect drift turns breaches and mismatches into drift events.
  • Repair proposes a fix, applies it automatically when it's marked safe, and hands the rest to a person.

Three ways to run it

The same runtime ran in three shapes, depending on how an organization was built.

  • Library. Import it and resolve in-process. Lowest latency, one deployment.
  • Sidecar. Run it as a service next to the application and call it over HTTP. The right choice for polyglot or microservice estates.
  • Platform. Make it the orchestration backbone; every intent flows through it and downstream systems consume bindings.

The sidecar surface was deliberately small:

text
POST /resolve                 intent → AX + HX + OX + governance
POST /generative/:intentId    intent → strict schema for a model
POST /apply                   write bindings into a project
GET  /intents                 registered intent IDs
GET  /drift?since=<ms>        drift events
GET  /governance/:intentId    rules for an intent
GET  /health  /ready  /metrics

/ready was specified to report healthy only once intents were loaded. A runtime that answers before it knows what intents exist will happily answer wrong.

Outcome contracts at runtime

The OX contract declares what the person must experience. At runtime, those declarations become checks.

ts
// When an execution completes, measure it against its contract.
const result = validateLatency(intentId, contract, actualMs);

emitTelemetry({ intentId, latencyMs: actualMs, trustSignalShown, channel });

onBreach((breach) => {
  // A breached outcome is drift, not just an alert.
  emitDrift("ox_threshold_breach", breach, intentId, {
    repair: "update_ox_threshold",
  });
});

The key design choice is the last one. An outcome breach isn't routed to a dashboard and forgotten; it becomes a drift event with a proposed repair attached. That repair might be to fix the implementation, or it might be to admit the contract was unrealistic and adjust the threshold — but either way, it's a decision someone has to make.

Four kinds of drift

Drift is the gap between the contract and reality. The runtime recognized four kinds, and each needed a different repair.

ui_patternPattern driftThe interface no longermatches its boundpattern or tokens.PROPOSED REPAIRre_applyagent_behaviorBehavior driftThe agent clarifies orescalates differentlythan its binding says.PROPOSED REPAIRre_bindintent_mismatchIntent mismatchThe action takendoesn’t match theintent that resolved.PROPOSED REPAIRRe-tag or blockox_threshold_breachOutcome breachLatency, trust, ortransparency fellbelow the contract.PROPOSED REPAIRupdate_ox_thresholdDETECT → SUGGEST → PROPOSE REPAIR → AUTO-APPLY WHEN MARKED SAFEEVERYTHING ELSE WAITS FOR A PERSON
Fig. 1Four kinds of drift, each with its own repair. Detection that doesn't propose a fix is just noise.
Drift typeWhat it looks likeTypical repair
ui_patternThe interface no longer matches its bound pattern or tokensRe-apply the binding
agent_behaviorThe agent clarifies, confirms, or escalates differently from its bindingRe-bind the agent to its contract
intent_mismatchThe action taken doesn't match the intent that was resolvedRe-tag the intent, or block the action
ox_threshold_breachLatency, trust, or transparency fell below the contractFix the implementation or update the threshold

Of the four, intent mismatch is the one I'd pay most attention to today. Pattern drift makes an experience inconsistent. Behavior drift makes it unpredictable. Intent mismatch means the system did something other than what was asked — which, for an agent with real authority, is the definition of an incident.

Every drift event was designed to produce the same four options: a suggestion, a proposed repair, the option to re-tag the intent, and the option to re-apply. Repairs flagged as safe could be applied automatically — mother repair --auto — and everything else waited for a person. The adoption plan had mother drift starting in CI as informational and graduating to a gate that fails the build on unhandled drift.

That was the plan more than the practice. Detection in this version was coarse, and the taxonomy was sharper than the checks behind it. The concrete version of drift detection turned out to be simpler and stricter: a deterministic check against a closed registry, run on every change, where anything undeclared fails.

Consequence as a number

The original runtime treated every intent's outcome contract as fixed. The later OX framework, built for operations where mistakes have physical and financial consequences, added a question the first one never asked: how much is at stake in this specific execution?

The answer is the Consequence Presence Score — a number from 0 to 1 that drives how intensely the interface and the process respond.

ts
const TIER_BASE = { T0: 0.0, T1: 0.15, T2: 0.35, T3: 0.6, T4: 0.85 };

// cps = tierBase + (1 - tierBase) × weightedFactors
const weightedFactors =
  compositeNorm         * 0.35 + // composite risk score, normalized
  irreversibilityFactor * 0.25 + // 1 if the operation can't be undone
  physicalFactor        * 0.2  + // 1 if it affects the physical world
  costFactor            * 0.2;   // log-scaled estimated cost, capped at $1M

const score = tierBase + (1 - tierBase) * weightedFactors;
cps = tierBase + (1 − tierBase) × weightedFactorsCOMPOSITE .35 · IRREVERSIBLE .25 · PHYSICAL .20 · COST .20 (LOG-SCALED, CAPPED AT $1M)MinimalStandard UIModerateAdditionalconfirmationElevatedExplicit warnings,review requiredMaximumCommitmentceremony00.30.60.851.0TIER FLOORST0 · 0.00T1 · 0.15T2 · 0.35T3 · 0.60T4 · 0.85A HIGHER TIER ALWAYS SCORES HIGHER — THE OPERATION ONLY MOVES IT WITHIN THE REMAINING RANGE
Fig. 2The consequence score. Each tier sets a floor, the operation's characteristics raise it, and the score decides the treatment.

Two details make it work. First, the tier sets a floor: a higher consequence tier always produces a higher score, and the operation's specific characteristics only move it within the remaining range. A trivial-looking T4 operation still starts at 0.85. Second, cost is scaled logarithmically, so the difference between $100 and $1,000 matters as much as the difference between $100,000 and $1,000,000.

The score maps to four treatments:

  • 0 to 0.3 — minimal: standard interface.
  • 0.3 to 0.6 — moderate: additional confirmation.
  • 0.6 to 0.85 — elevated: explicit warnings and a required review.
  • Above 0.85 — maximum: a commitment ceremony.

This is the runtime answer to the permission-model question from Part 2. Instead of fixing confirmation per intent, the level of scrutiny is computed per execution, from what the execution will actually do.

The commitment ceremony

Above 0.85, a confirmation dialog isn't enough. The OX framework required a commitment ceremony: five gates, in order, that can't be skipped or reordered.

REQUIRED WHEN CPS > 0.85 · IN ORDER · NO SKIPPINGGATE 01ReviewSimulation results,governance reportGATE 02AcknowledgeConsequencesexplicitly acceptedGATE 03VerifySystem checksevery prerequisiteGATE 04AuthorizeRequired approverssign offGATE 05CommitFinal andirreversibleEVERY COMPLETED GATE RECORDS WHO, WHEN, AND ON WHAT EVIDENCEABANDON AT ANY GATE → THE OPERATION MUST BE RE-INITIATED
Fig. 3The commitment ceremony. Five gates in strict order; each completed gate records who, when, and on what evidence.
  1. 01Review — the person reviews the simulation results and the governance report.
  2. 02Acknowledge — the person explicitly acknowledges the consequences.
  3. 03Verify — the system verifies that every prerequisite is met.
  4. 04Authorize — the required approvers sign off.
  5. 05Commit — the final, irreversible action.

The implementation is a small, strict state machine. A ceremony can't be created for an operation that doesn't need one, a gate can only be completed when it's the active gate, and every state change returns a new object rather than mutating the old one — which makes the whole history reconstructable.

ts
if (activeGate.gate !== gateName) {
  throw new Error(
    `Gate "${gateName}" is not the current active gate. Expected "${activeGate.gate}".`,
  );
}

Abandoning a ceremony is itself irreversible: the operation has to be started again from the beginning. That's intentional. If someone abandons it at the authorize gate, the approvals they gathered shouldn't still be sitting there a week later, waiting to be used against a changed situation.

Recovery mode

Things still go wrong. When an operation diverges from what was simulated, fails, halts, or times out, the runtime enters recovery mode and classifies the severity before suggesting anything.

DIVERGENCE FROM THE PLAN SETS THE LEVELMinor0–5%Moderate5–15%Critical15–50%Catastrophic> 50%THE TRIGGER SETS A FLOORfailurenever below moderatehaltnever below criticalSUGGESTED ACTIONS, BY PRIORITYemergency-haltnotify-safetyescalate-humanretry · resimulate · inspectreview-logs
Fig. 4Recovery severity. Divergence sets the level, the trigger sets a floor, and the suggested actions follow from both.

Severity starts from how far reality diverged from the plan: up to 5% is minor, up to 15% moderate, up to 50% critical, and beyond that catastrophic. Then the trigger applies a floor. A halt is always at least critical, and a failure is always at least moderate — no matter how small the measured divergence, because the fact that the system stopped is information in itself.

The suggested actions follow from both. Catastrophic severity triggers an automatic emergency halt and notifies the safety team. Critical severity requires a human operator before anything else happens. Divergence suggests recalibrating and re-simulating; a failure suggests a retry; a timeout suggests extending the window; a halt requires inspecting the physical state. Every recovery, at every severity, offers a review of the full operation log — and every recovery keeps a timeline, and ends explicitly as resolved or escalated.

The OX envelope

The final piece connects the runtime to the person. Every response involving a consequential operation is wrapped in an OX envelope that carries its own consequence context.

ts
interface OXEnvelope<T> {
  data: T;
  ox: {
    cps: number;
    tier: string;
    whatIsHappening: string;
    whatHappensNext: string;
    whatCouldGoWrong: string[];
    howToRecover: string;
    badges: { type: "simulation" | "physical" | "hybrid"; label: string }[];
  };
}
OXEnvelope<T>data: TTHE PAYLOAD, UNCHANGEDox: {cps: 0.91tier: "T4"whatIsHappening: "…"whatHappensNext: "…"whatCouldGoWrong: ["…"]howToRecover: "…"badges: [{ type: "physical" }]THE FOUR QUESTIONS EVERY RESPONSE ANSWERSWhat is happening right now?What will happen next?What could go wrong?How do I recover?THE CLIENT DOESN’T GUESS THE STAKES — THE RESPONSE CARRIES THEM
Fig. 5The OX envelope. The payload is unchanged; the response also carries the answers to the four questions every consequential action owes the person.

The four fields answer the four questions that The Observable Experience argues every system acting on intent owes the human: what is happening right now, what will happen next, what could go wrong, and how do I recover. Putting them in the response, rather than leaving the interface to reconstruct them, means no client — a screen, a Slack message, a voice response — has to guess the stakes. The badges make one more thing unambiguous: whether the person is looking at a simulation, a physical operation, or both.

What I'd measure

The protocol defined a small set of runtime metrics. It defined them better than it collected them, but I'd keep all of them:

  • The share of intents with complete AX and OX bindings.
  • Drift detection latency — how long a divergence goes unnoticed.
  • The auto-repair rate for drift.
  • The generative compliance rate — how often model output stayed inside its bindings.
  • Time to apply the protocol to a new repository.

Two things I planned and didn't finish, and would build first today: a feedback loop that turns patterns of outcome breaches into proposed contract changes, and an evaluation hook that checks whether a new intent's resolution is actually correct before it ships. Detection without learning is maintenance. Detection that improves the contracts is a system.

A contract that isn't measured at runtime is a suggestion. The runtime is what makes it a promise.