P004 · Technical series · Building AI Agent Experiences, part 3 of 3 · 14 September 2026 · 7 min read
The Runtime Beneath the Experience
Monitoring, drift, and safety for AI agents in production
Resolution, outcome contracts, drift, and the consequence machinery that keeps a governed experience honest after it ships
Part 3 of a three-part technical series on experience orchestration. Part 1: One Intent, Four Contracts. Part 2: Designing Agent Conduct.
Specifications describe the experience you intended. A runtime tells you which one you shipped.
The first two parts of this series covered the contracts: an intent that resolves to agent conduct, an interface pattern, an outcome contract, and governance rules, plus the design guidelines for the agent's conduct. None of that survives contact with production on its own. Interfaces drift. Agent behavior drifts every time a prompt or a model changes. Outcomes quietly breach their targets. A contract nobody measures is a suggestion.
This article covers the runtime in two generations. The first is the runtime the original protocol specified in February 2026: resolution, outcome telemetry, drift detection, and repair. It was more fully designed than enforced, and I'll say where that matters. The second is a later OX framework I built for operations with serious consequences — physical and financial — implemented as small, strict state machines: a consequence score, a commitment ceremony, and a recovery mode.
The loop
The runtime is a loop, not a pipeline. Every stage feeds the next, and the last stage feeds the first.
- Apply writes the resolved bindings into a project —
mother applyproduces per-intent binding files and can inject the runtime into an app's entry point. - Resolve turns an intent and its context into the four contracts, in-process or over HTTP.
- Execute is where the interface renders or the agent behaves, according to the bindings.
- Measure emits telemetry against the OX contract.
- Detect drift turns breaches and mismatches into drift events.
- Repair proposes a fix, applies it automatically when it's marked safe, and hands the rest to a person.
Three ways to run it
The same runtime ran in three shapes, depending on how an organization was built.
- Library. Import it and resolve in-process. Lowest latency, one deployment.
- Sidecar. Run it as a service next to the application and call it over HTTP. The right choice for polyglot or microservice estates.
- Platform. Make it the orchestration backbone; every intent flows through it and downstream systems consume bindings.
The sidecar surface was deliberately small:
POST /resolve intent → AX + HX + OX + governance
POST /generative/:intentId intent → strict schema for a model
POST /apply write bindings into a project
GET /intents registered intent IDs
GET /drift?since=<ms> drift events
GET /governance/:intentId rules for an intent
GET /health /ready /metrics/ready was specified to report healthy only once intents were loaded. A runtime that answers before it knows what intents exist will happily answer wrong.
Outcome contracts at runtime
The OX contract declares what the person must experience. At runtime, those declarations become checks.
// When an execution completes, measure it against its contract.
const result = validateLatency(intentId, contract, actualMs);
emitTelemetry({ intentId, latencyMs: actualMs, trustSignalShown, channel });
onBreach((breach) => {
// A breached outcome is drift, not just an alert.
emitDrift("ox_threshold_breach", breach, intentId, {
repair: "update_ox_threshold",
});
});The key design choice is the last one. An outcome breach isn't routed to a dashboard and forgotten; it becomes a drift event with a proposed repair attached. That repair might be to fix the implementation, or it might be to admit the contract was unrealistic and adjust the threshold — but either way, it's a decision someone has to make.
Four kinds of drift
Drift is the gap between the contract and reality. The runtime recognized four kinds, and each needed a different repair.
| Drift type | What it looks like | Typical repair |
|---|---|---|
ui_pattern | The interface no longer matches its bound pattern or tokens | Re-apply the binding |
agent_behavior | The agent clarifies, confirms, or escalates differently from its binding | Re-bind the agent to its contract |
intent_mismatch | The action taken doesn't match the intent that was resolved | Re-tag the intent, or block the action |
ox_threshold_breach | Latency, trust, or transparency fell below the contract | Fix the implementation or update the threshold |
Of the four, intent mismatch is the one I'd pay most attention to today. Pattern drift makes an experience inconsistent. Behavior drift makes it unpredictable. Intent mismatch means the system did something other than what was asked — which, for an agent with real authority, is the definition of an incident.
Every drift event was designed to produce the same four options: a suggestion, a proposed repair, the option to re-tag the intent, and the option to re-apply. Repairs flagged as safe could be applied automatically — mother repair --auto — and everything else waited for a person. The adoption plan had mother drift starting in CI as informational and graduating to a gate that fails the build on unhandled drift.
That was the plan more than the practice. Detection in this version was coarse, and the taxonomy was sharper than the checks behind it. The concrete version of drift detection turned out to be simpler and stricter: a deterministic check against a closed registry, run on every change, where anything undeclared fails.
Consequence as a number
The original runtime treated every intent's outcome contract as fixed. The later OX framework, built for operations where mistakes have physical and financial consequences, added a question the first one never asked: how much is at stake in this specific execution?
The answer is the Consequence Presence Score — a number from 0 to 1 that drives how intensely the interface and the process respond.
const TIER_BASE = { T0: 0.0, T1: 0.15, T2: 0.35, T3: 0.6, T4: 0.85 };
// cps = tierBase + (1 - tierBase) × weightedFactors
const weightedFactors =
compositeNorm * 0.35 + // composite risk score, normalized
irreversibilityFactor * 0.25 + // 1 if the operation can't be undone
physicalFactor * 0.2 + // 1 if it affects the physical world
costFactor * 0.2; // log-scaled estimated cost, capped at $1M
const score = tierBase + (1 - tierBase) * weightedFactors;Two details make it work. First, the tier sets a floor: a higher consequence tier always produces a higher score, and the operation's specific characteristics only move it within the remaining range. A trivial-looking T4 operation still starts at 0.85. Second, cost is scaled logarithmically, so the difference between $100 and $1,000 matters as much as the difference between $100,000 and $1,000,000.
The score maps to four treatments:
- 0 to 0.3 — minimal: standard interface.
- 0.3 to 0.6 — moderate: additional confirmation.
- 0.6 to 0.85 — elevated: explicit warnings and a required review.
- Above 0.85 — maximum: a commitment ceremony.
This is the runtime answer to the permission-model question from Part 2. Instead of fixing confirmation per intent, the level of scrutiny is computed per execution, from what the execution will actually do.
The commitment ceremony
Above 0.85, a confirmation dialog isn't enough. The OX framework required a commitment ceremony: five gates, in order, that can't be skipped or reordered.
- 01Review — the person reviews the simulation results and the governance report.
- 02Acknowledge — the person explicitly acknowledges the consequences.
- 03Verify — the system verifies that every prerequisite is met.
- 04Authorize — the required approvers sign off.
- 05Commit — the final, irreversible action.
The implementation is a small, strict state machine. A ceremony can't be created for an operation that doesn't need one, a gate can only be completed when it's the active gate, and every state change returns a new object rather than mutating the old one — which makes the whole history reconstructable.
if (activeGate.gate !== gateName) {
throw new Error(
`Gate "${gateName}" is not the current active gate. Expected "${activeGate.gate}".`,
);
}Abandoning a ceremony is itself irreversible: the operation has to be started again from the beginning. That's intentional. If someone abandons it at the authorize gate, the approvals they gathered shouldn't still be sitting there a week later, waiting to be used against a changed situation.
Recovery mode
Things still go wrong. When an operation diverges from what was simulated, fails, halts, or times out, the runtime enters recovery mode and classifies the severity before suggesting anything.
Severity starts from how far reality diverged from the plan: up to 5% is minor, up to 15% moderate, up to 50% critical, and beyond that catastrophic. Then the trigger applies a floor. A halt is always at least critical, and a failure is always at least moderate — no matter how small the measured divergence, because the fact that the system stopped is information in itself.
The suggested actions follow from both. Catastrophic severity triggers an automatic emergency halt and notifies the safety team. Critical severity requires a human operator before anything else happens. Divergence suggests recalibrating and re-simulating; a failure suggests a retry; a timeout suggests extending the window; a halt requires inspecting the physical state. Every recovery, at every severity, offers a review of the full operation log — and every recovery keeps a timeline, and ends explicitly as resolved or escalated.
The OX envelope
The final piece connects the runtime to the person. Every response involving a consequential operation is wrapped in an OX envelope that carries its own consequence context.
interface OXEnvelope<T> {
data: T;
ox: {
cps: number;
tier: string;
whatIsHappening: string;
whatHappensNext: string;
whatCouldGoWrong: string[];
howToRecover: string;
badges: { type: "simulation" | "physical" | "hybrid"; label: string }[];
};
}The four fields answer the four questions that The Observable Experience argues every system acting on intent owes the human: what is happening right now, what will happen next, what could go wrong, and how do I recover. Putting them in the response, rather than leaving the interface to reconstruct them, means no client — a screen, a Slack message, a voice response — has to guess the stakes. The badges make one more thing unambiguous: whether the person is looking at a simulation, a physical operation, or both.
What I'd measure
The protocol defined a small set of runtime metrics. It defined them better than it collected them, but I'd keep all of them:
- The share of intents with complete AX and OX bindings.
- Drift detection latency — how long a divergence goes unnoticed.
- The auto-repair rate for drift.
- The generative compliance rate — how often model output stayed inside its bindings.
- Time to apply the protocol to a new repository.
Two things I planned and didn't finish, and would build first today: a feedback loop that turns patterns of outcome breaches into proposed contract changes, and an evaluation hook that checks whether a new intent's resolution is actually correct before it ships. Detection without learning is maintenance. Detection that improves the contracts is a system.
A contract that isn't measured at runtime is a suggestion. The runtime is what makes it a promise.