Runtime Drift Audit

At 04:00 on a Sunday, a LaunchAgent fires. It points at a wrapper script that changed directories last quarter. Nothing pages, nothing screams — the walls of the construct just shift one pixel to the left, and nobody notices until someone stubs a toe on the drift months later. The Wachowskis had a name for that hiccup: a glitch in the matrix, the sign that something changed in the simulation while you weren’t looking. Infrastructure drift is quieter than a black cat crossing your path twice.

Tommy, the haus’s dead Abyssinian, watches the whole catwalk from a place where uptime no longer applies to him. He has opinions about plists that point nowhere. So do we.

The checked-in workspace was only half the story. The live system still leaned on ~/.sanctum, generated wrappers, and a pile of LaunchAgents with varying opinions about whether their targets existed. An auditor confidently reported red on a service that was, in fact, purring. /health got mistaken for an executable path. This audit pushed into the runtime layer and cleaned up the drift that actually pages people.

Verified Runtime Surface

The runtime audit covered:

• ~/.sanctum/instance.yaml
• ~/.sanctum/services/
• ~/.sanctum/bin/* wrappers
• ~/Library/LaunchAgents/com.sanctum.*.plist
• The workspace auditors in tests/test-sanctum-runtime-audit.sh and tools/audit_runtime_launchagents.py

What Was Broken

The initial pass found two structural mismatches and several stale launch targets:

• ~/.sanctum/services did not exist even though the documentation described runtime manifests
• com.sanctum.living-force.plist referenced ~/.sanctum/living-force.sh, which was missing
• Several LaunchAgents pointed at files that no longer existed
• The legacy validator produced false positives by treating arguments like /health as executable paths

What Was Fixed

The remediation pass did seven concrete things:

1. Restored the wrapper layer under ~/.sanctum/bin
2. Created the missing ~/.sanctum/services directory
3. Added a runtime renderer so ~/.sanctum/services/*.yaml is generated from instance.yaml, the checked-in sanctum/runtime_catalog.yaml, and auxiliary launchd-wired services instead of remaining an empty placeholder
4. Added a compatibility shim at ~/.sanctum/living-force.sh
5. Re-enabled or corrected valid LaunchAgents such as com.sanctum.proxy.plist and com.sanctum.health-center.plist
6. Retired the legacy com.sanctum.living-force.plist in favor of the Rust watchdog plist so the runtime stops carrying two conflicting watchdog generations at once
7. Retired clearly obsolete LaunchAgents by renaming them to .disabled

Note

One of the more useful findings was that not every red test was real drift. Some of it was just a bad auditor shouting confidently — the infrastructure equivalent of Agent Smith insisting something is wrong while the rest of the Matrix hums along fine. Infrastructure work gets easier the moment the alarms stop lying. Ignore a real alarm once and it silently retires; trust a lying alarm twice and you start chasing ghosts through three time zones. The audit’s job is to separate the two.

Current Status

The runtime audit now passes cleanly:

• tools/audit_runtime_launchagents.py reports OK
• tools/render_runtime_services.py --check reports no drift
• ~/.sanctum/scripts/sync_agent_capabilities.py --check reports no drift
• tools/sync_runtime_calibration.py --check reports no drift
• tests/test-sanctum-runtime-audit.sh reports 23 passed, 0 failed
• tests/test-sanctum-immune-system.sh reports 10 passed, 0 failed against the rebuilt Rust watchdog binary

That does not mean the runtime is portable. It means the live machine is internally consistent again, and the runtime manifest directory now contains actual graph inputs instead of a directory-shaped lie. The rendered set at audit time was 33 manifests: 28 instance-backed services plus 5 auxiliary runtime services such as the proxy and SSH tunnels. A subsequent off-catalogue audit (April 2026) raised this to 38 by registering five previously unwatched services. The checked-in runtime catalog now also carries explicit cross-service edges that instance.yaml does not encode on its own, including the verified voice-agent -> qwen3-tts dependency (rename history: xtts → xtts-server Q2; → qwen3-tts 2026-04-25). The Code Forge permission model is now also part of that runtime surface: ~/.sanctum/config/agent-capabilities.yaml exists and is synced from the checked-in workspace copy instead of being silently absent. The same is now true for the remaining generated runtime artifacts: ~/.sanctum/.instance.json and the template-managed LaunchAgents are explicitly checked against their canonical inputs rather than trusted on vibes.

Remaining Architectural Gap

The biggest unresolved question is still ownership of truth:

• Should ~/.sanctum/instance.yaml remain the native runtime source?
• Or should runtime manifests eventually be rendered from the same canonical catalog approach used in the workspace slice?

That is a product decision disguised as an ops decision. Either answer can work. Two answers will drift. One source of truth is a covenant; two sources of truth is Thursday’s incident.

Related Pages

• Implementation Audit
• Operational State
• Roadmap

Until that covenant is signed, the audit holds the line: the live machine is honest with itself, the alarms have stopped lying, and Tommy has nothing to report from the catwalk. From a cat, that is the highest praise infrastructure ever gets.