Skip to content

2026-07-31: The Reboot That Stayed Up

A stone keep at dawn, portcullis half-raised as a watchman cranks the gate wheel on the rampart. Old crack-scars across the walls glow with fresh amber repair seams — the only colour in the sketch. An Abyssinian cat sits on the drawbridge, watching the gate rise.

A Mini reboot is the only honest test of the haus. Everything can look green in a warm process table and still fail the first cold load of a 27B model under swap. This note is the pre-reboot land of that work: what broke, what was permanent, and what still requires one deliberate cold boot to call EETISMAD.

Cathedral (:1337). With --ablation-port set, a refusal-direction bank that failed to arm (wrong geometry for dense Qwen3.6-27B vs MoE [40,2048]) made sanctum-mlx fail-closed with exit(1) — and take the council seat down with it. launchd KeepAlive then crash-looped the load under multi‑tens of GiB of swap.

Force Flow (:4077). KeepAlive thrash without a singleton left EADDRINUSE storms and a process that answered HTTP while launchd said “not running.”

Health tunnel. SSH to the Lima VM before :22 was up → exit 255 spam.

Free-sub brains. Single-probe Gemini flaps and pre-boot state edges looked like CRITICAL outages.

Ark cutover. Backup canary still probed T9 after primary restic moved to /Volumes/Ark/sanctum-restic.

Kitchen lights. Green automation Tuya Reconnect Color Guard fired on unavailable → on and called light.turn_on with only color_temp — these Tuya bulbs snap to full brightness. Family comfort is not 3000K at bri 180; SmartLife truth is bri 100, hs [29, 88.2], rgb [255, 139, 30].

Permanent fixes (on disk, live on the Mini)

Section titled “Permanent fixes (on disk, live on the Mini)”
AreaFix
bin/sanctum-mlx-launchGeometry-validate ablation bank; strip --ablation-port if unarmed so :1337 always binds
bin/force-flow-launch.sh + failsafeSingleton on :4077; heal only when /health is down (PPID=1 is normal under launchd)
bin/health-tunnel-launch.shWait for VM :22; exit 75 TEMPFAIL instead of 255
bin/sanctum-boot-readyDefault wait 900s; --check revalidates live ports
scripts/free-sub-brain-sentinel.shStreak gate + AGY auto-heal; chat transient → degraded not out
Backup canary + scriptsPrimary Ark; scripts under ~/.sanctum/scripts/; plists rewired off ~/Backups/
bin/ha-green-bridge.pyComfort on kitchen + corner RGB + corridor; pin Tuya color-guard OFF
Albert summer curfewHoliday 23:00 restored (exam-week 22:30 rolled back)

Runbooks: runbooks/clean-reboot.md, runbooks/kitchen-comfort-lights.md, package ha-green/packages/kitchen_comfort.yaml.

Super-bright white was not “HA lying” — it was the wrong mode. Color-temp paths looked cold and harsh on these bulbs. The accepted default is the warm hs/rgb preset from SmartLife, applied on off / unavailable → on for:

  • light.kitchen_1_6
  • light.corner_rgb_1_6 + group
  • light.corridor_rgb_1_4 + group
LegStatus before cold boot
DeployedLive on Mini (bridge, launchers, devices.yaml, boot-ready env)
In docsThis field note + runbooks
MergedPathspec land to sanctum-config (+ related)
E2E testedSpot-tested warm; cold-boot checklist still required

After login, wait ~15–20 minutes for model load, then:

Terminal window
sanctum-boot-ready --check && echo READY
# expect READY only if 4040/1337/4077 listen
for p in 4040 1337 4077 3301; do
lsof -nP -iTCP:$p -sTCP:LISTEN >/dev/null && echo UP :$p || echo DOWN :$p
done
bash ~/.sanctum/scripts/pqc-status.sh | tail -8
launchctl print gui/$(id -u)/com.sanctum.mlx 2>&1 | grep -E 'runs =|state ='
df -h /Volumes/Ark /Volumes/ARK-GO
curl -sf http://127.0.0.1:4077/screen/schedule?child=albert | python3 -c 'import sys,json;d=json.load(sys.stdin);print(d["schedules"]["albert"])'
rg -n 'STRIP --ablation|ablation ARMED|FAIL-CLOSED' ~/.openclaw/logs/sanctum-mlx.log | tail -5

Pristine means: boot-ready true, core ports up, PQC all hops, mlx runs low (not a crash-loop), Ark mounted, Albert holiday curfew 23:00, no pqc/cathedral/free-sub CRITICAL in the boot window.

The receipt — the haus ran the missing E2E itself

Section titled “The receipt — the haus ran the missing E2E itself”

The deliberate cold boot never got scheduled; on 2026-08-13 the haus ran an unexpected one anyway. Hard reset / power loss at 16:56:28 (utmpx reboot with no preceding shutdown). Core eventually came up. Post-boot did not.

ClockWhat happened
16:56unexpected reboot
16:58com.sanctum.post-boot starts
17:04core-listener wait (240s) TIMEOUTs: 4040 8123 8199 8765 4077
17:06Phase 4+settle: 33/41, 6 fail, 1 heal (HA docker). Signal sent. exit 6
17:10sanctum-boot-ready READY (proxyd actually bound)
19:50apple-services --json --mac 35/35 healthy

The six “failures” were still starting. Three independent bugs made that look like an outage:

  1. port_bound used bert lsof only. proxyd listens as the sanctum user. netstat showed 127.0.0.1:4040 LISTEN; bert lsof was empty. sanctum-boot-ready already had the kernel fallback; post-boot did not.
  2. Firewalla heal skipped. Rule requires: firewallasanctum_enabled firewalla is no. Live key is firewalla_bridge.
  3. No remediations for mac/yoda-orchestrator, mac/ha-gateway, mac/sanctum-proxy / mac/proxyd. com.sanctum.ha-gateway and com.sanctum.firewalla plists live under ~/.sanctum/launchagents/live/, not ~/Library/LaunchAgents.

Same boot dropped WhatsApp ChatStorage.sqlite (container rebuilt at 16:59, drestore.mark). Desktop kept running. CRM whatsapp_history last row: 2026-08-12 23:00.

Shipped against that receipt: kernel port_bound fallback, 600s core wait + 180s settle-confirm, remediations for the six names, WhatsApp ChatStorage warn on apple-boot-check.

When something that worked breaks after reboot, ask what changed between boots — falsify before you fortify — and never let an optional seat (Heretic, a color-temp “guard,” a pre-VM SSH) take down the required one.