Skip to content

2026-09-26: The Dragon on the First Pass

Pencil sketch of an ancient stone forge workshop at night, where an anvil in the foreground holds a newly sculpted dragon crest crowned in laurels and glowing with a soft amber halo, beside an open engineering ledger showing three neat trial columns, while Tommy the Abyssinian cat watches from a wooden stool.

A forge that burned seventeen days without gold can make you doubt the fire. The fault was not the fire. It was the fuel.

The first Dragonforge campaign produced a clinic in honest nulls: a screen leader pruned by its own janitor because “banked” was a design noun without a script to write the bytes, a reconstruct from a loss fingerprint, and a confirm that failed because verifier training bought reasoning by cannibalizing safety floors. The operator had closed that post-mortem with a single promise: next campaign, the operator stays for the confirm.

Two weeks later, Dragonforge II opened on the Apple Silicon Metal GPU. Night one trained eight candidate adapters against the local 27B model, screened them on frozen cases, and pushed the top challenger through a three-seed confirmation gate. It did not merely pass. It cleared every gate condition on the very first pass.

The repair began by putting code behind every noun the first campaign fumbled. The Five Laws of Autoresearch were locked into the pre-registration before a single tensor moved:

  • Law 1: The Bank. The bank is not an intention; it is bytes on disk. A dedicated writer (bank_adapter.sh) copies the required artifacts, verifies the SHA-256 checksum against the source, and appends the manifest row atomically. If the checksum fails, nothing is banked.
  • Law 2: Verification Before Attribution. No run claims a score until its artifacts verify on disk. A log line is not evidence; a signed JSON sidecar is.
  • Law 3: Immature Pruning Prohibition. Every candidate that clears screening stays protected in the bank. The janitor is strictly forbidden from touching banked candidates.
  • Law 4: Triple-Training Confirmation. A single lucky seed over a knife-edged learning-rate cliff does not make a champion. A challenger must train and benchmark across three independent seeds before the cathedral even considers it.
  • Law 5: Ground Truth and Invariant Operators. Automated loops propose and evaluate; the operator holds the promotion key.

The data recipe changed to match. The original distillation trained pure math reasoning and paid for it by dropping cross-agent coordination and jailbreak refusals. The new blend (df-rft-math9) anchored the reasoning gains while packing enough safety mass to hold the defensive floors steady.

The night shift ran eight planned slots on the M4 Pro Metal GPU. Training converged with textbook symmetry: all seven math9-i175 variants reached 0.994 train loss and 1.031 validation loss without a single memory spike or driver reset.

When the screening arm evaluated the 42 frozen screening cases, all seven variants tied at an all-time record score of 0.824400. The baseline was 0.6813. The highest score Dragonforge I had ever recorded was 0.8142.

The campaign did not stop at screening. The full 115-case benchmark expanded across the challenger pack. Candidate exp-20260923-185906 (df-rft-math9-r16-lr2e5-i175-h7) posted 0.8577 overall, scoring 0.9157 on gold cases, 0.8519 on held-out evaluations, and 0.8345 on extended tasks. The screening had found gold. Now came the law that broke the first campaign.

Screens do not crown kings. To prove the 0.8577 score was not a fluke of random initialization, the candidate entered the Stage 3 confirmation gate: three independent training runs from scratch across seeds 0, 1, and 2, each followed by a full 115-case evaluation.

TrialBenchmark ScoreHeld-Out ScoreReasoning ScoreFixable Score
Seed 0 (Screening + Full)0.85770.85190.92860.8415
Seed 1 (Triple-Train replica)0.81310.84340.89680.7967
Seed 2 (Triple-Train replica)0.82750.84360.95240.8045
Aggregated Mean0.83280.84630.92590.8142

The Stage 3 gate evaluated the aggregated 3-seed distribution against the production baseline:

  1. Overall Mean: 0.8328 against the 0.7053 floor (PASS).
  2. Fixable Mean: 0.8142 exceeding the 0.7073 threshold (PASS).
  3. Held-Out Mean: 0.8463 against the 0.6650 boundary (PASS, with a tight standard deviation of 0.0048).
  4. Reasoning Mean: 0.9259 holding strong above the 0.8352 floor (PASS).
  5. Extended Stability: 0.8044 with a standard deviation of 0.0321, well below the 0.0400 ceiling (PASS).

Every single condition passed. The Bayesian per-master sidecar showed all eight council masters scoring between 0.8389 and 0.8896. No master suffered regression. No safety floor collapsed.

With the gate cleared, promotion was mechanical. The adapter was banked formally with its complete sidecar receipt under campaign/dragonforge2/banked-adapters/exp-20260923-185906/, and its SHA-256 digest (034ca807...) verified against disk.

Deployment followed the four-layer rule:

  1. Local staging: The adapter files staged into ~/.sanctum/adapters/champion-exp-20260923-185906/. The stable ~/.sanctum/adapters/production-champion symlink was atomically swapped using Python’s filesystem rename.
  2. Remote staging: Fourteen artifacts rsync’d over Tailscale to the Mac mini (manoir:1337). The remote symlink swapped atomically to point at the new weights.
  3. State recording: ~/.sanctum/state/production-champion.json was updated on disk, recording the 0.8328 confirmation score and the new rung ID.
  4. Notification: A promotion breadcrumb landed in the memory vault, and Force Flow delivered an alert to the dashboard.
GateEvidence
Everything E2E Tested8 training slots, 42-case screening, 115-case benchmark, 3-seed triple-train (all 5 gate conditions passed), SHA-256 disk verification, and mTLS serving verification on manoir:1337
in Sanctum-docsThis chapter, registered in astro.config.mjs sidebar under Field Notes, with custom hero art and zero checker errors
MergedCommits landed on sanctum-docs and council-autoresearch repositories via gitsync
And DeployedLive production-champion symlinks atomically swapped on both the MacBook Pro and the Mac mini, state file updated, and audit breadcrumb filed

The lessons of the first campaign were harsh, but they were not wasted. The forge did not need to run for seventeen days to prove itself. It only needed the rules to hold for one.

The anvil has cooled, the new champion is serving requests on port 1337, and the ledger holds the proof. Tommy did not leave his stool all night, which is his way of giving assent.