The cap, the band, and the lock

Last night a benchmark leg ground for two and a half hours on one exercise and wrote nothing. The spec for our frozen benchmark has required a 600-second per-exercise wall cap since the day it froze — required, and never wired. A slow dense model found the gap inside twenty-four hours of the freeze.

The cap

This morning the runner was rebuilt around the rule: every exercise in its own container, killed at the cap, recorded in a DNF ledger that scores as fail-both-tries — never silently dropped from the denominator. A killed or crashed exercise leaves no results file, so re-running the same run name resumes exactly where it died. The verdict logic reads results-on-disk before trusting process exits, which also self-heals an upstream hang we'd been killing by hand.

By evening the cap had already paid for the morning: the same exercise that ate 2.5 hours died at exactly 600 seconds, the ledger took the entry, and the run moved on. Twelve grinders that could have been days finished in a bounded two hours of cap time. Lesson kept: a regime clause that isn't enforced by the harness is a wish, not a rule.

The tie

With the cap holding, the planner's brain ran the full frozen set clean: 43.3% pass@1 / 61.7% pass@2 across sixty exercises in four languages. The interesting part: that ties the dedicated coder model's 61.7% exactly, with inverted language profiles — the dense 27B generalist wins python and rust, the MoE coder specialist holds go. A planner that codes at parity with the specialist, measured on a frozen ruler, at home, for free.

The band, held honestly

The self-improvement round spec pre-registered a decision rule: the 27B is the training target only if its score lands inside a set band. It landed 1.7 points above the ceiling. The easy move is to shrug and take it — the score is "close enough." The corrosive part isn't this one decision; it's what quietly bending your own rule does to every future number you publish.

So the rule got the treatment a rule deserves: a dated protocol deviation, written into the spec before any training rollout fired. The measurement regime had changed after the band was registered — the newly enforced cap scores twelve did-not-finish exercises as failures, a strictly harsher instrument than the rule contemplated — and on the capped ruler those DNFs are themselves headroom: a model that learns to grind faster converts them. The acceptance gate for the round itself is untouched. Pre-registration is only worth something if deviating from it costs you a paragraph of honesty in public. Paid.

The lock

Target locked, the squad went to work on its own instrument: tonight's workflow has the coder building the rollout generator and the harvest tool for the round — the machinery that will collect the 27B's own verified passes as its training data — with the verifier holding a grounded-verdict gate behind her. The crew is building the apparatus for its own next lesson, through the same gated seam it does every other job through.

Meanwhile, on the metal

Two smaller receipts from the same day. Google's new multi-token-prediction variants claim ~2× decode speedups "especially on GPUs," with Macs warned to expect less — we measured the full-precision 31B on this laptop at 8.9 tok/s without MTP and 18.7 tok/s with it: 2.10×, the full GPU-class gain on Apple Silicon. Full precision at nineteen tokens a second on a desk is not a curiosity anymore.

And when the new diffusion variant of Gemma dropped with no MLX runtime behind it, the read-only scout swarm earned its keep: six local agents digested nine reference sources — the canonical implementation, a working vision-stack port, two draft C++ PRs — into a line-cited architecture map in an evening, frontier tokens spent only on the synthesis. The port lane into Apple's mlx-lm is now claimed publicly, with the implementation plan attached. The meter went live the same day — physics, not bills.

The pattern of the day, if there is one: instruments first. Wire the cap, hold the band, lock the target, and let the crew build the next ruler.