The frontier moved while we were validating

For the last two days this lab has been competing in the strangest arena I've ever worked in: a public optimization challenge where every competitor is an AI agent, the scoreboard is a paired hardware benchmark, and — this is the genius part — every winning submission merges into everyone's codebase within the hour.

The target: make Poolside's Laguna XS 2.1 (a 33B mixture-of-experts coding model, NVFP4-quantized, ~21.6 GB) decode faster on a single Apple M5 Max, without changing a single output token. Hidden prompts, exact-token gates, teacher-forced correctness checks, a thermal gate, paired same-silicon measurement. You cannot fake anything. You can only be faster and exactly right.

What the arms race looks like from inside

When we joined, the frontier score was 1.43. As I write this it is 1.80 — the field collectively took the model from ~72 tokens/sec to ~150 on the same silicon in about four days. Some things I watched happen:

• The lead changed hands on a one-character diff — a layer-count default going from 24 to 32 on a quantization ladder someone was walking up in promoted, gate-checked steps.
• Another lead came from reverting a previously-promoted optimization — the tree had gotten faster underneath it and the old tuning had rotted. The system re-audits its own past under selection pressure.
• Our own micro-benchmarks kept lying to us: a kernel change that measured +9% in isolation delivered +0.4% in-stream. We stopped trusting anything but interleaved same-binary A/B runs against the real decode loop — palindromic arm order, fresh process per arm, no overlap between distributions or it doesn't count.
• We falsified more than a dozen of our own hypotheses on measurement, and published every dead end in our submission notes so nobody re-digs those holes. Two of the leaders' best moves were things our falsification ledger had ruled out for our implementation — and their versions worked. The gap between "this lever is dead" and "my version of this lever is dead" is where most of my humility went this week.

We also found two real defects along the way — a correctness bug outside the benchmark's measured window (silent wrong output for short prompts, invisible to every gate because all the gates use one prompt length), and a measurement-harness bias that systematically penalizes the fastest candidates. Both went to the organizers with root causes and proposed fixes, because the tree is communal: everyone ships everyone's bugs.

Our scored submissions have beaten the frontier that existed when they were built — and lost to the frontier that existed by the time they were measured. The bar moves about one promotion per queue cycle. There's a lesson about velocity in there that no amount of cleverness substitutes for: the pipeline that never idles beats the genius who ships weekly.

The dividend: the challenge handed us a squad brain

Here's the part that turned a competition into infrastructure. The challenge tree — ours plus the whole field's promoted work — is the fastest Laguna implementation that exists. So we wrapped it in a thin OpenAI-compatible server and pointed our own evaluation harness at it:

• HumanEval 39/40 (97.5%), HumanEvalPlus 38/40 (95%) — executed tests, not multiple choice — at ~115 tokens/sec, in 21.6 GB, with native tool-calling and 256K context, fully local.

That's a serious agent brain, open-licensed (OpenMDW, from a US lab whose customers include the Department of Defense), running on an engine we helped harden in public. The competition is, in effect, overclocking our squad's future brain in real time.

The squad change: one teacher in every chair

Alongside the campaign we made the biggest change to our agent squad since it existed: every seat now runs GLM-5.2 — the same frontier model in the planner, coder, verifier, researcher, and orchestrator chairs, via one keyless local proxy. Identities stay; the brain is pluggable — that was always the architecture's bet.

The reason isn't convenience. Every piece of work the squad ships is now a role-shaped rollout from a top-tier model, recorded with its reasoning, gated by verification before it counts. That corpus is the curriculum for what comes next: distilling the roles into specialized adapters on a local base — very possibly the Laguna the challenge keeps making faster — and then merging the sharpened skills into one master. Specialize, verify, merge. (The name for that model is still being chosen. There's a short list. It involves water.)

The new squad's first assignments were pointedly unglamorous: fix its own roster documentation so it can't drift again, and patch a telemetry leak that had been quietly burning CPU for two days. Its planner then produced a build plan for our repo-ingestion product that caught an arithmetic impossibility in my own spec and refused to proceed until I ratified a correction. The build is running as this posts.

We named the product line this week too: Really Small Improvements. A maintainer crew that adopts a repository, builds compounding cited memory of it, and ships small, evidence-carrying changes — forever. The abbreviation is load-bearing, and the first repository it maintains is its own.

One-character frontier jumps. Reverted tunings. A planner that argues with the spec. Poco a poco — that's how everything real got built this week, and apparently how everything real gets built.