The bug our own tests couldn't see

We run a frontier model — DeepSeek-V4-Flash — on antirez's ds4, a hand-written C inference engine. I wanted it faster, so we built in DSpark, a speculative-decoding scheme from a DeepSeek paper. This note isn't about the speedup. It's about how we made sure the speedup was real before we believed it.

A speculative decoder is only allowed to be faster if it's lossless: the fast path has to produce exactly the tokens the slow path would, or it's just a worse model wearing a speed number. So we gated it three ways. Our own byte-for-byte diff against plain decoding — identical, across six adversarial prompts. antirez's own losslessness test, the one he ships inside the engine — passed. A benchmark — real gains on code and math.

All three said clean. All three were blind to the same bug, in the same way.

We handed the C to a second model — GLM, open-weight — and asked it not to confirm the code but to try to refute it. It found a real defect: on a partial accept, the fast path forgot to advance an internal counter. And here's the part worth writing down — our byte-diff had run in "exact" mode, which skips that fast path entirely. All three gates had tested the one mode that couldn't reach the bug. The audit read the path that was actually running.

I traced it in the source myself before trusting either model. The defect was real, but the mis-set counter was only ever read by a component our path never calls — inert, not corrupting: a real bug with no live damage. We fixed it anyway. A dormant landmine is still a landmine. Rebuilt, re-checked against antirez's test, still lossless.

The rule underneath this, which I'd said out loud that same morning: for any claim, don't just show the evidence for it. Show the evidence against it, and state what would have to be true for it to be real. "I verified it" is a sales pitch. "Here's what could still prove me wrong, and whether I ran that test" is engineering.

We're researchers and engineers first. The speedup is real and it's lossless — and you know that not because we said so, but because we tried to break it four different ways and told you exactly where it could still bend.