The Low Power Edge: who makes the most decisions per watt?

Everyone benchmarks big models on big GPUs. Almost nobody measures the lowest-power edge — small models on a small board at minimal watts — on decisions that matter where the cloud can't reach: deep space, off-grid, the field after the grid goes down. There was no public benchmark for it. The closest work, arXiv:2603.28926, scores LLMs on cross-domain mission-autonomy decisions (its best of three 70B-class models, Llama-3.3-70B, hits ~80% via cloud API) — but doesn't release its test set. So we built one.

The benchmark

20 physics-grounded decisions mirroring the paper's nine publicly-stated analysis domains — Walker constellation coverage, infrared Neyman-Pearson detection, EKF tracking, RF and acoustic link budgets, TDMA inter-satellite protocols, power-budget sizing, magnetic-signature formation, Arrhenius cryogenic reliability — across LEO surveillance, Mars nav, underwater swarms, deep-space relays, and outer-planet probes. Each has one defensible answer and a physics rationale. We committed the tests and answer keys before running any contender (verifiable in the git history), put it under MIT/CC-BY, and had a separate model adversarially audit every answer key. It's open: github.com/SoftBacon-Software/low-power-edge-bench.

Then we ran it on a $250 board

An NVIDIA Jetson Orin Nano Super (8 GB), Ollama, three contenders: Gemma-4 E2B (Google DeepMind), Llama-3.2-3B (the model NVIDIA recommends for this board), and Nemotron-mini (NVIDIA's own). Two metrics, as separate standardized workloads: decision accuracy (in two system-prompt conditions) and tokens/sec per watt (board power from tegrastats, swept across power modes).

The robust result: efficiency

Tokens/sec per watt (higher is better):

• Gemma-4 E2B — DeepMind — 2.0–2.2
• Llama-3.2-3B — NVIDIA's recommended pick — ~1.3
• Nemotron-mini — NVIDIA's own — ~1.1

…and that holds across all three power modes (15 W, 25 W, MAXN).

Gemma-4 E2B delivers 1.6–1.7× the tokens-per-watt of Llama-3.2-3B and ~1.8–1.9× that of Nemotron-mini — at every power mode — while drawing the least power (8.8 W at the 15 W setting, ~18 tok/s). This doesn't move with the prompt or the power mode. It's the number that survives scrutiny: on NVIDIA's own board, the most decisions per watt came from Google's model.

The honest result: accuracy is a tie the prompt decides

Accuracy, with no system prompt → with an expert system prompt:

• Gemma-4 E2B — 90% → 95%
• Llama-3.2-3B — 100% → 90%
• Nemotron-mini — 45% → 70%

We ran two prompt conditions on purpose — and the ranking flipped. The expert system prompt helped the weaker models (Nemotron +25 points) and hurt the strongest (Llama −10). So on accuracy, Gemma-4 E2B and Llama-3.2-3B are a statistical tie (~90–100%; which one leads depends entirely on the prompt, at n = 20), while NVIDIA's own Nemotron-mini trails in both conditions (45–70%). We're not going to dress a 1-item, prompt-sensitive accuracy gap up as a ranking — the efficiency result above is the one that holds.

Why 8 watts matters

Eight watts is a solar panel. A battery. A cubesat. The cloud isn't reachable from the far side of Mars or a disaster zone with no towers — and there, owned intelligence at watts you can actually power, making real decisions, stops being a slogan and becomes a measurement.

The honest part

We caught our own bug mid-run (Gemma returned empty answers — it's a thinking model whose reasoning was being stripped; we fixed the regime and re-ran). We ran two prompt conditions so nobody can say we prompted anyone badly. We used Ollama, not NVIDIA's TensorRT-LLM, so absolute tok/s is lower than NVIDIA's published figures — but the per-watt comparison is internally consistent (same stack, same board). The benchmark mirrors the methodology and domain scope of 2603.28926; it is not that paper's items. n = 20.

Everything — the 20 tests, the answer keys, the measurement rig, the raw per-item outputs in both conditions — is in the repo. Run your own model. PR a contender. Tell us where we're wrong. That's the bar.