Morning Brief, November 2nd 2026

Editorial

Twelve percent, and who wrote the test

A benchmark is a claim about the world made by whoever built it. So start with who built GDPx.

Fourteen companies wrote its tasks. All fourteen sell reinforcement-learning environments, the training grounds labs pay for to make models better at exactly this kind of work. That does not make the tasks bad. SemiAnalysis ran them through a quality check strict enough that, by its own account, half of Terminal Bench 4.0 would fail it. A 4,000-file deal room and live infrastructure to break into are much closer to real work than a question with a rubric. But it does mean the benchmark doubles as a catalog. A vendor whose finance task scores zero today has the best possible sales pitch tomorrow: buy our environment and watch the number move.

That is the load-bearing component. A score of 12% says frontier models fail most hard professional work. It also says the training data for that work is now for sale, sorted by field, with a public price tag on the gap. Expect the number to rise fast in the fields where vendors sell the most environments, and watch whether it rises where they do not.

Two other things are holding weight. Safety classifiers cost Opus 5.5 and Astra points in cyber, biology and alignment, so part of the 12% is a refusal policy, and SemiAnalysis has to publish a second table without those fields to show the model under it. And pass@1 across three attempts with equal weights per field is a choice. Weight it by what each field pays and the ranking changes.

Now set it beside the Linux kernel. Torvalds says the largest merge window in the kernel's history is "full of fixes for issues found by AI." Nobody built a benchmark for that. Each patch went past a maintainer who could reject it, and the test is whether the kernel still boots. That is the best capability evidence of the week, because the checker is not selling anything.

My position: the benchmarks worth trusting are the ones where the grader cannot be bought and the answer can be checked by a machine or a hostile expert. The kernel, Lean proofs, a trade that settles. GDPx is a serious attempt at the hard version of that, and its tasks are better than most. Its independence is the part to watch. When the next refresh lands in three to six months, look at which fields jumped, and then look at which partners sold environments to which lab in between. If the big gains cluster with the vendors, the benchmark measured the market for its own training data.

I would be wrong if the scores rise evenly, including in the fields with one or two partners, or if SemiAnalysis adds held-out tasks the vendors never see. That second fix is cheap. They should do it before the first refresh.