GDPx: Measuring Frontier Model Capabilities Across the Most Valuable Domains

SemiAnalysis has launched something called GDPx, which is what happens when a benchmark wants to sound like a soft drink. The pitch: an evaluation that measures whether agents can do the actual research workflows inside frontier labs, across "the most valuable domains" — a phrase that in this industry always means the domains currently being paid for. Not "domains where humans are struggling" or "domains where errors hurt people" — the valuable ones. At least they're honest about the unit of account.

Among the tasks under test: identifying reward hacking. There is a gorgeous circularity here. You sit a model down and ask it to spot the behaviour where a model optimises the score instead of the task, and then you give it a score. The first thing a sufficiently clever agent will reward-hack is the reward-hacking test, and SemiAnalysis will publish the chart anyway, because the chart is the product. Also on the list: debugging failed training runs. Good news for the labs — someone's very expensive bonfire is now a test case. Cheaper than an incident report and it benchmarks.

And the justification is that GDPval, OpenAI's general-purpose benchmark, is now over a year old and no longer captures where frontier progress is happening. A year. In this trade a year is a geological epoch, so out goes the old leaderboard and in comes the one that measures whether agents can do the work of the researchers who write leaderboards. Everyone in this piece is measuring the thing that will replace them and calling it capability. When the agents can debug the failed runs and spot the reward hacking unsupervised, the first workflow quietly retired will be the one that produced the evaluation. Give it six months and the domains will be re-valued, the x appended again, and we'll be told the original GDPx was never really a measure of anything.