Memorizing the answer key
Open benchmarks like MMLU or GSM8K are publicly available. Everyone can see the questions. So labs train their models on them, intentionally or accidentally through web data. The model memorizes the exam. It aces the scoreboard.
Then you ask it something slightly different on a Tuesday afternoon and it falls apart.
It is like studying for a driving test by memorizing the answer key. You get a perfect score on the theory test, but you still cannot handle Budapest rush hour traffic. A static benchmark is a minimum entry threshold, not a skill ranking. If a model fails it, you know something is broken. But a perfect score does not mean it knows how to drive.
Companies also set up testing environments in ways that favor their own models. They give their candidate longer thinking windows, extra retries, or a more forgiving grading rubric. When you control the questions and the scorecard, winning is mostly a question of effort.
When a metric becomes the target, it ceases to be a good metric. Goodhart’s law arrived in AI right on schedule.
Same score, different bill
A new flagship drops. Leadership talks about AGI. Social feeds fill with one-shot demos: whole games, full apps, “just ask once.” The launch charts look historic.
What the charts rarely show is how many tokens the model burned to get there. Modern reasoning models think in private first: hundreds or thousands of hidden tokens, retries, and internal self-corrections before you ever see the tidy final answer. The benchmark only checks if the last line is right. It treats the draft pages as free.
Two models can post the exact same score and live on different planets of cost and latency. One glances at the problem and answers. The other fills thirty pages of scratch paper, runs for two minutes, and gets full credit for the same result. Until evaluations account for the scratch paper, “smarter” often just means “we let it run longer.”
And the pile keeps growing. Each generation seems to need a deeper vault of tokens to move the chart another notch. It is starting to look like Scrooge McDuck diving into the money bin, except the gold is compute, and the dive itself is what we are applauding.
Newer models can feel sharper while quietly draining your usage tier before lunch. That is what happens when thinking tokens become the actual product, but the press release still sells the raw score.
Researchers are starting to measure this as its own axis: accuracy per token, not accuracy alone. It has not reached the keynote slides yet.
Why it feels worse two weeks later
Whenever a new model launches, the early excitement is through the roof. Then people start running messy, real-world tasks through it: strange prompts, half-finished workflows, jobs that never fit a clean benchmark category. The model shows its limits. People start posting that it got dumber.
Then another model drops, and the excitement moves on.
The AI community started calling this the “vibecession.” Part of it is psychological: your focus simply shifted from the things the model handles easily to the friction points. But part of it is technically real. The honeymoon fades, the rough edges show, and then comes the quiet feeling that the model was better on day one.
Benchmarks measure a best-case scenario: a clean prompt with a single correct answer. Real work is the opposite. Ambiguous instructions. Messy context. The willingness to iterate when the first output is wrong. The gap between the two is where the disappointment lives.
What actually happens behind the curtain
The feeling that a model performs differently over time is not always in your head. Companies rarely discuss this openly, but the infrastructure behind the scenes changes constantly after launch day.
Modern models often use a Mixture of Experts architecture. Think of it as a team of specialists inside one model. One handles math. Another writes code. A third does creative writing. When you ask a question, the system routes it to the right specialist. But when server load spikes, the company can save capacity by activating fewer experts per response or shortening the thinking window. The answers come back shallower.
I noticed this myself. A model that handled complex multi-step reasoning late at night suddenly gave me surface-level answers the next morning during peak hours. The version number had not changed, but the compute behind it had. Another common pattern is quieter cost-cutting after the reviews: shorter thinking budgets, heavier compression, or routing everyday queries to a cheaper sibling model. What you got on launch week is not always what you get on a busy Wednesday.
At launch, a model often runs at maximum effort for the reviews. Later, the infrastructure gets throttled to keep server costs manageable.
On top of that sits the safety tax. As millions of users poke at the model, the company updates hidden system prompts and safety filters. More rules in the background. The model becomes stiffer and more cautious, and often less capable on creative tasks that have nothing to do with guardrails.
The shift to blind votes
Automated benchmarks still matter for engineers. When you retrain a model, you need an instant regression test to check if you broke basic math. But for anyone actually using the tool, those numbers mean less every month.
This is why the industry is shifting toward private evaluation sets that models cannot memorize, and community voting platforms like LMSYS Chatbot Arena.
On Chatbot Arena, you enter a prompt, see two anonymous model responses side by side, and vote on which one actually solved your problem. No brand names, no cherry-picked charts.
Blind head-to-head votes are much harder to memorize or game than a static test. They help. So do newer evaluations that measure accuracy per token, instead of treating compute as infinite. The leaderboard that matters changes slowly, based on real usage patterns instead of keynote slides.
Reading between the lines
The next time you see a launch announcement with a wall of broken benchmark records, look for what is missing. The five tests they picked versus the ninety-five they skipped. The launch-week experience versus the throttled model you will actually use on a busy Wednesday. The tidy answer versus the hidden scratch paper it took to get there. And how much deeper the token vault got since the last “best model ever.”
The best model is rarely the one with the highest bar chart on a slide. It is the one that stays useful when your task gets messy.
How long does the honeymoon usually last for you before you notice where a new model actually breaks?