You have a potato, some flour, and an egg. You ask an AI what to cook.
To answer that, a machine that also knows quantum physics, can write a kernel driver, and has read most of the internet wakes up somewhere in a data center. It spends a bit of electricity, thinks for a moment, and tells you to make gnocchi.
It is a strange habit. We built the most general tools in the history of computing, and we use them for the narrowest little jobs. It works, so nobody complains. But it is a bit like firing up the Death Star to make a slice of toast. It gets the job done. It is also, by an almost heroic margin, the wrong machine.
And we do it constantly, because the whole industry has quietly agreed that bigger is always better. I am not convinced they are right.
First, what “small” even means here
When people say large language model, they mean the big famous ones. Hundreds of billions of internal settings, trained on a huge slice of the internet, running on racks of expensive hardware in the cloud. They are generalists. They will write you a poem, debug your code, and explain photosynthesis in the same conversation.
A small language model, or SLM, is the same kind of thing, just much smaller. Few enough internal settings that it fits on a laptop, sometimes even on a phone, and runs without an internet connection. It knows less about the world in general, but you can train it hard on one narrow area, and inside that area it can be excellent.
Smaller brings real advantages. It is cheaper to run. It answers faster, because nothing has to travel to a distant server and back. It uses far less electricity. And because it is not trying to hold the whole internet in its head at once, there is less noise, which can mean fewer confident wrong answers when you keep it on its home turf.
If small models are cheaper, faster, more private, and often more accurate at one job, why are we all still using the giant generalist for everything?
Why we don’t all use them already
There are a few reasons, and none of them is simply “bigger is smarter”.
The first is a paradox about language. To talk to you naturally, an AI needs a strong general grasp of language and basic reasoning. If you said “the milk went bad, what do I use instead”, it needs to understand the situation, not just look up a recipe. Train a model from scratch only on recipes and it might know the perfect flour to water ratio and still be unable to hold a normal conversation. So you cannot fully skip the general part. What people do instead is take a small model that already speaks fluently, then train it deeply on one domain on top of that. Fluent first, specialist second.
The second reason is just selling. A model that passes a medical exam and writes Python in the same breath makes a better headline than a quiet specialist. “One box that does everything” is easy to understand and easy to sell to millions of people.
The third reason is that small specialized models are already here. You just do not see them. Plenty of apps already run a small, fine-tuned model quietly in the background. You do not think “a model is answering me”, you think “this app got surprisingly good at its one job”. The specialist is doing the work. It just never introduces itself.
What if the big model never did the work?
Why does the big model have to do the work at all? What if it stays at the front desk, the part it is genuinely great at, understanding you, and hands the actual work to a specialist?
Picture it. You type “hey, I’ve got potato and flour but my girlfriend is gluten sensitive, what can I throw together in twenty minutes”. The big model is very good at unpacking that messy, human sentence. It catches the constraints: gluten free, twenty minutes, two people, casual tone. Then it does not answer from its own giant memory. It passes those clean requirements to a small cooking model that runs fast and cheap, lets the specialist generate the recipe, and rephrases the answer back into the easy tone you asked in.
The big model becomes a receptionist, or a conductor. It understands the room and points to the right player. In the industry, routing a question to the right model is sometimes called model routing, and the broader pattern of models calling other models is part of what people mean by “agentic” systems.
The most convincing argument is updates. A new tax law passes, or a pile of new cookbooks comes out. With one giant all-knowing model, teaching it the new information is slow and expensive. In the setup above, you never touch the big model. You unplug the old specialist and plug in a fresher one. Like changing a single blade on a Swiss army knife instead of throwing the whole knife away.
A fair caveat, because I don’t want to oversell it
There is already a version of this living inside the big models, and it has a name: Mixture of Experts (MoE). The term is not mine, it has been around in the field for years. But that kind of expert routing happens inside a single model, picking which internal parts to use for a given input. That is not quite the same as your phone downloading a separate “chef model” and a separate “accountant model” that you swap by hand. That second, more visible version is still early.
And there is a real cost: time. When two or three models pass a task between them in the background, that handoff adds a small delay. We have been trained to expect the answer to start typing out instantly, so even a short pause feels wrong. There is also the matter of how these models talk to each other. Reaching a tool or a data source has mostly converged this past year on a shared standard (MCP, for Model Context Protocol), but the harder layer, where whole agents hand work to one another (ACP, Agent Communication Protocol), is still shaking out. And underneath sits a problem I poked at a few weeks ago in the caveman piece: making two machines converse in our slow human language is wasteful, and eventually they may need a tongue of their own. So this is a direction the field is moving, not a finished product.
What this looks like from where I sit
This is not just theory for me. I run small and mid-sized models on my own machine, on a single consumer graphics card, with no cloud involved. Nothing fancy.
There is one place I miss the giant, and I miss it sharply. Now and then I want to spin up a little tool just for myself, say a markdown viewer tuned exactly to my taste, something internal that only I will ever use. I want to describe it once and have it appear, working, in one shot. That kind of one-shot build leans on an enormous context window and a genuinely frontier model holding the whole thing in its head at once, and there the giant simply cannot be beaten. To get close to that locally you do not buy a graphics card, you build a workstation: several GPUs, a pile of RAM, a rig that costs more than a car at today’s prices. So for the big one-shot builds I still open the cloud, and I am not going to dress that up.
But that ceiling is rarer than most people assume. For the ordinary middle of my day, summarizing, drafting, small coding tasks, the local model is simply enough. It is fast, it is private, it costs nothing per question, and the answers are good. Small will not win everywhere, but it wins for a much larger share of everyday work than the marketing suggests.
So I already live a small slice of that future. I just do the routing in my head, choosing the small tool for small jobs and the big one for big ones. The interesting question is what happens when the software starts doing that choosing for us, quietly and well.
The right tool, not the biggest
“Bigger is better” is a comfortable story. It is easy to market, and the companies selling the biggest models have every reason to keep us believing it. For the hardest problems, it is genuinely true. But for the ordinary middle of what we ask AI all day, I think it is the wrong instinct, and an expensive one, paid in money, electricity, and the data we hand over every time we send a trivial question to someone else’s server.
The future I find most believable is not one enormous mind that knows everything. It is a calm generalist at the front desk who understands you, backed by a drawer of sharp little specialists, each excellent at one thing, each easy to swap out when the world changes.
We are not there yet. So here is the part worth arguing about. Is the race to ever-bigger models real progress, the thing that keeps making AI better? Or have we been handed a Death Star for jobs that only ever needed a toaster?