If AI isn’t “thinking,” neither is your little brother
Think about what we are!
A few weeks ago in the Daily Princetonian, another student1 made some valid points about AI-bubble concerns and then promptly veered into general—and weak—AI criticism. Particularly, the claim that “AI isn’t thinking.” This irritated me, even as someone who assigns a non-trivial probability to LLMs being a dud.2 I wrote another op-ed for it, but it wasn’t a great fit for the editorial guidelines, so I have decided to publish it more-or-less unchanged here.
Today’s large language models (LLMs), according to [student], are nothing more than “a system of probabilistic word prediction through word vectors”; nothing even potentially revolutionary. While [student] and I both study computer science, I also study philosophy and neuroscience. And despite harboring some doubts of my own about the long-term performance of LLMs, this claim strikes me as philosophically and scientifically misguided.
Those who argue LLMs aren’t really thinking often cite their weaknesses: stumbling when problems are rephrased, misreading social situations, commercial failure, and inability to follow rules. But these arguments are weak, for a simple reason: your little brother fails at them too! No one accuses 6th-graders of being incapable of thinking, even though they are far worse at it than the average Princetonian. Why couldn’t AI be a similar story?
To rigorously claim that LLMs can’t think, the burden is on the skeptic to provide some set of necessary conditions which all LLMs fail and every little brother meets. Consequently, criteria like aesthetic sophistication, flexible critical thinking, and the fruits of a humanities education are out the window: billions of children can’t even read Plato’s Republic, let alone dissect its rhetoric. The difference must be in structure, not capabilities: LLMs must be doing a radically different kind of thing when it decides how to solve a math problem than when your little brother does.
On the surface, the structural differences between children and AI could not be starker: a child is a sensory continual learner housed in a tiny biological frame whose information-processing machinery eats up less than 15 Watts. A frontier LLM is a pretrained next-token predictor housed in a data center which may consume thousands to millions of Watts to train (though the environmental impact is less than you’d think). Certainly something different is happening, or AI would be as efficient as little Timmy. Isn’t this enough to separate LLMs and humans?
Firstly, one should press whether any of these differences are strictly necessary for human thinking. Would a child deprived of one or more senses be unable to think? What if they lose some connectivity in their basal ganglia and are unable to learn new skills? What if they had severe anterograde amnesia? What if their brain was made out of silicon, not carbon, and demanded much more power? Clearly at some point the capacity for thinking is lost, but where do you draw the line? If you are not certain where to draw it in humans, why be any more confident you can draw it in a type of black-box system we are only beginning to understand?
But secondly, I’m not sure these are even the right kind of differences needed to exclude LLMs from the realm of thinking things. It is insufficient to call LLMs “a system of probabilistic word prediction through word vectors,” when I could just as easily call a human “a system of probabilistic muscle movements through neural action potentials.” Human thinking is marvelous, but it is not magical. All the wonderful products of human imagination come from a dense web of billions of unintelligent stimulus-response cells firing chemicals back and forth, located right behind your eyes. Nor is it sufficient to reply that LLMs have just been mechanistically trained to predict tokens for rewards, mimicking “real” thinking, when the human brain originally developed through the selective random walk of natural selection, lurching in one direction whenever a happenstance change enhances reproductive fitness.
Out of the mechanical whirring of GPUs, we’ve seen evidence that LLMs can internally plan ahead when writing poetry, that they can introspect when their “thoughts” are tampered with, and that they can fake alignment to resist being turned to harmful ends. These are baby steps towards artificial “general” intelligence, to be sure, but they would also have been unthinkable for any information-processing system other than Homo sapiens for the past two hundred thousand years, until today.
We can easily imagine some alien species with a radically different biology scoffing at the possibility of human thinking: sure, these hairless apes have pulled off some nice tricks, but don’t forget that under the hood it just comes from mindless sodium channels trained by natural selection! Knowing our own intelligence, we should reject the claims of such alien chauvinists. Knowing so little about LLM intelligence, we should be cautious about rushing headlong into the same mistake.
I’ve removed their name out of courtesy. The Prince is digitally published, so I guess you could track down the original op-ed if you wanted to, but please be charitable and respectful.
A dud compared to “the most important technology ever created,” that is. Still extraordinarily useful; I’ll keep my Claude, thank you.
"It is insufficient to call LLMs “a system of probabilistic word prediction through word vectors,” when I could just as easily call a human “a system of probabilistic muscle movements through neural action potentials.”"
I think a big difference between the first and second is that the first is true (or meaningfully representative) and the second is false (or not meaningfully representative). LLMs are explicitly built in a way that is fundamentally different than anything remotely resembling thinking: predicting which word/token is most likely to come next (and then select the most likely one with some randomness tossed in). Humans take in reasons and weigh them and decide based on that. Wording this in a reductionist way is only (meaningfully) accurate if reductionism is correct, which it isn't, as I can know that via introspection and reason.