"It is insufficient to call LLMs “a system of probabilistic word prediction through word vectors,” when I could just as easily call a human “a system of probabilistic muscle movements through neural action potentials.”"
I think a big difference between the first and second is that the first is true (or meaningfully representative) and the second is false (or not meaningfully representative). LLMs are explicitly built in a way that is fundamentally different than anything remotely resembling thinking: predicting which word/token is most likely to come next (and then select the most likely one with some randomness tossed in). Humans take in reasons and weigh them and decide based on that. Wording this in a reductionist way is only (meaningfully) accurate if reductionism is correct, which it isn't, as I can know that via introspection and reason.
How can you know that reductionism isn't correct by introspection and reason? I don't think I can. I'm sympathetic to some non-physicalist views, but panpsychism and idealism can both still be reductive.
Anyway, for a non-reductionist to object to my overall point, they have to explain:
- What is the relation between the brain, which looks extremely mechanistic, and reasons?
- What is the relation between the brain and LLMs, which look reasonably similar?
- How are we confident that the relation between the brain and reasons does not hold for LLMs and reasons?
"It is insufficient to call LLMs “a system of probabilistic word prediction through word vectors,” when I could just as easily call a human “a system of probabilistic muscle movements through neural action potentials.”"
I think a big difference between the first and second is that the first is true (or meaningfully representative) and the second is false (or not meaningfully representative). LLMs are explicitly built in a way that is fundamentally different than anything remotely resembling thinking: predicting which word/token is most likely to come next (and then select the most likely one with some randomness tossed in). Humans take in reasons and weigh them and decide based on that. Wording this in a reductionist way is only (meaningfully) accurate if reductionism is correct, which it isn't, as I can know that via introspection and reason.
How can you know that reductionism isn't correct by introspection and reason? I don't think I can. I'm sympathetic to some non-physicalist views, but panpsychism and idealism can both still be reductive.
Anyway, for a non-reductionist to object to my overall point, they have to explain:
- What is the relation between the brain, which looks extremely mechanistic, and reasons?
- What is the relation between the brain and LLMs, which look reasonably similar?
- How are we confident that the relation between the brain and reasons does not hold for LLMs and reasons?