Discussion about this post

User's avatar
JerL's avatar

I'd be careful of conflating "computers can't think", which is about substrate dependence, with "LLMs can't think", which might be more about architecture, etc.

For example, I'd expect that for something to be conscious it has to have some coherent notion of the passage of time for it--it has to entangle with entropic processes in some non trivial way. So a lookup table doesn't count, not because it's written on paper instead of brain tissue, but because the information contained in it is too static.

So a sim-brain could be made out of silicon, and still have this property; or you could have a lookup table written in brain tissue which wouldn't.

Whether your imagined sim brains would pass this test I'm not sure... But I don't think the fact that they're made out of sim-components is the source of my doubts.

I also wouldn't be too quick to write off the relevance of quantum stuff here; suppose you believe in something like Scott Aaronson's freebit idea for free will: that free will is possible because human beings can entangle with a source of previously-unentangled, low-entropy qubits left lying around from the big bang. If you think that, say, free will is a necessary component of consciousness, you might think anything incapable of using freebits as a resource can't be conscious, which might (though, also might not) rule out your sim-brains.

Lydia Nottingham's avatar

i recently got a lot out of sandberg’s ‘simulation’ vs. ‘emulation’ distinction:

“The term emulation originates in computer science, where it denotes mimicking the function of a program or computer hardware by having its low-level functions simulated by another program. While a simulation mimics the outward results, an emulation mimics the internal causal dynamics (at some suitable level of description). The emulation is regarded as successful if the emulated system produces the same outward behaviour and results as the original (possibly with a speed difference).”

from https://www.openphilanthropy.org/wp-content/uploads/SandbergandBostrom2008.pdf

8 more comments...

No posts

Ready for more?