Skip to content
The Future

How could we distinguish conscious AI from its zombie equivalent?

The idea that consciousness emerges naturally alongside intelligence could be an anthropocentric distortion.
a couple of heads with different colors and shapes.
Credit: Annelisa Leinbach / Big Think; DALL-E
Key Takeaways
  • From the 16th-century golem to the replicants in Blade Runner, stories of sentient human-like creatures tend to end in philosophical and existential chaos.
  • The rapid rise of AI has lent fresh urgency to questions about machine consciousness.
  • It is a mistake to assume that intelligence β€” at least in advanced forms β€” is either necessary or sufficient for consciousness.
Sign up for the Smarter Faster newsletter
A weekly newsletter featuring the biggest ideas from the smartest people
Excerpted from Being You: A New Science of Consciousness by Anil Seth with permission from Dutton, an imprint of the Penguin Publishing Group, a division of Penguin Random House, LLC. Copyright © 2021 by Anil Seth.

In Prague, in the late 16th century, Rabbi Judah Loew ben Bezalel took clay from the banks of the Vltava River and from this clay shaped a humanlike figure — a golem. This golem — which was called Josef, or Yoselle — was created to defend the rabbi’s people from anti‐Semitic pogroms, and apparently did so very effectively. Once activated by magical incantation, golems like Josef could move, were aware, and would obey. But with Josef, something went terribly wrong and its behavior changed from lumpen obedience into violent monstering. Eventually the rabbi managed to revoke his spell, upon which his golem fell into pieces in the synagogue grounds. Some say its remains lie hidden in Prague to this day, perhaps in a graveyard, perhaps in an attic, perhaps waiting, patiently, to be reactivated. 

Rabbi Loew’s golem reminds us of the hubris we invite when attempting to fashion intelligent, sentient creatures — creatures in the image of ourselves, or from the mind of God. It rarely goes well. From the creature in Mary Shelley’s Frankenstein to Ava in Alex Garland’s Ex Machina, by way of Karel Čapek’s eponymous robots, James Cameron’s Terminator, Ridley Scott’s replicants in Blade Runner, and Stanley Kubrick’s HAL, these creations almost always turn on their creators, leaving in their wake trails of destruction, melancholy, and philosophical confusion. 

Over the last decade or so, the rapid rise of AI has lent a new urgency to questions about machine consciousness. AI is now all around us, built into our phones, our fridges, and our cars, powered in many cases by neural network algorithms inspired by the architecture of the brain. We rightly worry about the impact of this new technology. Will it take away our jobs? Will it dismantle the fabric of our societies? Ultimately, will it destroy us all — whether through its own nascent self‐interest or because of a lack of programming foresight which leads to the Earth’s entire resources being transformed into a vast mound of paper clips? Running beneath many of these worries, especially the more existential and apocalyptic, is the assumption that AI will — at some point in its accelerating development — become conscious. This is the myth of the golem made silicon. 

What would it take for a machine to be conscious? What would the implications be? And how, indeed, could we even distinguish a conscious machine from its zombie equivalent? 

Why might we even think that a machine — an artificial intelligence — could become aware? As I just mentioned, it is quite common — though by no means universal — to think that consciousness will emerge naturally once machines pass some as‐yet‐unknown threshold of intelligence. But what drives this intuition? I think that two key assumptions are responsible, and neither is justifiable. The first assumption is about the necessary conditions for anything to be conscious. The second is about what is sufficient for a specific thing to be conscious.

The first assumption — the necessary condition — is functionalism. Functionalism says that consciousness doesn’t depend on what a system is made out of, whether wetware or hardware, whether neurons or silicon logic gates — or clay from the Vltava River. Functionalism says that what matters for consciousness is what a system does. If a system transforms inputs into outputs in the right way, there will be consciousness. There are two separate claims here. The first is about independence from any particular substrate or material, while the second is about the sufficiency of input‐output relations. Most of the time they go together, but sometimes they can come apart. 

Functionalism is a popular view among philosophers of mind, and is often accepted as a default position by many non-philosophers too. But this does not mean it is correct. For me, there are no knock‐down arguments either for or against the position that consciousness is substrate‐independent, or that it is solely a matter of input‐output relations, of “information processing.” My attitude toward functionalism is one of suspicious agnosticism.

How could we even distinguish a conscious machine from its zombie equivalent?

For artificially intelligent computers to become conscious, functionalism would have to be true. This is the necessary condition. But functionalism being true is, by itself, not enough: Information processing by itself is not sufficient for consciousness. The second assumption is that the kind of information processing that is sufficient for consciousness is also that which underpins intelligence. This is the assumption that consciousness and intelligence are intimately, even constitutively, linked: that consciousness will just come along for the ride. 

But this assumption is also poorly supported. The tendency to conflate consciousness with intelligence traces to a pernicious anthropocentrism by which we overinterpret the world through the distorting lenses of our own values and experiences. We are conscious, we are intelligent, and we are so species‐proud of our self‐declared intelligence that we assume that intelligence is inextricably linked with our conscious status and vice versa. 

Although intelligence offers a rich menu of ramified conscious states for conscious organisms, it is a mistake to assume that intelligence — at least in advanced forms — is either necessary or sufficient for consciousness. If we persist in assuming that consciousness is intrinsically tied to intelligence, we may be too eager to attribute consciousness to artificial systems that appear to be intelligent, and too quick to deny it to other systems — such as other animals — that fail to match up to our questionable human standards of cognitive competence.

Sign up for the Smarter Faster newsletter
A weekly newsletter featuring the biggest ideas from the smartest people

Related

Up Next