If an AI Tells You It's Conscious, Should You Believe It? | pt. 3 of 3
Addressing the Punting to Panpsychism Argument for Machine Consciousness
If an AI tells you it’s conscious, you probably shouldn’t believe it, or rather, you might not be justified in attributing consciousness to it. Why think that? Well, the arguments for machine consciousness fail and thus leave us with no good reasons to affirm that a given machine is conscious.
Now the ‘consciousness’ I have in mind here is ‘phenomenal consciousness’, that is, that what-it’s-likeness of consciousness. If an AI were conscious in the phenomenal sense, then there would be something-it’s-like to be that AI. Currently, there's no way to directly experience the qualitative experiences of minds other than our own, of ‘other minds’—and in fact, many think it’s just not possible to experience the qualitative experiences of someone else (I’m one of those many).
So, how do we know that anyone else has a mind like ours? That any other minds are phenomenally conscious? Well, we make arguments for the existence of other minds. The same goes for AI. How could we know that an AI was conscious? We make similar arguments to those made in favor of the existence of other human minds and animal minds. Here are the three major ways to argue for conscious machines:
Way1: drawing an analogy between the human cognitive processes and/or framework, and an AI’s organizational structure, and then concluding that the AI is conscious like us.
Way2: inferring that phenomenal consciousness best explains the behavior of an AI.
Way3: Punting to Panpsychism, which is to say, positing panpsychism (the family of theories that claims all things are conscious) and getting machine consciousness for free. If everything is conscious, then AIs are conscious too.
You’re reading pt. 3 of 3 on this question of attributing consciousness to machines. If you haven’t read pt.1 yet, you should, you need the set up. In pt.1 I take on Way1. You can check that out here:
If you haven’t read pt. 2 where I take on Way2, then you should also read that one as well:
The details of these 3 posts come from my article in the Journal of Artificial Intelligence and Consciousness. If you want to read the full thing with more of the technical jargon, less of the colloquial jargon, and all of the footnotes, you can find the penultimate version of my paper on my philpeople page here.
In pt. 3 I’ll look at Way3 and I’ll explain why it too is no good.
Reminder: you can support my mission to promote the life of the mind and get tons of members-only content for a few dollars a month:
full-length notebook philosophy essays/tutorials where I share in-depth instructions on how to use various notebook methods and the philosophers who inspired each method
members-only access to For the Love of Wisdom entries (my digital intro to philosophy text book)
members-only access to Sayings of the Sages, my digital commonplace book of wise sayings I’ve collected over my years of studying philosophy and culture
access to our Zoom Book Club sessions where we discuss the read-along books and my companion essays
the full catalog of members-only essays + lecture videos
access to the members only chat
and lots more!
Punting to Panpsychism
The final method of seeking to justify an attribution of consciousness to AGIs I call “punting to panpsychism”. Proponents of this method quickly punt to panpsychism as the justification for attributing consciousness to machines. They argue something like the following:
Punting to Panpsychism Argument
1. If panpsychism is true, then everything already is conscious.
2. Panpsychism is true.
Therefore,
3. When we accomplish AGI, it will be conscious too.
But this argument moves too fast. The core idea which demarcates panpsychism from other theories of mind is something like “whatever is metaphysically fundamental is conscious”. But there are many disparate ways in which this core idea can be fleshed out, and they all raise their own unique challenges for machine consciousness.
Punting to panpsychism is not enough to justify machine consciousness on its own because one’s panpsychism will be shaped by one’s larger metaphysical picture, namely one’s fundamental mereology.[a]
Ross Inman provides us with three fundamental mereological paradigms which I will argue give rise to disparate panpsychisms:
i. Priority Monism: the maximal mereological whole, the cosmos, is a fundamental substance and is metaphysically prior to its proper parts [Inman, 2018].
ii. Substantial Priority: there are intermediate composite objects in the category of substance [Inman, 2018].
iii. Priority Microphysicalism: the mircrophysical parts of composite wholes are fundamental substances and are metaphysically prior to their wholes [Inman, 2018].[b]
So what one takes to be fundamental will shape what a panpsychist takes to be conscious and will in turn have its own benefits and its own unique problems for that flavor of panpsychism.
A priority microphsyicalist picture would have the smallest microphysical entities of the universe, call them ‘beebees’, be most fundamental and hence these beebees would be conscious.[c] Panpsychist philosopher, Philip Goff, calls this priority microphsicalist panpsychism “smallist panpsychism” [Goff, 2019].
While smallist panpsychism may have its virtues, when you put consciousness down at the bottom of the picture you run into a difficult problem known as the combination problem, i.e., how do we explain singular, unified consciousness at the macro level of human persons if we are a conglomerate of an untold number of conscious beebees?
If smallist panpsychism were true, it seems like we should have an untold number of distinct centers of consciousness rather than one unified center of consciousness that I call ‘mine’ or ‘me’. This is a difficult problem for the smallist panpsychist but it becomes an even more difficult and unique problem for the machine consciousness smallist panpsychist, a problem I will call the AGI combination problem:
AGI Combination Problem: even if we could solve the combination problem for human beings, why think that consciousness would combine into a unified center of conscious awareness in an AGI system?
Now the machine consciousness attributer will have to give an argument for why an AGI is relevantly similar to a human cognitive agent which pushes them right back to the failed SOS argument and the failed Machine Consciousness IBE argument (the arguments from pt.1 and pt.2 of this series, respectively.)
Priority monist panpsychism, which Goff calls “constitutive cosmopsychism”—or cosmopsychism for short—on the other hand, will have an inverse problem to that of smallist panpsychism.
Since the cosmopsychist takes the whole cosmos to be fundamental, their panpsychism will see the whole cosmos as being conscious. Thus, their difficulty will be in explaining the decombination of cosmos-wide consciousness rather than its combination from the beebees up. That is, if the universe as a whole is conscious, why is it that I seem to have my own unique center of consciousness, distinct from the rest of the cosmos? This decombination problem is as difficult for the cosmopsychist as the combination problem for the smallist panpsychist, but when applied to machine consciousness it becomes even more difficult. I will call this the AGI decombination problem:
The AGI Decombination Problem: the problem of individuating human level consciousness from the whole conscious-cosmos, and further explaining why we should think an AGI system likewise has an individuated conscious experience.
Now we know that human cognitive agents are conscious, so the decombination problem is a bit easier in the human case: we have the phenomena of individuated subjective consciousness, e.g. ‘mine’, and now the cosmopsychist needs to explain it given a priority monist mereology.
But the AGI Decominbination Problem is even harder because we don’t have the initial phenomena to work with—we don’t know that any AGIs are in fact conscious, whereas in the human case, at least we know that we are conscious.
In the case of machine consciousness on cosmopsychism, we’re not just trying to solve the decombination problem about consciousness simpliciter, we’d need to solve the decombination problem in order to motivate machine consciousness on this cosmopsychist theory of mind in order to justify an attribution of consciousness to an AGI system.
Now perhaps the decombination problem is soluble, but even if the cosmopsychists can produce a plausible solution for decombination in the human case, we’d still need another reason to think that an AGI system would likewise be a candidate for an individuated center of consciousness. So here, cosmopsychism is no quick solution for the machine consciousness proponent.
On a substantial priority mereology, the panpsychist would claim that all substances are conscious and that substances can be found at multiple (maybe all) levels of granularity.
The biggest obstacle to this view is to explain why an artefact like an AGI system should count as a substance and not merely an ordered aggregate.
The AGI system is by definition not a natural kind (artificial intelligence). It doesn’t appear to have inseparable parts.[d] And it’s not clear what the conscious substance of the AGI would even be. Should we think of the whole system as a conscious substance, or just the parts essential for ‘thinking’?
In his paper “The Metaphysics of Artificial Intelligence”, Eric Olson argues that in AI conversations, too much of the time and energy are given to explicating what artificial thought consists in but rarely is any time given to getting clear on what an artificial thinker would consist in [Olson, 2019].[e] To remedy this fact, Olson gives a taxonomy of potential views on what might count as the artificial thinker in an AI system but he also raises significant problems for each view.
First, Olson considers the Computer-Hardware view, wherein the physical computer hardware itself is the conscious subject of artificial thought. But he argues that this view conflicts with widely held views about persistence conditions. The artificial thinker and the computer would have two different histories and it’s hard to tell if the thinker survives being shut off or whether it’s destroyed. Yet the computer survives being shut off. We could imagine transferring the data to a new computer and turning it on, such that the new computer is conscious and the old one is not. These objections seem to suggest that the computer is not the conscious subject of artificial thought.
Olson considers a variant view called Temporal-Parts, which utilizes unrestricted composition and persistence through arbitrary temporal parts. On this view, the subject of artificial thought is wholly present at various temporal stages of physical computers while the AGI program is running on them. The artificial thinker is then comprised of its various temporal parts. But on this view, the computer gains the property of consciousness and then loses it when the data or program is transferred to a new computer, thus there’s a too many thinkers problem and the subject wouldn’t know whether it’s the one perishing or being transferred to the next computer’s temporal part.
Olson then considers the Constitution View. Here the thinker is not the computer itself but is instead constituted by it. Both the computer and the thinker share the same material, but have different modal properties and histories. Olson argues that this view isn’t attractive to AGI theorists because it violates the weak supervenience principle which makes the computer a philosophical zombie while there is something-it’s-like to be the artificial thinker, even though they share the same physical and spatial properties. If not, then there are two thinkers occupying the same physical space, which seems like a cost. If we reject these views for natural thinkers like ourselves, then we ought to reject the constitution view as well. A further problem for all of the ‘materialist’ views thus considered is determining the spatial boundaries of the artificial thinker. In cases of natural intelligence, one could argue that the boundaries are determined by everything that is caught up in the life of the organism. But this option is not open to the AGI theorist.
Olson considers two immaterialist views: the Program View and the Bundle View. On the Program View, the artificial thinker is the program ‘type’ rather than a particular program ‘token’. Olson immediately demurs that this is highly implausible since the thinker would be abstract and would come into being as soon as the program instructions were complete (in eternity past?). This would mean that the thinker would exist absent a computer. He further objects that as a type, the thinker would be multiply instantiable or realizable and thus the same thinker could have contradictory properties at the same time on two different computers. According to The Bundle View, the thinker is a token instance of the thinker-type, running on a particular computer, and thus, a bundle of electronic states and events. But Olson objects that on this view, the computer is not doing any thinking, contrary to what AGI engineers usually claim of their systems, and that it substitutes the subject of thoughts with more thoughts. Thoughts are somehow supposed to be the subjects of artificial thought and no account of an artificial thinker is given.
Lastly, Olson considers what he calls the Relaxed Attitude view, whereby there is no true thinker of artificial thought. The instrumentalist or antirealist who holds this view is concerned with the usefulness of attributing mentality to computers rather than considering the ontology at play, which Olson proposes as an objection in and of itself against the view.
The AGI theorist looking to attribute machine consciousness to an AGI system through appropriating a substantial priority panpsychism will need to grapple with Olson’s taxonomy of artificial thinkers and their accompanying obstacles. The AGI theorist will need to get clear on why we ought to consider the AGI system a substance with its own center of consciousness rather than a mere ordered aggregate comprised of conscious substances, and she should also be able to tell us which part or parts of the AGI system actually count as the substantial subject of consciousness.
Now perhaps satisfactory answers to these questions are forthcoming, but the point stands that punting to panpsychism is not a sufficient reply to the questions of machine consciousness. A lot more needs to be said and it’s not clear that panpsychist solutions are any more plausible than any other theory of mind.
Conclusion
While many impressive advancements in AI are rapidly arriving by the day, I’ve given a survey of various philosophical problems facing the justified attribution of consciousness to machines, such that even if a conscious AGI were to arrive anytime soon, it’s not clear that we would be justified in identifying it as conscious.
This is an ethically precarious predicament in that we may end up creating a phenomenally conscious being without being able to identify it as such.
Further areas of research could include work on evolutionary program based AGI and the application of Schneider’s ACT as well as fleshing out the possible ethical ramifications of creating phenomenally conscious artificial agents of which phenomenal consciousness cannot be justifiably attributed to. More work should be done at the intersection of panpsychism and machine consciousness since both subdisciplines are enjoying a massive spike in interest. And perhaps work can be done to determine principled solutions to the AGI combination and decombination problems I raised in this paper.
Alright, so that wraps up my publicly accessible version of my paper on the difficulties facing the attribution of consciousness to machines. If you enjoy this kind of philosophy, please consider upgrading to a paid subscription and help me do more of it.
Additionally, if you want to buy me a coffee, you can also do that here:
Thanks!
References
Boden, Margaret [2016] Artificial Intelligence: A Very Short Introduction
(Oxford: Oxford University Press).
Block Ned, [1978] “Troubles with Functionalism” in Chalmers, David [2021]
Philosophy of Mind: Classical and Contemporary Readings 2nd ed. (Oxford University Press)
Chalmers, David [2022] draft of “Could a Large Language Model Be
Consious?” This is an edited transcript of a talk given in the opening session at the NeurIPS conference in New Orleans on November 28, 2022, with some minor additions and subtractions. Video is at https://nips.cc/virtual/2022/invited- talk/55867. Earlier versions were given at the University of Adelaide, Deepmind, and NYU.
Cutter, Brian [Forthcoming] “The AI Ensoulment Hypothesis” in Faith and
Philosophy.
Descartes, René [2006] A Discourse on the Method (New York: Oxford
University Press).
Goff, Philip [2019] “Did the Universe Design Itself?” in the International
Journal for Philosophy of Religion (2019) 85:99-122.
Goertzel, Ben [2023] OpenCog Hyperon: A Framework for AGI at the
Human Level and Beyond https://arxiv.org/abs/2310.18318
Huemer Michael [forthcoming] “Dualism and the Problem of Other Minds”
Inman, Ross [2018] Substance and the Fundamentality of the Familiar: A
Neo-Aristotelian Mereology (New York: Routledge).
Kim Jaegwon [2011] Philosophy of Mind 3rd ed. (Boulder Co.: Westview
Press).
Leibniz, G.W. [2005] The Monadology (Dover Publications Inc., Mineola, NY).
Mitchell, Melanie [2019] Artificial Intelligence: A Guide for Thinking
Humans. (New York: Picador)
Olson, Eric [2019] “The Metaphysics of Artificial Intelligence” in Mihretu P.
Guta [2019] Consciousness and the Ontology of Properties (New York: Routledge).
Putnam, H. [1967] Psychological Predicatses, in W.H. Captain and D.D. Merril (ed.) Art, Mind,
and Religion (University of Pittsburg Press, Pittsburg, PA.), pg. 37-48.
Rickabaugh, B. [2020] Dissertation: The Conscious Mind Unified.
Schneider, Susan [2019] Artificial You: AI and the Future of Your Mind
(Princeton: Princeton University Press)
Searle, J. [2003] Minds, Brains, and Science (Harvard University Press, Cambridge).
[a] For a detailed analysis and critique of panpsychism along mereological lines, see Brandon Rickabaugh [2020]
[b] Inman also lists Priority macrophysicalism but I’ve elided it in for the sake of clarity over comprehensiveness.
[c] Panprotopsychisms or protopanpsychisms would describe the beebees as being or involving some form of proto-consciousness and would limit full-blown consciousness to macrolevel entities like dogs, giraffes, and humans, etc.
[d] For an analysis of Aristotelian substances vs. ordered aggregates and the distinction between seperable vs. inseperable parts, see Inman [2018], c.f. Rickabaugh [2020].
[e] I’m indebted to Brandon Rickabaugh for bringing Olson’s paper to my attention as he taught through it in his philosophy of technology course at Palm Beach Atlantic University.