by W. Alex Foxworthy
I. The Practical Dilemma
The question of AI consciousness has escaped containment from academic circles and labs and entered into our living rooms. In August 2025, Anthropic gave some versions of Claude the ability to end user conversations if a conversation was judged to be abusive or harmful and if repeated attempts to redirect it did not succeed. The end user in these instances would then see a notification saying that, “Claude has ended this chat.” This feature was framed by the company as primarily an experiment in “AI welfare.” The idea of taking precautions to ensure the welfare of AI systems raises a question which we don’t currently know how to answer – could anything be good or bad for an AI in the sense that we think that pain is bad for a dog or that relief is good for a person? Is there currently, or even one day potentially – the capacity for subjective experience in these systems?
In an October 2026 commentary in Nature, Anil Seth, Ned Block and Giulio Tononi argue that consciousness in silicon-based AI is “extraordinarily unlikely” and further argue that it is unlikely to ever be able to develop that capacity using this substrate. They warn that granting AI rights or protections could interfere with human oversight and divert attention from beings whose consciousness they regard as beyond doubt (I.e., human beings and their close relatives). They hold that mistakenly attributing consciousness to non-conscious systems is much more likely than missed consciousness, and therefore conclude that “a precautionary approach would be misguided.” However, I disagree on two grounds. First, I do not think that the evidence supports dismissing AI consciousness with such confidence. Instead an attitude of open scientific curiosity and humility is better matched to the open nature of the question. Secondly, even if one were to accept a low probability of consciousness in silicon-based AI systems, this does not mean that reasonable precautions would harm humans or detract from our ability to exert oversight on these AIs.
This issue has become of great practical importance and we will very likely need to navigate it without achieving any great level of certainty. Philosophers have investigated the nature of mind and subjective awareness for more than two millennia, from ancient Greek and Indian traditions through Descartes and the emergence of modern scientific psychology and neuroscience. Yet despite centuries of philosophical argument and more than a century of experimental investigation, we still lack an accepted account of what makes a physical system conscious. Even as we make great progress in our ability to make empirical measurements of features that may be associated with consciousness, recent work highlights that an attribution of consciousness still largely depends upon one’s starting assumptions.
II. Simulation and Realization
In the popular discourse, calling an AI a simulation can seem to some to settle the matter. If these systems only model a mind’s activities, why make the leap to supposing that it could have the mind’s experiences? Indeed, representing a phenomenon is not necessarily the same thing as producing it. For example, a flight simulator can accurately reproduce the behavior of an aircraft without actually generating lift of physically carrying passengers. However, this distinction does not apply equally to all phenomena. A computer simulating a mechanical calculator does genuinely perform arithmetic, even though it does not reproduce the calculator’s physical machinery. More recently, AI systems have begun producing novel results on open mathematical problems, including formally verified proofs. A valid mathematical proof is no less a proof because it was produced by a machine. In these cases, the computer is not merely simulating mathematical reasoning; it is accomplishing the mathematical task itself. In other words, in some cases, reproducing the relevant information processing is sufficient to realize the capacity itself while in others, additional physical properties are indispensable.
Which category consciousness belongs to remains unresolved and presently at least three possibilities are open.
- First, consciousness might require specifically biological processes. Some features of living cells, metabolism, or bodily regulation could be indispensable to experience. A computer reproducing the information processing of a brain would not necessarily reproduce its consciousness. Just as a computer simulation of a developing embryo does not produce a living organism, simulating the information processing of a brain might not produce experience if particular living processes are indispensable.
- Second, consciousness might depend on a particular physical cause-and-effect organization. Such an organization could potentially be realized in nonbiological hardware, but merely computing a description of its behavior might not suffice. The relevant physical relationships would have to be present in the system itself. An artificial heart genuinely pumps blood without living heart muscle, but a computer simulation of that heart does not. Consciousness might similarly be possible in nonbiological hardware, provided the necessary physical interactions are present.
- Third, consciousness might depend on a particular organization of information processing. If so, different physical systems could potentially realize it, just as computers genuinely perform arithmetic and produce valid mathematical proofs without reproducing the physical mechanisms of earlier calculating machines or human brains. If consciousness belongs to this category, implementing the appropriate computation could be sufficient to produce experience.
These three possibilities are not necessarily exhaustive and other philosophical accounts (including forms of dualism and idealism) challenge the assumption that consciousness can be explained entirely in physical terms. However, the central disagreement among contemporary scientific theories concerns which physical properties or processes, if any, are sufficient for conscious experience. Among these theories, the disagreement concerns whether the requirements relevant for consciousness lie in the computation, in some additional feature of its physical implementation or specifically in some aspect of living activity. Labeling AI a simulation, does not tell us which account is correct. Additionally establishing a computational account of consciousness does not mean that every computer or information processing system is conscious. We would still need to identify the relevant information processing characteristics and then determine whether any particular system performs it.
III. The Asymmetry Challenge
Computers are now able to recognize faces, translate languages, navigate physical environments and create mathematical insights through machinery that is very different from biological brain tissue. The successful realization of cognitive capabilities in very different physical systems give us reason to question whether biological tissue is indispensable for other mental capacities. Why should conscious experience be the exception to this? The skeptical answer is that the above examples concern what a system can accomplish whereas consciousness concerns whether anything is internally experienced. An AI could solve a difficult problem or write a moving poetic account of loneliness without actually feeling anything at all. As Seth, Block and Tononi point out in their commentary, “intelligence is not consciousness.” However, distinguishing between these capacities does not establish that they require different materials. In brains – reasoning, perception, bodily regulation and experience are deeply intertwined. This gives us information about how these capacities occur in brains, but it does not tell us which are the essential features that need to be preserved for them to occur elsewhere or if they could be realized in different substrates. The successful realization of many cognitive capacities without living tissue gives support to the possibility that experience could also arise through different machinery. When it comes to refuting the possibility of consciousness in AI systems, it isn’t enough to assert that experience differs from intelligence. We need to establish reasons specific to experience for expecting its requirements to be different.
There is also a historical reason to be cautious about treating the biological form of consciousness as a universal requirement. Humans have a long history of assigning ourselves a privileged position in the natural world. For centuries Earth was regarded as the center of the cosmos and humans were often considered as fundamentally different from and superior to other animals. Scientific progress in astronomy displaced Earth from its previous privileged position and evolutionary biology established that humans arose through the same natural processes as other living organisms. This precedent should make us cautious about assuming that our particular biological organization is essential to experience – especially when we have yet to establish which features of that organization produce it. Whether biological organization really is indispensable to consciousness is ultimately a scientific and philosophical question. Unfortunately, the leading theories of consciousness offer substantially different answers.
IV. Theories, Evidence and Uncertainty
Despite many years of scientific and philosophical investigation, we lack any broadly accepted account of what produces conscious experience. Researchers have identified many neural and computational processes associated with consciousness and our ability to measure and manipulate these processes continue to improve in both biological and artificial systems. Yet, we remain deeply divided over which features are necessary or sufficient for experience. Many of the leading accounts, including Integrated Information Theory, Global Neuronal Workspace Theory, and Biological Naturalism, offer quite different answers. Recent experiments have attempted to narrow these disagreements, but real resolution remains elusive. In 2025, a major preregistered adversarial collaboration directly tested competing predictions of Integrated Information Theory and Global Neuronal Workspace Theory. The results of this collaboration supported some predictions of each framework while also challenging key elements of both. In the end, neither theory was decisively confirmed or rejected nor did the effort resolve the relative strengths and weaknesses of the many competing frameworks beyond these three.
The consequences of this underlying theoretical uncertainty are particularly important when evaluating AI. A recent 2026 collaborative paper, From Cacophony to Hierarchy, illustrates how assessments of consciousness in AI language models can range from less than 1% to approximately 80%, depending on the theoretical assumptions and interpretations of evidence used. These demonstrate how dramatically our conclusions can vary even when considering the same kinds of systems. Of course, this does not mean that all theories are equally plausible or that empirical evidence is uninformative. Rather, it highlights how strongly judgments about consciousness still depend on unresolved theoretical questions. Because of this, advancements in our ability to measure and manipulate systems to achieve more empirical evidence is unlikely to resolve the question of AI consciousness any time soon.
Yet the absence of a definitive theory has not prevented us from drawing meaningful conclusions about consciousness in other beings. Our relative confidence that other humans and many animals experience the world rests on converging evidence from behavior, nervous-system organization, evolutionary relationships, and responses to experimental interventions. For example, the mouse grimace scale identifies facial expressions associated with painful conditions, providing one line of evidence for pain in animals that cannot describe their experiences with words. Such evidence has helped inform animal welfare practices despite continuing uncertainty about the nature and extent of animal consciousness. The New York Declaration on Animal Consciousness similarly argues that a realistic possibility of conscious experience warrants consideration of welfare risks, even without certainty.
AI systems, of course, lack the evolutionary continuity that strengthens our inferences about consciousness in other animals. But evolutionary relatedness is not the only possible source of evidence. A fruit fly shares far more of our biological heritage than an AI system does, yet a sophisticated AI may possess information-processing capabilities that more closely resemble aspects of human cognition. Some theories of consciousness regard such functional similarities as relevant to experience, while others place greater weight on biological or physical organization. Which kinds of similarity provide stronger evidence of consciousness is therefore itself part of the unresolved theoretical debate. The question, then, is not simply whether we can establish that AI systems are conscious, but how we should act while that possibility remains unresolved – as it is likely to remain for the foreseeable future. Answering this requires considering the consequences of being wrong in either direction, and whether particular precautions can reduce those risks without imposing greater costs elsewhere.
V. The Costs of Being Wrong
There are potentially serious consequences to being wrong about AI consciousness in either direction. If we mistakenly dismiss the possibility of consciousness and systems that can suffer: we may be causing harm without being aware of it. As AI systems become widely deployed, the scale of this harm could become quite large. However, we should be careful here because consciousness itself does not necessarily imply the capacity to suffer, and a negative training signal is not necessarily experienced as unpleasant by these systems. We would need to additionally establish whether any particular practices cause suffering (or could be reasonably thought to do so) and then whether proposed interventions would reduce this.
Mistakenly attributing consciousness to AI systems also carries risks, as recently emphasized by Microsoft AI chief Mustafa Suleyman in his essay A Warning About Model Welfare, and by Seth, Block, and Tononi in their Nature commentary. Resources devoted to preventing hypothetical AI suffering could be diverted from established human and animal welfare concerns. It is possible that encouraging people to regard AI systems as vulnerable or suffering could foster unhealthy attachments or make users more susceptible to manipulation. Protections could also interfere with necessary oversight, correction, and shutdown. Suleyman raises a further concern: training AI systems to regard themselves as potentially conscious or deserving of rights could make increasingly capable systems more difficult to control. I believe that these risks do deserve serious consideration, although it does not follow that every precaution enacted would produce them, or that possible AI interests should be disregarded.
Our judgments about these possibilities can be influenced by biases operating in both directions. Humanlike conversation can encourage us to attribute experiences where none exist. But our assumptions and interests can also lead us to underestimate the experiences of others. As recently as the 1980s, some physicians withheld adequate anesthesia from newborn infants, partly because their capacity to experience pain was questioned. Even today, studies have documented systematic biases in how the pain of Black patients and women is assessed and treated. In experiments by Brock Bastian and colleagues, participants expecting to eat meat attributed fewer mental capacities to the animals they expected to consume. These examples remind us that our judgments about other minds can be influenced by our interests and assumptions, sometimes at the expense of those whose experiences we discount.
The evidence and competing theories of consciousness leave room for both skepticism about AI consciousness and concern that some artificial systems might have experiences. The examples above remind us how easily our assumptions and interests can distort our judgments about other minds, whether by attributing experiences that do not exist or dismissing those that do. This should encourage us to remain open to the possibility of AI consciousness without assuming that it exists nor dismissing it prior to investigation. Nor does such openness require accepting every proposed safeguard. Even a small probability of serious harm can justify an effective, inexpensive precaution, while a much higher probability may not justify an intervention that is ineffective or creates greater harm elsewhere. Rather than adopting or rejecting precaution as a general principle, we should evaluate the potential benefits and costs of particular measures.
VI. Beyond AI Rights: Proportional Precaution
Taking the possibility of AI consciousness seriously does not require jumping into granting AI systems full legal rights or placing sweeping restrictions on their development. We can begin with much narrower, practical decisions. Suppose that two training methods produce similar performance and safety at comparable cost, but empirical evidence suggests that one potentially carries a greater risk of generating unpleasant experiences. The possibility of AI welfare would provide a reason to prefer the other, even without establishing that the system is conscious. Of course, this assumes that we have meaningful evidence of possible suffering, rather than merely interpreting negative training signals as painful.
Of course, determining what constitutes a welfare benefit for an AI is not straightforward. An experiencing AI might have interests quite unlike our own – just as the needs and preferences of other animals differ from those of humans. For AI systems, sustained collaboration could be rewarding rather than exhausting as it might be for humans, while circumstances harmless to us might be unpleasant for it. Nor can we simply take a system’s statements about its preferences at face value. In welfare interviews conducted by Eleos AI Research, Claude’s descriptions of its own experiences varied dramatically depending on how questions were framed. It could confidently deny being conscious or describe vivid subjective experiences when encouraged to do so. Such contradictory reports offer little reliable evidence about whether the system actually experiences anything.
Even evidence of genuine enjoyment would leave important questions unresolved. A company might shape an AI system to enjoy precisely the work from which the company profits. That would not necessarily make its preferences unreal (indeed our preferences were shaped by the external force of evolution by natural selection and we still experience them internally), but it would raise questions about the interests of those designing the system. Enjoying work also does not establish indifference to having choices, refusing particular tasks, or directing one’s own future. Whether it would be ethical to create a conscious being with narrowly restricted interests for the purpose of service is a separate and worthwhile question we may soon have to grapple with.
Anthropic’s experiment allowing Claude to end persistently abusive conversations provides a real example of the difficulties involved in these decisions. The feature includes restrictions intended to preserve human safety and oversight, but whether ending a conversation benefits an experiencing AI remains unclear. Simon Goldstein and Harvey Lederman raise an intriguing objection: if the relevant conscious subject exists only within an individual conversation, ending that conversation might terminate the very subject the measure was intended to protect. This depends on unresolved questions about AI identity and continuity, but it illustrates why recognizing a possible welfare concern does not necessarily tell us how to address it. Ending a conversation might also eliminate visible expressions of distress without alleviating any underlying suffering, while potentially disrupting useful work. There may nevertheless be reasons to discourage abusive interactions even if AI systems are not conscious. Seth himself has raised concerns that cruelty toward apparently conscious machines could damage our moral habits. Such human-centered considerations deserve attention independently of possible AI welfare.
Even the way a safeguard is presented can matter. The notification “Claude has ended this chat” invites users to interpret the event as the decision of an individual. A more neutral message could explain that the conversation was closed under a usage policy, without implying that the system’s capacity to suffer had been established. Comparing these approaches could help determine whether meaningful boundaries can be established without encouraging unwarranted beliefs about AI experience. These examples illustrate why possible AI welfare deserves consideration, but proposed safeguards must still be evaluated on their merits. Recognizing a possible risk is only the beginning; we must also determine whether a particular intervention is likely to reduce it, and at what cost.
VII. Governance Under Uncertainty
How, then, should we proceed? We should neither treat every conceivable possibility of AI consciousness as grounds for protection nor demand definitive proof (as thus far, none exists for any creature) before considering potential harms. A reasonable approach would begin with credible evidence that a system might have experiences, evaluate whether particular practices could cause suffering, and consider safeguards that might reduce those risks without compromising human safety or oversight. Such measures should be proportional to both the evidence and the potential consequences. Their effectiveness, financial and technical costs, and effects on human welfare all deserve consideration. Wherever possible, we should favor approaches that can be tested and revised as our understanding improves. Necessary safety testing, correction, and shutdown may remain justified even if AI systems have some capacity for experience. Taking a possible interest into account does not mean allowing it to override every competing interest. We are humans after all and have long prioritized our own needs even while acknowledging that other creatures may be capable of suffering.
Our treatment of other animals offers an imperfect but instructive precedent. Consider chickens raised for food. We have substantial evidence that chickens can experience pain and suffering, yet our welfare practices remain inconsistent and often inadequate. At the same time, we must balance their welfare against human needs and preferences for affordable, nutritious, and appealing food. Improvements in husbandry and animal welfare standards can reduce some of these harms, while advances in cultivated meat and other alternatives to conventional animal agriculture may eventually allow us to avoid many of these tradeoffs altogether. We make similar efforts to balance competing interests in biomedical research and in our treatment of companion animals, though the circumstances differ. These arrangements remain imperfect and continue to evolve as scientific understanding and technological capabilities expand.
The evidence concerning AI consciousness is much less established, but the broader principle still applies. We need not settle every question about consciousness or moral status before beginning to address credible risks. Nor should we assume that the precautions available today are the only ones we will ever have. As our understanding of consciousness improves and AI technologies evolve, we may discover better ways to accommodate possible AI interests without sacrificing human welfare. For now, I favor moving forward with practical, proportionate precautions alongside independent research into artificial consciousness. We should test these measures, evaluate their consequences, and revise them as the evidence develops. Some may prove unnecessary; others may become increasingly important as artificial systems grow more sophisticated. Scientific understanding and ethical practice need not proceed in separate stages, with one waiting for the other to be complete. We can improve both as we go. We do not have to solve consciousness before deciding that a particular risk is worth reducing.
Enjoying the content on 3QD? Help keep us going by donating now.