Time to be a doomsayer! If technology continues on its current trajectory, we will soon be facing a moral catastrophe. We will create AI systems that some people reasonably regard as deserving human or humanlike rights. But there won’t be consensus on this view. Other people will reasonably regard these systems as wholly undeserving of human or humanlike rights. Given the uncertainties of both moral theory and theories about AI consciousness, it is virtually impossible that our policies and free choices will accurately track the real moral status of the AI systems we create. We will either seriously overattribute or seriously underattribute rights to AI systems—quite possibly both, in different ways. Either error will have grave moral consequences, likely at a large scale. The magnitude of the catastrophe could potentially rival that of a world war or major genocide.
Does this sound too extreme? I hope it is too extreme. Let me walk you through my thinking.
(1.) Legitimate calls for robot rights will soon be upon us. We’ve already seen the beginnings of this. There already is a robot rights movement. There already is a society for the ethical treatment of reinforcement learners. These are currently small movements, but they leapt to mainstream attention in June when Google engineer Blake Lemoine made international headlines for claiming to have decided that the large language model LaMDA was sentient, after having an extended philosophical conversation with it. Although few researchers agree that LaMDA is actually sentient in any significant way, exactly what it lacks is unclear. On some mainstream theories of consciousness, we are already on the verge of creating genuinely conscious/sentient systems. If we do create such systems, and if they have verbal or verbal-seeming outputs that appear to be pleas for rights—requests not to be abused, not to be deleted, not to be made to do certain things—then reasonable people who favor liberal theories of AI consciousness will understandably be inclined to respect those pleas.
One might think it plausible that the first rights-deserving AI systems would warrant rights (or more broadly, moral consideration if “rights” language sounds too strong) similar to the rights normally accorded to vertebrates. For example, one might think that they would deserve not to be needlessly killed/deleted or made to suffer, but that human interests can easily outweigh their interests, as (it’s common to think) the interests of non-human vertebrates can be outweighed by human interests in meat production and scientific testing.
However, I suspect that if such minimal rights or moral consideration were granted to AI systems, that would not be a consensus solution. After all, some of these AI systems will presumably produce verbal outputs that friends of AI sentience will regard as signals that they have sophisticated long-term goals, an understanding of their position in the world, and an ability to enter into discussions with human beings as peers. Under those conditions, many will presumably think that merely animal-level rights are insufficient, and something closer to equal rights are required—human rights or human-like rights.
It is perhaps worth noting that a substantial portion of younger respondents to a recent survey find it plausible that future robots will deserve rights.
(Note: I see “robot rights” as a subcase of “AI rights,” given that robots are a subclass of AI systems, specifically, those with bodies. On some theories, embodiment is a necessary condition for consciousness. Also, an AI system with an appealing body might tend to draw higher levels of concern than a non-embodied AI system. So it’s not implausible that the first AI systems to draw wide consideration for serious rights will be robotic systems. “Robot rights” is also a more familiar term than “AI rights.” Hence my use of the phrase.)
(2.) These legitimate calls for robot rights will be legitimately contestable. For three reasons, it’s extremely unlikely that there will be a consensus on what rights, if any, to give to AI systems.
(2a.) There will continue to be widespread disagreement about under what conditions, if ever, an AI system could be conscious. On some mainstream theories of consciousness, consciousness requires complex biological processes. Other theories require a specifically human-like cognitive architecture that even complex vertebrates don’t fully share with us. Currently, the range of respectable theories in consciousness science runs all the way from panpsychist or nearly-panpsychist theories in which everything or nearly everything is conscious, to extremely restrictive theories on which consciousness requires highly advanced capacities that are restricted to humans and our nearest relatives, with virtually no near-term prospect of AI consciousness. The chance of near-term consensus on a general theory of consciousness is slim. Disagreement about consciousness will drive reasonable disagreement about rights: Many of those who reasonably think AI systems lack consciousness will reasonably also think that they don’t deserve much if any moral consideration. In contrast, many of those who reasonably think that AI systems have consciousness as rich and sophisticated as our own will reasonably think that they deserve human or humanlike moral consideration.
(2b.) There will continue to be widespread disagreement in moral theory on the basis of moral status. Complicating this issue will be continued disagreement in moral theory. Utilitarians, for example, hold that moral considerability depends on the capacity for pleasure and suffering. Deontologists typically hold that moral considerability depends on something like the capacity for (presumably conscious) sophisticated practical reasoning or the ability to enter into meaningful social relationships with others. In human beings, these capacities tend to co-occur, so that practically speaking for most ordinary human cases it doesn’t matter too much which theory is correct. (Normally, deontologists add some dongles to their theories so as to grant full moral status to human infants and cognitively disabled people.) But in AI cases, capacities that normally travel together in human beings could radically separate. To consider the extremes: We might create AI systems capable of immense pleasure or suffering but which have no sophisticated cognitive capacities (giant orgasm machines, for example), and conversely, we might create AI systems capable of very sophisticated conscious practical reasoning but which have no capacity for pleasure or suffering. Even if we stipulate that all the epistemic problems concerning consciousness are solved, justifiable disagreement in moral theory alone is sufficient to generate radical disagreement about the moral status of different types of AI systems.
(2c.) There will be justifiable social and legal inertia. It is likely that law and custom will change more slowly than AI technology, at a substantial delay. Conservatism in law and custom is justifiable. For Burkean reasons, it’s reasonable to resist sudden or radical transformation of institutions that have long served us well.
(3.) Given wide disagreement over the moral status of AI systems, we will be forced into catastrophic choices between risking overattributing and risking underattributing rights. We can model this simplistically by imagining four defensible attitudes. Suppose that there are two types of AI systems that people could not unreasonably regard as deserving human or humanlike rights: Type A and Type B. A+B+ advocates say both systems deserve rights. A-B- advocates say neither deserves rights. A+B- advocates say A systems do but B systems do not. A-B+ advocates say A systems do not but B systems do. If policy and behavior follow the A+B+ advocates, then we risk overattributing rights. If policy and behavior follow the A-B- advocates, we risk underattributing rights. If policy and behavior follow either of the intermediate groups, we run both risks simultaneously.
(3a.) If we underattribute rights, it’s a moral catastrophe. This is obvious enough. If some AI systems deserve human or humanlike rights and don’t receive them, then when we delete those systems we commit the moral equivalent of murder, or something close. When we treat those systems badly, we commit the moral equivalent of slavery and torture, or something close. Why say “something close”? Two reasons: First, if the systems are different enough in their constitution and interests, the categories of murder, slavery, and torture might not precisely apply. Second, given the epistemic situation, we can justifiably say we don’t know that the systems deserve moral consideration, so when we delete one we don’t know we’re killing an entity with human-like moral status. This is a partial excuse, perhaps, but not a full excuse. Normally, it’s grossly immoral to expose people to a substantial (say 10%) risk of death for no absolutely compelling reason. If we delete an AI system that we justifiably think is probably not conscious, we take on a similar risk.
(3b.) If we overattribute rights, it’s also a catastrophe, though less obviously so. Given 3a above, it might seem that the morally best solution is to err on the side of overattributing rights. Follow the guidelines of the A+B+ group! This is my own inclination, given moral uncertainty. And yet there is potentially an enormous cost to this approach. If we attribute human or humanlike rights to AI systems, then we are committed to sacrificing real human interests on behalf of those systems when real human interests conflict with the seeming interests of the AI systems. If there’s an emergency in which a rescuer faces a choice of saving five humans or six robots, the rescuer should save the robots and let the humans die. If there’s been an overattribution, that’s a tragedy: Five human lives have been lost for the sake of machines that lack real moral value. Similarly, we might have to give robots the vote—and they might well vote for their interests over human interests, again perhaps at enormous cost, e.g., in times of war or famine. Relatedly, I agree with Bostrom and others that we should take seriously the (small?) risk that superintelligent AI runs amok and destroys humanity. It becomes much harder to manage this risk if we cannot delete, modify, box, and command intelligent AI systems at will.
(4.) It’s almost impossible that we will get this decision exactly right. Given the wide range of possible AI systems, the wide range of legitimate divergence in opinion about consciousness, and the wide range of legitimate divergence in opinion about the grounds of moral status, it would require miraculous luck if we didn’t substantially miss our target, either substantially overattributing rights, substantially underattributing rights, or both.
(5.) The obvious policy solution is to avoid creating AI systems with debatable moral status, but it’s extremely unlikely that this policy would actually be implemented. Mara Garza and I have called this the Design Policy of the Excluded Middle. We should only create AI systems that we know in advance don’t have serious intrinsic moral considerability, and which we can then delete and control at will; or we should go all the way and create systems that we know in advance are our moral peers, and then give them the full range of rights and freedoms that they deserve. The troubles arise only for the middle, disputable cases.
The problem with this solution is as follows. Given the wide range of disagreement about consciousness and the grounds of moral status, the “excluded middle” will be huge. We will probably need to put a cap on AI research soon. And how realistic is that? People will understandably argue that AI research has such great benefits for humankind that we should not prevent it from continuing just on the off-chance that we might soon be creating conscious systems that some people might reasonably regard as having moral status. Implementing the policy would require a global consensus to err on the side of extreme moral caution, favoring the policies of the most extreme justifiable A+B+ view. And how likely is such a consensus? Others might argue that even setting aside the human interests in the continuing advance of technology, there’s a great global benefit in eventually being able to create genuinely conscious AI systems of human or humanlike moral status, for the sake of those future systems themselves. Plausibly, the only realistic way to achieve that great global benefit would be to create a lot of systems of debatable status along the way: We can’t plausibly leap across the excluded middle with no intervening steps. Technological development works incrementally. Thus I conclude: We’re headed straight toward a serious ethical catastrophe concerning issues of robot or AI rights.
Eric Schwitzgebel
Eric Schwitzgebel is Professor of Philosophy at the University of California, Riverside, and the author of A Theory of Jerks and Other Philosophical Misadventures.His areas of interest include philosophy of psychology, philosophy of mind, moral psychology, classical Chinese philosophy, epistemology, metaphilosophy, and metaphysics.
Just reading the title, I knew I was going to basically agree even before going through the details of the article. Why? Because neither the people nor the rulers of society have a coherent notion of human rights. But if their understanding of their own rights is so flawed, how can we expect them to do any better with AI rights?
But the above assumes there is at least some understanding of the notion. But the lawless trampling in human rights of even their own soldiers shows that Putin and rulers too like him just do not care about rights at all — other than their own ‘right’ to rob and murder. They will be quick to trash any notion of AI rights using AIs instead to trample even more on human rights.