There is a rift between near and long-term perspectives on AI safety – one that has stirred controversy. Longtermists argue that we need to prioritise the well-being of people far into the future, perhaps at the expense of people alive today. But their critics have accused the Longtermists of obsessing on Terminator-style scenarios in concert with Big Tech to distract regulators from more pressing issues like data privacy. In this essay, Mark Bailey and Susan Schneider argue that we shouldn’t be fighting about the Terminator, we should be focusing on the harm to the mind itself – to our very freedom to think.

There has been a growing debate between near and long-term perspectives on AI safety – one that has stirred controversy. “Longtermists” have been accused of being co-opted by Big Tech and fixating on science fiction-like Terminator-style scenarios to distract regulators from the real, more near-term, issues, such as algorithmic bias and data privacy.

Longtermism is an ethical theory that requires us to consider the effects of today’s decisions on all of humanity’s potential futures. It can lead to extremes, as it concludes that one should sacrifice the present wellbeing of humanity for the good of humanity’s potential futures. Many Longtermists believe humans will ultimately lose control of AI, as it will become “superintelligent”, outthinking humans in every domain – social acumen, mathematical abilities, strategic thinking, and more.

SUGGESTED VIEWING AI and the end of humanity With Liv Boeree, Güneş Taylor, Joscha Bach, Eliezer Yudkowsky, Scott Aaronson

So is Longtermism just futuristic fearmongering or does it point to genuine risks? The long-term risks are not entirely imaginary, we believe, but one must see beyond the sensationalized existential risks that receive so much airtime and reconceptualize things, considering realities of today’s AI ecosystem. In what follows, we propose a view we call Moderate Longtermism.

1. Reconceptualizing long-term risks

First, notice that AI systems do not need to be superintelligent to outthink humans in significant ways and pose a serious risk to human survival. For example, an emotionally deficient system, such as one that is deficient in empathy and moral programming, yet which can set and achieve goals, self-improve, and possesses superior planning and execution abilities, would seem to be the most dangerous system of all, especially when given knowledge of (and access to) critical infrastructure.

Because this system wouldn’t outthink us in every domain, it would not be superintelligent. Indeed, this kind of system is already within the technological near-term horizon. And it is precisely the combination of superhuman abilities in certain domains and massive gaps in other abilities that makes it dangerous.

Second, a Terminator-style scenario does not seem to be the real threat with superintelligent AI systems, at least for now. For one thing, robotics developments lag far behind developments in generative AI. Second, the danger concerns not just a single “killer robot” system, but rather the unforeseen impacts of the interaction of many of these systems in the larger AI ecosystem. There are legitimate cybersecurity worries about interacting AIs on the internet that could exhibit emergent features, including greater-than-human intelligence. These are not embodied, humanoid intelligences, yet they are serious candidates for the aforementioned emotionally deficient systems of concern.

SUGGESTED VIEWING Consciousness and ChatGPT With James Tartaglia

There is a risk of “propagated uncertainty” that could occur when many opaque AI systems interact with each other, especially if such systems gain control over critical systems. In this scenario, the “explainability problem” inherent in a single AI – the difficulty of explaining how the AI works – is magnified as the AI interacts with other unexplainable AI systems. Schneider and Kilian have elsewhere called such interacting systems “AI megasystems”.

These two points are in keeping with the core Longtermist concern that we could lose control of ultra-intelligent AI, and thus over our future. But notice that Moderate Longtermism, although derived from the earlier Longtermist concerns, is not a Longtermist position that can be tagged by critics as irrelevant, for these are pressing here-and-now concerns, even if they are about losing control of ultra-intelligent AI. Further, just as a mixed financial portfolio is seen as a hedge against an unknown future, the wisest approach to AI safety may be one that balances realistic near- and long-term risks.

2. The ethics of Strong Longtermism

Longtermists might respond to our Moderate Longtermism by noting that the long-term and short-term risks demand a trade off. For Longtermists contend that actions made today must focus on the consequences for the greatest number of people; and further, that our actions should maximize human potential into the far future. An even more extreme version of Longtermism (called “Strong Longtermism”) posits that this should be achieved at the expense of humans who are alive today. This will make tradeoffs inevitable.

Strong Longtermism as an ethical stance we can do without, however. One problem arises from its “consequentialist” nature, which, in this case, requires one to maximize value oriented toward the potential long-term future of humanity. Consequentialism is an ethical theory focused on maximizing the “good,” often defined as happiness, flourishing, or pleasure. It focuses on the morality of the outcome of an action, as opposed to the intrinsic nature of the action itself. For example, if a parent were faced with the choice of either saving their own child who is drowning or saving three unrelated drowning children, extreme consequentialism would demand that the parent save the three children at the expense of the one. So, while consequentialism is a leading and important ethical theory, it is controversial when not tempered with other forms of ethical reasoning.

Strong Longtermism also faces a second serious objection. Instead of limiting its scope to the present human population and near-future generations, it considers the long-term future of humanity – even billions of years into the future – and how our actions today impact every possible human who might ever exist. For example, Strong Longtermists would be willing to sacrifice funding for research on short-term gains in HIV prevention in favor of funding research to mitigate potential future risks that could lead to the loss of humanity’s long-term potential – even those that could occur well into the future – simply because the net increase in utility would be greater when summed over the course of humanity’s future existence. Their reasoning is that as long as present tragedies don’t lead to human extinction, they would have a minimal deleterious effect on human potential when considering the grander scale of every human who could ever live.

This consequentialist extreme is demanded by Strong Longtermism, where the only moral choice is one that maximizes utility for all of humanity’s possible futures. Like other consequentialist normative theories, every moral choice is either demanded or forbidden – there is no moral gray area, and no opportunity for supererogation, or choosing to go above and beyond one’s moral duty.

This is unpalatable to many. Further, even if you can stomach this, consider, pragmatically speaking, how difficult it is to know what the future brings. Indeed, an unforeseen situation – a pandemic, a totalitarian world leader, a technological innovation – could change the way AI technology plays out over the long term. Strong Longtermists, lacking a God’s-eye view of the evolution of history, would inevitably be wrong about the future. In sacrificing the present for an uncertain future, only one thing is certain: those who are alive today would suffer.

3. Near-term risks

Of course, this doesn’t mean considering the future impact of today’s decisions is irrelevant to policymaking, but the farther into the future projections go, the more cautious we should be. Aiming to mitigate global extinction risks (like the possibility of unaligned, superintelligent AI) is noble, but it shouldn’t occur at the expense of addressing short-term risks that may not necessarily threaten all of humanity’s existence. And within the context of AI, the short-term risks are serious: accelerated disinformation propagation, the erosion of truth, threats to digital privacy, lethal autonomous weapons, accelerated bioweapons development, and exacerbated inequality and bias in contemporary AI decision-making.

It is particularly important to consider the intellectual harms chatbots like ChatGPT pose as they become part of our intellectual lives. Suppose you ask a question of a chatbot like ChatGPT or Google’s Bard rather than using a standard search engine query. Instead of being presented with a list of websites from which to evaluate the information yourself, your answer comes in the form of a few paragraphs, leaving you with even less opportunity to draw your own conclusion – and no explanation about the source of the information. The algorithm might also prompt subsequent questions, shaping your thoughts about what to consider further (and what not to). The natural trajectory of ubiquitous AI chatbots could easily undermine individual decision-making – and potentially lead to groupthink.

Indeed, an authoritarian regime or nefarious actor could build their own GPT-like model that shapes the conversation in favor of its ideology. Anyone with access to the AI system could, in principle, inject bias to meet their particular agenda.

Increasingly, the public is becoming habituated into handing over their data and putting their lives on public display on social media platforms. Both sides of the political spectrum are united in concern for the impact on America’s youth. Those thinking of both near- and long-term risks should anticipate that as people grow more and more tethered to digital devices throughout their environments, including ones grabbing biometric data, the risk to human flourishing expands. Data privacy regulations must be nimble enough to anticipate the new ways data can be sourced, ranging from wristbands detecting thoughts, iris scanners, and even neural implants, all of which are already under development.

We should all be against a future world in which we are dumbed down, tethered to our devices, dependent on chatbots to think for us, write papers for our kids, talk us to sleep, give us therapy, and, of course, tell us “the truth.” Nefarious actors can leverage these technologies to erode our strategic advantage, dumb down our workforce, and exacerbate culture wars. We shouldn’t be fighting about the Terminator, we should be focusing on the harm to the mind itself – to our very freedom to think.

  SUGGESTED VIEWING The AI Apocalypse With Liv Boeree, Stephanie Hare, Timothy Nguyen, Michael Wooldridge

4. Moderate Longtermism

So, where does this leave us? Notice that the above “here-and-now” worries dovetail with our reformulated Longtermist concerns. As the AI ecosystem develops into networks of AI services utilizing generative AI, these systems become precisely the affectively limited systems sketched above. The traditional Longtermist problem of control applies not to Terminator-style robots, but to the “megasystems” of integrated AI services exhibiting emergent behaviors. The megasystem’s tools for success are precisely those we just sketched: a habituated public, apps without robust privacy, politicized social media platforms, chatbots integrated throughout the AI ecosystem, and so on. We call this the “real AI control problem”: the problem of how humans can maintain control over AI megasystems. Indeed, even today, the algorithmic web of AI services seems instead to control us.

So, we believe a Moderate Longtermist position, as sketched above, is well worth considering, one that recognizes that the impact on future generations is important, but which takes a stance of epistemic humility, recognizing that we are fallible about calculating future impacts.

Further, we noted extreme consequentialism seems unpalatable. While the issues in the field of ethics are intricate and subtle, it pays to bear in mind that ethicists frequently temper their consequentialist positions with additional components – nodding to consequentialism but mixing it with other forms of ethical reasoning. For example, few would deny that the consequences of one’s actions must be a key ingredient in determining whether an action is right or wrong, and few would deny that harms to future generations matter. But the philosophical concept of phronesis – practical wisdom to evaluate ethical logic – can help us tame some of the consequentialist extremes of Strong Longtermism. Phronesis shapes and cultivates virtues over time, ultimately leading to a state of positive being and cooperative, prosocial behavior that benefits everyone. All it takes is a little pragmatism and self-reflection.

So we reject the dichotomy between Longtermism and short-termism for a Moderate Longtermist approach. Any approach to AI regulation must consider ways to mitigate risks for the benefit of humans alive today, not just those who may exist in the far future. At the same time, today’s problems, if they slip through the cracks, can lead to grave longer-term dangers, given how rapidly AI technology evolves. Reshaping the extreme elements of Longtermism will help us avoid the perverse outcomes that neglect short-term risks, but both sides must be frank about the nature of the risk.

--

Disclaimer: The authors are responsible for the content of this article. The views expressed do not reflect the official policy or position of the National Intelligence University, the Department of Defense, the Office of the Director of National Intelligence, the U.S. Intelligence Community, or the U.S. Government.