Preventing an AI Apocalypse

by Seth Baum

Seth Baum is the executive director of the Global Catastrophic Risk Institute (GCRI), a think tank focused on extreme global risks. He is also affiliated with the Blue Marble Space Institute of Science and the University of Cambridge Centre for the Study of Existential Risk.

 

NEW YORK – Recent advances in artificial intelligence have been nothing short of dramatic. AI is transforming nearly every sector of society, from transportation to medicine to defense. So it is worth considering what will happen when it becomes even more advanced than it already is.

The apocalyptic view is that AI-driven machines will outsmart humanity, take over the world, and kill us all. This scenario crops up often in science fiction, and it is easy enough to dismiss, given that humans remain firmly in control. But many AI experts take the apocalyptic perspective seriously, and they are right to do so. The rest of society should as well.

To understand what is at stake, consider the distinction between “narrow AI” and “artificial general intelligence” (AGI). Narrow AI can operate only in one or a few domains at a time, so while it may outperform humans in select tasks, it remains under human control.

AGI, by contrast, can reason across a wide range of domains, and thus could replicate many human intellectual skills, while retaining all of the advantages of computers, such as perfect memory recall. Run on sophisticated computer hardware, AGI could outpace human cognition. In fact, it is hard to conceive an upper limit for how advanced AGI could become.

As it stands, most AI is narrow. Indeed, even the most advanced current systems have only limited amounts of generality. For example, while Google DeepMind’s AlphaZero system was able to master Go, chess, and shogi – making it more general than most other AI systems, which can be applied only to a single specific activity – it has still demonstrated capability only within the limited confines of certain highly structured board games.

Many knowledgeable people dismiss the prospect of advanced AGI. Some, such as Selmer Bringsjord of Rensselaer Polytechnic Institute and Drew McDermott of Yale University, argue that it is impossible for AI to outsmart humanity. Others, such as Margaret A. Boden of the University of Sussex and Oren Etzioni of the Allen Institute for Artificial Intelligence, argue that human-level AI may be possible in the distant future, but that it is far too early to start worrying about it now.

These skeptics are not marginal figures, like the cranks who try to cast doubt on climate-change science. They are distinguished scholars in computer science and related fields, and their opinions must be taken seriously.

Yet other distinguished scholars – including David J. Chalmers of New York University, Yale University’s Allan Dafoe and Stuart Russell of the University of California, Berkeley, Nick Bostrom of Oxford University, and Roman Yampolskiy of the University of Louisville – do worry that AGI could pose a serious or even existential threat to humanity. With experts lining up on both sides of the debate, the rest of us should keep an open mind.

Moreover, AGI is the focus of significant research and development. I recently completed a survey of AGI R&D projects, identifying 45 in 30 countries on six continents. Many active initiatives are based in major corporations such as Baidu, Facebook, Google, Microsoft, and Tencent, and in top universities such as Carnegie Mellon, Harvard, and Stanford, as well as the Chinese Academy of Sciences. It would be unwise simply to assume that none of these projects will succeed.

Another way of thinking about the potential threat of AGI is to compare it to other catastrophic risks. In the 1990s, the US Congress saw fit to have NASA track large asteroids that could collide with the earth, even though the odds of that happening are around one in 5,000 per century. With AGI, the odds of a catastrophe over the upcoming century could be as high as one in a hundred, or even one in ten, judging by the pace of R&D and the strength of expert concern.

The question, then, is what to do about it. For starters, we need to ensure that R&D is conducted responsibly, safely, and ethically. This will require a deeper dialogue between those working in the AI field and policymakers, social scientists, and concerned citizens. Those in the field know the technology and will be the ones to design it according to agreed standards; but they must not decide alone what those standards will be. Many of the people developing AI applications are not accustomed to thinking about the social implications of their work. For that to change, they must be exposed to outside perspectives.

Policymakers also will have to grapple with AGI’s international dimensions. Currently, the bulk of AGI R&D is carried out in the United States, Europe, and China, but much of the code is open source, meaning that the work potentially can be done from anywhere. So, establishing standards and ethical ground rules is ultimately a job for the entire international community, though the R&D hubs should take the lead.

Looking ahead, some efforts to address the risks posed by AGI can piggyback on policy initiatives already put in place for narrow AI, such as the new bipartisan AI Caucus launched by John Delaney, a Democratic congressman from Maryland. There are many opportunities for synergy between those working on near-term AI risks and those thinking about the long term.

But regardless of whether narrow AI and AGI are considered together or separately, what matters most is that we take constructive action now to minimize the risk of a catastrophe down the road. This is not a task that we can hope to complete at the last minute.


Seth Baum is the executive director of the Global Catastrophic Risk Institute (GCRI), a think tank focused on extreme global risks. He is also affiliated with the Blue Marble Space Institute of Science and the University of Cambridge Centre for the Study of Existential Risk.

Copyright: Project Syndicate, 2018.
www.project-syndicate.org

 


This article is brought to you by Project Syndicate that is a not for profit organization.

Project Syndicate brings original, engaging, and thought-provoking commentaries by esteemed leaders and thinkers from around the world to readers everywhere. By offering incisive perspectives on our changing world from those who are shaping its economics, politics, science, and culture, Project Syndicate has created an unrivalled venue for informed public debate. Please see: www.project-syndicate.org.

Should you want to support Project Syndicate you can do it by using the PayPal icon below. Your donation is paid to Project Syndicate in full after PayPal has deducted its transaction fee. Facts & Arts neither receives information about your donation nor a commission.

 

 

Browse articles by author

More Current Affairs

Added 12.07.2018
The cabinet members who resigned this week apparently feared that politics is taking May toward a “soft Brexit,” their worst of all possible worlds........“soft Brexit,” maintains the status quo, more or less, letting Europeans freely circulate into British labor markets and allowing European firms to operate easily in the UK. The problem with “soft Brexit” is that it raises questions about why the UK is leaving at all, since it will still have the same obligations to Europe as before, it just won’t have a voice when the remaining 27 members of the European Union meet to make decisions.
Added 12.07.2018
One study on the 2010 World Cup found that there was a 37.5% rise in admission rates across 15 accident and emergency departments on England match days........Examining reports of domestic abuse in Lancashire (a county of approximately 1.5m people in Northern England), across the 2002, 2006 and 2010 World Cup tournaments, we discovered a 26% increase in reports of domestic abuse when England won or drew, and a 38% increase when England lost. Reports were also more frequent on weekends, and reached their peak when England exited the tournament.
Added 10.07.2018
If, back in the 1980s and 1990s, the US government, rather than arguing for Chinese economic opening, had prohibited any US company from investing there, China’s rise would have been significantly delayed, though not permanently prevented. Because that did not happen, China’s rise is now self-sustaining. A huge and increasingly affluent domestic market will make exports less vital to growth.
Added 10.07.2018
Comparing today’s demagogues with Adolf Hitler is almost always unwise. Such alarmism tends to trivialize the actual horrors of the Nazi regime, and distracts attention from our own political problems. But if alarmism is counterproductive, the question remains: At what point are democracies truly in danger? What was unimaginable only a few years ago – a US president insulting democratic allies and praising dictators, or calling the free press “enemies of the people,” or locking up refugees and taking away their children – has become almost normal now. When will it be too late to sound the alarm?
Added 09.07.2018
In view of such actions, expectations for Trump’s behavior at the upcoming summit have gone from prickly to dangerous. The sense of foreboding has been heightened by the announcement that, just four days after the summit ends, Trump will meet with Russian President Vladimir Putin, in Helsinki. The nightmare scenario is easy to imagine: Trump lays bare NATO’s fractures, including by questioning mutual defense, before selling his allies down the river by publicly embracing Putin. But this does not need to be the outcome.
Added 09.07.2018
After 2027 (or maybe even 2025, only 7 years from now), the number of EVs will rapidly accelerate, as virtually all new vehicles bought will be electric (an effect of rapidly falling battery and other component costs and of the fuel for electric cars being essentially free; you can power one off your rooftop solar array).
Added 03.07.2018
Most pundits interpret Trump’s outbursts as playing to his political base, or preening for the cameras, or blustering for the sake of striking future deals. We take a different view. In line with many of America’s renowned mental-health experts, we believe that Trump suffers from several psychological pathologies that render him a clear and present danger to the world.
Added 03.07.2018
In the United Kingdom, Brexit looms large, with everyone from government ministers to tabloid newspapers frothing daily about the deal that will be struck with the European Union and the effects that it will have. But the EU faces too many pressing challenges to be obsessing about Britain. The UK’s concern is understandable: evidence is mounting of the likely damage a departure from the single market and customs union will do to the UK economy. According to new research from the Centre for European Reform, the UK economy is already 2.1% smaller than it would have been had voters chosen to remain. The hit to public finances totals £440 million ($579 million) per week.
Added 26.06.2018
Nowadays, Britain’s words and actions on the world stage are so at odds with its values that one must wonder what has happened to the country. Since the June 2016 Brexit referendum, British foreign policy seems to have all but collapsed – and even to have disowned its past and its governing ideas. Worse, this has coincided with the emergence of US President Donald Trump’s erratic administration, which is pursuing goals that are completely detached from those of Britain – and of Europe generally. 
Added 26.06.2018
With each passing day, it becomes increasingly evident that US President Donald Trump’s administration cares less about economics and more about the aggressive exercise of political power. This is obviously a source of enormous frustration for those of us who practice the art and science of economics. But by now, the verdict is self-evident: Trump and his team continue to flaunt virtually every principle of conventional economics.
Added 26.06.2018
The sights and sounds of Central American children being ripped from their parents by US Border Patrol officers have, by now, spread across the globe. The experience has been traumatizing to its victims and deeply painful to watch. It has also done incalculable damage to the very idea of America. This is June when we are supposed to be celebrating "Immigrant Heritage Month". Each year, I have taken this opportunity to recall my family's immigrant story - the opportunity and freedom they sought, the hardships they endured, and the remarkable progress they made in just one generation. 
Added 24.06.2018
State terrorism comes in many forms, but one of its most cruel and revolting expressions is when it is aimed at children. Even though U.S. President Donald Trump backed down in the face of a scathing political and public outcry and ended his administration’s policy of separating migrant children from their parents, make no mistake: His actions were and remain a form of terrorism.
Added 22.06.2018
It is now clear that the twenty-first century is ushering in a new world order. As uncertainty and instability associated with that process spread around the globe, the West has responded with either timidity or nostalgia for older forms of nationalism that failed in the past and certainly will not work now. Even to the most inveterate optimist, the G7 summit in Quebec earlier this month was proof that the geopolitical West is breaking up and losing its global significance, and that the great destroyer of that American-created and American-led order is none other than the US president. To be sure, Donald Trump is more a symptom than a cause of the West’s disintegration. But he is accelerating the process dramatically.
Added 20.06.2018
Sessions quoted a line written by the apostle Paul to a small community of Christians living in Rome around 55AD to defend the Department of Justice’s approach. He said: "I would cite you to the Apostle Paul and his clear and wise command in Romans 13, to obey the laws of the government because God has ordained them for the purpose of order." Sessions used the Bible because one of the most vocal opponents of the crackdown on asylum cases has been the Catholic Church. It’s no surprise that Sessions appealed to Romans chapter 13 verse 1 in response: not only did he hope to undermine Catholic authority by using the Bible against them, he cited a statement so broad that one might use it to defend anything a government does, good or bad. Picture below St Paul writing his epistles, by Valentin de Boulogne, via Wikimedia Commons.
Added 19.06.2018
 

I find it exceptionally irritating when I hear liberals worry about whether Israel will be able to remain a "Jewish and Democratic State" if it retains control of occupied Palestinian lands.

Added 18.06.2018
Daniel Wagner: "My prediction Korean War will be formally ended, the peninsula will be denuclearised, and a lasting peace will be the result."
Added 14.06.2018
Extract: PiS [ the ruling Law and Justice party] has established the most significant addition to the Polish social safety net since 1989: the Family 500+ program. Launched in 2016, Family 500+ embodies the nationalism, traditional family values, and social consciousness that the PiS seeks to promote. The program pays families 500 złoty ($144) per month to provide care for a second or subsequent child...........The program has been enormously popular. Some 2.4 million families took advantage of it in the first two years. The benefit, equivalent to 40% of the minimum wage, has almost wiped out extreme poverty for children in Poland, reducing it by an estimated 70-80%........... Liberal pro-European politicians and policymakers are not convinced. They complain that such a generous family benefit will weaken work incentives and blow up the government budget. But initial evidence suggests that Family 500+ has actually increased economic activity. It has also reversed the post-communist decline in fertility, increased wages (particularly for women), and enabled families to buy school materials, take vacations, buy more clothes for their kids, and rely less on high-priced credit for basic household needs. And, thanks to rapid economic growth, the government deficit has steadily fallen, not grown.
Added 12.06.2018
The depths of hypocrisy of the Republican Party in supporting Trump’s meeting with the North Korean dictator in Singapore are hard to plumb. This is a party whose leading members adopted the Ostrich Foreign Policy Principle for decades. If you don’t like a country’s government or political and economic system, pretend it does not exist.
Added 12.06.2018
US Defense Secretary Jim Mattis has spoken out against China’s strategy of “intimidation and coercion” in the South China Sea, including the deployment of anti-ship missiles, surface-to-air missiles, and electronic jammers, and, more recently, the landing of nuclear-capable bomber aircraft at Woody Island. There are, Mattis warned, “consequences to China ignoring the international community.” But what consequences?
Added 12.06.2018
With a general election approaching in September, Swedish voters are being warned that now it’s their turn to be targeted by Russian interference in the democratic process. According to Sweden’s Civil Contingencies Agency (MSB), which is leading the country’s efforts to counter foreign-influence operations, such interference is very likely, and citizens should be on the lookout for disinformation and fake news.