Preventing an AI Apocalypse

by Seth Baum

Seth Baum is the executive director of the Global Catastrophic Risk Institute (GCRI), a think tank focused on extreme global risks. He is also affiliated with the Blue Marble Space Institute of Science and the University of Cambridge Centre for the Study of Existential Risk.

 

NEW YORK – Recent advances in artificial intelligence have been nothing short of dramatic. AI is transforming nearly every sector of society, from transportation to medicine to defense. So it is worth considering what will happen when it becomes even more advanced than it already is.

The apocalyptic view is that AI-driven machines will outsmart humanity, take over the world, and kill us all. This scenario crops up often in science fiction, and it is easy enough to dismiss, given that humans remain firmly in control. But many AI experts take the apocalyptic perspective seriously, and they are right to do so. The rest of society should as well.

To understand what is at stake, consider the distinction between “narrow AI” and “artificial general intelligence” (AGI). Narrow AI can operate only in one or a few domains at a time, so while it may outperform humans in select tasks, it remains under human control.

AGI, by contrast, can reason across a wide range of domains, and thus could replicate many human intellectual skills, while retaining all of the advantages of computers, such as perfect memory recall. Run on sophisticated computer hardware, AGI could outpace human cognition. In fact, it is hard to conceive an upper limit for how advanced AGI could become.

As it stands, most AI is narrow. Indeed, even the most advanced current systems have only limited amounts of generality. For example, while Google DeepMind’s AlphaZero system was able to master Go, chess, and shogi – making it more general than most other AI systems, which can be applied only to a single specific activity – it has still demonstrated capability only within the limited confines of certain highly structured board games.

Many knowledgeable people dismiss the prospect of advanced AGI. Some, such as Selmer Bringsjord of Rensselaer Polytechnic Institute and Drew McDermott of Yale University, argue that it is impossible for AI to outsmart humanity. Others, such as Margaret A. Boden of the University of Sussex and Oren Etzioni of the Allen Institute for Artificial Intelligence, argue that human-level AI may be possible in the distant future, but that it is far too early to start worrying about it now.

These skeptics are not marginal figures, like the cranks who try to cast doubt on climate-change science. They are distinguished scholars in computer science and related fields, and their opinions must be taken seriously.

Yet other distinguished scholars – including David J. Chalmers of New York University, Yale University’s Allan Dafoe and Stuart Russell of the University of California, Berkeley, Nick Bostrom of Oxford University, and Roman Yampolskiy of the University of Louisville – do worry that AGI could pose a serious or even existential threat to humanity. With experts lining up on both sides of the debate, the rest of us should keep an open mind.

Moreover, AGI is the focus of significant research and development. I recently completed a survey of AGI R&D projects, identifying 45 in 30 countries on six continents. Many active initiatives are based in major corporations such as Baidu, Facebook, Google, Microsoft, and Tencent, and in top universities such as Carnegie Mellon, Harvard, and Stanford, as well as the Chinese Academy of Sciences. It would be unwise simply to assume that none of these projects will succeed.

Another way of thinking about the potential threat of AGI is to compare it to other catastrophic risks. In the 1990s, the US Congress saw fit to have NASA track large asteroids that could collide with the earth, even though the odds of that happening are around one in 5,000 per century. With AGI, the odds of a catastrophe over the upcoming century could be as high as one in a hundred, or even one in ten, judging by the pace of R&D and the strength of expert concern.

The question, then, is what to do about it. For starters, we need to ensure that R&D is conducted responsibly, safely, and ethically. This will require a deeper dialogue between those working in the AI field and policymakers, social scientists, and concerned citizens. Those in the field know the technology and will be the ones to design it according to agreed standards; but they must not decide alone what those standards will be. Many of the people developing AI applications are not accustomed to thinking about the social implications of their work. For that to change, they must be exposed to outside perspectives.

Policymakers also will have to grapple with AGI’s international dimensions. Currently, the bulk of AGI R&D is carried out in the United States, Europe, and China, but much of the code is open source, meaning that the work potentially can be done from anywhere. So, establishing standards and ethical ground rules is ultimately a job for the entire international community, though the R&D hubs should take the lead.

Looking ahead, some efforts to address the risks posed by AGI can piggyback on policy initiatives already put in place for narrow AI, such as the new bipartisan AI Caucus launched by John Delaney, a Democratic congressman from Maryland. There are many opportunities for synergy between those working on near-term AI risks and those thinking about the long term.

But regardless of whether narrow AI and AGI are considered together or separately, what matters most is that we take constructive action now to minimize the risk of a catastrophe down the road. This is not a task that we can hope to complete at the last minute.


Seth Baum is the executive director of the Global Catastrophic Risk Institute (GCRI), a think tank focused on extreme global risks. He is also affiliated with the Blue Marble Space Institute of Science and the University of Cambridge Centre for the Study of Existential Risk.

Copyright: Project Syndicate, 2018.
www.project-syndicate.org

 


This article is brought to you by Project Syndicate that is a not for profit organization.

Project Syndicate brings original, engaging, and thought-provoking commentaries by esteemed leaders and thinkers from around the world to readers everywhere. By offering incisive perspectives on our changing world from those who are shaping its economics, politics, science, and culture, Project Syndicate has created an unrivalled venue for informed public debate. Please see: www.project-syndicate.org.

Should you want to support Project Syndicate you can do it by using the PayPal icon below. Your donation is paid to Project Syndicate in full after PayPal has deducted its transaction fee. Facts & Arts neither receives information about your donation nor a commission.

 

 

Browse articles by author

More Current Affairs

Added 23.05.2018
The good news is that the United States and China appear to have backed away from the precipice of a trade war. While vague in detail, a May 19 agreement defuses tension and commits to further negotiation. The bad news is that the framework of negotiations is flawed: A deal with any one country will do little to resolve America’s fundamental economic imbalances that have arisen in an interconnected world.
Added 21.05.2018
The cryptocurrency revolution, which started with bitcoin in 2009, claims to be inventing new kinds of money. There are now nearly 2,000 cryptocurrencies, and millions of people worldwide are excited by them. What accounts for this enthusiasm, which so far remains undampened by warnings that the revolution is a sham? One must bear in mind that attempts to reinvent money have a long history. As the sociologist Viviana Zelizer points out in her book The Social Meaning of Money: “Despite the commonsense idea that ‘a dollar is a dollar is a dollar,’ everywhere we look people are constantly creating different kinds of money.” Many of these innovations generate real excitement, at least for a while. As the medium of exchange throughout the world, money, in its various embodiments, is rich in mystique. We tend to measure people’s value by it. It sums things up like nothing else. And yet it may consist of nothing more than pieces of paper that just go round and round in circles of spending. So its value depends on belief and trust in those pieces of paper. One might call it faith.
Added 19.05.2018
The protests that rippled across Russia ahead of Vladimir Putin’s fourth inauguration as president followed a familiar script. Police declared the gatherings illegal, and the media downplayed their size. Alexey Navalny, the main organizer and Russia’s de facto opposition leader, was arrested in dramatic fashion, dragged out of a rally in Moscow by police. On May 15, he was sentenced to 30 days in prison. More than 1,600 protesters across the country were beaten and detained.
Added 16.05.2018
Many knowledgeable people dismiss the prospect of advanced AGI [=Artificial General Intelligence]. Some, ..........,argue that it is impossible for AI to outsmart humanity........Yet other distinguished scholars........do worry that AGI could pose a serious or even existential threat to humanity. With experts lining up on both sides of the debate, the rest of us should keep an open mind.
Added 15.05.2018
The world’s most important bilateral relationship – between the United States and China – is also one of its most inscrutable. Bedeviled by paradoxes, misperceptions, and mistrust, it is a relationship that has become a source of considerable uncertainty and, potentially, severe instability. Nowhere is this more apparent than in the brewing bilateral trade war.
Added 15.05.2018
Viewed from Palestine, it’s hard to disagree that we’ve perhaps seen one of the most inflammatory weeks in recent memory. In just a few days, several extremely sensitive events have coincided to devastating effect: the culmination of weekly protests in the Gaza Strip, the relocation of the US embassy from Tel Aviv to Jerusalem, the 70th anniversary of the 1948 Nakba (from the Arabic, “Immense Catastrophe”) and the start of the holy month of Ramadan. Throw in for good measure Israel and Iran’s recent clash over the occupied Golan Heights and it seems that more than ever, the region is something of a tinderbox.
Added 14.05.2018
The irony, of course, is that this is exactly the type of “grand bargain” Iran proposed to the Bush administration in May 2003. Bush rejected the offer, vowing never to talk with a member of the “axis of evil.” As Vice President Dick Cheney put it in reference to North Korea – another member of that fanciful “axis” – Americans “don’t negotiate with evil; we defeat it.” But by trading diplomacy for saber-rattling, the Bush administration slammed the door on a solution with Iran. Today, as Trump embraces the same tactics, it’s hard to fathom how the outcome will be any different.
Added 12.05.2018
Quote: "If you take out a piece of paper and put 10 political issues that are important to you on one side and the words “Republican” or “Democrat” on the top, then categorize them, most of you are likely to find that you, too, are a hybrid of both. If my contention is correct, and enough voters get fed up enough with the political status quo in the mid-terms and the next presidential election to say that neither Party actually represents their belief system, we may just find that the 40% of American voters who are already Independent turns into 50% or 60% in the near term. I am willing to bet that in spite of all the hype about most Americans being firmly in one political camp or the other, that most Americans are actually middle ground voters who form part of the emerging hybrid political class. Whether they choose to remain affiliated with either the Democrat or Republican Parties in the future remains to be seen. Stay tuned".
Added 10.05.2018
Quote: "China, clearly, is emerging as a world power, even more quickly than it otherwise would, to the extent that the US is coming to be seen as an unreliable partner concerned only with advancing its own interests – at the expense, if necessary, of other countries. But the belief that China will continue growing at mid-single-digit rates for an extended period violates the first rule of forecasting: don’t extrapolate the present into the future. At some point, China will hit bumps in the road, and there is no guarantee that its leaders will admit their failures and adjust policy accordingly."
Added 08.05.2018
The decertification of the Joint Comprehensive Plan of Action (JCPOA) by Trump is most unfortunate. It seems that Trump was not swayed by either France’s President Macron or Germany’s Chancellor Merkel to preserve the deal. Instead, he appears to have taken Israeli Prime Minister Netanyahu’s advice to decertify the deal, even though Iran continues to fully adhere to all of its provisions. It is dangerous that neither Trump nor Netanyahu appears to fully grasp the dire regional and international implications of the unilateral decertification of the deal by the US.
Added 04.05.2018
Quote fromthe article: "Iran has been instrumental in the survival of the Assad regime since the Syrian uprising began in March 2011. Since then, Israel has used its air power to disrupt weapons supplies to Hezbollah and maintain a buffer zone in the southwest, near the Israeli-occupied Golan Heights. But now it seems things are heating up. Is this now a far bigger, more direct confrontation in the Syrian arena?"
Added 03.05.2018
As signs of an incipient slowdown in the European economy begin to multiply – coincident indicators suggest that industrial production has slowed sharply in 2018 – the case for agreeing on a Brexit deal and refocusing attention on capital markets union is becoming more powerful and more urgent. The commissioner now responsible, Valdis Dombrovskis, said in London in late April that the “building blocks” will be in place early next year, to “help our companies to better cope with the departure of Europe’s largest financial center from the single market.” That is a laudable goal, but it could well be too little, too late.
Added 03.05.2018
Gaza has often been invaded for its water. Every army leaving or entering the Sinai desert, whether Babylonians, Alexander the Great, the Ottomans, or the British, has sought relief there. But today the water of Gaza highlights a toxic situation that is spiralling out of control. A combination of repeated Israeli attacks and the sealing of its borders by Israel and Egypt, have left the territory unable to process its water or waste. Every drop of water swallowed in Gaza, like every toilet flushed or antibiotic imbibed, returns to the environment in a degraded state.
Added 24.04.2018
Renewable electricity costs have fallen faster than all but the most extreme optimists believed possible only a few years ago. In favorable sunny locations, such as northern Chile, electricity auctions are being won by solar power at prices that have plummeted 90% in ten years. Even in less sunny Germany, reductions of 80% have been achieved. Wind costs have fallen some 70%, and battery costs around 80%, since 2010.........[The] estimate that the cost of going green will be very small has proved too pessimistic – the cost will actually be negative.
Added 23.04.2018
Electric Vessels, EVs don’t rise or fall on Tesla — Nissan and the Chinese brands are arguably way ahead, and the Chevy Bolt is comparable to the Tesla 3. But it is unarguable that a price spike in petroleum would certainly help the company get past its current Tesla 3 production problems by substantially bolstering investor and consumer confidence.
Added 20.04.2018
 

As the US, Russia and China test each other’s patience and strategic focus, speculation about the chances of a world war has hit a new high.

Added 18.04.2018

HONG KONG – At the beginning of this century, when China launched its “going out” policy – focused on using foreign-exchange reserves to support overseas expansion and acquisitions by Chinese companies – few expected the country quickly to emerge as a leading economic player in Latin America.

Added 18.04.2018

BEIJING – Last month, US President Donald Trump enacted steel and aluminum tariffs aimed squarely at China. On April 2, China retaliated with tariffs on 128 American products. Trump then announced 25% tariffs on another 1,300 Chinese products, representing some $50 billion of exports.

Added 17.04.2018

WASHINGTON, DC – Last week was a most unusual one for President Donald Trump’s administration.

Added 16.04.2018

All the attention being given to Facebook and Cambridge Analytica regarding their breach of public trust and exploitation of personal data is richly deserved.