May 16th 2018

Preventing an AI Apocalypse

by Seth Baum

Seth Baum is the executive director of the Global Catastrophic Risk Institute (GCRI), a think tank focused on extreme global risks. He is also affiliated with the Blue Marble Space Institute of Science and the University of Cambridge Centre for the Study of Existential Risk.

 

NEW YORK – Recent advances in artificial intelligence have been nothing short of dramatic. AI is transforming nearly every sector of society, from transportation to medicine to defense. So it is worth considering what will happen when it becomes even more advanced than it already is.

The apocalyptic view is that AI-driven machines will outsmart humanity, take over the world, and kill us all. This scenario crops up often in science fiction, and it is easy enough to dismiss, given that humans remain firmly in control. But many AI experts take the apocalyptic perspective seriously, and they are right to do so. The rest of society should as well.

To understand what is at stake, consider the distinction between “narrow AI” and “artificial general intelligence” (AGI). Narrow AI can operate only in one or a few domains at a time, so while it may outperform humans in select tasks, it remains under human control.

AGI, by contrast, can reason across a wide range of domains, and thus could replicate many human intellectual skills, while retaining all of the advantages of computers, such as perfect memory recall. Run on sophisticated computer hardware, AGI could outpace human cognition. In fact, it is hard to conceive an upper limit for how advanced AGI could become.

As it stands, most AI is narrow. Indeed, even the most advanced current systems have only limited amounts of generality. For example, while Google DeepMind’s AlphaZero system was able to master Go, chess, and shogi – making it more general than most other AI systems, which can be applied only to a single specific activity – it has still demonstrated capability only within the limited confines of certain highly structured board games.

Many knowledgeable people dismiss the prospect of advanced AGI. Some, such as Selmer Bringsjord of Rensselaer Polytechnic Institute and Drew McDermott of Yale University, argue that it is impossible for AI to outsmart humanity. Others, such as Margaret A. Boden of the University of Sussex and Oren Etzioni of the Allen Institute for Artificial Intelligence, argue that human-level AI may be possible in the distant future, but that it is far too early to start worrying about it now.

These skeptics are not marginal figures, like the cranks who try to cast doubt on climate-change science. They are distinguished scholars in computer science and related fields, and their opinions must be taken seriously.

Yet other distinguished scholars – including David J. Chalmers of New York University, Yale University’s Allan Dafoe and Stuart Russell of the University of California, Berkeley, Nick Bostrom of Oxford University, and Roman Yampolskiy of the University of Louisville – do worry that AGI could pose a serious or even existential threat to humanity. With experts lining up on both sides of the debate, the rest of us should keep an open mind.

Moreover, AGI is the focus of significant research and development. I recently completed a survey of AGI R&D projects, identifying 45 in 30 countries on six continents. Many active initiatives are based in major corporations such as Baidu, Facebook, Google, Microsoft, and Tencent, and in top universities such as Carnegie Mellon, Harvard, and Stanford, as well as the Chinese Academy of Sciences. It would be unwise simply to assume that none of these projects will succeed.

Another way of thinking about the potential threat of AGI is to compare it to other catastrophic risks. In the 1990s, the US Congress saw fit to have NASA track large asteroids that could collide with the earth, even though the odds of that happening are around one in 5,000 per century. With AGI, the odds of a catastrophe over the upcoming century could be as high as one in a hundred, or even one in ten, judging by the pace of R&D and the strength of expert concern.

The question, then, is what to do about it. For starters, we need to ensure that R&D is conducted responsibly, safely, and ethically. This will require a deeper dialogue between those working in the AI field and policymakers, social scientists, and concerned citizens. Those in the field know the technology and will be the ones to design it according to agreed standards; but they must not decide alone what those standards will be. Many of the people developing AI applications are not accustomed to thinking about the social implications of their work. For that to change, they must be exposed to outside perspectives.

Policymakers also will have to grapple with AGI’s international dimensions. Currently, the bulk of AGI R&D is carried out in the United States, Europe, and China, but much of the code is open source, meaning that the work potentially can be done from anywhere. So, establishing standards and ethical ground rules is ultimately a job for the entire international community, though the R&D hubs should take the lead.

Looking ahead, some efforts to address the risks posed by AGI can piggyback on policy initiatives already put in place for narrow AI, such as the new bipartisan AI Caucus launched by John Delaney, a Democratic congressman from Maryland. There are many opportunities for synergy between those working on near-term AI risks and those thinking about the long term.

But regardless of whether narrow AI and AGI are considered together or separately, what matters most is that we take constructive action now to minimize the risk of a catastrophe down the road. This is not a task that we can hope to complete at the last minute.


Seth Baum is the executive director of the Global Catastrophic Risk Institute (GCRI), a think tank focused on extreme global risks. He is also affiliated with the Blue Marble Space Institute of Science and the University of Cambridge Centre for the Study of Existential Risk.

Copyright: Project Syndicate, 2018.
www.project-syndicate.org

 


This article is brought to you by Project Syndicate that is a not for profit organization.

Project Syndicate brings original, engaging, and thought-provoking commentaries by esteemed leaders and thinkers from around the world to readers everywhere. By offering incisive perspectives on our changing world from those who are shaping its economics, politics, science, and culture, Project Syndicate has created an unrivalled venue for informed public debate. Please see: www.project-syndicate.org.

Should you want to support Project Syndicate you can do it by using the PayPal icon below. Your donation is paid to Project Syndicate in full after PayPal has deducted its transaction fee. Facts & Arts neither receives information about your donation nor a commission.

 

 

Browse articles by author

More Current Affairs

Sep 18th 2018
To understand them, it is worth looking at three reputable leaders who died this summer: former United Nations Secretary-General Kofi Annan, former British Foreign Secretary and NATO Secretary-General Peter Carrington, and US Senator John McCain. Having worked with Annan and for Carrington, I can vouch for their grace, honor, and commitment to truth. McCain plainly had the same qualities, not to mention a level of personal bravery far beyond what is expected of most of us (though it should be noted that Carrington was also a war hero). These leaders’ combination of honor and commitment to truth – two attributes that are intrinsically connected – is nowhere to be seen in Trump or Johnson.
Sep 18th 2018
From controlling the media to stoking nationalism, Russian President Vladimir Putin has always known how to keep his approval ratings high. But Russians’ lives are not getting any better, especially after the latest round of Western economic sanctions – and Putin’s declining approval rating shows it.
Sep 15th 2018
As we mark the decennial of the collapse of Lehman Brothers, there are still ongoing debates about the causes and consequences of the financial crisis, and whether the lessons needed to prepare for the next one have been absorbed. But looking ahead, the more relevant question is what actually will trigger the next global recession and crisis, and when. The current global expansion will likely continue into next year, given that the US is running large fiscal deficits, China is pursuing loose fiscal and credit policies, and Europe remains on a recovery path. But by 2020, the conditions will be ripe for a financial crisis, followed by a global recession. There are 10 reasons for this. First, the fiscal-stimulus policies that are currently pushing the annual US growth rate above its 2% potential are unsustainable. By 2020, the stimulus will run out, and a modest fiscal drag will pull growth from 3% to slightly below 2%.
Sep 12th 2018
Next month, a judge in Oregon will begin hearing a case brought against the United States government on behalf of 21 young people, supported by the non-profit organization Our Children’s Trust, who allege that the authorities’ active contributions to the climate crisis violate their constitutional rights. The government defendants have repeatedly tried – so far without success – to have the case thrown out or delayed, and the trial is currently scheduled to start on October 29.
Sep 5th 2018
Wars are expensive, as the Russian people are now learning. The Kremlin is pursuing military adventures in Eastern Ukraine and Syria, and though these conflicts are limited in scope, one wonders if the country can really afford them.
Sep 1st 2018
This week, the California state legislature voted to mandate that all the state’s electricity come from non-carbon sources (chiefly wind, solar and hydro) by 2045. Since California if it were a country would have the world’s fifth largest economy, and since so many other states are economically integrated with it, this plan, if signed by governor Jerry Brown, could help transform the entire country. The goal is less difficult than it seems on the surface. California had already committed to getting one third of its electricity from renewables by 2020, and reached that goal in 2017. It committed to getting 50% of its electricity from renewables by 2030, and in fact will likely reach that goal 10 years early, in 2020.
Aug 29th 2018
Quote: "This may ultimately result in creation of a new accounting standard - the Enterprise Value of Data – which could become an integral part of financial statements, capturing the value of the largest and most ignored corporate asset: data."
Aug 29th 2018
What comes through clearly in polling on US public opinion is that there is a deep partisan divide on the Israeli/Palestinian issue, with key demographic groups increasingly more supportive of Palestinian rights and antagonistic to hardline Israeli policies. In some ways, the Netanyahu/Trump "marriage" has also helped to fuel the partisan divide. A Pew poll from earlier this year found that support for Palestinians far surpasses support for Israel among self-described "progressive" and "liberal" voters. And a recent Gallup poll shows that only 17% of Democrats now have a favorable view of the Israeli leader. 
Aug 27th 2018
History suggests that current-account imbalances ultimately matter a great deal. A still-unbalanced global economy may be forced to relearn that painful lesson in the coming years.
Aug 27th 2018
The United States economy is doing well. But the next recession – and there is always another recession – could be very bad. The US Bureau of Economic Analysis estimates that GDP growth in the second quarter of 2018 reached 4.1% – the highest since the 4.9% seen under President Barack Obama in 2014. Another year of growth will match the record ten-year expansion of the 1990s. Add to that low unemployment, and things are looking good. But this cannot continue forever.
Aug 24th 2018
If Britain leaves the European Union with no deal in place to govern trade with its biggest partner, it will fall back on World Trade Organisation rules. The same set of rules would apply to EU countries and non-EU trade partners. This is why the UK government has published a series of “technical notices” detailing preparations for a no-deal Brexit. Here are seven reasons that sum up why a no-deal Brexit and defaulting to WTO rules would be bad for British businesses and the wider economy.
Aug 23rd 2018
Unlike today’s aspiring strongmen, a truly tough leader would stand up for international cooperation, and seek to persuade voters why it matters. One hopes that French President Emmanuel Macron, German Chancellor Angela Merkel, and Japanese Prime Minister Shinzo Abe will do precisely that in the coming months. In the meantime, we must pray that wannabe tough guys like Trump and Erdoğan do not do too much damage to their respective countries, and to the rest of us. It is time to make cooperation great again.
Aug 23rd 2018
Historians will mark August 21, 2018, as a turning point in American history. President Donald Trump’s personal lawyer pleaded guilty to 8 counts of criminal wrongdoing that could carry a sentence of up to 65 years. Most significantly, he pleaded guilty to having attempted illegally to interfere in an election “in coordination with and at the direction of a federal candidate for office.”
Aug 22nd 2018
As the usual political inferno between parties and private firms rages on, the phantom threat of mafia involvement in Italian construction has resurfaced. The region of Liguria sadly scores quite high in the assessments of mafia infiltration. In the area, Calabrian mafia clans of the ‘ndrangheta – Italy’s most powerful mafia today – have heavily invested in the construction sector, in public tenders and in the exploitation of the port of Genoa and the roads to France and to the rest of the Italian north, for the purposes of illegal trafficking.
Aug 15th 2018
Why do conspiracy theories and general charlatanism so often receive their strongest support from the world’s dictators? Sure, dictators are almost always oddballs themselves, but that cannot be all there is to it. In fact, it is worth asking whether quackery is a necessary feature of authoritarian rule.
Aug 15th 2018
Across the global economy, the potential for automation seems huge. Adidas’ “Speedfactory” in Bavaria will employ 160 workers to produce 500,000 pairs of shoes each year, a productivity rate over five times higher than in typical factories today. The British Retail Consortium estimates that retail jobs could fall from three million to 2.1 million within ten years, with only a small fraction replaced by new jobs in online retailing. Many financial-services companies see the potential to cut information-processing jobs to a small fraction of current levels. And yet, despite all this, measured productivity growth across the developed economies has slowed. One possible explanation.....
Aug 11th 2018
US President Donald Trump’s erratic unilateralism represents nothing less than abdication of global economic and political leadership. Trump’s withdrawal from the Paris climate agreement, his rejection of the Iran nuclear deal, his tariff war, and his frequent attacks on allies and embrace of adversaries have rapidly turned the United States into an unreliable partner in upholding the international order. But the administration’s “America First” policies have done more than disqualify the US from global leadership. They have also created space for other countries to re-shape the international system to their liking. The influence of China, in particular, is likely to be enhanced.
Aug 9th 2018
Wall Street is full of turnaround talk following Tesla’s second-quarter earnings results, sending shares in the futuristic electric car pioneer up almost 10%. Investors welcomed news that the company was burning through cash at a lower rate and revenues were considerably higher, yet the results also showed that Tesla’s financial and operational weaknesses have not gone away. From my reading of the figures, this is a company in real trouble.
Aug 1st 2018
By seeking to disrupt virtually all that has defined the West since the end of World War II, Trump has brought the world to a historical turning point. At stake is not the US-EU relationship, which remains strong, but rather the West’s dominant position on the world stage. Trump is accelerating a shift in the global balance of power that will leave both America and Europe weaker in relative terms. As income and wealth shift from the West to the East, China will increasingly be able to challenge the US as the world’s leading geopolitical, economic, and technological power.
Jul 28th 2018
Alarming stories about the diabetes epidemic that threatens millions of lives have become commonplace, and with good reason.......The link between cheap, sugary and fatty food and obesity and type 2 diabetes is indisputable. The healthy/unhealthy food cost ratio has to change because the evidence is that education – while valuable – is not enough by itself. The evidence from many countries shows that in most chronic, lifestyle-related diseases, legislation is faster and often more effective.