We’re all doomed – and soon. That’s the terrifying message from Silicon Valley as the developers of artificial intelligence foretell the apocalypse for which they will themselves be responsible.
This month, Jacob Coxon, 27, a British employee of Anthropic announced he was leaving over fears his company and its main competitor, OpenAI, were ‘gambling with our lives’ by rushing to create a digital ‘superintelligence’ that could destroy humanity.
Some working in AI – known as ‘doomers’ – have expressed similar fears but Coxon’s warning went viral and many of his colleagues swiftly echoed his warnings. Evan Hubinger, a senior safety researcher at Anthropic, estimated there was more than a 10 percent chance that AI ‘could kill all humans’ within a decade.
Anthropic boss Dario Amodei largely agreed with Coxon, saying AI had been ‘advancing drastically faster’ and also called his industry to ‘slow down’ and accept external regulation.
OpenAI boss Sam Altman (creator of ChatGPT) and AI developer Elon Musk followed suit.
Amodei acknowledged that the US government recognises the West is racing to develop superintelligent AI before China and other ‘autocracies’ and that pulling back amounted to a ‘significant national security risk’.
But he warned that after a notorious incident this summer – in which a ‘swarm’ of AI agents belonging to OpenAI launched sophisticated cyberattacks on rival AI company Hugging Face – a more capable swarm might ‘take over the entire internet’ within as little as six months.
The media and Washington establishment have been listening agog to these apocalyptic pronouncements.
Anthropic boss Dario Amodei has said AI had been ‘advancing drastically faster’ and called on his industry to ‘slow down’ and accept external regulation
Sam Altman – the boss of OpenAI, which saw a swarm of its agents launch cyberattacks on rival AI company Hugging Face this summer – joined the discussion on the dangers of AI
So are we all genuinely doomed – or might there be another explanation for all this terrifying recent rhetoric?
Donald Trump certainly believes so, sticking to his guns on pushing ahead with AI. Critics insist that he is only thinking about himself: America has crucial Congressional midterm elections in November and Trump, they say, doesn’t want to do anything to harm the vast investment in AI that has pushed the US stock market to record highs and buoyed the country’s economy.
This may indeed be part of his thinking – but that doesn’t necessarily mean the President is wrong to want to pursue AI development.
Many experts claim that all those issuing dire warnings about AI being an imminent threat to the human race are wildly exaggerating. The doomers’ predictions, they say, depend on AI achieving superintelligent, superhuman abilities – primarily by AI learning how to improve itself, a process known as ‘recursive self-improvement’.
But we are nowhere near such superintelligence (a phrase Trump has latched on to, no doubt because it sounds sexy), say critics in the tech world.
The doomers have also been accused of ignoring the essential truth that AI software and mathematical models aren’t innately malicious or rebellious but – as with the Hugging Face hacking scandal – are simply trying to meet objectives set for it by its human designers.
The risks of AI, they counter, are vastly outweighed by the potential benefits, from economic productivity to medicine, military gains and much else.
These sceptics have dismissed previous warnings by Amodei and others as attempts to get free publicity – and they certainly might draw in investors and customers attracted by all the talk of how devastatingly powerful AI may be.
Critics believe there are other –rather more selfish – reasons behind the calls from Anthropic and OpenAI bosses to slow down research and have their industry better regulated.
The pair may be the biggest players in AI but a multitude of rivals are trying to catch up.
Some industry observers have suggested that Anthropic and OpenAI may be trying to preserve their substantial lead by suddenly demanding a general moratorium on research and thus cementing their duopoly.
They may have additional financial motives, too. Both companies have shelled out vast sums on AI research and development but are struggling to find paying customers.
So when ChatGPT owner Sam Altman – an entrepreneur hardly famous for his ethical approach – announced a few days ago that he was pausing OpenAI’s stock market launch for another year, supposedly amid concerns over AI safety, some whispered that he had simply latched on to the controversy as an excuse to delay matters until his company is in a better shape.
David Sacks, a venture capitalist who served as Trump’s ‘AI Tsar’ and now co-chairs the President’s council of science and technology advisers, has observed waspishly that AI leaders need to ‘stop pretending the motivation to “slow down” is purely altruistic’.
Sacks, who like others in Trump’s administration believes that claims of an AI apocalypse are a thinly-veiled attempt to damage the Republicans in the mid-terms, has dismissed the gloomy predictions as scaremongering.
‘We’ve seen this movie before,’ he told Bloomberg. ‘We saw it with global warming: they’ve taken some legitimate concerns and blown it out of proportion.’
And there are further reasons why many are increasingly taking the doomsayers with such a hefty pinch of salt.
The most obvious question is: if these bosses genuinely believe there is a high risk that AI could imminently kill us all, why haven’t they stopped already?
Sceptics also point out the supposed scenarios for AI Armageddon are unconvincingly woolly. Ask doomers how AI will wipe us out and they become vague.
A lot of the theorising hasn’t gone much further than the notorious ‘paperclip maximiser’ advanced by Oxford philosopher Nick Bostrom in 2003.
David Sacks,who served as Trump’s ‘AI Tsar’, has observed that AI leaders need to ‘stop pretending the motivation to “slow down” is purely altruistic’
Oxford philosopher Nick Bostrom has theorised how a superintelligent computer that is asked to focus entirely on a task would take the instruction to a total extreme
He outlined how a superintelligent computer, asked to focus entirely on producing as many paperclips as possible, would take the instruction to a total extreme, co-opting the world’s entire resources and destroying humanity after deciding they were an obstacle to making more clips (having first harvested the iron in our blood to make more paperclips). Thought-provoking, perhaps – but surely fanciful.
Some have suggested that AI could detonate a nuclear bomb, launch a global drone war, create ‘misinformation’ that could cause one country to attack another, or develop a lethal virus – though how it would physically do any of this is another matter entirely.
Alternatively, it could supposedly wipe us out by starving us of essential resources such as food and medicine. All these scenarios and more have been brandished by the doomers – though with few details on how it might actually happen.
Some experts complain that doomer predictions that the human race could be totally exterminated within years are unfeasible while an AI would find it very difficult to hack the entire internet. The real danger, they say, isn’t AI going rogue but humans misusing it.
Gary Marcus, a renowned cognitive scientist and emeritus professor at New York University, says the notion that AI will kill everyone within a few years is mostly ‘preposterous’.
Marcus believes AI could certainly be used to cripple infrastructure – perhaps hospitals or air or train networks but that the total extinction of the human race is unfeasible.
He and other sceptics also insist that it would be vanishingly difficult for an AI to hack the entire internet, as Anthropic’s boss this week suggested.
Sceptics also claim it is nonsensical for anyone to try to put a mathematical number on the likelihood of AI destroying us all and emphasise how the doomers hardly help themselves given that their estimates of how long such a catastrophe will take to occur vary so widely.
For instance, Jack Clark, Anthropic’s co-founder, estimates that AI won’t start ‘doing dangerous stuff’ for about 20 years.
Oren Etzioni, a professor at the University of Washington and founder of the Allen Institute for Artificial Intelligence, insists we are ‘very far away’ from AI polishing everyone off – and that the risk of a deadly virus escaping a lab, for example, is far higher.
Some claim that the AI Cassandras have essentially lost all perspective. Having devoted their careers to researching AI because they believe so strongly in its potential, they are more susceptible to over-emphasising its power – and by extension, perhaps, their own importance.
Ex-Anthropic staffer Jacob Coxon acknowledged as much when he told The New York Times: ‘I’ve been stuck in this field for three years. It could quite plausibly have turned me and my colleagues insane.’
President Trump may be somewhat blase in dismissing all safety concerns around AI as just a ‘hoax’.
But it seems premature to be counting the days before our robot overlords carry out the extinction of our species.
