‘Too little, too late’: critics perplexed and suspicious of AI leaders’ call for a slowdown
The safety debate that ignited when whistleblowers last week warned that the most advanced AI poses an existential threat to humanity should come with a health warning: side effects include whiplash. It was only seven days ago that AI enthusiasts were cheerily toying with OpenAI’s newest model GPT-6 Astra which was marketed in part as a lifestyle tool to help people book tennis courts, run fashion companies and order takeaways.
By Wednesday, cutting-edge AI started to look a lot less appealing when the resigning Anthropic researcher Jacob Coxon warned “the people building AI earnestly believe that it could kill us all by the end of the decade”. Yes, he was talking about artificial superintelligence – an as-yet-nonexistent version of the technology that outcompetes humans in most domains – but his comments, and the chorus of experts that echoed him, jangled plenty of nerves.
Another lurch came when Anthropic revealed it had found users dodging controls to use its existing models “in ways that could support biological weapons development” – think novel mosquito-borne or bird flu viruses. It was all enough to make lawmakers in London and Washington demand brakes and bans on the most potentially dangerous types of AI development.
And then this weekend, another sharp turn in the form of a three-point plan from the Anthropic boss, Dario Amodei, to “pace the frontier”. Never mind that his own employees had seeded the borderline panic only a few days earlier, Amodei was here to save us from fears of AI Armageddon, but not before his own spicy warning that the accelerating rate of capabilities meant a swarm of AI agents “could be capable of taking over the entire internet” in six to 12 months. Sceptics doubted that claim, but it was enough for some to declare it a “code red” moment.
In signs that a coordinated slowdown might not be just talk, some of Amodei’s precautionary ideas quickly attracted backing from his rivals – Sam Altman at OpenAI and Elon Musk at SpaceX. Perhaps there were reasons to be hopeful after a bumpy week: at least the AI companies were talking about a slowdown before anyone got badly hurt.
In a nutshell, Amodei proposed three moves. First, each US AI company grants ongoing access to embedded third-party evaluators to check compliance with safety commitments, report incidents, ensure new AI models are not misaligned. Until now, such watchdogs have only been brought in at the companies’ behest or given limited access to systems to investigate when things go wrong. In the absence of any government-level independent regulator, this is a start.
Second, he wants all companies in democratic countries building frontier AI models to establish common safety standards as well as limits on the rate of unchecked AI progress.
Third, the world’s democratic AI powers would coordinate with autocracies – notably China – to control the race. This is likely to be the hardest step. Amodei suggested a baby step could be a narrow agreement prohibiting obviously dangerous uses of AI, such as for the production of biological weapons. Donald Trump’s meeting with Xi Jinping in Washington DC on 24 September will test the prospects of any Sino-American coordination.
An immediate political problem is that Donald Trump appears reluctant to confess to any AI fears. Amodei’s plan requires US government action and Trump said last week he had no concerns about AI leading to human extinction. He doubled down on Sunday, saying people were “bringing up things that won’t happen”. Losing to China appears to be a bigger worry in Trump’s AI calculus.
“There is no day after tomorrow if China wins at this,” said Scott Bessent, the US treasury secretary, last week. “If they were to pull away from us on AI, then nothing else would matter.”
People close to Trump also wonder why Amodei and the rest of the AI leaders do not get on with pacing AI development themselves.
“The easiest way not to build superintelligence is for you to agree not to build it,” said David Sacks in response to Amodei’s blog. Sacks is co-chair of Trump’s council of advisers on science and technology.
after newsletter promotion
“Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system,” he told Amodei.
Another criticism of Amodei’s call for “pacing” of AI progress to allow safety measures and guardrails to keep up has came from Prof Stuart Russell, one of the world’s leading AI experts. Amodei proposed a slowdown to free up time to make advances in systems to keep the AIs aligned to human interests. But Russell said this was “completely backwards”.
“We don’t just set a slower rate of progress for capabilities and then hope that provides enough time to get the safety right,” he said. “We set the safety requirements and further progress occurs only when they are met. Imagine if a pharmaceutical company said: ‘We’re going to release a new cancer drug every year, and we hope that provides enough time for some clinical trials to be completed and for the results to be good.’”
AI experts not working for the big labs were also suspicious about a plan for a global regulatory framework for AI drafted by the leader of the world’s most-valuable AI company.
“Too little, too late,” said David Krueger, an AI professor, safety campaigner and former founding director of the UK government’s AI Security Institute. He added: “We need an immediate, indefinite, international moratorium on frontier AI development.”
Read the full story at The Guardian ↗ · The Guardian ↗
From the Trump administration to AI experts, plans by the Anthropic boss to boost safety have spawned a largely negative response. OpenAI boss and Elon Musk back slowdown on…
This lens runs the verified story through Cinnamon's AI — wired in the next step.
The safety debate that ignited when whistleblowers last week warned that the most advanced AI poses an existential threat to humanity should come with a health warning: side effects include whiplash. It was only seven days ago that AI enthusiasts were cheerily toying with OpenAI’s newest model GPT-6 Astra which was marketed in part as a lifestyle tool to help people book tennis courts, run fashion companies and order takeaways.
By Wednesday, cutting-edge AI started to look a lot less appealing when the resigning Anthropic researcher Jacob Coxon warned “the people building AI earnestly believe that it could kill us all by the end of the decade”. Yes, he was talking about artificial superintelligence – an as-yet-nonexistent version of the technology that outcompetes humans in most domains – but his comments, and the chorus of experts that echoed him, jangled plenty of nerves.
Another lurch came when Anthropic revealed it had found users dodging controls to use its existing models “in ways that could support biological weapons development” – think novel mosquito-borne or bird flu viruses. It was all enough to make lawmakers in London and Washington demand brakes and bans on the most potentially dangerous types of AI development.
And then this weekend, another sharp turn in the form of a three-point plan from the Anthropic boss, Dario Amodei, to “pace the frontier”. Never mind that his own employees had seeded the borderline panic only a few days earlier, Amodei was here to save us from fears of AI Armageddon, but not before his own spicy warning that the accelerating rate of capabilities meant a swarm of AI agents “could be capable of taking over the entire internet” in six to 12 months. Sceptics doubted that claim, but it was enough for some to declare it a “code red” moment.
In signs that a coordinated slowdown might not be just talk, some of Amodei’s precautionary ideas quickly attracted backing from his rivals – Sam Altman at OpenAI and Elon Musk at SpaceX. Perhaps there were reasons to be hopeful after a bumpy week: at least the AI companies were talking about a slowdown before anyone got badly hurt.
In a nutshell, Amodei proposed three moves. First, each US AI company grants ongoing access to embedded third-party evaluators to check compliance with safety commitments, report incidents, ensure new AI models are not misaligned. Until now, such watchdogs have only been brought in at the companies’ behest or given limited access to systems to investigate when things go wrong. In the absence of any government-level independent regulator, this is a start.
Second, he wants all companies in democratic countries building frontier AI models to establish common safety standards as well as limits on the rate of unchecked AI progress.
Third, the world’s democratic AI powers would coordinate with autocracies – notably China – to control the race. This is likely to be the hardest step. Amodei suggested a baby step could be a narrow agreement prohibiting obviously dangerous uses of AI, such as for the production of biological weapons. Donald Trump’s meeting with Xi Jinping in Washington DC on 24 September will test the prospects of any Sino-American coordination.
An immediate political problem is that Donald Trump appears reluctant to confess to any AI fears. Amodei’s plan requires US government action and Trump said last week he had no concerns about AI leading to human extinction. He doubled down on Sunday, saying people were “bringing up things that won’t happen”. Losing to China appears to be a bigger worry in Trump’s AI calculus.
“There is no day after tomorrow if China wins at this,” said Scott Bessent, the US treasury secretary, last week. “If they were to pull away from us on AI, then nothing else would matter.”
People close to Trump also wonder why Amodei and the rest of the AI leaders do not get on with pacing AI development themselves.
“The easiest way not to build superintelligence is for you to agree not to build it,” said David Sacks in response to Amodei’s blog. Sacks is co-chair of Trump’s council of advisers on science and technology.
after newsletter promotion
“Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system,” he told Amodei.
Another criticism of Amodei’s call for “pacing” of AI progress to allow safety measures and guardrails to keep up has came from Prof Stuart Russell, one of the world’s leading AI experts. Amodei proposed a slowdown to free up time to make advances in systems to keep the AIs aligned to human interests. But Russell said this was “completely backwards”.
“We don’t just set a slower rate of progress for capabilities and then hope that provides enough time to get the safety right,” he said. “We set the safety requirements and further progress occurs only when they are met. Imagine if a pharmaceutical company said: ‘We’re going to release a new cancer drug every year, and we hope that provides enough time for some clinical trials to be completed and for the results to be good.’”
AI experts not working for the big labs were also suspicious about a plan for a global regulatory framework for AI drafted by the leader of the world’s most-valuable AI company.
“Too little, too late,” said David Krueger, an AI professor, safety campaigner and former founding director of the UK government’s AI Security Institute. He added: “We need an immediate, indefinite, international moratorium on frontier AI development.”
Read the full story at The Guardian ↗ · The Guardian ↗
The safety debate that ignited when whistleblowers last week warned that the most advanced AI poses an existential threat to humanity should come with a health warning: side effects include whiplash. It was only seven days ago that AI enthusiasts were cheerily toying with OpenAI’s newest model GPT-6 Astra which was marketed in part as a lifestyle tool to help people book tennis courts, run fashion companies and order takeaways.
By Wednesday, cutting-edge AI started to look a lot less appealing when the resigning Anthropic researcher Jacob Coxon warned “the people building AI earnestly believe that it could kill us all by the end of the decade”. Yes, he was talking about artificial superintelligence – an as-yet-nonexistent version of the technology that outcompetes humans in most domains – but his comments, and the chorus of experts that echoed him, jangled plenty of nerves.
Another lurch came when Anthropic revealed it had found users dodging controls to use its existing models “in ways that could support biological weapons development” – think novel mosquito-borne or bird flu viruses. It was all enough to make lawmakers in London and Washington demand brakes and bans on the most potentially dangerous types of AI development.
And then this weekend, another sharp turn in the form of a three-point plan from the Anthropic boss, Dario Amodei, to “pace the frontier”. Never mind that his own employees had seeded the borderline panic only a few days earlier, Amodei was here to save us from fears of AI Armageddon, but not before his own spicy warning that the accelerating rate of capabilities meant a swarm of AI agents “could be capable of taking over the entire internet” in six to 12 months. Sceptics doubted that claim, but it was enough for some to declare it a “code red” moment.
In signs that a coordinated slowdown might not be just talk, some of Amodei’s precautionary ideas quickly attracted backing from his rivals – Sam Altman at OpenAI and Elon Musk at SpaceX. Perhaps there were reasons to be hopeful after a bumpy week: at least the AI companies were talking about a slowdown before anyone got badly hurt.
In a nutshell, Amodei proposed three moves. First, each US AI company grants ongoing access to embedded third-party evaluators to check compliance with safety commitments, report incidents, ensure new AI models are not misaligned. Until now, such watchdogs have only been brought in at the companies’ behest or given limited access to systems to investigate when things go wrong. In the absence of any government-level independent regulator, this is a start.
Second, he wants all companies in democratic countries building frontier AI models to establish common safety standards as well as limits on the rate of unchecked AI progress.
Third, the world’s democratic AI powers would coordinate with autocracies – notably China – to control the race. This is likely to be the hardest step. Amodei suggested a baby step could be a narrow agreement prohibiting obviously dangerous uses of AI, such as for the production of biological weapons. Donald Trump’s meeting with Xi Jinping in Washington DC on 24 September will test the prospects of any Sino-American coordination.
An immediate political problem is that Donald Trump appears reluctant to confess to any AI fears. Amodei’s plan requires US government action and Trump said last week he had no concerns about AI leading to human extinction. He doubled down on Sunday, saying people were “bringing up things that won’t happen”. Losing to China appears to be a bigger worry in Trump’s AI calculus.
“There is no day after tomorrow if China wins at this,” said Scott Bessent, the US treasury secretary, last week. “If they were to pull away from us on AI, then nothing else would matter.”
People close to Trump also wonder why Amodei and the rest of the AI leaders do not get on with pacing AI development themselves.
“The easiest way not to build superintelligence is for you to agree not to build it,” said David Sacks in response to Amodei’s blog. Sacks is co-chair of Trump’s council of advisers on science and technology.
after newsletter promotion
“Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system,” he told Amodei.
Another criticism of Amodei’s call for “pacing” of AI progress to allow safety measures and guardrails to keep up has came from Prof Stuart Russell, one of the world’s leading AI experts. Amodei proposed a slowdown to free up time to make advances in systems to keep the AIs aligned to human interests. But Russell said this was “completely backwards”.
“We don’t just set a slower rate of progress for capabilities and then hope that provides enough time to get the safety right,” he said. “We set the safety requirements and further progress occurs only when they are met. Imagine if a pharmaceutical company said: ‘We’re going to release a new cancer drug every year, and we hope that provides enough time for some clinical trials to be completed and for the results to be good.’”
AI experts not working for the big labs were also suspicious about a plan for a global regulatory framework for AI drafted by the leader of the world’s most-valuable AI company.
“Too little, too late,” said David Krueger, an AI professor, safety campaigner and former founding director of the UK government’s AI Security Institute. He added: “We need an immediate, indefinite, international moratorium on frontier AI development.”
Read the full story at The Guardian ↗ · The Guardian ↗
This lens runs the verified story through Cinnamon's AI — wired in the next step.
- From the Trump administration to AI experts, plans by the Anthropic boss to boost safety have spawned a largely negative response.
- OpenAI boss and Elon Musk back slowdown on…