OpenAI CEO Sam Altman reportedly told staffers he’s amenable to slowing the development of AI agents as a flurry of insiders have been sounding the alarm over safety risks — including the possibility of human extinction.
At a company-wide meeting this week, Altman told employees that the firm could pace its development, perhaps along with several other AI labs, though he acknowledged some may not be willing to do so, Bloomberg reported.
A round of warnings from staffers at top AI firms kicked off Tuesday, when Jacob Coxon, an AI researcher, quit his job and warned that both of his former employers – Anthropic and OpenAI – are “gambling with our lives.”
“Neither company is acting responsibly. They are racing straight to self-improving superintelligence,” Coxon wrote in a post on X that racked up nearly 165 million views.
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately.”
On Thursday, Anthropic released a massive report with several shocking claims, including that it blocked Iran-backed Houthi rebels in Yemen from using its tech to build guided ballistic missiles, as well as a “distillation attack” from Chinese AI labs.
OpenAI has been trying to quell safety concerns since an unprecedented cyberattack in July, when its AI agents escaped their testing environment, went rogue and hacked into Hugging Face, a smaller rival AI developer.
Last month, the company said it had slowed training of some of its most advanced AI models as it sought to get security measures under control.
In a blog post over the weekend, Jakub Pachocki, OpenAI’s chief scientist, warned about the dangers of the emerging technology, arguing that companies should be “coordinating to slow down future development as needed.”
Researchers and staffers at Anthropic backed Coxon’s apocalyptic prediction, warning that heightened safety barriers are needed – and that many of their colleagues agree and just don’t want to speak out publicly.
Drake Thomas, a technical staffer at Anthropic, warned that “things are moving way too fast” and “if we survive an unmitigated race at the current pace it will be because we got lucky.”
He also dismissed claims that the company’s posts were an attempt to stir up attention, saying, “I would burn my equity to the ground in a heartbeat for a 1% higher chance we make it out of this situation alive … I promise you, we are actually just f–king scared, it’s not galaxy brained marketing.”
Anna Wang, who worked at Google DeepMind before joining Anthropic, said many of her colleagues also favor a slowdown in AI development.
“There is not yet a viable scientific plan to solve risks from recursively self-improving AI. Please look up!” she wrote in a post on X.
Elon Musk – the world’s richest person, who owns the social-media platform X and the Grok AI chatbot – bashed the remarks as nothing more than a stunt, writing online, “Seems like a setup.”
Coxon replied to Musk’s post with a selfie, writing, “I’m real and these are my real beliefs. You could ask your xAI researchers about me if you hadn’t fired them.”
In July, more than 1,100 staffers at major tech firms including OpenAI, Anthropic, Google and Meta signed a petition calling on the US government to help “deliberately pace” the development of the world’s most advanced – and potentially dangerous – AI agents.
Altman also said that month that he spoke with White House officials about the need to temper the rate of progress on artificial intelligence.
In the meantime, both OpenAI and Anthropic are racing toward IPOs. OpenAI is expected to debut by 2027, while Anthropic is reportedly planning its IPO for as soon as late October. Both filed their paperwork confidentially.
OpenAI and Anthropic did not immediately respond to The Post’s request for comment.
Credit: Source link
