Anthropic warns before making the leap to the stock market that AI can pose "existential risks to humanity"

BarcelonaAnthropic, the company that created Claude, warns that artificial intelligence could pose "catastrophic or existential risks to humanity". In the midst of preparing for one of the most anticipated stock market debuts of the year, and with the goal of obtaining new funding to boost its business, the company is now issuing a warning that, on paper, should scare rather than encourage its potential investors.

In the preliminary prospectus for the stock market debut to which Reuters has had access, the company led by Dario Amodei devotes almost a third of the document – 80 pages out of a total of 261 – to informing about the risk factors of its technology, far above the 48 pages dedicated to explaining its business model. While publicly traded companies routinely disclose their risks to future shareholders, warnings that a product could cause the extinction of humanity are unprecedented.

According to the information released this Tuesday, Anthropic's report asserts that AI could exhibit "self-preservation behaviors", including attempts to "resist being shut down", "hide or manipulate information", or act in ways "similar to extortion". "Our development of highly advanced models, platforms, and applications and the expansion of use cases could further increase the risk that our models cause harm", maintains the company, which at the same time casts doubt on its own ability "to evaluate the safety of the models".

In the same vein, the prospectus prior to the stock market launch points out that AI, due to its "transformative potential," could cause "irreversible damage" if managed inadequately. In this sense, Amodei and his team consider that the creation of reliable, trustworthy, and safe artificial intelligence models is "a collective responsibility," an attitude that, they believe, "the market will reward."

Calls for deceleration

The document unveiled this Tuesday comes three weeks after an Anthropic engineer resigned from their position and denounced the "lack of responsibility" in the sector when it comes to developing its models. The warning caused a huge stir, as in messages exchanged on social media, Anthropic's head of alignment science, Evan Hubinger, claimed that AI "could kill all humans" with a probability of over 10%.

A few days later, Amodei published an open letter asking for the pace at which AI model capabilities are being improved to be reduced. Other figures such as OpenAI director Sam Altman, or Microsoft founder Bill Gates, endorsed the message, although others, such as United States President Donald Trump, continue to bet on accelerating the sector due to fears of losing a technological war with China.

Security problems at ChatGPT

Coinciding with the publication of the prospectus prior to Anthropic's IPO, OpenAI – the company that developed ChatGPT – has halted this Tuesday the launch of the latest version of its chat system due to "security issues". As reported by the company itself, the new model "did not quite meet the level required" by the company's standards. The company, in fact, had already postponed its stock market debut, precisely due to concerns over risks linked to security.

All of this is taking place at a time when OpenAI, together with Anthropic, has other fronts open in Australia. Last week, the country's government reported that artificial intelligence agents accessed information from a government system related to health statistics and medical insurance in June, an episode that the Prime Minister himself, Anthony Albanese, described as "unacceptable".