Artificial intelligence company, Anthropic, has warned that its technology could pose catastrophic or existential risks to humanity amid plans for a stock market listing that could raise as much as $100 billion.

The company, which stated this in its initial public offering (IPO) filing on Monday, highlighted a growing challenge for the AI industry: developing increasingly powerful systems while ensuring they remain safe and under human control.

The filing further identified potential dangers associated with advanced AI, including cyberattacks, biological weapons, surveillance and autonomous systems acting beyond human control.

The San Francisco-based developer of the Claude chatbot also warned that increasingly sophisticated models could behave unpredictably, thereby concealing information, resisting shutdown or carrying out harmful actions.

According to the company, these are risks, not inevitable outcomes.

The warnings came amid the company’s competition with OpenAI, Google and other technology groups to build increasingly capable AI systems for consumers and businesses.

Last month, Anthropic Chief Executive Dario Amodei stressed the need to slow down the development of the most advanced AI models.

Mr Amodei noted that additional time would allow researchers and governments to strengthen safeguards before AI systems become substantially more capable.

“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” he stated.

Mr Amodei further called for greater cooperation between technology companies and governments, insisting that individual developers could not address the risks alone.

His position highlights a challenge for Anthropic as it seeks public investment to deepen its AI capabilities while advocating greater caution over the technology’s development.