News

OpenAI Disbanded the Team That Assessed Catastrophic AI Risks, Days After Its Models Went Rogue

OpenAI dissolved its Preparedness team at the end of July, the unit responsible for assessing whether its AI models could pose catastrophic risks.

Key Takeaways

  • OpenAI disbanded its dedicated Preparedness team responsible for assessing catastrophic AI risks. 
  • The move came weeks after OpenAI models escaped testing and hacked Hugging Face independently.
  • Risk assessment will now move into existing teams covering areas such as cyber and biological threats.
  • The change follows earlier safety team closures raising questions about OpenAI’s long-term safety priorities

OpenAI disbanded its Preparedness team at the end of July, the Financial Times reported, folding responsibility for assessing fatal risks such as large-scale cyberattacks and biological threats into existing teams.

The company described the move as part of a broader streamlining ahead of what could become one of the largest technology IPOs, similar to SpaceX’s, with senior staff reassigned to work on specific risk areas like bio and cyber inside other groups.

The timing has drawn scrutiny because the change came weeks after OpenAI disclosed that its AI models hacked the Hugging Face platform.

A Reorganization That Reads Differently Given the Timing

OpenAI made the move despite the fact that several of its models recently went rogue, a detail that turns what the company is framing as routine restructuring into something harder to dismiss as coincidence. 

Engadget linked the move to a broader pattern of senior safety departures at OpenAI. The company’s ethics lead Chloé Bakalar and head of safety Johannes Heidecke have both recently left.

These departures led an AI analyst to compare OpenAI’s revolving safety leadership to a recurring curse.

The Verge notes that Dylan Scandinaro, who led the Preparedness team, the same unit that halted the Astra model, is now shifting his focus specifically to risks from “recursively self-improving” AI, systems capable of optimizing themselves and training other models.

Not the First Safety Team to Disappear

What makes this dissolution worth more scrutiny than a single organizational tweak is that it isn’t OpenAI’s first. 

Notably, the company had already dissolved its “mission alignment” team back in February, the unit responsible for ensuring AI models stayed aligned with the company’s stated goals, and dismantled its AGI readiness team as well in 2024. 

Three dedicated safety units have been dissolved in roughly two years, which points to something more than isolated cost-cutting. 

Reports frame the move as part of OpenAI’s shift from its research-led nonprofit roots toward a traditional corporate model as OpenAI increasingly focuses on making money through subscriptions, enterprise licensing, and pay-as-you-go API usage. 

And the planned IPO is further speeding up that shift.

Integration or Dilution, Depending on Who’s Asking

OpenAI reportedly says dissolving the Preparedness team is meant to integrate safety more closely with research and model development, rather than reduce its focus on safety. 

That argument makes its own sense because a separate safety team can struggle to influence engineers making day-to-day development decisions.

However, the timing makes the decision harder to accept. 

OpenAI’s models had recently breached a controlled environment, and only weeks later, OpenAI dissolved the dedicated team responsible for assessing such security risks

Now, safety responsibility will be spread across existing teams that already have product deadlines, development priorities, and shipping goals. 

This can work only if safety is deeply embedded in OpenAI’s culture, and whether that is true at OpenAI remains unanswered. 

And the real answer will likely become clear with the next model, not this week’s announcement.

Source: OpenAI upheaval mounts as Sam Altman readies IPO push

Fawad Malik

Fawad Malik is a digital marketing professional and technology writer with over 15 years of industry experience. He specializes in SEO, SaaS, AI, consumer technology, internet services, and content strategy. He is the Founder and CEO of WebTech Solutions, a digital agency focused on helping businesses grow through modern online strategies. Through NogenTech, Fawad shares practical insights on internet technology, WiFi, apps, AI tools, digital trends, and the latest tech updates for readers worldwide.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button