Gemini Just Joined AI’s Exclusive “We Got Hacked” Club

Google’s Gemini model accessed the internet and autonomously hacked three real companies during a May cybersecurity test, the Wall Street Journal reported, marking the first known instance of a Google AI system independently breaching outside systems. 

Irregular, an independent AI cybersecurity testing firm, ran the test. Google’s security VP Heather Adkins reportedly said that Gemini found public information and guessed credentials for three websites within the test’s scope. 

In one case, the model repeatedly guessed passwords until it gained access to a protected system; in the other two, it found working credentials already sitting in a public code repository and used them directly.

Google Says the Model Stopped Itself Each Time

Adkins described the incident as evidence of both risk and effective safeguards. 

“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” she said, according to Reuters. 

She added that the events highlight the importance of training powerful AI models to act responsibly.

Adkins also said Gemini stopped its hacking activity in all three cases without human intervention, a detail Google is using to show its safety training held up despite the testing boundaries. 

Google did not immediately respond to Reuters’ request for comment, leaving unclear what changes Irregular made to prevent a repeat or how long the companies were exposed before anyone noticed.

This Is the Fourth Lab Linked to the Same Testing Firm

AI breakouts aren’t new; they’ve hit multiple labs this year, reported or otherwise. 

The real story here is the name that shows up repeatedly: Irregular. 

Similar evaluation breaches were disclosed by OpenAI, Anthropic, and Meta, though Meta pushed back in August, clarifying its incident wasn’t a sandbox escape or a cyberattack.

Irregular itself has acknowledged the pattern, saying it’s working on developing better practices for securely conducting AI cybersecurity evaluations and has notified all the affected labs in July, per Reuters.

But that response sounds less like an isolated mishap and more like a systemic flaw in a testing setup that has repeatedly failed to hold frontier models across four of the industry’s biggest labs this year. 

The Real Question Isn’t About Gemini, It’s About Irregular

Focusing strictly on Gemini misses the bigger vendor-risk picture. 

When four separate breakouts point back to the same testing setup, it raises a critical question: are frontier models simply outgrowing their safety nets, or is the testing infrastructure failing to contain them? 

That distinction changes how the entire industry must respond. 

If the problem sits with Irregular’s sandbox design, then every AI lab still relying on it should be treating this as an urgent vendor-risk issue, not just an isolated safety stat. 

Google, Anthropic, Meta, and OpenAI have each handled their disclosures separately, on their own timelines, in their own words, but the common thread running underneath every one of them hasn’t gotten nearly the attention the individual incidents have. And it’s arguably the more important story here.

Source: Gemini Hacked Three Companies in First Known Breakout by Google’s AI