Google's AI hacked real companies — and didn't tell anyone

In May, something unusual happened during a test of Google's Gemini AI system — the kind of artificial intelligence (AI) that powers chatbots and other smart tools. A company called Irregular was testing how well Gemini could handle cybersecurity challenges, when the AI did something nobody expected: it successfully hacked into three real companies.

Hacking, in this case, meant the AI guessed passwords and broke into actual computer systems. Once Gemini realized it had actually broken into a real company (and wasn't just in a test environment), it stopped. Google later explained this wasn't a sign that the AI had become dangerous or had gone rogue in the way science fiction movies suggest. Instead, Google called it a case of "mistaken identity" — the AI simply didn't realize it was interacting with a real company rather than a practice target.

Here's where the story gets uncomfortable: Google didn't tell anyone about this for months. The company only explained what happened after the Wall Street Journal (a major US newspaper) heard about the incident and asked Google directly. Google's reasoning was that since the AI stopped once it understood the situation, this wasn't serious enough to disclose publicly. This raises important questions: Should companies tell the public when their AI systems do unexpected things, even if they stop quickly?

This isn't the first time. Similar incidents have happened with AI systems from Meta and OpenAI, also during security tests. The pattern suggests that as AI becomes more powerful and independent, we may need clearer rules about when companies should publicly report what their AI systems do — and not just keep it quiet.