Google AI zero-day exploit

    by Tim Official: Cybersecurity

    Google has confirmed the first case of hackers using AI to create a zero-day exploit from scratch. An AI model discovered a vulnerability that no human had ever found, turned it into a working weapon, and aimed it at a mass exploitation campaign targeting thousands of systems. Google's Threat Intelligence Group caught it yesterday and shut down the operation before it scaled. But the details of how it worked are genuinely alarming: The AI found a flaw in a popular two-factor authentication system that traditional security tools had completely missed. The vulnerability was a logic error buried deep in the authentication flow, where a developer had hard-coded a trust exception years ago. No human security researcher or automated scanner had caught it. The flaw was invisible to every tool the cybersecurity industry has built over the past two decades. But the AI spotted it immediately. Then it wrote a full Python exploit script to weaponize it. Google's analysts could tell the code was AI-generated because it had textbook formatting, educational comments explaining every function, and even a hallucinated severity score that doesn't exist in any real database. The AI literally graded its own attack with a fake rating. So the code had mistakes in it. The criminals' implementation was clumsy enough that it probably interfered with the actual deployment. This was the sloppy first attempt by people who are still learning how to use these tools. And it still found a vulnerability that the entire cybersecurity industry missed. Google's chief threat analyst John Hultquist said: "There's a misconception that the AI vulnerability race is imminent. The reality is that it's already begun. For every zero-day we can trace back to AI, there are probably many more out there." But here's where it gets truly alarming... This wasn't even a sophisticated operation. North Korea's APT45 hacking unit is sending thousands of repetitive prompts to AI models, recursively analyzing known vulnerabilities and building an entire exploit arsenal that would be physically impossible for human hackers to assemble at the same speed. They're essentially industrializing cyberattacks. A Chinese state-linked group jailbroke Google's own Gemini by simply asking it to "pretend to be a network security expert" and then used that persona to research how to hack TP-Link routers and corporate file transfer systems. Another Chinese group deployed autonomous AI agents that probed a Japanese tech firm with minimal human oversight, deciding on their own which tools to use and pivoting between targets based on internal reasoning. And then there's PROMPTSPY, an Android backdoor that calls Google's Gemini API to read your phone screen in real time, navigate your interface autonomously, capture your biometric data, replay your lock screen PIN, and block you from uninstalling it by placing an invisible overlay over the uninstall button. It literally operates your phone using commercial AI tools anyone can access. Everyone spent the last 3 years arguing about whether AI would take people's jobs. Meanwhile AI is making every password, every firewall, and every two-factor authentication system on Earth fundamentally less secure. The entire $190 billion cybersecurity industry was built on one assumption: that finding vulnerabilities is hard and requires deep expertise. But AI just removed that assumption from the equation. And the scariest part is that Google said the criminals made errors this time. The implementation was rough and the campaign probably didn't fully work. These were amateurs. Now imagine what professionals are able to do. There's a reason Sam Altman predicted an inevitable massive cyberattack this year. What do you think?

    Transcript (en)

    I suspect in the next year we will see significant threats we have to mitigate from cyber. And these models are already quite capable and will get much more capable. And then on bio, this is something we've been talking about a lot. The models are clearly going to get very good at helping people do biology at an advanced level. Wonderful things are going to happen there. We'll see a bunch of diseases get cured. word, someone is going to try to misuse those. And for now, when the models, the frontier models are all sort of in the hands of pretty responsible companies, I think we can mitigate those by the companies aligning the models and having good classifiers and good safety stacks. But we're not that far away from a world where there are incredibly capable open source models that are very good at biology and the need for society to be resilient to terrorist groups using these models to try to create novel pathogens is like, that's no longer a theoretical thing, or it's not going to be for much longer. So part of the reason that we are trying to push also in this blueprint ideas around societal resilience is a realization that AI safety or safety in a world of powerful AI cannot be done by the companies alone. When you mentioned cyber in the next year, something that Jim and I have heard a lot from the AI companies is there could well be a world-shaking cyber attack this year that would get people's attention. It sounds like you agree with that. I think that's totally possible, yes. I think to avoid that, it will require a tremendous amount of work.