HalluSquatting: How AI Hallucinations Are Being Turned Into a Doorway for Malware
Security researchers have found a way to turn a known quirk of AI chatbots into a method for delivering malicious software directly to developers' computers, without hacking the AI itself.
Key points
- Researchers demonstrated a technique called "HalluSquatting" that turns AI hallucinations into a delivery route for botnet malware.
- AI hallucinations, meaning moments when an AI assistant confidently invents something that does not exist, can be weaponised to point developers toward fake software packages.
- Attackers register fake packages using names the AI makes up, then load those packages with malicious code.
- The attack can result in remote code execution, meaning criminals gain the ability to run any command they choose on a victim's computer.
- No breach of the AI system itself is required.
AI assistants like ChatGPT or GitHub Copilot occasionally make things up. They might recommend a software library, a tool, or a package that sounds plausible but simply does not exist. This is called a hallucination, and until now it was mostly treated as an embarrassing quirk.
Researchers have found a way to make it dangerous.
The technique, which SecurityWeek reported under the name "HalluSquatting", works in two steps. First, researchers identified software package names that popular AI coding assistants regularly invent during hallucinations. Second, they registered those made-up names on real software repositories, the public libraries where developers download code, and filled the packages with malicious software.
Why should developers be worried?
Developers are the direct target here, and the risk is real. When a developer asks an AI assistant to help write code, the assistant might recommend a package by a name it invented. The developer searches for it, finds a real-looking package sitting in a legitimate repository, downloads it, and runs it. At that point the attacker's code runs too.
That outcome is called remote code execution. It means the criminal can control the victim's machine: copying files, installing further malware, or pulling the computer into a botnet, which is a network of hijacked machines criminals use to send spam or launch attacks on other targets.
No password was stolen. No phishing email, where criminals send fake messages to trick people into handing over credentials, was needed. The AI's habit of inventing plausible-sounding names did the groundwork for free.
The attack is a variation on typosquatting, a well-known trick where criminals register package names that are one letter away from popular, legitimate ones, hoping developers mistype. HalluSquatting removes the need for a typo entirely. The AI provides the bait on its own.
Organisations whose developers use AI coding tools should treat AI-generated package recommendations with the same scepticism they apply to an email from an unknown sender. Verify a package exists and is maintained before running it. Several software supply-chain security tools can scan packages before installation and flag ones with no history or suspicious code.
No regulator has issued guidance on this specific technique yet, though supply-chain security broadly falls within scope of guidance from bodies including the US Cybersecurity and Infrastructure Security Agency.
What developers should do now:
- Cross-check any package name an AI suggests against the official repository before downloading it.
- Prefer packages with long histories, many downloads, and verified maintainers.
- Use a dependency-scanning tool, software that checks packages for known malicious code, before including anything new in a project.
- Report hallucinated package names to your AI vendor so they can filter them.
