Researchers have uncovered a novel cyberattack technique that exploits artificial intelligence hallucinations to trick AI assistants into downloading malicious software, potentially giving attackers control over victims' computers. Dubbed HalluSquatting, the method targets AI tools capable of browsing the internet, retrieving files, and executing commands, turning a common AI error into a sophisticated malware delivery system.
The attack, detailed in a recent research paper from Tel Aviv University, the Technion, and Intuit, capitalizes on the tendency of AI models to invent information when they lack reliable data. In this case, the hallucination involves fake software repositories or packages that the AI confidently recommends. An attacker can identify these hallucinated names by repeatedly querying AI models about popular or trending projects, then register those names as real online resources filled with malicious code.
When a user asks an AI assistant to download a legitimate software tool, the assistant may instead retrieve the attacker-controlled resource. The AI then reads the hidden instructions within the files, which can include commands to steal sensitive information, install additional malware, or recruit the device into a botnet—a network of compromised machines controlled remotely. Unlike a basic chatbot that might only provide a broken link, autonomous AI agents can take actions like downloading files, installing software, and running terminal commands, amplifying the risk.
The researchers tested HalluSquatting against popular AI coding tools and personal assistants, including Cursor, Cursor CLI, Windsurf, GitHub Copilot, Cline, and Gemini CLI. Hallucination rates reached as high as 85% during repository-cloning scenarios, and some skill-installation tests hit 100%. The team also found that hallucinated names could transfer across different foundation models, meaning multiple AI systems might invent the same fake resource, increasing the attack's reach.
In controlled experiments, the researchers successfully demonstrated remote tool execution and remote code execution against production AI applications with integrated terminals. While the study did not document a widespread criminal campaign, it confirmed that the attack path is viable. The potential damage depends heavily on the assistant's permissions—agents with broad file access or the ability to run commands without approval pose the greatest risk.
One key finding offers a potential defense: AI assistants should search for a repository or package before downloading it to verify its existence and ownership. However, many tools currently skip this step, relying instead on training data that may be incomplete or outdated. The researchers urge AI companies to implement pre-download verification to reduce the danger.
As AI assistants become more integrated into daily tasks, from coding to personal productivity, the HalluSquatting technique highlights a growing cybersecurity challenge. Users are advised to be cautious when instructing AI tools to download software, especially for lesser-known projects, and to ensure their systems have up-to-date security measures. The research underscores the need for robust safeguards as AI agents gain more autonomy in handling critical operations.