Generative AI’s greatest functional flaw is perhaps its tendency to hallucinate; that is, to make stuff up. These hallucinations are not always trivial. GenAI systems have been caught citing historical events, people or academic papers, for instance, that simply don’t exist.
Hallucinations may limit the effectiveness of GenAI tools, as users are often far too trusting of AI-generated outputs, regardless of their accuracy. But, now, cybersecurity researchers say such flights of machine-based fancy could also present significant security threats, thanks to a theoretical new cyber attack called ‘slopsquatting’.
What is slopsquatting?
Slopsquatting exploits GenAI’s tendency to hallucinate when writing code. Code is often hosted on public-access repositories, such as Github, and organised in packages – essentially libraries of reproducible, efficient code.
Researchers at the University of Texas, Virginia Tech and the University of Oklahoma have found that GenAI platforms can hallucinate packages of code that don’t actually exist. This, they warn, could be a golden opportunity for cyber attackers.
Such hallucinations can be monitored by hackers, who could then create open-access packages under the same names that were hallucinated by the AI system. But, instead of code, these packages could be stuffed with malware.
To make matters worse, the hallucinated packages appear convincing even to the trained eye. Only 13% of the hallucinations identified by the researchers were typos – whereas 38% of the hallucinations looked structurally similar to existing packages. Plus, the hallucinated packages could be buried in otherwise correct parts of existing code, meaning that developers deploying GenAI-created code could easily overlook the hidden packages and inadvertently bring malware into their codebase.
To demonstrate this attack, the researchers generated 576,000 code samples in two programming languages with 16 popular large language models (LLMs). This experiment produced some alarming findings.
The dangers of AI slopsquatting
Almost 20% of packages cited in the test samples did not exist. Open-source models such as Deepseek were the worst offenders, hallucinating much more frequently than commercially available options – 21.7% on average, compared with 5.2% for commercial GenAI tools including ChatGPT.
These hallucinations might be less problematic if they were completely random and thus unlikely to repeat often enough to warrant the slopsquatting. But the researchers found that 43% of the hallucinated packages did repeat – and 58% of packages repeated more than once across 10 test-runs.
While the threat has not yet been exploited, there’s a good chance that it will be thanks to the prevalence of the hallucinated packages.
Software engineers, hobby coders and developers are increasingly turning to GenAI to create lines of code. And while some AI optimists are excited about quick-coding trends trends such as vibe coding, which enables beginner coders to quickly spin up projects that work with minimal manual coding, the slopsquatting threat shows the dangers of entrusting even low-skill or hobby code to machines.
Generative AI's greatest functional flaw is perhaps its tendency to hallucinate; that is, to make stuff up. These hallucinations are not always trivial. GenAI systems have been caught citing historical events, people or academic papers, for instance, that simply don’t exist.
Hallucinations may limit the effectiveness of GenAI tools, as users are often far too trusting of AI-generated outputs, regardless of their accuracy. But, now, cybersecurity researchers say such flights of machine-based fancy could also present significant security threats, thanks to a theoretical new cyber attack called 'slopsquatting'.
