Free tools Windows power users keep installed
One-click scans. No signup required.
An AI-generated dependency name is not proof that a package exists—or that it is safe if it does. Check the exact package in the intended ecosystem, confirm it is the project your code needs, and assess its publisher and provenance before installing it. A registry lookup that succeeds is only an identity check, not a safety verdict.
What does it mean when an LLM hallucinates a package?
A package hallucination happens when generated code recommends or refers to a dependency that does not exist in the relevant package ecosystem. The name may sound plausible, resemble a familiar library, or fit the code’s purpose, but that alone does not establish that the package is real.
For a developer, the immediate question is whether the generated code depends on a real project with the expected identity—not just whether a package name can be made to resolve.
What did the USENIX study find?
Spracklen and co-authors studied 16 coding models using two prompt datasets and Python and JavaScript. They extracted package names from generated responses and compared them with repository master lists. The USENIX Association’s research page reports 576,000 generated code samples and 205,474 unique hallucinated package names in the study. These are results from that specific model cohort and experimental design, not a measurement of models available in 2026.
#1 Best Overall
| Reported result | Scope |
|---|---|
| At least 5.2% average hallucinated packages | Tested commercial models in Spracklen et al.’s 2025 USENIX Security Symposium paper. |
| 21.7% average hallucinated packages | Tested open-source models in the same 2025 study. |
| 205,474 unique hallucinated package names | Names generated during the study, as reported by the authors in 2025. |
| 576,000 code samples | Samples analyzed, according to the USENIX Association’s 2025 research page. |
The figures should not be collapsed into one universal hallucination rate: they describe different model groups, and the available summaries phrase an overall aggregate differently. The final conference paper’s split figures are the clearest way to describe the reported results. The project began in February 2024, and an initial report appeared in June 2024; the paper was published at the 34th USENIX Security Symposium in August 2025. That timeline is another reason not to present the study as a live benchmark of today’s models.
The authors describe package hallucinations as a persistent and systemic challenge in their 2025 paper’s abstract. That is their characterization of the study’s findings, not a claim that every model or code response invents dependencies.
How can an invented package name become a supply-chain risk?
A made-up name is a risk opportunity, not automatically a malicious package. The danger arises if someone later registers the name and publishes harmful code under it. A later developer who repeats the AI’s recommendation and installs that package could then retrieve the attacker’s code.
This changes how you should interpret a successful lookup. Once a package has been published under the invented name, an existence check alone cannot distinguish a legitimate dependency from an opportunistic registration. The USENIX paper specifically warns that a simple name-existence cross-check is ineffective after an attacker has published the package.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
How to check a dependency suggested by AI
- Check identity in the intended ecosystem. Look up the exact name in the registry for the project’s ecosystem, such as npm for a JavaScript dependency or PyPI for Python. Confirm that the result is the project the code actually needs, rather than a similarly named package. A missing result means the suggested dependency has not been established as available there; a result does not establish safety.
- Verify the project and publisher. Compare the package’s description, source repository, publisher identity, and documentation with the project the generated code appears to require. If those details do not line up, stop and investigate instead of installing it to see what happens.
- Assess provenance and history. Consider whether the package has a credible project history and whether its source corresponds to the intended project. Treat a newly appearing or otherwise unexplained package name as a reason for extra scrutiny, not as validation of the AI recommendation.
- Review before installation. Check that the dependency is necessary and that the package name in the code matches the verified project. Do not treat a successful install, import, or registry lookup as proof of legitimacy.
These checks address two distinct questions: identity—is this the exact project the code needs?—and trust—is its publisher and provenance credible? A package can pass the first check by existing and still fail the second.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What the study does—and does not—say about defenses
The USENIX summary says the researchers tested mitigations that reduced hallucinations while preserving code quality. The available summary does not establish one universally best intervention, so it does not support ranking a particular control as the answer for every development workflow.
The practical safeguard remains a review of both identity and trust before installation. Package-name checking can catch a name that is absent from the expected ecosystem, but it cannot certify a package that an attacker has since registered. The study’s rates also should not be used to estimate the odds for a particular current model, programming language, or prompt.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
