In recent years, software supply chain attacks have emerged as a significant threat to organizations worldwide. One particular technique gaining notoriety is the dependency confusion attack. In this blog post, we will:
- Delve into the technique, its variants and the associated risks.
- Explore detection methods, and provide prevention strategies and best practices to safeguard your development processes and environments.
- Insights on the attack from scanning over 54,000 repositories of companies of all sizes and industries.
Dependency confusion attacks
The problem of dependency confusion arises when there exists a public package with the same name as one of an organization’s private packages, causing ‘confusion’ for the package manager in use on which of the packages to fetch, use and trust: the one on the internal, private registry or the one on the official, default public registry. That package on the public registry can potentially be unsafe and include malicious code, sometimes targeting your specific organization with data exfiltration, injecting backdoors, gaining persistent and so on.
In automated CI/CD environments, this problem might eventually lead to malicious code being injected into production systems automatically and silently – without any indication of an undergoing attack taking place.
The package managers have to make a decision, and the root of the problem lies in the default behavior of the package managers. For example, PIP, the package manager for Python, defaults to fetch the package with the higher version. Exploiting this mechanism into an actual dependency confusion attack took place in the last week of 2022, when PyTorch, a widely used open-source project by Meta, suffered a software supply chain compromise due to a dependency confusion attack.
NPM, the node package manager, on the other hand, has a slightly different mechanism but is still vulnerable. You can upload packages to the official public NPM registry under an organization scope, e.g. @oxappsec/poc-npm-dependency-confusion, and add a configuration file to your repository which will direct the package manager on where the packages under each scope will be fetched from – the private registry or the official public one (npmjs). Misconfigurations in that file or repository, CI/CD misconfigurations, a private registry fallback mechanism or a simple mistake of a developer will allow attackers to utilize dependency confusion attacks on your organization.
A deeper look at how such an attack can still happen using NPM:
- Leaving your organization scopes available for registration on the public registry
- Misconfigured repos or without .npmrc will always fetch the public package
- Using a private registry with a fallback mechanism that fetches the public registry if a package was not found (some fetches the public registry on any error). For example, if by mistake someone updated the package version in package.json, without realizing that this version hasn’t even been published, it wouldn’t be found on the internal, private registry and it will fall back to the public registry and get the adversary’s created package.
- It also applies to devs who might not add the configuration file (.npmrc file) at first when working on a new project, or add a misconfigured .npmrc file at first when working locally.
Once the public package with the same name is downloaded from the public registry, even if a correct .npmrc file is added later – NPM will keep looking for the downloaded public package. A full POC of this case will be presented in this post.
Example use case utilizing NPM as a package manager for importing node dependencies into internal servers, developers’ workstations or CI/CD pipelines:
A malicious package can cause havoc on all these systems and might lead to a full compromise of your software. E.g. CI/CD pipelines that might execute the malicious code with elevated permissions or deploy it into production systems, effectively causing numerous implications, and even getting your users to run the malicious code you transferred them.
To better understand how an attack might take place, practically, even while using organization scopes in NPM, we will delve into the proof of concept:
– We have created a private organization “oxappsec” under Google Artifact Registry, which will serve as our internal, private registry for this POC.
Attack flow:
An attacker registers the internal organization name (in this case, “oxappsec”) on the official NPM public registry, at: https://www.npmjs.com/
Then, the attacker searches and identifies the internal package names under that organization scope on your internal registry. How he does that, and the intelligence work and techniques that have to be done in this step are out of the scope of this POC. There are many ways to publicly identify internal package names.
The attacker then uploads a malicious package named the same as an internal package name, specifies it with a higher version number, and uploads it under the organization scope that was created in the first step.
Thus, developers, internal servers, and automated build processes will fetch, import and execute the malicious package uploaded by the attacker.
As for Python, the PIP package manager will always prefer an official PyPI package if it’s with a higher version.
PoC implementation of a successful dependency confusion attack
Let’s assume our organization is using a private registry where internal packages are uploaded and consumed or being used across the SDLC (Software development lifecycle). These internal packages are transferred from our internal registry into developers’ workstations, internal and external servers and our CI/CD pipelines using NPM as the package manager:
An internal package named “poc-npm-dependency-confusion” with version @0.1.1337 was uploaded under the “@oxappsec” organization scope. It’s sole purpose is to print “Hello world from a private package!” to the console.
Our organization has a super important repo named “dummy-repo”. It utilizes that internal package, as can be seen in the package.json file:
To emulate an adversary’s actions and perform the dependency confusion attack on our systems, the organization name “@oxappsec” was registered on NPM official website and we published a (non malicious) research-purposes-only package under that organization scope, named (you guessed it) “poc-npm-dependency-confusion”, with a higher version: @0.1.11337
(If you missed the extra ‘1’, you might also fall for Typosquatting, you can read more here)
You can view and investigate this package for yourself as it is currently sitting at the NPM public registry under “@oxappsec” organization scope. https://www.npmjs.com/package/@oxappsec/poc-npm-dependency-confusion
We run ‘npm i’ or ‘npm i @oxappsec/poc-npm-dependency-confusion’, and the public package will be downloaded. This can be shown in the package-lock.json file or within the node_modules/ folder.
We then notice and add an “.npmrc” file, in which we will specify that @oxappsec scoped packages should be fetched from our internal GAR (Google artifact registry) only.
We run ‘npm i’ again, and all packages published under @oxappsec will be fetched from the private registry assuming there are no errors or misconfigurations.
To our surprise, the @oxappsec/poc-npm-dependency-confusion package is still fetched from the public registry!
Let’s assume we didn’t re-check the dependencies file or looked at where it was resolved from.
We now run once again the “helloWorld” function, which is inside @oxappsec/poc-npm-dependency-confusion:
“Hello World from public package!”
Only now, we notice it was still fetched from the public registry, a bit too late (!), as the adversary’s code was already executed.
This package, which could have been uploaded by an adversary could cause havoc on internal systems, exfiltrate environment variables during CI/CD pipeline executions, expose credentials, tokens and secrets, or do any of that silently without ever noticing, if properly disguised as the legitimate private package.
On the risks, prevention, and mitigations against dependency confusion attacks:
Risks:
- Compromised CI/CD Pipelines:
Attackers attempt to upload malicious code disguised as internal dependencies to public repositories. If successful, these malicious packages may be unknowingly downloaded and incorporated into the CI/CD pipeline, leading to the introduction of backdoors, malware, or other malicious actions into the software.
- Developers’ Exposure: Developers relying on package managers may inadvertently install external packages with the same names as internal dependencies. This exposes organizations to the risk of unintentionally including vulnerable or malicious code from public repositories.
- Service Disruption and Downtime: Infiltrating the software supply chain with malicious packages can lead to service disruptions and downtime. The compromised code can trigger errors, crashes, or introduce vulnerabilities that affect the availability and reliability of the software. This can result in financial losses, customer dissatisfaction, and damage to the organization’s reputation.
- Data Breach and Intellectual Property Theft: When developers unknowingly incorporate these malicious packages into their projects, it can lead to the introduction of backdoors, spyware, or other forms of malware into the software. As a result, these malicious components can compromise sensitive data, intellectual property, and even grant unauthorized access to attackers, leading to data breaches and theft of valuable proprietary information.
- Supply Chain Compromise and Chain of Trust: Attackers exploit the blind spots between internal and public package repositories, potentially substituting legitimate internal dependencies with malicious ones in public repositories. This breach compromises the chain of trust, allowing malicious code to infiltrate the software development process. Consequently, the compromised code can be unknowingly integrated into CI/CD pipelines, leading to the introduction of backdoors, malware, or other malicious actions. This not only jeopardizes the security of the software but also undermines the foundation of the supply chain’s trust, potentially causing significant service disruptions, downtime, financial losses, and reputational damage.
- Compliance Violations and Regulatory Penalties: By infiltrating public repositories with deceptive internal package names, attackers can introduce unauthorized and potentially malicious code into an organization’s codebase. This unauthorized code may violate compliance standards and regulatory requirements, exposing the organization to severe penalties, legal actions, and reputational damage. Ensuring the integrity of the software supply chain is crucial to prevent such attacks and maintain compliance with industry regulations.
Prevention & Best Practices:
- Always publish internal packages under organization scopes.
- Register your organization’s scopes on the public registries to avoid attackers from being able to take advantage of these attacks.
- Registering your organization scope on the public registry will PREVENT the attack from taking place.
- Since PIP and PyPI currently do not support organizations or scopes, you should always check if your internal package name can be registered also on the PyPI as a placeholder to prevent adversaries from taking advantage of these design flaws of PIP. E.g. Yandex are using placeholders to prevent dependency confusion attacks on their internal packages:
And, use the – – index-url flag on your requirements.txt files when specifying an alternative registry to fetch packages from, instead of – -extra-index-url flags.
- Use OX Security platform (Free if <25 devs) to continuously monitor all of your internal packages or internal organization scopes for any exposure to dependency confusion and more, with full application context, prioritization engines, recommendations and even a chatGPT integration for faster issue resolution.
Organizations of all sizes are exposed:
Analyzing results from over 1,000 organizations, and over 54,000 repositories scanned by OX Security reveals the broad exposure of organizations of all sizes and from all sectors to dependency confusion attacks:
- Small (less than 1,000 employees), mid-size and large organizations (1,000+ , 8,000+ and 80,000+ employees) are exposed and are using vulnerable packages or utilizing free-to-register (on the public registries) organization scopes. The fact that an organization scope is free for registration on the public registry implies that all of the internal organization’s packages under that scope are exposed to dependency confusion attacks.
- An astonishing fact we’ve seen by analyzing the results shows that among the organizations that are exposed, 73% of their assets are vulnerable.
- We define an ‘asset’ as an internal package of an organization, which might be used internally as a development dependency or deployed to production applications.
There was no specific concentration on one sector; exposure to dependency confusion exists on applications and organizations from all sectors – gaming, tech, media, consulting agencies, and even in the investment & banking sector, which could have devastating consequences.
Furthermore, applications of all sizes were vulnerable; applications serving millions of users are not protected, and the majority of applications with 1B+ users scanned were vulnerable to dependency confusion attacks.
Let OX Security do the AppSec hard work, and take action now for free: https://app.ox.security/login
)