Will We Survive The Transitive Vulnerability Locusts

Understanding and Mitigating the Risks of Transitive Vulnerabilities in Application Security

At OX Security, our mission is to eliminate manual AppSec processes. Our latest research, presented at Black Hat 2024, highlights a critical issue: the often-overlooked risks associated with transitive dependencies in software. These dependencies, integral to any software build, can introduce significant vulnerabilities into your application ecosystem. Developers may be familiar with standard output (stdout) that shows vulnerabilities in their projects. While an important element of AppSec vulnerability management, it’s not enough to truly help developers see the full scope of risk.

Using the popular Parse Server application which, despite being well-maintained, our research showed vulnerability to a specific open-source security flaw in the ‘ws’ module that stemmed from a null reference issue. We ran an exploit against it and managed to crash the application relatively easily…. demonstrating how vulnerabilities become easily transitive. Although each vulnerability may have a slight chance of exploitation, the sheer number of transitive dependencies significantly amplifies the risk. This data underscores the importance of this discussion and the need for scalable strategies to mitigate these risks in software projects. Although there are many routes to triage, we show how building an automated model to address these issues is essential.

Our key takeaways from this research include:

• • Different open-source vulnerability classes exhibit varied likelihoods of exploitation.
  • The depth of transitive vulnerabilities significantly affects exploitability likelihood.
  • A mature DevSecOps and AppSec culture is crucial in reducing risk exposure.

By focusing on these highlights and using the model to identify, fix, and track transitive vulnerabilities in code, we can achieve more secure software. The battle against transitive vulnerabilities is ongoing and requires a combination of careful analysis, automated tools, and a mature security culture. Understanding the different behaviors of vulnerabilities and implementing a robust risk calculation can significantly reduce the risk posed by these often-overlooked threats.

Acknowledging the Problem of Transitive Dependencies

Before we dive into our research process, we have to start by acknowledging a common reality: many of us have encountered an overwhelming number of vulnerabilities when dealing with JavaScript projects. One of our first experiences with this happened when we encountered over a hundred vulnerabilities in a JavaScript object—some critical, others high-risk. When the software engineer responsible was approached, their response was alarmingly casual: “Don’t worry, nobody cares; just ignore it.”

This mindset is precisely why transitive vulnerabilities are some of the most persistent issues in application security today. Many vulnerabilities go unnoticed or unaddressed simply because they appear too frequently, often as false positives, leading to a “boy who cried wolf” scenario. However, it only takes one overlooked vulnerability to compromise an entire organization.

Understanding Software Composition Analysis (SCA)

It’s also important to understand that the challenge we’re tackling is the Software Composition Analysis (SCA) problem. The issue today is that modern software development resembles constructing with Legos, where applications are built using numerous open-source dependencies — no one writes frameworks from scratch anymore. With each dependency comes the very real probability of inherited vulnerabilities. When unique applications are then built on top of those frameworks, it turns into a patchwork of potential vulnerability dependencies that are stitched together with our own proprietary code, without any mitigation of the existing vulnerabilities.

The Research Process

Demonstration of Exploit Vulnerability in Parse Server Application

For our research, we used the Parse Server application, which is a popular backend framework. Despite being well-maintained, this project was vulnerable to a specific open-source security flaw in the ‘ws’ module that stemmed from a null reference issue. We ran an exploit against it and managed to crash the application relatively easily. 

The process was straightforward. We used a malicious payload and it turned out to be really simple because we just utilized the exploit example provided in the original advisory. This meant that we didn’t even need to develop a new exploit. This highlights a critical point: these vulnerabilities are real, and the potential for exploitation is not hypothetical. View the demo below. 

Again, the exploit, which was part of the original advisory, attacked open-source code that contained a vulnerability that had not been patched or updated. In other words, the vulnerability was imported from the base code without mitigation. The image in figure 3 shows the code on the left, and the trimmed packet capture of the exploit that was snipped during our exploit exercise. This is a perfect example of how vulnerabilities easily become transitive. If left unchecked, every new application that uses the vulnerable piece of code is also vulnerable to this exploit, and potentially other vulnerabilities as well.

Research Question Number One

What is the likelihood of an open-source direct vulnerability exploit?

The next step in the research process was to ask: what is the likelihood of an open-source direct vulnerability exploit? To address these vulnerabilities, AppSec engineers typically follow a structured triage process, which we also applied for our answers:

1. Run an SCA Scan: Identify vulnerabilities in the manifest file.
2. Import and Verify Vulnerability: Ensure the identified version is actually in use.
3. Assess Usage: Confirm the vulnerable dependency is actively used in the code.
4. Analyze the Risk: Prior to going to the developer understand how impactful the risk is. 
5. Accept the Risk or Remediate: Determine the effort required to fix the vulnerability or decide to accept the risk.

In our case, despite the ‘ws’ module in Parse Server being patched quickly, an SCA scan still flagged the same vulnerability. This led us to investigate deeper, revealing multiple references to the vulnerable version within different modules. The main challenge with this process of exploitation analysis is that it is time-consuming and not scalable.

Triage at Scale

Our next step was to tackle this problem in a scalable way using a five-step methodology we developed. Each step has its own input, culminating in the final step where a Large Language Model (LLM) will decide whether the vulnerability is actually exploitable or not. The steps are:

1. Build a Dependency Graph: Analyze manifest files to build a comprehensive graph of dependencies.
2. Identify Vulnerabilities: Correlate each dependency and its version with known vulnerabilities.
3. Check Imports: Ensure the vulnerable dependency is actually imported in the code.
4. Verify Usage: Confirm the dependency is actively used.
5. Final Verdict with LLM: Use an LLM to analyze the context and decide on exploitability.

The Final Verdict

Using OpenAI, we analyzed the NVD description of the ‘ws’ vulnerability and the codebase from Parse Server. We incorporated all relevant context and data from the previous steps, including NVD descriptions, advisories, GitHub commits, and more. This information was compiled and sent to a prompt builder, which created an engineered prompt. We then fed this prompt into a Large Language Model (LLM). The LLM assessed whether the vulnerability was exploitable, taking into account the input, context, and codebase details, including usage requirements, flags, and arguments for the dependency. The LLM provided the final verdict on the exploitability of the vulnerability.

Risk Analysis and Mitigation

Despite privacy concerns about sending code to OpenAI, we validated our approach with local LLMs like LLaMA 3 and Mistral models, which produced satisfactory results.

At the end of our 5-step analysis, we moved the data into the LLM, which helped determine whether the vulnerability — given the inputs, context, codebase, requirements, etc. — is exploitable. In this case, we determined that it is, along with details on “why.”

The Challenge of Remediation at Scale

With a proposed solution, it would be easy to conclude that we have fixed the problem. Given this vulnerability, we could just patch it and be secure, right? But after we updated the manifest file, and theoretically removed the transitive vulnerability, it still showed up in the SCA scan. After two tries at remediating the problem, we recognized that two variable versions were present. Using the SCA scan, we determined the root cause of the vulnerability had been imported and used.

This is a fine manual fix, but reproducing this process manually at scale is near-impossible. We therefore decided to test whether we could group CVE behavior by their common weakness enumeration (CWE) classification. In short, we embarked on a new research analysis to check if the exploit probability changes or behaves differently with different CWEs.

In the next section, we’ll show how we dove into the CWE of a regular expression to determine the risk of a transitive vulnerability.

Second Research Question: What is the likelihood of Open-Source Transitive Vulnerability Exploit?

Statistical Estimation of Exploit Likelihood for Transitive Vulnerabilities

For our second approach, we applied a statistical approach to estimate the likelihood of exploiting a transitive vulnerability. By analyzing tens of thousands of applications, both open-source and closed-source, we observed that most vulnerabilities are transitive rather than direct. Although running an LLM to determine the exploit likelihood of a transitive vulnerability is an option, it requires significant resources and compute power. Therefore, we opted for a statistical analysis.
Referring back to the CVE advisory used in the exploit described above, the vulnerability is clearly defined with a description, affected products, and a CWE. Many companies manually analyze CVEs in a given repository, but this method is not scalable. Instead, we tested grouping the vulnerabilities by behavior classification, specifically by CWEs.

Analyzing the vulnerabilities this way, we determined that the probability of an exploit varies based on the CWE. The image below shows that the green line in the middle represents the median exploit probability. Exactly fifty percent of CVEs under the tested CWE-1333 were below the median, and fifty percent were above.

The key insight is not that the exploitability is “average,” but that, with additional context, we found that 50% of those particular CVEs (the interquartile range, or IQR) were exploitable in 60-80% of the applications analyzed. To fully understand the exploitability per vulnerability class, we need additional information.

We developed a formula to calculate the probability of bringing a vulnerable dependency — and all its security implications — to calculate the probability of exploit.

To better understand the methodology around it, we used the hypothesis that a transitive vulnerability at depth 1 is a direct vulnerability of a direct dependency, where we utilize the exploitability likelihood determined by the CWE. 

Looking into deeper transitive vulnerabilities, those can be calculated as well. For example, a transitive vulnerability at depth 2 is a direct vulnerability of a dependency at depth 1.

To be comprehensive, we used insight from the CVE to provide greater context, and also introduced a variable we called “alpha” which scales the risk contribution based on its depth. What happens when we apply this formula on the direct vulnerability? 

If we simply use the exploit likelihood of the CVE, itself, with the power of “1,” we reduce the risk — the depth falls to zero. If we increase “alpha,” the risk score increases. Once we have those scores, we can implement a policy that determines the level at which transitive risk is acceptable and the level at which it is not. This will help the AppSec team avoid importing transitive dependencies that carry a higher risk. 

It’s important to note that “alpha” is a variable for each company to determine based on AppSec maturity, risk tolerance, criticality of the application, and so on. 

This underscores the most important aspect of this research, which is that different vulnerability classes have different exploitation likelihood and that transitive vulnerabilities — vulnerabilities from open-source code — are a major contributing factor to risk and must be managed.

Transitive Vulnerabilities Survival Guide

The original idea for this talk was to address how we will survive “the transitive vulnerability locusts.” Our analysis provided some good insight, especially given that a preponderance of application vulnerabilities — open-source vulnerabilities — come from transitive vulnerabilities rather than direct vulnerabilities. After our deep dive, we’ve come up with a few tips for AppSec team, a survival guide of sorts, so you can avoid the locusts: 

1. Apply Virtual Patching: Traditional approaches like putting a WAF or API tool in front of your apps could save a lot of trouble. Even with a generic signature-based tool, you can pick off the “low-hanging fruit,” i.e., the obviously malicious packets.
2. Patch: More mature AppSec programs will have the capability to apply patches, even if the open-source dependency is unmaintained or if a public patch doesn’t exist. In many cases, all you need to do is upgrade to the most recent version — provided you’ve examined it and seen that the upgrade also does not contain a vulnerability.
3. Scheduled Rebuilds: Plan for rebuilds that give you the time and bandwidth to update applications. Most vulnerabilities from open source do get a patch at some point, so check your sources and plan to update at a regular cadence.
4. Choose Versions Wisely: Don’t just assume that the latest version of software is the safest. Make sure you’re assessing everything you put into your software as you’re building it and as it’s running throughout its entire lifecycle.

Key Takeaways

From our analysis, we learned that transitive vulnerabilities are a major contributing factor to software and software supply chain compromise. We believe it deserves more emphasis, but we also understand that this is a complex problem and that many organizations may not yet have the AppSec maturity or bandwidth to tackle these problems at scale. This is why we were so committed to creating this model and presenting it to the security community at large. We believe that not only a better understanding of the scope of the problem is necessary, but that developing methods to make it easier for Dev and AppSec teams to accurately identify issues is the key to reducing the number and severity of transitive vulnerabilities that reach runtime. As such our key takeaways from this research include:

1. Different open-source vulnerability classes exhibit varied likelihoods of exploitation.
2. The depth of transitive vulnerabilities significantly affects exploitability likelihood.
3. A mature DevSecOps and AppSec culture is crucial in reducing risk exposure.

If you can focus on these highlights, and use the model to identify, fix, and track transitive vulnerabilities in your code, we will all have more secure software.

The battle against transitive vulnerabilities is ongoing and requires a combination of careful analysis, automated tools, and a mature security culture. By understanding the different behaviors of vulnerabilities and implementing a robust risk calculation, we can significantly reduce the risk posed by these often-overlooked threats.

Remember, in application security, the wolves are always at the door — but with the right processes, we can keep them at bay.