Risk, Severity, Threat Modeling, and Why You Need A Pentest

CVSS — A Misunderstood Scoring System That Leads to Inefficiencies in Defensive Teams

Through my experience as a penetration tester at Novacoast, I have observed that organizations often struggle with timely patching and accurate threat modeling of their assets. This is not due to a lack of expertise or commitment, but rather to limited visibility or a threat model built on the wrong parameters.

CVSSv3 is a widely used severity scoring system, but it is commonly misused to assess risk and prioritize patching efforts for vulnerabilities. Such a patching strategy has been shown to be severely inefficient—almost as ineffective as patching at random.

This suggests a gap between how vulnerability severity is viewed (or perhaps misunderstood or confused with risk) and how organizations can effectively translate that information into actionable security decisions.

In this blog post, we explore why CVSS is not as useful as you might think, what alternatives exist, and what we recommend for accurate threat modeling of a system.

TL;DR

• CVSS assesses severity—how bad something would be if exploited. It does not and cannot measure the likelihood of exploitation, so using it as the sole basis for patching is inefficient.
• EPSS assesses the likelihood of exploitation within 30 days. A score of 0.8 means an 80% likelihood of exploitation, making prioritization based on this score more efficient.
• The major limitation of any of these scores is that they only capture CVEs (catalogued software vulnerabilities). A wide range of issues—such as misconfigurations or dangerous defaults—are not captured by these automated tools.
• There is no substitute for actual penetration testing, which can uncover vulnerabilities involving defaults, misconfigurations, systemic issues, bad practices, and more.

Understanding the Fundamentals: Key Terms for Effective Threat Modeling

To understand why CVSS falls short for threat modeling, we need clear definitions of some crucial terms that are often confused or misused in security discussions.

Threat – A destructive circumstance or person with malicious intent that can cause loss. This loss can affect integrity, confidentiality, reputation, and availability. The most significant threats in cybersecurity include cybercriminals, malware, and advanced persistent threats.

Vulnerability – A bug or unintended feature in software (whether an application or a protocol) that allows someone with malicious intent to perform unintended actions, such as gaining remote control over computer systems. In simple terms, vulnerabilities are what attackers look for to compromise a system.

Severity – A subjective measure of how bad a vulnerability could be if it were exploited. Various scoring systems attempt to quantify vulnerability severity, with CVSS being the most commonly used.

Risk – How likely it is for a threat to cause some type of loss. In the context of software, risk refers to the likelihood that a threat actor will exploit a vulnerability.

A Practical Example: Why Severity ≠ Risk

Let’s illustrate this critical distinction:

• Company A has assets vulnerable to vulnerability X, which scores 3 out of 10 in severity.
• Company B has assets vulnerable to vulnerability Y, which scores 10 out of 10 in severity.

Assume, for simplicity, that each point in the severity metric corresponds to a hypothetical loss of $1 million.

Now consider this:

• Attackers have exploited vulnerability X in 850 out of 1,000 companies since it was published.
• The same threat groups have exploited vulnerability Y in only 10 out of 1,000 companies.
• Both vulnerabilities were disclosed at the same time.

The result: Company A faces higher risk despite a lower severity score, while Company B faces lower risk despite having a more severe vulnerability.

In financial terms, an 85% chance of losing $3 million represents a greater risk than a 1% chance of losing $10 million.

What Is Threat Modeling?

Threat modeling is a complex task that aims to objectively assess software (applications, operating systems, etc.) to identify threats and quantify risk. While the core objective is consistent, motivations can differ. Threat modeling approaches typically fall into three categories: asset-centric, attacker-centric, or software-centric.

There is no universally “best” approach—it is a complex undertaking with inherent trade-offs, and the chosen method should align with specific organizational goals.

Threat modeling expert Adam Shostack points out that threat models fail when organizations add steps without understanding their associated costs, benefits, and potential issues. They also fail when models become overly complex or introduce excessive subjectivity. An effective threat model should enable quick and accurate assessment while clearly defining its intended users and required skill levels.

Current Threat Modeling Solutions: A Critical Overview

Several established frameworks attempt to formalize threat modeling, each with strengths and weaknesses.

DREAD is a mnemonic for Damage, Reproducibility, Exploitability, Affected Users, and Discoverability. Each category is scored from 1 to 10, with the final score being the arithmetic mean. A key flaw is that the Discoverability component encourages security through obscurity, which is why many practitioners remove it entirely.

STRIDE originated at Microsoft in the late 1990s and stands for Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege. STRIDE is effective for categorizing threats but provides no numerical scoring and is difficult to automate.

Attack Trees are diagram-based models in which the root represents the attacker’s ultimate goal and branches represent possible attack paths. While intuitive and visual, they are not sufficient on their own for quantifying risk.

Other models, such as PASTA or TRIKE, exist but often suffer from excessive complexity or limited adoption.

CVSS and Its Fundamental Misuse

Many automated tools exist to help defensive teams assess their environments and prioritize remediation. Unfortunately, CVSS is often the parameter teams fixate on.

When a vulnerability with a CVSS score of 10.0 is disclosed, defensive teams feel compelled to patch it immediately. However, CVSS only tells us how severe the vulnerability would be if exploited—not how likely exploitation is, nor what the real-world business impact would be.

Lower-severity vulnerabilities can still pose significant risk when attackers chain them together. Conversely, some high-severity vulnerabilities may require conditions that make exploitation unlikely or impractical.

At the end of the day, a penetration test is what allows vulnerabilities to be evaluated in proper operational context.

EPSS: A Quantitative Alternative

The Exploit Prediction Scoring System (EPSS) was designed to estimate the likelihood of exploitation based on empirical data.

Because EPSS aligns more directly with risk-oriented goals, it enables more efficient prioritization than CVSS alone. In practice, prioritizing vulnerabilities above a certain EPSS threshold yields significantly better remediation efficiency and coverage.

The Limitations of Automated Scoring

Even advanced scoring systems cannot identify misconfigurations, dangerous defaults, poor network segmentation, weak monitoring, or flawed operational practices.

It is not uncommon for environments to appear clean in automated scans, yet be fully compromised by a skilled tester within hours due to configuration weaknesses.

What You Can Do

• Do not rely solely on automated scores to prioritize remediation.
• Use penetration testing to gain contextual and operational insight.
• Improve both the breadth and frequency of testing across environments and attack surfaces.

Closing Words

Threat modeling and penetration testing extend far beyond automated metrics. A future article will explore real-world exploitation examples involving weak defaults and misconfigurations that bypass traditional scoring systems. Stay tuned.