Summary

This report is based on 56,000 iterations performed over 56 risk scenarios and 63 capabilities on 2019-04-07 14:39:44+0000.

Total yearly losses are estimated to exceed the organization’s major risk threshold of $50,000,000 0% of the time.

The following table shows the maximum, 95th percentile Value at Risk (VaR), mean, and minimum annual losses.

Total Annual Loss Exposure
Value at Risk Maximum Loss Mean Loss Minimum Loss
$22,282,758 $42,171,518 $13,820,108 $5,166,501

Loss Exceedance Curve

The following loss exceedance curve is a common way to review the expected losses in a year. This figure shows how often total losses exceed any particular level during a given year. The 80% line shows that a loss of at least $4,223,248 occurs every four out of five years when outlier scenarios are excluded, or at least $10,142,053 when the outliers are included.

Loss Exceedance Curve

Loss Exceedance Curve

Risk Exposure By Domain

The top three information security program domains with the largest likely losses are Information Security Incident Management, Physical and Environmental Security, and Compliance. The losses associated with each domain of the program are described in the following table.

Annual Loss by Domain
Domain Value at Risk Maximum Mean (Average) Minimum Standard Deviation
Information Security Incident Management $6,691,612 $9,367,780 $4,465,097 $1,513,809 $1,302,761
Physical and Environmental Security $4,644,610 $11,578,463 $430,896 $0 $1,759,657
Compliance $4,077,983 $6,145,570 $2,598,394 $743,762 $869,087
Information Security Management Program $3,963,943 $5,468,451 $2,474,047 $663,743 $807,195
Organization of Information Security $3,237,394 $5,326,646 $1,907,368 $280,905 $758,155
Human Resources Security $1,120,515 $1,855,731 $622,980 $44,730 $275,654
Access Control $1,083,095 $1,450,198 $599,813 $15,054 $266,695
Communications and Operations Management $533,819 $1,151,143 $251,152 $43,936 $148,927
Privacy Practices $436,371 $786,643 $159,156 $0 $147,745
Asset Management $58,655 $321,407 $27,463 $1,138 $32,002
Risk Management $15,102 $321,986 $5,762 $0 $30,204
Information Systems Acquisition, Development, and Maintenance $0 $304,163 $4,741 $0 $27,386
Business Continuity Management $0 $12,660,540 $273,071 $0 $1,414,699
Security Policy $0 $8,246 $169 $0 $937

Simulation Outcomes by Domain

Each scenario generates a number of threat contact events where the threat community has the opportunity to act against the organization’s assets and result in a loss. Whenever the threat community acts and the organization’s capabilities prevent the attack, no loss occurs and a contained event is recorded in the simulation. Each threat community action that is not prevented by the organization’s capabilities is recorded as a loss event. The distribution of loss vs. contained events, and the average amount of control strength gap/surplus is displayed below.

Top Risk Scenarios

All of the scenarios are ranked against one another based upon their value at risk. The top five scenarios are:

Top Five Scenarios by Value at Risk
Scenario ID Scenario Median Annual Loss Value at Risk
PHY - RS-42 Damage to or loss of physical facility through natural disaster. $0 $4,500,548
IM - RS-50 Inadequate response results in inappropriate internal use of data. $2,500,580 $4,073,422
ISMP - RS-54 Key areas of the security program are not managed. $2,410,077 $3,963,943
COMP - RS-11 External attackers locate previously unknown weaknesses in the information security program not revealed through internal controls. $2,314,387 $3,874,253
IM - RS-22 Undetected and unremediated security incidents result in unmitigated access. $1,173,944 $2,157,974

A list of all evaluated risk scenarios is in Appendix A.

Key Capability Weaknesses

Threats most frequently overcome the control capabilities, resulting in losses, in the domains of Access Control, Information Systems Acquisition, Development, and Maintenance, and Asset Management.

Domain Weaknesses
Domain Succesful Threat Events Control Gap Surplus Control Strength
Access Control 99.0% 34.5% 25.1%
Information Systems Acquisition, Development, and Maintenance 67.2% 11.4% 6.0%
Asset Management 99.7% 36.7% 4.1%
Business Continuity Management 0.7% 11.9% 50.8%
Compliance 45.1% 16.7% 16.2%
Human Resources Security 50.2% 30.7% 15.2%
Information Security Incident Management 57.8% 15.3% 13.2%
Information Security Management Program 95.8% 11.8% 2.2%
Communications and Operations Management 47.7% 46.8% 22.3%
Organization of Information Security 33.3% 3.8% 6.4%
Physical and Environmental Security 2.5% 20.5% 32.6%
Security Policy 0.1% 6.5% 29.2%
Privacy Practices 4.8% 3.8% 32.9%
Risk Management 0.4% 13.5% 46.7%

Focus Risk Scenarios

The focus section allows in depth coverage of any scenarios that are of particular leadership interest. By highlighting those scenarios of particular interest to your decission makers (e.g. ransomware), you can address hot topics of interest without losing sight of the overall risk environment. You can delete this section if there are no particular areas of focus.

Key Scenario A

Scenario: Unauthorized access to or use of information and systems.

Scenario RS-51 Overview
Value at Risk $1,083,095
Vulnerability (% of events resulting in loss) 100%
Mean Control Gap 34.5%
Maximum Annual Loss $1,450,198
Median Annual Loss $579,625
Maximum Single Loss $440,835
Median Single Loss $80,786

Key Scenario B

Scenario: External auditors find compliance issues with regulations and standards not identified via internal processes.

Scenario RS-12 Overview
Value at Risk $0
Vulnerability (% of events resulting in loss) 7.81%
Mean Control Gap 2.36%
Maximum Annual Loss $152,703
Median Annual Loss $0
Maximum Single Loss $152,703
Median Single Loss $0

Outliers

Some scenarios have values at risk that are significantly higher than the population mean of $497,503. These scenarios are outliers. When viewed next to non-outlier scenarios, the rest of the risk scenarios may be lost. Portions of this report exclude outliers to avoid distorting the results. Graphs and tables are clearly noted when they display filtered data. The outlier scenarios are:

Scenario ID Description Value at Risk Median Maximum
PHY - RS-42 Damage to or loss of physical facility through natural disaster. $4,500,548 $0 $11,437,634
IM - RS-50 Inadequate response results in inappropriate internal use of data. $4,073,422 $2,500,580 $5,572,632
ISMP - RS-54 Key areas of the security program are not managed. $3,963,943 $2,410,077 $5,468,451
COMP - RS-11 External attackers locate previously unknown weaknesses in the information security program not revealed through internal controls. $3,874,253 $2,314,387 $5,432,649

Methodology

The security strategic risk assessment process implemented by the Evaluator toolkit is based upon the industry standard OpenFAIR methodology. Expert opinion is polled on the threats, capabilities, and probable loss magnitudes associated with key risk scenarios. The Evaluator toolkit implements a Monte Carlo model on this information to generating a dollar-quantified exposure for each scenario.

Risks are ranked by the economic Value at Risk (VaR). VaR is a summary statistic (the 95 percentile) and should only be used to rank items at a similar level of granularity. The VaR totals for domains should not be looked at directly with the VaR total for the individual simulations that make up those domains.

Domains

The organization categorizes its security program into 14 domains in order to systematically review risk. These domains are:

Domain Listing
Domain ID Domain
AC Access Control
ASSET Asset Management
BC Business Continuity Management
OPS Communications and Operations Management
COMP Compliance
HR Human Resources Security
IM Information Security Incident Management
ISMP Information Security Management Program
ADM Information Systems Acquisition, Development, and Maintenance
ORG Organization of Information Security
PHY Physical and Environmental Security
PRI Privacy Practices
RISK Risk Management
POL Security Policy

Capabilities

The security team and key subject matter experts formed a consensus opinion on the maturity level of the 63 capabilities which make up the 14 security program domains. The group assessed each capability against a five-level capability maturity model (patterned after the CMMI model), ranging from initial (level 1) through optimizing (level 5). These capability ratings are used to create a distribution of simulated capability effectiveness over the course of a year, ranging from 100% (completely effective) to 0% (completely ineffective).

The full capabilities catalog is included as Appendix B.

Risk Scenarios

Each domain of the security program has one or more risk scenarios addressed by that portion of the program. These scenarios are made up of:

  1. The threat community (e.g. internal workforce members, nature, partners) performing the action.
  2. The action taken by the threat community.
  3. The program capabilities that resist harm by the threat community’s action.
  4. The consequences of the action, should it overcome the capabilities.

Working through the scenario list, the security team assigned qualitative ratings to each of these frequency, strength, and loss elements.

Simulation

Each of the qualitative labels is mapped to a set of parameters describing a beta pert distribution. These distributions are used to run simulations over each risk scenario. Within a given iteration, a scenario is evaluated for potential losses using:

  1. The number of times the threat community acts against the organization.
  2. The force of the threat community applies against the organization.
  3. The difficulty the relevant controls present to the threat community. For scenarios which have multiple controls applied, difficulty is the arithmetic mean of all the applicable controls.

This process generates several outputs:

  • Threat Events: The number of times per year the threat presents itself
  • Loss Events: The number of times the threat results in a loss (the threat community overcomes the controls)
  • Single Loss Expected (SLE): The size range of individual losses from each loss event
  • Annual Loss Expected (ALE): The annualized sum of all individual losses.

Total risk is the sum of annual expected losses across all 56 scenarios within an iteration.

Recommendations

Recommendations are left for the analyst to complete. Include security improvement (increasing the strength of controls) and analysis improvement projects (increase data input quality) projects.

Project Recommendation

Document the approved or proposed key risk management projects for the coming planning period (typically yearly). These projects should address the findings from the simulated scenarios by improving controls, reducing loss impact, or transferring risk. Describe each project in terms of its cost versus the expected amount of reduced loss exposure.

  • FOO
    • Description
    • Cost
    • Expected Loss Reduction
  • BAR
    • Description
    • Cost
    • Expected Loss Reduction
  • BAZ
    • Description
    • Cost
    • Expected Loss Reduction
  • QUX
    • Description
    • Cost
    • Expected Loss Reduction
  • UIER
    • Description
    • Cost
    • Expected Loss Reduction

Analysis Improvement Opportunities

The objective of a risk analysis is to provide better information and to reduce uncertainty in making strategic resource allocation decisions. As part of this decision process, consider if additional information is needed to perform a better risk analysis. Additional data or higher confidence data may reduce the variability in your projections.

Typical areas for additional data include:

  • Control Effectiveness Refinement - Can we benchmark our controls against other organizations or otherwise better understand the maturity of our capabilities?
  • Scenario Refinement - Have we clearly identified the threat communities, their capabilities and their frequency of action?
  • Loss Magnitude Refinement - Can we leverage either our own incident data or data publicly (such as the VCDB) or semi-publicly (industry CIRTs) to better understand our potential economic losses?

Supplemental Analysis

Scenarios should be treated based upon size of the value at risk (VaR) calculation. Ranking scenarios by VaR creates a prioritized list of scenarios to address.

The following figure shows each scenario’s mean single loss size plotted a gainst the median number of loss events. The size of the circle represents the median annual loss total for that scenario.