Analyzing Performance Data
Performance measurement alone is not enough to add value; learning from the information and applying that learning to drive changes that improve performance are important steps. Optimizing the benefits of performance measurement can be achieved by performing analysis of the data collected. Data analysis transforms the performance information making it useful input, which can help business leaders to make better risk-informed decisions. There are many types of analysis that can be used, and the choice will vary based on the objectives of the analysis.
While this list is not exhaustive, here are some examples of commonly used analyses:
• Failure mode and effects analysis (FMEA).
FMEA helps to identify potential failure points based on certain conditions. The consequences of failures are further analyzed to understand their impact on other parts of a system or process. FMEA can help to design more comprehensive risk mitigation efforts.
• Regression analysis.
Regression analysis provides information on the relationship between one dependent variable and one or more independent variables. This type of analysis can be helpful in understanding the correlation between different risks.
• Pareto analysis.
Pareto analysis measures the frequency of issues, from most to least frequent. This type of analysis is useful in making decisions that provide the greatest results – for example, targeting resources to address issues in a specific component of a process with the greatest number of errors or control failures.
• Root cause analysis.
Root cause analysis is designed to identify and correct the fundamental cause of a problem. It helps focus remediation not on merely correcting symptoms but on preventing the recurrence of problems. This type of analysis is especially useful as a method to proactively forecast probable events before they occur.
• Scenario analysis.
Scenario analysis uses discrete scenarios to understand the potential outcome. Typically the worst case, best case, and most likely case are considered. Single-point estimates or a Monte Carlo simulation model using a range of values can be used. This type of analysis is useful to enhance readiness and strengthen response capabilities.
Benchmarking compares a company's current practices to best practices. This type of analysis facilitates development of strategies to improve processes and performance measures.
• Threat analysis.
Threat analysis can be used to evaluate a broad spectrum of areas such as natural disasters, criminal activity, legal or regulatory factors, technology trends, internal capabilities, and market forces. Using this type of analysis to gain insights into potential threats is useful to enhance readiness and strengthen response capabilities, as well as to enhance risk mitigation strategies.
Analyses such as these can be used to perform a deep review of a specific risk area to understand effectiveness of current risk mitigation strategies, or can be used broadly to understand potential emerging risks.
Using Key Risk Indicators to Understand Potential New Risks or Changing Risks
Most organizations use key performance indicators to monitor progress in meeting corporate objectives. Those indicators provide valuable information, including insights into risks. However, key performance indicators primarily provide insights into risks already well known by the organization. With ever-changing business environments challenging companies to take a longer-term view into potential risks, there is increased focus on understanding emerging risks. Key risk indicators are used to provide an early warning signal by not just looking at current risks but looking for leading indicators or triggers in the business environment. These triggers can be used to develop strategies that better position the company to manage new risks as they arise. Development of risk indicators can come from analysis of previous risk events to understand their root cause and triggers that can be used in the future as risk indicators. External information, such as economic indicators, industry benchmarks and trends, competitor actions, and the like, can all be utilized in developing key risk indicators. Just as with key performance indicators, key risk indicators are most effective if they are tangible, flexible, standardized, and outcome or objective focused.
Exhibit 12.3 provides some examples of key risk indicators.
Exhibit 12.3 Key Risk Indicators
Examples of Key Risk Indicators
Industry trends in customer attrition Frequency of critical process failures
Trends in gasoline or other critical commodity prices in relevant geographies Unexpected significant change in number of competitors or suppliers Spreads on debt for comparably rated companies