DEVELOPING A MORE SOPHISTICATED APPROACH TO RISK ANALYSIS AND EVALUATION

According to ISO 31000, an essential part of developing any risk management framework is defining the criteria for evaluating risk. Risk criteria are used to reduce subjectivity and to communicate risk tolerance, and should lead to consistency across different assessments. In common with many nonfinancial organizations, BCLC uses risk tables with qualitative descriptions of a variety of potential impacts.

Over the past 10 years, a variety of risk tables and evaluation approaches have been adopted.

When BCLC conducted its initial enterprise risk management exercise in 2003, generic consequence and likelihood and management effectiveness scales with a 1 to 5 range were provided to BCLC by the consultants. The impact ratings focused on monetary and service provision consequences, while the likelihood ratings considered the chance of occurrence over the next three years.

For this initiative, risk workshops were used for the majority of risk analysis, with risk statements either predetermined or defined in advance using interviews with key internal stakeholders and then voted on by the Executive Committee, the ERMAC team, or a specific project team depending on the context. Voting technology was used at each workshop, with each participant independently rating each risk. After each vote, the software calculated the average score and derived an overall risk rating for each risk. Using voting has a number of benefits, principally allowing a large number of risks to be assessed in a relatively short period of time. Advocates also claim that voting reduces group bias, as results can be presented anonymously and any variations can be discussed.

Voters at each facilitated workshop were asked to rate the likelihood that a particular event would occur in the absence of any controls in place to mitigate the risk (known as the inherent likelihood). Each risk was then mapped to one of four categories (see Exhibit 10.8). An additional exercise considered the effectiveness of current control levels for each risk and also the desired level of control in order to identify any risks where it was considered that additional levels of control were required.

The Internal Audit-led exercise in 2006 initially used a very simple scale (high, moderate, low, and very low) when asking participants to identify/report their top three risks, and then introduced a new BCLC-specific impact and likelihood table to assess inherent impact and likelihood, using the same voting and averaging methodology as used in 2003. The new risk criteria considered a range of potential consequences, from threats to product integrity, to media reports, sales, stakeholder relations, regulatory noncompliance, and budgetary impact. The new likelihood ratings included both an assessment of the probability of occurrence and reference to historical incidence and common root causes and control effectiveness. The risks were again grouped into four categories, as can be seen in Exhibit 10.9.

The 2007 enterprise assessment developed the risk assessment framework further, reflecting the additional resources now available to the ERM program with the appointment of a dedicated manager and the engagement of the new ERMAC team. The criteria were revised once more, with metrics developed for each

Exhibit 10.8 2003 Risk Mapping Approach

category of consequence, a cleaner likelihood table with measures of both probability and frequency, and a new management effectiveness rating table.

Assessment participants were asked to vote on the impact if the risk event were to occur and the inherent likelihood of that event occurring. As with the previous assessments, the overall rating assigned to each risk was taken as the average, giving a score from 1 to 5 for each risk. A further vote was then conducted

Exhibit 10.9 2006 Internal Audit Risk Matrix

Exhibit 10.10 2008 ERM Residual Risk Rating Matrix

on how effective the ERMAC team considered current controls to be for each risk (the "current management effectiveness"). The two scores were then compared and any risks with a high-risk rating and lower management effectiveness rating were identified as requiring management attention.

The two enterprise risk assessments in 2008 in February and November used a very similar approach to the 2007 assessment, except that, instead of reporting on the inherent risk ratings and highlighting any significant gaps between the inherent risk rating and the management effectiveness rating, the management effectiveness metric was used to place each risk in a residual risk matrix, according to the size of the gap. Where the gap showed that controls were insufficient, this was termed a risk (better described as intolerable residual risk), and where the gap showed that controls were excessive, this was classified as an opportunity (to reduce control levels). The final outcome of the exercise is shown in Exhibit 10.10.

This approach was adopted partly in recognition that BCLC had not always put in place sufficient controls for the level of risk, but also because there was a perception that in some areas excessive controls had been implemented, partly in response to the Ombudsman report and subsequent recommendations and partly because some areas of the organization were considered to be risk averse.

From 2009, there was a change in emphasis from primarily inherent to residual risk assessments. This was partly due to the different approach of the new manager, partly due to difficulties with accurately assessing inherent risk, and partly because of a new opportunity with the development of new organizational goals. BCLC had been exploring the concept of balanced scorecards^[1] as part of developing a more mature approach to performance management, and in early 2009 new risk criteria were introduced based on the new goals. This reinforced the link between risk and wider business and strategic planning, and enabled the development of a smaller set of risk impact categories that resonated with both management and senior leadership. The impact criteria were developed with key managers and validated with the executives, with an annual update incorporated into the risk management planning timetable.

At this time also BCLC ceased to use the voting technology for a variety of reasons, including cost and geographical limitations, and moved to an approach where group workshops prioritized risk but did not undertake formal analysis or evaluation. A variety of visual mapping techniques were introduced with a more hands-on style adopted, requiring workshop participants to engage more directly through the use of techniques such as using Post-its, voting cards, target placement, assigning spots, and drawing process maps. Formal analysis moved to the appropriate subject matter expert with quality assurance provided by the risk manager and then confirmation of risk scoring provided by the relevant member of the executive or project steering group.

In 2011, as an outcome of the Internal Audit ERM review, it was agreed that the criteria were not sufficiently aligned with leadership attitudes to risk, and that too many risks were being reported with a high rating and thus being escalated in the quarterly report. An exercise was conducted with executives to better align the existing risk criteria to organizational tolerance, and to discuss the perception that the organization, or at least some parts of it, was overly risk averse. Perspective was provided through discussion of the balance between risk aversion and excessive risk appetite and the use of the "as low as reasonably practical" principle (sometimes referred to as ALARP or ALARA [as low as reasonably achievable], and described in ISO 31010).

Two activities were undertaken, each designed to look at the four dimensions of impact in the ERM framework to ascertain whether current levels were an accurate representation of the attitude of BCLC leadership toward risk, and to initiate discussion where that attitude varied among the executives.

The first exercise (see Exhibit 10.11) used a poster showing the existing impact criteria, and each executive was asked to mark where he or she believed the current catastrophic or level 5 impact should truly fall on the scale. This clearly shows that the scales in use at the time were generally felt to be misaligned with organizational risk tolerance, in particular for financial/operations and people impacts.

The second exercise took a small number of existing and well-understood risks, all currently assessed at a similar risk rating but with impacts across the different dimensions. Each executive was asked to place the risk where he or she believed it lay on the current impact table, again displayed as a large poster. Exhibit 10.12 depicts the mapping for two of the risks, showing both the spread of opinion, and the disparity between the rating at the time and the risk attitude of the executives both as individuals and collectively.

The exercises were successful in generating discussion about relative risk tolerances and showed both that the overall evaluation tools were escalating risk at too low a level and also that the risk criteria across the different impact

Exhibit 10.11 Impact Scale Evaluation Exercise

dimensions were not completely aligned to the collective executive risk perception and attitudes.

The impact criteria and the risk evaluation table were adjusted after the executive meeting, and the new approach adopted for the next risk review in March 2011. As a result of changing the criteria, the number of risks escalated to the executive declined from 33 to 10, allowing a much greater focus on the most significant risks, while risks now rated as having a moderate risk level continued to receive focus at the divisional risk review meetings.

In early 2012, a new risk framework was put in place describing BCLC's now maturing approach to enterprise risk management. The framework contained a section on determining appropriate risk responses, including a formal statement that BCLC had adopted the ALARP approach to determine the appropriate response to risk. This approach divides risks into three regions or zones:

1. An acceptable region, where further treatment may be undertaken but is not required

Exhibit 10.12 Specific Risk Impact/Likelihood Evaluation Exercise

2. A tolerable region where treatment should be undertaken dependent on cost/benefit analysis

3. An unacceptable region where treatment to lower the risk is mandated

Taking an ALARP approach to risk response allows for flexibility when determining the best approach to managing risk, and reflects that organizations may on occasion choose to adopt higher-risk strategies where the potential reward is deemed to be sufficient, or may elect to carry significant risk where the cost of treatment is felt to be prohibitive.

The relationships between criteria, severity, escalation, and tolerance are set out in Exhibit 10.13.

The next significant risk assessment and evaluation development was the expansion of the risk consequence criteria in August 2012 to include positive outcomes. Consideration of positive outcomes from uncertainty was introduced in ISO 31000, but has long been recommended by project management, for example in the Project Management Institute (PMI)'s Practice Standard for Project Risk Management. The concept was introduced for two reasons: to better engage those parts of the organization that were aiming to become highly innovative, and to better assess the risks associated with new initiatives. The new approach enables the comparison of risk with potential reward, and establishes the idea that both threats and opportunities are associated with uncertainty.

The new consequence table was based as previously on the key BCLC goals but for the first time included consideration of both positive and negative impacts, with benefits considered as opportunity and loss/harm as threat. The table has

Exhibit 10.13 Implementing the ALARP Approach to Risk Response

four levels of positive outcomes and four levels of negative outcomes (with a neutral zone bridging the two). BCLC has opted for a symmetrical approach so that a given level of negative outcome in any of the dimensions is balanced by the equivalent level of positive outcome. For example, one of the existing financial criteria references the possibility of making a loss of up to $5 million. Therefore, the parallel positive consequence is a potential gain of up to $5 million. Likewise, in the overall severity matrix, the appetites and tolerances for positive risk follow the same principles already in use for negative risk.

The new table was incorporated into the business case template, with simple graphical maps produced as an outcome of a detailed assessment showing the overall risk profile of any proposed initiative. These maps are used as one of the factors determining both the selection of initiatives and the level of risk management support and monitoring subsequent to approval. The approach has proved very helpful for both risk mitigating proposals to be able to demonstrate value more clearly and for those initiatives that have a more balanced profile to incorporate risk treatment plans from a much earlier stage, allowing for better risk planning and resourcing.

Exhibit 10.14 shows an example of the summary charts produced as an outcome of a business case risk assessment exercise. The business case is for an initiative that is primarily designed to reduce existing risks across a number of organizational objectives. The bars show the current threat and opportunity assessment, while the lines show the anticipated effect of the initiative on the organizational risk profile. The matrix looks at the overall balance between threat and opportunity, with the pre- and post-treatment statuses showing very positive changes. This initiative was approved and is proceeding. Because of the high levels of uncertainty, monitoring of threat mitigation and benefit realization will be important.

Exhibit 10.15 shows another example, this time for an initiative with very low levels of uncertainty. The overall effect of the initiative on the organization's risk profile is broadly neutral. This initiative was also approved and is proceeding. As levels of uncertainty are low, monitoring will be minimal.

Although there was a significant learning curve both for the teams participating in the risk assessments and for senior management in interpreting the results,

Exhibit 10.14 Business Case Risk Assessment Output Example 1

the new approach was endorsed by management and was used again in 2013 with some minor improvements to increase consistency.

Linking discussion of potential rewards with potential problems has supported the development of a more nuanced view of risk across BCLC and proved more culturally acceptable to individuals and groups tasked with developing innovative practices, as there is less of a focus on asking "What could go wrong?" and more emphasis on "What is not certain?" This has helped the ERM program to counter the viewpoint held by some groups that managing risk is a necessary but uninspiring and possibly bureaucratic exercise required by a risk-averse corporation, and has led to a better understanding that becoming risk-aware helps in embracing change and achieving objectives.

Exhibit 10.15 Business Case Risk Assessment Output Example 2

• [1] The balanced scorecard originated by Drs. Robert Kaplan and David Norton as a performance measurement framework that added strategic nonfinancial performance measures to traditional financial metrics to give managers and executives a more balanced view of organizational performance.