FORCE Value Metrics for Operations
In addition to using the assessment scores of the Security FORCE Survey to gauge the security value of operations, an organization can track the Security FORCE Metrics associated with operations to provide additional measures of HRSP behavioral alignment. These five metrics are shown in Figure 12-2.
Using the FORCE Operations Value Metrics
The five FORCE Metrics associated with the value of operations track the organization's capabilities for improved visibility into a broader range of operational information security behaviors, and for identifying discrepancies between what is expected operationally within the InfoSec program and what is actually taking place within organizational systems and processes. As with the other FORCE Metrics, there is no “right” way to measure and the measures I have created, including suggested time intervals, are not exhaustive. The organization should use them and adapt them as appropriate.
Figure 12-2 FORCE Value Metrics for operations value behaviors
Level of security staff coverage for the organization (size of program, breadth of responsibility, systems managed, etc.) I've known big companies that had large, centralized InfoSec teams who were responsible for every aspect of protecting systems and data throughout the organization. I've known others of comparable size where the security team was two or three people. Every organization must decide for itself the best structure for organizing information security, but the operational fact is that fewer people cannot observe, explore, or test as much as larger teams, assuming enterprises of equal size. Automation can help, but for reasons I discussed earlier in the chapter, automated security operations carry their own visibility risks. This metric is not prescriptive, and does not imply a magic number for effective security staffing. But it can help an organization understand why operational visibility may be lacking. Like anything else, information security is something of a numbers game, and you can only do so much more with so much less for so long.
Number of security operations reviews completed in the past year This metric does not refer to detailed operational reporting, but rather to overall reviews of InfoSec operational effectiveness. Several respected InfoSec governance frameworks, including ISO 27001, require regular and comprehensive reviews of the security program as a best practice for information security management. Organizations collect a lot of tactical data every day, but it is necessary sometimes to consider all of this from a strategic perspective. Is the data giving us what we need, in terms of visibility and in terms of actionable intelligence or predictive evidence? How can we make InfoSec operations better, or improve and expand sources of visibility?
Most organizations tend to do this sort of review annually, although in large organizations comprehensive reviews may be broken down into components or capabilities and conducted on a quarterly or (more rarely) a monthly basis.
Ratio of formally documented security operations or processes If managing something you don't measure is a challenge, measuring something you haven't defined is an even greater one. Those familiar with the concept of capabilities maturity models will recognize the benefits of formalizing and standardizing processes and operations within an enterprise. A lack of formal, documented processes makes it difficult to replicate behaviors and share or transfer knowledge. It also makes accurate operational visibility and comparison between what should happen and what does happen nearly impossible. Low ratios of documented processes indicate potential blind spots, spaces where failures can occur and grow larger without anyone noticing. By identifying all the processes associated with information security operations and identifying which are written down, an organization can begin to determine how formalized (and, by extension, how mature) their security program is.
Ratio of security operational assessments shared outside the security group Measuring how often the InfoSec program shares operational assessments with outsiders is similar to measuring how they share failure data. The goal is to elicit valuable feedback and insight from others who may have other needs, priorities, or concerns. Sharing sensitive operational data about security does not require total transparency. But organizations that seek a higher level of reliability will welcome feedback from interested advisors elsewhere in the enterprise (and maybe even outside of it, in certain cases), and they will track how often this sharing and elicitation of feedback takes place and in what contexts.
Average time to address operational instabilities When an organization finds a disconnect between what it thinks is happening in terms of information security and what is occurring operationally every day, it has several choices of response. One is to do nothing, for whatever reason seems most logical. Maybe the problem seems small, or maybe everyone already knows about it. Maybe change requires political or senior management support that simply doesn't exist. Another option is to take action to address the discrepancy. In either case, understanding how long this process takes can be valuable to the InfoSec program and to other stakeholders. Improving visibility provides less return on the security value of operations if the average time to fix problems the organization might find approaches forever. In situations where operational instabilities and problems are addressed, then the time necessary to address them becomes another useful InfoSec operations metric to add to the security program's toolkit.
