ISM3 Brings Greater Standardization to Security Measurement Across Enterprise IT

Updated: February 06, 2010

The security metrics panel and sponsored podcast discussion are coming to you from The Open Group's Enterprise Architecture Practitioners Conference in Seattle on Feb. 2, 2010. The goal is to determine the strategic imperatives for security metrics, and to discuss how to use them to change the outcomes in terms of IT's value to the business.

Our panel consists of a security executive from The Open Group, as well as two experts on security who are presenting at the consortium's Security Practitioners Conference: Jim Hietala, Vice President for Security at The Open Group; Adam Shostack, co-author of The New School of Information Security, and Vicente Aceituno, director of the ISM3 Consortium. The discussion is moderated by Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Hietala: We think there's a contribution to make from The Open Group, in terms of developing the ISM3 standard and getting it out there more widely. [Being a data-driven security organization means] using information to make decisions, as opposed to what vendors are pitching at you, or your gut reaction. It's getting a little more scientific about gathering data on the kinds of attacks you're seeing and the kinds of threats that you face, and using that data to inform the decisions around the right set of controls to put in place to effectively secure the organization.

A presentation we had today from an analyst firm talked about people being all over the map [on security practices]. I wouldn't say there's a lot of rigor and standardization around the kinds of data that's being collected to inform decisions, but there is some of that work going on in very large organizations. There, you typically see a little more mature metrics program. In smaller organizations, not so much. It's a little all over the map.

... The important outputs of a good metrics program can be that it gives you a different way to talk to your senior management about the progress that you're making against the business objectives and security objectives.

That's been an area of enormous disconnect. Security professionals have tended to talk about viruses, worms, relatively technical things, but haven't been able to show a trend to senior management that justifies the kind of spending they have been doing and the kind of spending they need to do in the future. Business language around some of that is needed in this area.

Shostack: We have an opportunity to be a heck of a lot more effective than we have been. We can say, "This control that we all thought was a really good idea -- well, everyone is doing it, and it's not having the impact that we would like." So, we can reassess how we're getting real, where we're putting our dollars.

The big change we've seen is that people have started to talk about the problems that they are having, as a result of laws passed in California and elsewhere that require them to say, "We made a mistake with data that we hold about you," and to tell their customers.

We've seen that a lot of the things we feared would happen haven't come to pass. We used to say that your company would go out of business and your customers would all flee. It's not happening that way. So, we're getting an opportunity today to share data in a way that's never been possible before.

Aceituno: The top priority should be to make sure that the things you measure are things that are contributing positivity to the value that you're bringing to business as a information security management (ISM) practitioner. That's the focus. Are you measuring things that are actually bringing value or are you measuring things that are fancy or look good?

Because metrics are all about controlling what you do and being able to manage the outputs that you produce and that contribute value to the business ... you can use metrics to manage internal factors.

I don't think it brings a bigger return on investment (ROI) to collect metrics on external things that you can't control. It's like hearing the news. What can you do about it? You're not the government or you're not directly involved. It's only the internal metrics that really make sense.

Basically, we link business goals, business objectives, and security objectives in a way that's never been done before, because we are painfully detailed when we express the outcomes that you are supposed to get from your ISM system. That will make it far easier for practitioners to actually measure the things that matter.

Business value approach


Shostack: Vicente's point about measuring the things you can control is critical. Oftentimes in security, we don't like to admit that we've made mistakes and we conceal some of the issues that are happening. A metrics initiative gives you the opportunity to get out there and talk about what's going on, not in a finger pointing way, which has happened so often in the past, but in an objective and numerically centered way. That gives us opportunity to improve.

Hietala: There's some taxonomy work to be done. One of the real issues in security is that when I say "threat," do other people have the same understanding? Risk management is rife with different terms that mean different things to different people. So getting a common taxonomy is something that makes sense.

The kinds of metrics we're collecting can be all over the map, but generally they're the things that would guide the right kind of decision making within an IT security organization around the question, "Are we doing the right things?"

Today, Vicente used an example of looking at vulnerabilities that are found in web applications. A critical metric was how long those vulnerabilities are out there before they get fixed by different lines of business, by different parts of the business, looking at how the organization is responding to that. We're trying to drive that metric toward the vulnerabilities being open for less time and getting fixed quicker.

Shostack: We've seen over the last few years that those security programs that succeed are the ones that talk to the business needs and talk to the executive suite in language that the executives understand.

We've seen over the last few years that those security programs that succeed are the ones that talk to the business needs and talk to the executive suite in language that the executives understand.



The real success here and the real step with ISM3 is that it gives people a prescriptive way to get started on building those metrics.

You can pick it up and look at it and say, "Okay, I'm going to measure these things. I'm going to trend on them." And, I'm going to report on them."

As we get toward a place, where more people are talking about those things, we'll start to see an expectation that security is a little bit different. There is a risk environment that's very outside of people's control, but this gives people a way to get a handle on it.


Related Categories
Featured Research
  • Securing Enterprise Information Technology

    In the 1980s and 1990s, business applications and data were largely confined within and protected by a Local Area Network (LAN). The 2000s introduced a significant change. Download this white paper now to learn why the shift to the cloud is changing how companies think about and manage their IT infrastructure. more

  • Office365 Adoption eGuide

    Microsoft moved to the cloud in 2014, and, as a result, Office 365 is taking off. Now, Okta customers are connecting to Office 365 in increasing numbers. This eGuide explains why IT departments should plan and deploy solutions around identity and mobility management in concert with their Office 365 roll out to get maximum user adoption. more

  • Okta Directory Integration

    For most companies, Active Directory (AD) or Lightweight Directory Access Protocol (LDAP) play a central role in coordinating identity and access management policies. When on-premise applications are integrated to Active Directory or LDAP, users get the best possible experience. That's why Okta's cloud-based identity and access management service provides a highly useful single integration point. more

  • Top 8 Identity and Access Management Challenges with Your SaaS Applications

    With more and more businesses adopting Software-as-a-Service (SaaS) applications, enterprise IT is fundamentally changing. This whitepaper presents the eight biggest Identity and Access Management (IAM) challenges associated with adopting and deploying cloud and SaaS applications, and discusses best practices for addressing each of them. more

  • Better BYOD with Pulse Secure and MDM Partners

    Learn how Pulse Secure and leading MDM product partners are transforming the way employees and IT benefit from the productivity and flexibility of BYOD — without compromising security or increasing management complexity. more