Quelle: CSO, USA
Jeff Nigriny wants to believe that patch management software is a goodinvestment. but he can't. until nigriny, chief of security foraerospace and defense supply chain exchange network Exostar, can provea positive return on his security investment, or ROSI, he willcontinue to manually patch systems. He will download the patches,perform regression testing, deploy them in a staging area, determinewhat machines need patches and then, finally, spit them out onto hisnetwork.
"Patch management software seems like the perfect candidate to show aneasy return," says Nigriny. "Everyone kind of feels like it's theright thing to do. But I haven't procured a system. And I won't--yet.Why? Because right now the ROSI for it isn't working."
He calls this particular scenario "the most difficult and abstract interms of risk and return" that he's worked on. It's nothing like 24/7monitoring, which he said was a cinch to bring to the brass,especially since after he proved an ROSI for monitoring, he alsoshowed that he could cut costs another threefold by outsourcing it.
But with patching, he continues to build and then rebuild his ROSImodels, looking for that elusive positive return, all the while fixinghis systems the old-fashioned way.
Many of you might be snickering by now because you don't shareNigriny's idealism about the necessity of an ROSI to sell security tothe CEO and CFO. In fact, it seems you are legion in your resistance.
It's understandable, in a way. As CISO Tina LaCroix of insurancebroker and consultancy Aon points out, "This elusive packaging of theROI formula to validate our existence is one that may take us down anendless path," a path that probably looks to many CSOs like the oneNigriny's put himself on now with patch management.
But, in fact, it's not an endless path, and we're here to suggest notonly that you can use ROSI to sell security internally but that youmust. As good a reason as any for the mandate is this: Economist FrankBernhard's research shows about six cents of every revenue dollar isat risk due to a lack of information security, whereas many companiesspend barely a dime of their IT dollar on security.
"I'm not sure why IT tends to disregard these tools; it's a bitfrustrating to keep hearing you can't do it accurately," says BobJacobson, founder and president of International Security Technology(IST), which handles physical and logical security risk assessment."It's not true. The tools are there. Nuclear uses them. Pharma usesthem. The whole world has used ROI in security for a long time. [CSOs]have an opportunity to make a major contribution in theirorganization, if they have the willingness to learn this."
None of which is to say ROSI isn't hard work for a security executive;it is. But it's not hard like calculus--plenty of researchers andeconomists have taken care of sigmas and mus and other esotericeconomic math already. It's hard like running a marathon--ROSIrequires legwork, and lots of it.
We'll set you on the path to succeed in building and using ROSI as atool to sell security, with a simple three-step primer. Trust us, yourCEO will think it's worth it.
Step 1: Rethink Your Assumptions
Exostar's Nigriny is clearly not in the majority when it comes tosecurity professionals and ROSI. The defeatist shrugs that accompanyconversations about ROSI have become conventional wisdom. "Most execswant hard numbers to make financial decisions, and we live in a worldwhere you can't always have that," says Rich Mogull, research directorat Gartner G2 Cross-Industry Research. "I mean, what's the ROI of afire extinguisher?"
According to one study the American Society of Safety Engineers (ASSE)cites, the ROI of fire extinguishers is in fact about a $3 return forevery $1 invested if you take fire extinguishers as part of a largercorporate health and safety initiative--which you should, since fireextinguishers (like IT security) rarely show up as a discrete securitypurchase. (For the sake of our argument, ignore that Mogull's exampleis hamstrung by the fact that, often, regulation mandates fireextinguishers.)
The point here is ROSI can be calculated and is being calculated. Todo so with information security, though, there needs to be adeliberate effort to rethink some of the industry's assumptions andcultural biases. Specifically, there are two biases that need to beeliminated:
PRECISION IS NOT THE GOAL. One of the reasons that ROSI might feellike an endless path comes from the fact that there has been a naturaltendency in the tech sector toward approaching problems with theprecision a software engineer would expect. The "hard numbers" Mogullassumes are required.
"This is a classic problem that technologists have," says Kevin SooHoo, a researcher at security consultancy @Stake doing ROSI studies,and who at Stanford University wrote his thesis, dense with economictheory, on the subject. "They don't understand that you can make roughguesses to work out a problem. We dive into an ROSI study, and theengineers are focused on the minutiae and want to argue for dayswhether some variable should be .6 or .55. It doesn't matter," Soo Hoosays emphatically, as if he's been through this more than a few times."Choose one!"
With ROSI, like all risk assessment, the goal instead needs to beaccuracy, which isn't at all the same thing as precision. Notice thatthe ASSE study suggested about $3 for every $1. There was no attempthere to delineate the exact return, because that's not the point. Thepoint is to provide a set of guiding principles from which you, yourCEO and CFO can make good decisions about what's acceptable. In otherwords, the CEO doesn't (or shouldn't) care if a return is precisely$3.13 for every $1 spent or $2.97. He cares that it's accurate tosuggest about a 3-to-1 return, and not a 1-to-1 return or, worse, a1-to-3 return.
THE DOGMATIC I.T. MIND-SET MUST BE ELIMINATED. It's obvious why ITtends to approach problems with binary thinking. It is, after all, thelanguage of the trade. But an on-off, "either we've been hacked or wehaven't" view of the problem will make ROSI an impossible task. (Somebelieve it helps to eliminate binary terms from their discussions sothat security becomes risk management and threats aren't eliminated,they're mitigated and so forth.)
Back to the fire extinguishers. A binary thinker might suggest that,since there was no fire last year, there was no ROSI. If that is theattitude at your company, it's time to initiate some awareness andeducation because that's not how risk mitigation works. Think of itthis way: If you wear your seat belt but don't get in a car accident,does that mean you ought not invest in a seat belt because there wasno return?
No. You did get a return, because return is not measured in a dogmaticworld of what did or did not occur, but in the stochastic world ofwhat might occur and how likely it is to occur. That is the game ofrisk; prepare for something to happen by investing in ways to stop itfrom happening.
"You can't get from the cost of security incidents directly to areturn on investment," says Thomas Koulopoulos, president, CEO andfounder of Delphi Group, an information technology research andconsulting company. "You need to focus on the intermediate step. Theprobability."
Step 2: Do The Legwork
Here's just a portion of the effort Nigriny put into his patchmanagement ROSI: "I am throwing into it how many patches per year Iapply, based on three years of data. I sit down with the network teamand talk about the types of patches, their criticality level. I lookat how long it takes to vet the patch. How many rollouts result in arollback because of problems with the patch. Then I look at how manypatches I should have installed, based on all the patches on all themailing lists I subscribe to. I dedicate a day to that, but I couldtake weeks. Eventually, I come up with total time I was atX-percentage risk level before the patches were installed. Here's theaverage cost of an incident to us; that's my baseline number. Youabsolutely have to have that. There are industry baselines for thisyou can find. You can talk to peers at other companies about theirbaselines and massage them for your situation."
You get the idea. ROSI is labor-intensive. In his partial history ofthe patch management ROSI above, though, Nigriny demonstrates much ofwhat you need to do to prepare to use ROSI. Here it is:
FIND AND USE DATA THAT'S OUT THERE. The most common misconception CSOshave about ROSI is that there isn't any data available to even startan ROSI study. There's a ton of it, and the body of usable statisticsis growing. Some is free for the taking, other data you might have topay for, but the actuarial figures do exist. (CSOs who come from aphysical security world probably know this, as they've dealt with riskof theft, natural disasters and so forth for a long time and havesought out the data on the probability of such events.)
CERT and Riptech, for example, have combed over data to discover someincredibly useful facts. They measured attacks per company, whichright now come in at a rate of 2,112 attacks over two years. What'smore, at current growth, that number will grow to 8,403 attacks percompany over two years. That's a fourfold increase--which strengthensthe ROI argument. Mitigation now will protect against a growingthreat. In addition, CERT built some complicated math that showssecurity spending is a diminishing-return game; that is, as you spendmore, the probability of attack goes down but at an ever-slowing rate.By crossing this data with what are called indifference curves (toocomplicated to get into here), you can actually determine a kind ofsweet spot of security spending for your organization.
Consultancy @Stake has published well-known numbers that prove thatthe earlier you build security into applications, the higher thereturn. The company's researchers now believe they probably lowballedtheir 21 percent ROI for incorporating security from the start.
You need to cull as much of this kind of data as possible and keep itin your toolbox because the more you set out to show returns onsecurity, the more you'll be coming back to these kinds of figures.
CANVASS TO GET WHAT'S NOT OUT THERE. If the first piece of advice is"go to the library," then this is "play detective." You must developcertain numbers, like the cost of incidents to your organization andthe probability that a given incident will occur. While these numberscan be based on research, to hone them for your situation requirescanvassing of the relevant players--including business managers withinyour company, peers at similar companies, economists, consultants andso on.
"My experience is that the business managers have clear ideas aboutloss, risk and what it will cost them and probably more experiencethan the security guys know," says Jacobson of IST. "You have to go toMr. Jones and ask him what it would cost him to be down, what is hisoptimum recovery time. He will have better answers than you think,especially as he thinks about it more."
KNOW THYSELF. With all of this data in hand, you can start to build athreat profile. You'll need to know the threats specific to yourindustry, the probabilities of certain types of attacks based on thekind of company you have or the kind of infrastructure you use. Crudebut true example: Financial services companies face more attacks thanmanufacturing companies. Companies in the news endure spikes inattempted incidents. The Riptech statistics actually do somedemographic breakdowns based on industry sector.
CALCULATE CONSERVATIVELY. We're moving from how and where to get datato how you're going to present it. When pulling together numbers for aROSI study, always play it safe. Don't assume costs or benefits you'renot sure of. If someone says the probability of an attack is between10 percent and 20 percent, use 20 percent. If they say the cost of anattack is $50,000 to $100,000, take the bigger number.
And use "soft returns" as gravy. Soft returns are generally thehardest elements of a security investment to quantify. An improvedbrand image due to increased security is a soft return. Trying to addthese to the equation is difficult--some skeptical CFOs might evendismiss your ROSI argument as "fudged" because of these variables.Therefore, soft returns are more effectively used as an added benefiton top of ROSI when selling executives.
KNOW YOUR AUDIENCE. And when selling the bosses, the CSO should learnwhat those executives are looking for in terms of return. "I can'ttell you how many times these things are rejected out of hand, becauseIT is selling something that the executives aren't even looking tobuy," says Delphi's Koulopoulos.
Know how the executives want the ROSI positioned--cash savings,productivity gains, increase in security--and move forward that way.Many sources also report that making the ROSI case interactive forexecutives--allowing them to tweak variables and watch what happens tothe ROSI--is by far the single most effective selling tool you canuse. "The key is not to be defensive about the data, as I think ITsometimes can be," IST's Jacobson says. "Don't defend the model;explain it."
Nigriny thinks there are other, underrated sales skills CSOs need tofoster in themselves. A general familiarity with accounting ispriceless, he says. Also, "You have to be good at public speaking andat PowerPoint engineering. If you're speaking to the CFO, expect himto do some number crunching; have your numbers ready for him. The CEO?The executive summary is far more important. Talk to the CFO ahead oftime; you'll have his support, and the CEO won't have to sit throughthe numbers discussion," says Nigriny.
We weren't kidding when we said this is laborious, intensive work. ToNigriny, ROSI is fractallike, in that the closer he examines hissituation, the more intricate it becomes. "Every time I thought I hadit covered, a raft of new variables came up. I've just got this swagof numbers here I have to deal with," a nonplussed Nigriny says.
It's up to the CSO to set the thresholds of what's really needed for aparticular scenario. You can make ROSI as simple or as complicated asyou think is necessary, and an obvious tenet that emerges is that asimpler ROSI will be somewhat less accurate than a detailed ROSI, butthe detailed version will require ever more legwork.
Step 3: Do The Math
In the end, the math is simple. You subtract cost from benefits. Apositive number is good: a return on investment. A negative number isbad: You're spending more than you're getting.
Of course, the math behind the variables and coefficients that go intothe costs and benefits is massively complex. Fortunately, if you'vegot raw data from your legwork, someone else has done or will do thedifficult computations for you. Still, there are some basic riskcomputations you should know. Here they are:
ANNUAL LOSS EXPECTANCY. ALE is the foundation of risk assessment. Itis what it sounds like: how much money you expect to lose per year dueto some sort of security incident. Note that this is different thanthe raw cost of an incident (which, remember, you should always keepas a baseline). It's actually the raw cost times the probability of anevent in the next year. So the ALE of a security breach that costs $1million and has a 40 percent chance of happening is:
Incident cost X Probability of incident = ALE
$1,000,000 X 0.4 = $400,000
MODIFIED ALE. mALE is the same equation, but with the probabilityaffected by mitigation measures you take. Imagine the above scenariowere a virus attack. You introduce antivirus software that cuts inhalf the probability of a successful attack, to 20 percent. Or, youstart an awareness program that reduces probability 5 percent. (Theseare arbitrary, but if you've done the legwork from Step 2, you'll havereal numbers to plug in here.) Then:
Probability X Mitigation A = Modified probability
Probability X Mitigation B = Modified probability
A: 0.4 X 0.5 = 0.2
B: 0.4 X 0.95 = 0.38
You must consider each mitigation separately. Once you've gone throughthe process for several types of mitigation, you can pick which onesyou feel are most important or provide the best return. (Of course,some mitigation measures will have overlapping effects. We're notputting that into this math.)
At any rate, adding mitigation measures produces modified ALEs:
Incident cost X Modified probability = mALE
A: $1,000,000 X 0.2 = $200,000
B: $1,000,000 X 0.38 = $380,000
So, in each case you've reduced your ALE.
ALE - mALE = Savings
A: $400,000 - $200,000 = $200,000
B: $400,000 - $380,000 = $20,000
This is the step at which executives will want to interact with themodel, seeing how different measures that they take affect their mALE.
Now, to get a basic return, you simply subtract the cost to implementeach mitigation measure from your savings on your mALE by implementingthe mitigation. Let's say mitigation A, antivirus software, costs$120,000. And mitigation B, an awareness program, costs $8,000. Then:
Savings - Mitigation cost = ROSI
A: $200,000 - $120,000 = $80,000
B: $20,000 - $8,000 = $12,000
Both mitigation measures provide a ROSI (if the final number came outnegative, then you're spending more than you're getting back).Awareness actually has a higher return; or put another way, you getthe most bang for the buck. (Your savings are 2.5 times what youspend, whereas in the antivirus case, they are 1.7 times what youspend.)
This is a simple model. No doubt CSOs, consultants and vendors withtheir own ideas will hue and cry that we've presented ROSI in thisparticular, facile way. But we're only trying to provide a guidingprimer. To attempt more in this space would be a fool's errand. (Forexample, we didn't even approach the concept of Net Present Value,which takes into account costs and benefits over time as if all themoney were here now. Ask your CFO.)
Don't take this as a final "how to" but rather as a starting point todevelop your own ROSI. But don't forget: The most important message isto do the homework. Collect as much data as possible so that there'splenty to crunch.
ROSI is empirical, but in many ways it's emotional, believe it or not.It is about coming up with numbers, but those numbers are only usefulin the context of how executives feel about them. ROSI is riskeconomics that paints a picture of your organization's attitude towardsecurity. What level of risk is the enterprise comfortable with? Howdoes the company prioritize its limited resources? Is technology orawareness more valuable as a tool? Suddenly you're answering businessquestions based on the security numbers.
"The numbers right now show patch management automation doesn'tprovide a positive return for this organization," Nigriny says. "Sowhy would I do it? It just doesn't make sense." Just by coincidence,it seems, ROSI has aligned Nigriny with the business.