Given the upcoming public release of ISECOM‘s Open Source Security Testing Methodology Manual (OSSTMM) version 3, I took the opportunity to have a closer look at it. While we at ERNW never adopted the OSSTMM for our own way of performing security assessments (mostly due to the fact that performing assessments is our main business since 2001 and our approach has been developed and constantly honed since then so that we’re simply used to doing it “our way”) I’ve followed parts of ISECOM’s work quite closely as some of the brightest minds in the security space are contributing to it and they come up with innovative ideas regularly.
So I was eager to get an early copy of it to spend some weekend time going through it (where I live we have about 40 cm of snow currently so there’s “plenty of occasions for a cosy reading session” ;-))
One can read the OSSTMM (at least) two ways: as a manual for performing security testing or as a “whole philosophy of approaching [information] security”. I did the latter and will comment on it in a two-part post, covering the things I liked first and taking a more critical perspective on some portions in the second. Here we go with the first, in an unordered manner:
a) The OSSTMM (way of performing tests) is structured. There’s not many disciplines out there where a heavily structured approach is so much needed & desirable (and, depending on “the circumstances” so rarely found) so this absolutely is a good thing.
b) The OSSTMM has a metrics-based approach. We think that reasonable decision taking in the infosec space is greatly facilitated by “reducing complexity to meaningful numbers” so this again is quite valuable.
c) One of the core numbers allows to display “waste” (see this post why this is helpful).
d) It makes you think (which, btw, is exactly why I invited Pete to give the keynote at this year’s Troopers). Reading it will certainly advance your infosec understanding. There’s lots of wisdom in it…
In many aspects, the OSSTMM is another “step in the right direction” provided by ISECOM. Stay tuned for another post on the parts where we think it could be sharpened.
Some days ago my old friend Pete Herzog from ISECOM posted a blog entry titled “Hackers May Be Giants with Sharp Teeth” here which – along with some quite insightful reflections on the way kids perceive “bad people” – contains his usual rant on (the uselessness of) risk assessment.
Given that this debate (whether taking a risk-based infosec approach is a wise thing or not) is a constant element of our – Pete’s and mine – long lasting relationship I somehow feel enticed to respond 😉
Pete writes about a 9-yr-old girl who – when asked about “bad people” – stated that those “look like everybody else” and, referring to risk assessment, he concludes “that you can’t predict the threat realibly and therefore you can’t determine it.”
I fully agree here. I mean, if you _could_ predict the threat realibly why perform risk assessment at all? Taking decisions would simply be following a path of math then. Unfortunately most infosec practitioners do neither dispose of a crystal ball – at least not a dependable one – nor of the future prediction capabilities this entity seems to have …
So… as long as we can’t “determine reliably” we have to use … risk assessments. “Risk” deals with uncertainty. Otherwise it would be called “matter of fact” 😉
Here’s how the “official vocabulary of risk management” (ISO Guide 73:2009) defines risk: “effect of uncertainty on objectives”.
Note that central term “uncertainty”? That’s what risk is about. And that’s, again, what risk assessments deal with. Deal with uncertainty. In situations where – despite that uncertainty – well-informed decisions have to be taken. Which is a quite common situation in an ISO’s professional life 😉
Effective risk assessment helps answering questions in scenarios characterized by some degree of uncertainty. Questions like “In which areas do we have to improve in the next 12 months?” in (risk assessment) inventory mode or “Regarding some technology change in our organization, which are the main risks in this context and how do they change?” in governance mode. [See this presentation for an initial discussion of these terms/RA modes].
So asking for “reliable threat prediction/determination” from risk assessments is just not the right anticipation. In contrast, structured RA can certainly be regarded as the best way to “take the right decisions in complex environments and thereby get the optimal [increase of] security posture, while being limited by time/resource/political constraints and, at the same time, facing some uncertainty”.
Btw: the definition from ISO 73:2009 – that is used by recently published ISO 31000 (Risk management — Principles and guidelines) too – nicely shows the transformation the term “risk” has undergone in the last decade. From “risk = combination of probability and consequence of an event [threat]” in ISO 73:2002 through ISO 27005:2008’s inclusion of a “vulnerability element” (called “ease of exploitation” or “level of vulnerability” in the appendix) to the one in ISO 73:2009 cited above (which, for the first time, does not only focus on negative outcomes of events, but considers positive outcomes as well. which in turn reflects the concept of “risk & reward” increasingly used in some advanced/innovative infosec circles and to be discussed in this blog at another occasion).
Most (mature) approaches used amongst infosec professionals currently follow the “risk = likelihood * vulnerability * impact” line. We, at ERNW, use this one as well.
Which brings me to the next inaccuracy in Pete’s blog entry. He writes: “Threats are not the same for everyone nor do they actually effect us all the same. So why do we put up with risk assessments?”.
Indeed (most) threats _are_ the same for everyone. Malware is around, hardware fails from time to time and humans make errors. Point is all this does _not_ affect everyone “the same way” (those with the right controls will not get hit hardly by malware, those with server clusters will survive failing hardware more easily, those with evolved change control processes might have a better posture when it comes to consequences from human error). And all this is reflected by the – context-specific – “vulnerability factor” in risk assessments (and, for that matter, sometimes by the “impact factor” as well). So while threats might be the same, the _associated risks_ might/will not be the same.
which, again, is the exact reason for performing risk assessments ;-))
If they _were_ the same one just would have to look up some table distributed by $SOME_INDUSTRY_ASSOCIATION.
So, overall, I’m not sure that I can follow Pete’s line of arguments here. Maybe we should have a panel discussion on this at next year’s Troopers 😉