[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[ISN] Book Review: Security Metrics: Replacing Fear, Uncertainty, and Doubt



Security Metrics: Replacing Fear, Uncertainty, and Doubt
Author: Andrew Jaquith
Pages: 336
Publisher: Addison-Wesley
Rating: 10
Reviewer: Ben Rothke
ISBN: 0321349989
Summary: Authoritative text on information security metrics

One could write a book on how FUD sells security products. One of the 
most memorable incidents was in 1992 when John McAfee created widespread 
panic about the impending Michelangelo virus. The media was all over him 
as he was selling solutions for the five million PCs worldwide he said 
would be affected. The end result is that the Michelangelo virus was a 
non-event. Nonetheless, it was far from the last time that FUD was used 
to sell security.

The allure of FUD is that companies can spend huge amounts of money 
fighting nebulous digital adversaries and feel good about it. They can 
then put all of that fancy hardware in dedicated racks in their data 
center, impressing the auditors with the flashing lights giving off an 
aroma of security and compliance.

And that is the chaos that security metrics comes to solve. Security 
metrics, if done right, can help transform a company from a nebulous 
perspective on security to an effective one based on formal security 
risk metrics.

Security Metrics is a fabulous book that should be in the hands of every 
security professional. The book demonstrates that companies must 
establish metrics based on their unique requirements, as opposed to 
simply basing their requirements on imprecise industry polls, 
best-practices and other ill-defined methods.

So why don't companies do that in the first place? If security metrics 
can provide even a quarter of the benefits that Jaquith states, 
companies should run to implement them. Real security metrics require an 
organization to open up their security hood and dig deep into the engine 
that runs their security infrastructure. It necessitates understanding 
the internal requirements, unique organizational risks, myriad strengths 
and weaknesses, and much more. Very few companies are willing to 
dedicate the time and resources for that, and would rather build their 
security infrastructure on thick layers of FUD. History has shown that 
the security appliance of the month almost always beats a formal risk 
and needs assessment.

Chapter 1 lays out the problem with approaches that most companies take 
to risk management. The main problem is that traditional risk management 
is far too dependant on identification and fixing, as opposed to 
quantification and triage based on value. Quantifying and valuing risk 
is much more difficult than simply identifying, since the software tools 
used do not have an organization context or knowledge of the specific 
business domain.

Chapter 2 sets out the foundation of security metrics. The goal of these 
metrics are to provide a framework in which organizations can quantify 
the likelihood of danger, estimate the extent of possible damage, 
understand the performance of their security organizations and weigh the 
costs of security safeguards against their expected effectiveness.

The time has come for security metrics since information security is one 
of the few management disciplines that have yet to submit itself to 
serious analytical scrutiny. The various chapters provide many different 
metrics that can be immediately used in most organizations to address 

The author defines various criteria for what makes a good metric. One of 
his pet peeves is the use of the traffic light as a metaphor for 
compliance. Jaquith feels that traffic lights are not metrics at all, 
since they don't contain a unit of measure or are a numerical scale. He 
suggests using traffic lights colors sparingly, and only to supplement 
numerical data or draw attention to outliers. He astutely notes that if 
your data contains more precision than three simple gradations, why 
dilute their value by obscuring them with a traffic light.

The chapter concludes on what makes a bad metric, defined as any metric 
that relies too much on the judgment of a person. These metrics can't be 
relied on since the results can't be guaranteed to be the same from 
person to person. Also, security frameworks such as ISO-17799 should not 
be used for metrics. The book also tackles the sacred cow of risk 
management, namely ALE (annualized loss expectancy), and how it is 
significantly misused and misunderstood in the industry.

The book states that in developing metrics, there must be formal 
collaboration between the business units and the security staff. This 
collaboration serves to increase awareness and acceptance of security. 
In addition, it ensures that security requirements are incorporated into 
the lifecycle early on. This is needed as business units generally have 
no clue as to what the needed security requirements are.

Chapter 5 is a short course on analysis techniques and statistics. The 
author quotes George Colony who stated that "any idiot can tell you what 
something is. It is much harder to say what that thing means". With 
that, the book details a number of techniques for analyzing security 
data (average, median, time series, etc.) and how each one should be 

Chapter 6 is about visualization and notes that most information 
security professionals have no real idea how to show security, both 
literally and figuratively. Part of the problem is that security is 
proliferated with esoteric terminology and concepts, and the lack of 
understanding risk management amongst the masses. Part of the reason for 
this difficulty in sharing the security message with management is that 
many security practitioners lack simple metaphors for communicating 
priorities. This is compounded by the fact that the message is often 
focused exclusively on technical security issues, as opposed to the 
underlying business issues, which is was management is concerned with. 
The chapter is invaluable as it weans one off the malevolent pie chart 
and traffic light PowerPoint presentation.

Marcus Ranum notes that people seem to want to treat computer security 
like its rocket science or black magic. In fact, computer security is 
nothing but attention to detail and good design. FUD is all about 
emphasizing the black magic aspect of hackers and other rogue threats. 
Metrics are all about the attention to detail that FUD lives to 

Security Metrics: Replacing Fear, Uncertainty, and Doubt is one of the 
more important security books of the last few years. Jaquith turns much 
of the common security wisdom on its head, and the world will be a 
better place for it. Security metrics are a necessity whose time has 
come and this invaluable book shows how it can be done.


Ben Rothke, CISSP is a security consultant with BT INS and the author of 
Computer Security: 20 Things Every Employee Should Know.

Attend Black Hat USA, July 28-August 2 in Las Vegas, 
the world's premier technical event for ICT security 
experts. Featuring 30 hands-on training courses and 
90 Briefings presentations with lots of new content 
and new tools. Network with 4,000 delegates from 
70 nations.   Visit product displays by 30 top
sponsors in a relaxed setting. Rates increase on 
June 1 so register today. http://www.blackhat.com