Anil John
Making Digital Services Secure and Trustworthy

Anil John

Likelihood of Alien Invasions and Assurance Levels

 Tweet  Share  Comment  Print  Email

One of the first steps taken to protect a system from authentication errors is the determination of its assurance level requirement. That risk assessment process takes as input potential harm and likelihood of harm. This blog post looks at the applicability of the likelihood factor when assessing assurance level requirements for Internet connected systems.

The classic “E-Authentication Guidance for Federal Agencies (OMB-M04-04) [PDF]” defines risk from authentication error as a function of two factors: (a) potential harm or impact and (b) the likelihood of such harm or impact. The categories of harm and impact and how to apply them, per OMB-04-04, can be found in my earlier blog post on HOW-TO Conduct a Risk Assessment to Determine Acceptable Credentials.

The key point to note is that most risk assessment methodologies allow for “tuning” the risk using a “likelihood of harm/impact” factor, which looks something like this:

Risk of Authentication Error = Potential Impact/Harm * Likelihood of Impact/Harm

But how does one determine the “likelihood of harm” number? The two classic approaches are to explore “base rates” or to consult with experts. But there is a gotcha with experts:

The simplest and most intuitive advice we can offer [...] is that when you’re trying to gather good information and reality-test your ideas, go talk to an expert. Here’s what is less intuitive: Be careful what you ask them. Experts are pretty bad at predictions. But they are great at assessing base rates.

Decisive: How to Make Better Choices in Life and Work

So a prediction by an expert may not be all that valuable. But what about the base rates? My concern there is the constantly evolving threat environment that is the Internet, and how base rates that are based on past data are an unreliable predictor of the future.

So my recommendation in this particular case is rather simple. In this type of evaluation set the “likelihood” factor equal to 1. DO NOT discount the likelihood of harm, and ALWAYS assume there is a likelihood of harm:

Risk of Authentication Error = Potential Impact/Harm * 1

What that means is that, if as part of your assurance assessment you need to factor in the impact or harm from an alien invasion, do not discount the likelihood! Stand firm, fully account for it, and put into place compensating controls to mitigate the consequences.


Did you find this interesting? Don't miss any new posts. Sign up to automatically receive them now!

I will never share, rent, or sell your information to anyone. Cancel anytime.

This blog post first appeared on Anil John | Blog ( The opinions expressed here are my own and do not represent my employer’s view in any way.

By on |

Continue The Conversation ...

I would love to know your thoughts on this blog post. Please leave a comment below!

I am a Public Interest Technologist. I help technical leaders make digital services secure and trustworthy. Learn more »

Free Updates

I will never share, rent, or sell your information to anyone