S4x20 Video: Lessons Learned from Norsk Hydro on Loss Estimation and Cyber Insurance

I gave a talk at S4X20 in January on the Norsk Hydro ransomware attack.  The full video has now been posted on YouTube:




Like all great presentations, it includes a Seinfeld reference :-)

Splattered Swan: Collateral Damage, Friendly Fire, and Mis-fired Mega-systems

Like Curly from the "Three Stooges" said,
"I'm a victim of circumstances!"

There is a type of "...Swan" that is not surprising or extreme in its aggregate effect, but is extremely surprising to a particular entity that was considered to be outside the scope of the main process.  A bomb is supposed to kill enemy soldiers, not your own soldiers. 

I call this type "Splattered Swans".

Context: Rethinking "Black Swans"

This post is nineth in the series "Think You Understand Black Swans? Think Again". The "Black Swan event" metaphor is a conceptual mess.

Summary: It doesn't make sense to label any set of events as "Black Swans".  It's not the events themselves, but instead they are processes that involve generating mechanisms, our evidence about them, and our method of reasoning that make them unexpected and surprising.

Definition 

A "Splattered Swan" is a process where:

• The generating process involves a very powerful force (i.e. penal, constraining, or damaging force) with less than perfect aim.
• The evidence are official rules, specifications, or scope, or experience that is limited what is intended;
• The method of reasoning are based on the assumption that the aim will be perfect and error-free, or that errors will be "well behaved".

 

Main Features

A Splattered Swan arises when a very powerful system is prone to misfiring in very bad ways, causing damage to some entities that are considered "safe" or "out of bounds" by normal reasoning.  That these outcomes are extreme or surprising are basically due to failures to understand the total system and the ways it can fail.  

Two key features of  Splattered Swans are 1) critical error conditions are excluded from reasoning on principle and 2) those errors are potentially severe, even the first time.  A lot of systems adapt by trial and error, but that only works of the magnitude of errors (i.e. aim) is relatively small and the magnitude of collateral damage is also relatively small.  Consider airplane bombers from World War II aiming to kill enemy troops that are located near allied troops.  Even though they had bomb sights, the bombers were notoriously inaccurate.  With ordinary large bombs, the risk of "friendly fire" (i.e. killing your own troops) is high.  If the bomber is carrying a single atomic bomb, then the risk of "friendly fire" becomes extremely high, because you only get one chance to aim and drop and there is no feedback from previous attempts.  Plus the damage process is extreme.  In the other direction, if there are several bombers, and the first bombers drop flairs instead of bombs, then the cost of error is small and the opportunity of corrective feedback has the potential to dramatically reduce the risk of "friendly fire".

Another important feature of Splattered Swans are the blind spots created by the "official" or "intended" definition of the system of interest.  This can lead analysts and decision-makers to never even consider the possibility of collateral damage or unintended consequences. 
 

One Example

"Offensive cyber" , a.k.a. "hack back" is an example from the domain of cyber security.  There are many flavors of offensive cyber, but the most extreme involve damaging the targets, either physically or digitally or both.  Such extreme attacks might also be considered acts of war, a.k.a. "cyber war".  Putting aside the ethics or advisability of offensive cyber, there is immense potential for collateral damage.  First, it might be hard or impossible to attribute a given attack to the "real"  threat agents or groups (a.k.a. "Black Hat").  They might operate through affiliates, mask or disguise their tools and infrastructure, and might even intentionally implicate a different agent or group in the "Indicators of Compromise" and other forensic evidence.  Even if you can correctly identify the attacking group, it may be hard to attack them in a way that doesn't also do harm to socially-important entities or resources (e.g. cloud computing resources, networks, etc.).  Finally, in corner-case situations there is also a non-zero potential for self-harm, where an offensive cyber attack backfires on the "White Hat" attacker.

From a planning and on-going management viewpoint, it is much harder to anticipate and control the side-effects of cyber attacks than it is for physical attacks. 
 

How to Cope with Splattered Swan

 It is relatively simple to cope with Splattered Swan systems.  Don't take the "official" or "intended" system as a strict definition of what behavior or outcomes are possible.  Use Scenario Planning or "What If?" analysis to look outside the "official" or "intended" to identify potential for collateral damage.

Then look for ways to introduce error-correcting feedback or damage mitigations for the collateral damage.  Another good mitigation is to reduce the intensity of the damage/punishment process.

Swarm-as-Swan: Surprising Emergent Order or Aggregate Action

A flock of swans in swan-shaped formation

In complex systems with many interacting elements or agents, most of the time the actions of individual agents "average out" and remain local.  In some systems and in some circumstances, surprising aggregate or emergent behavior patterns arise.

Emergence is a common characteristic of such systems.  But that alone doesn't qualify them as a "Swan".  It requires several other factors that could, under the right circumstances, yield very surprising or cataclysmic outcomes.  I call this the "Swarm-as Swan", since swarming behavior (birds, fish, insects) is one well-known type of emergent phenomena, but this category is explicitly not limited to swarm phenomena.

Context: Rethinking "Black Swans"

This post is eighth in the series "Think You Understand Black Swans? Think Again". The "Black Swan event" metaphor is a conceptual mess.

Summary: It doesn't make sense to label any set of events as "Black Swans".  It's not the events themselves, but instead they are processes that involve generating mechanisms, our evidence about them, and our method of reasoning that make them unexpected and surprising.

 

Definition 

A "Swarm-as-Swan" is a process where:

• The generating process involves a large-scale Complex Adaptive System that has regions in the state space where collective and/or emergent phenomena become dominant, leading to collective behavior that is dramatically different from the behavior in the "normal" regions of state space.
• The evidence are patterns of system behavior and interaction at various scales (individual, group, collective) and especially surprisingly different patterns, including downward causation and varieties of self-organization and information processing;
• The method of reasoning are mental models of the system, whether formal or informal, sophisticated or common sense, and the implications of those models on what behaviors are "normal" and expected vs. what is surprising.

Main Features

The field of Complexity Science has grown and blossomed over the last 30 years.  This Wikipedia article gives a good summary, along with the central theme of Complex Adaptive Systems (CAS).  From that article, the common characteristics of CAS:

• The number of elements is sufficiently large that conventional descriptions (e.g. a system of differential equations) are not only impractical, but cease to assist in understanding the system. Moreover, the elements interact dynamically, and the interactions can be physical or involve the exchange of information
• Such interactions are rich, i.e. any element or sub-system in the system is affected by and affects several other elements or sub-systems
• The interactions are non-linear: small changes in inputs, physical interactions or stimuli can cause large effects or very significant changes in outputs
• Interactions are primarily but not exclusively with immediate neighbors and the nature of the influence is modulated
• Any interaction can feed back onto itself directly or after a number of intervening stages. Such feedback can vary in quality. This is known as recurrency
• The overall behavior of the system of elements is not predicted by the behavior of the individual elements
• Such systems may be open and it may be difficult or impossible to define system boundaries
• Complex systems operate under far from equilibrium conditions. There has to be a constant flow of energy to maintain the organization of the system
• Complex systems have a history. They evolve and their past is co-responsible for their present behavior
• Elements in the system may be ignorant of the behavior of the system as a whole, responding only to the information or physical stimuli available to them locally

While all CAS are, in principle, capable of emergent behavior, not all are capable of big surprises in behavior that we require for our "...Swans" series.  Roughly, there are three levels of emergent phenomena

• Level 1: Emergent behavior -- behavior of large numbers of individuals becomes interdependent and mutually influencing, far beyond the range of causal and information interaction, including some downward causation where the collective shapes the individuals.  Examples: flocks of birds, schools of fish. 
• Level 2: Emergent functional structures -- the formation of stable networks of individuals that constitute functional subsystems. "Functional" means they do some work beyond just collective behavior of Level 1. An excellent example is the "glider" phenomena in Conway's Game of Life (a type of cellular automata).

A single Gosper's glider gun creating "gliders"

•  Level 3: Emergence against a model -- similar to Level 2, but the "stable functional subsystems" have information processing and self-sustaining capabilities (including possibly metabolizing energy and repairing/regenerating structures).  In a real sense, Level 3 systems "take on a life of their own", at least for an extended period.  Example: emergent subsystems that function as regulators (e.g. thermostat), communication systems (e.g. encoding, decoding, transmission), pattern matching, optimization, etc.  The human immune system has some of these capabilities.

As you move up these levels, the nature of emergent phenomena changes dramatically, from mildly surprising and interesting at Level 1 up to "HOLY COW" at Level 3.   But our understanding of CAS is most at Level 1, some at Level 2, and only a little at Level 3. Put another way, if you create a hundred or thousand different CAS in a computational laboratory, most would only exhibit Level 1 emergent phenomena, a few would exhibit Level 2 emergent behavior, and only a very few, under narrow circumstances, would have Level 3 capabilities.  (In his book A New Kind of Science, Stephen Wolfram did exactly this investigation for all simple cellular automata.)

Like all of the "...Swans", it's just as important to understand the evidence that we use to understand these systems (i.e. CAS), and also our methods of reasoning. It's the combination of all three that give rise to the surprising/shocking/extreme behavior that we associate with "...Swans".

The most common evidence people pay attention two is either individual-based behaviors and interactions and the most common collective behavior patterns and distributions.  If the states of the CAS are in "low complexity" regions of the state space (i.e. not in one of the three Levels, above), then people may not even recognize that the CAS is capable of complex emergent phenomena.  The reverse is also true.  If the CAS is normally in a highly coherent, highly functional state then people may not observe or understand the micro-level behavior that supports that phenomena.  The evidence we need most is the location of "phase transitions" in state space, where the CAS shifts dramatically from one regime to another.  Unfortunately for us mortal humans, it's almost impossible to know in advance where the important phase transitions are in CAS, especially the Level 2 and 3 CAS.

Our methods of reasoning about CAS fall into three categories: 1) Intuitive (i.e patterns of "normal" behavior with small deviations, naïve causal models, "folk wisdom", etc.);  2) Linear Models (i.e. the standard tools of science up to ~1990);  3) Non-linear Models, including Agent-based Modeling.

Methods 1) and 2) are most common, and work well in "normal" circumstances, but are very prone to catastrophic failures of reasoning when the CAS enters a new, unfamiliar regime of emergent behavior.  Method 3) is specifically designed to understand CAS in all their complexity, but they aren't a "magic bullet" that completely eliminate the potentials for surprise or extreme outcomes.

One huge difference between Method 2) "Linear Models" and Method 3) "Non-linear Models" is that the that Method 3) usually does not  yield a forecast or prediction of system behavior in the same way that Method 2) does.  Instead it can help us understand when and why the CAS will change regimes, which is still very useful information to understand potential surprises.

Examples

In the previous section I  mentioned some illustrative examples. But here I'll mention two "biggies".

The  modern economy has been characterized and studied as a Complex Adaptive System (CAS), especially to understand innovation and crises, especially societal/economic collapse.  Some of the first books published by the Santa Fe Institute in the late `80s were titled "The Economy as a Complex Adaptive System".

Mass uprisings and mass revolutions are other classes of phenomena that benefit from study as CAS.  I won't go into detail here, but if you are curious, you might read up on these cases:

• Revolutions of 1848
• The Gordon Riots in London, 1780
• The Haitian Revolution

How to Cope with Swarm-as-Swan

The first step is to recognize that the system you are dealing with has the characteristics of a Complex Adaptive System.  Start with the Wikipedia page, then read some of the general references listed at the bottom.  This will give you the basic knowledge plus some exposure to many types of CAS.

The second step is to characterize the types of emergent behavior and structures that are within the "possibility space" of the CAS.  But stay away from "magical thinking" and "conspiracy theories".  

The third step is to apply modeling tools that are appropriate to the complexity of the CAS.  Linear models are fine for what they do, but don't try to use them to identify "phase transitions" from simple to complex behavior, etc.

If you aren't mathematically inclined or are not comfortable programming your own Agent-based Models (ABM), you can at least read books and papers that utilize these models and learn from the pros who built them and analyzed them.  Even better, you can play with them yourself using the Model Library that comes with NetLogo (free and open source).  Each model is controlled by sliders and buttons, and comes with documentation that guides you how to use it, how to interpret it, and how to explore it.

 

 

 

Talk Like a Cyber Insurance Risk Analyst

In a recent class on catastrophe risk modeling, I learned the definition of terms that are common in insurance but not so well understood elsewhere:

• Peril
• Exposure
• Hazard
• Ground-up Loss
• Risk

Read on for definitions, ending with an analogy that, hopefully, ties them all together.

RESET: "Data-driven Security Smashup" will launch in Fall 2019

Big change of plans for the "Data-driven Security Smashup":

We are canceling the live event in Las Vegas, August 3 - 5. 

Instead, we aim to launch one or more Virtual Smashup projects in the Fall of 2019, followed by one or more live events early in 2020, perhaps one in the US and one in UK.

Why?

Basically, we ran out of time as we were trying to organize the event: sponsorship, organizer recruiting and on-boarding, Call for Participation, legal structure, venue.  No fault to anyone.  We started relatively late, and our standards are high.  We didn't want to just throw it together and risk having things fall apart during the event.

Benefits

This new schedule gives us time to do it right, starting with the basics.  For example, we will secure a "fiscal sponsorship" relationship so we have the legal, financial, and operational infrastructure to take donations, manage risk, and to spend money responsibly.

Another "basic" that needs attention is contact and relationship management for all the people who have expressed interest, asked questions, or need responses.  This includes a dedicated website instead of this blog.

The new schedule gives us the lead time to recruit organizers and collaborators in academia, professional associations, industry, independent consultants, and government, both in US and internationally (mostly UK, Europe, Switzerland).

Personally, I'm not disappointed. The core idea is solid.  Lots of interest.  This change makes some space for some of my other priorities (dissertation!).

Stay tuned!

SIRAcon: "Probabilistic models of breach impact – combining theory and empirical data"

Here are my slides from my SIRAcon talk.

Here's a slide that got a lot of attention.  (Humorous, of course)

Announcing: Data-driven Security Smashup

Data-driven Security Smashup

A Hackathon + Supercollider of Talent, Ideas, & Resources

Fall 2019

Las Vegas, NV; Saturday - Monday August 3-5, 2019

[updated June 14, 2019, see "RESET..." for more info]

• Venue: rented house*, well off the Strip Working on it.  Aiming for UNLV
• Timing: just before B-Sides LV/Black Hat/Defcon
• Organizers: Me, Jon Hawkes, plus 2-6 others to be named (interested? Contact me)
• On-site capacity: ~30 30 - 60
• Remote/virtual participation? Yes. Details TBD  Also several Satellite locations
• Call for Participation: coming soon, mid May
• Call for Sponsorship: coming soon, mid May
• Other locations: if this first Smashup goes well, we'd like to 'step-and-repeat' it soon in the EU, UK, Switzerland, elsewhere in US, and maybe more
• Updates and news:  follow @dds_smashup on Twitter

Summary

The Data-driven Security Smashup (DDS Smashup) is a combination of hackathon and ‘supercollider’ of talent, ideas, and resources, aiming for breakthrough innovations in data-driven cyber security, especially solutions to problems that span domains of people, process, technology, institutions, and culture.  

Why Is Breakthrough Innovation in Cyber Security So Hard?

Short answer: Innovation activities tend to focus on just a few pieces at a time, treating it as a simple problem. That doesn't create breakthroughs because the system* is too complicated.

* "system" = technology, information, people, processes, organizations, institutions, economics,...

In Sciences of the Artificial, Herbert Simon argued that most evolved systems (natural and artificial) were "partially decomposable" (if not fully decomposable) into units or subsystems that could be studied and understood in isolation. While cyber security is partially decomposable for many purposes, it is my conjecture that it is much less decomposable than we believe or desire.

What this means is that breakthrough innovations will depend on many, simultaneous inventions, including crossing system levels.

Does Modern Portfolio Theory (MPT) apply to cyber security risks?

Many months ago, my colleague David Severski asked on Twitter how Modern Portfolio Theory (MPT) does or does not apply to quantified cyber security risk:

I replied that I would blog on this "...soon".  Ha!  Almost four months later.  Well, better late than never.

Short answer: No, MPT doesn't apply.  Read on for explanations.

NOTE: "Cyber security risk" in this article is quantified risk -- probabilistic costs of loss events or probabilistic total costs of cyber security.  Not talking about color-coded risk, categorical risk, or ordinal scores for risk.  I don't ever talk about them, if I can help it.

The Swan of No-Swan: Ambiguous Signals Tied To Cataclysmic Consequences

What do you see? colored blocks, or a Black Swan, or both?
This is figure-ground reversal, a type of ambiguity.

We are in the middle of the 100th anniversary of the Great War (a.k.a. World War I).  None of the great powers wanted a long total war. Yet the unthinkable happened anyway.

Surprisingly, historians are still struggling to understand what caused the war.

One of the biggest causal factors was ambiguous signals that precipitated cascading actions and reactions. When tied to cataclysmic consequences, this represents a distinct class of "Black Swan" systems.

(Here are some great lectures for those interested in a full analysis of causes of the Great War: Margret MacMillan, Michael Neiburg, Sean McMeekin)

Rethinking "Black Swans"

As I have mentioned at the start of this series, the "Black Swan event" metaphor is a conceptual mess. (This post is seventh in the series "Think You Understand Black Swans? Think Again".) 

It doesn't make sense to label any set of events as "Black Swans".  It's not the events themselves, but instead they are processes that involve generating mechanisms, our evidence about them, and our method of reasoning that make them unexpected and surprising.

Definition

A "Swan of No-Swan" is a process where: 

• The generating process is some set of large opposing forces that can be triggered by a set of signals or decisions tied to ambiguous signals;
• The evidence are signals -- communications, interpreted actions, interpreted inaction, rhetoric/discourse, irreversible commitments -- that have ambiguous interpretations, either intentionally or unintentionally;
• The method of reasoning either rational expectations (normative Decision Science) or biased expectations (Behavioral Psychology and Economics).  The key feature is lack of attention or awareness that one might be mis-perceiving the signals, combined with a strategic preference for precautionary aggressiveness.

Main Features

First, let us recognize that ambiguity is pervasive in social, business, and political life.  Ambiguous signals and communication have many pro-social functions: keeping our options open, saving face, avoiding insult or offense, optimistic interpretation of events, and so on.  They are especially prevalent in the lead-up to major commitments -- romance+marriage in personal life and big ticket sales in commercial life.

Most of the time, ambiguity has a smoothing effect.  It reduces the probability of extreme/rare events because of the flexibility of action and response associated with ambiguous signals.  Therefore, most people would not associate ambiguous signals with any type of "Black Swan" phenomena.

But when tied to "large opposing forces", things change and that's why this deserves to be a separate type of Black Swan.  Ambiguous signals become dangerous when they are linked to cataclysmic processes via certain types of reasoning processes.  It's not rational vs. biased.  Instead, it's committed self-confidence vs. self-aware fallibility. In committed self-confidence, there is lack of attention or awareness that one might be mis-perceiving the signals, combined with a strategic preference for precautionary aggressiveness.  "Shoot first, ask questions later".

Examples

Military forces leading to total war are the obvious case, and most common in history.   But we are now in a new age -- the Cyber Age!  (Yes.  I said it. Cyber)  Here are some cyber examples.

• Offensive cyber capabilities -- By "offensive" I mean everything from "hack back" to punitive or disabling cyber attacks on critical infrastructure. If it becomes common for nation states and various non-nation actors to develop and deploy offensive capabilities, they everyone faces the strategic dilemma as to when and how much to deploy/trigger each capability.  This depends critically on the ability of each actor to detect and accurately interpret a wide variety of signals and evidence related to normal and abnormal activity, including breach events, threat actor attribution, signs of escalation, and so on.  These are all swimming in ambiguity, including intentional ambiguity (spoofing, camouflage, etc.)
• Remote kill switches -- What if Internet of Things (IOT) makers build "remote kill switches" in their devices? After all, we'd like to prevent our toaster, pacemaker, automobile, or drone from doing harm in the case when it starts malfunctioning catastrophically.  Are there scenarios where one or more IOT manufacturers decide to remotely kill at the same time?  What if their monitoring instruments make it appear that some threat actor(s) are making self-driving cars intentionally crash into crowds of people?  Out of an abundance of caution, they might remotely kill the IOT devices to cut off the apparent disaster as it is unfolding.  But maybe the threat actor is only spoofing the signals because they pwned the monitoring devices and infrastructure.  Or maybe it's the precautionary action of some other IOT device system owner that is causing your monitoring system to go bonkers.  I could go on but you get the idea.

How to Cope with Swan of No-Swan

It would be good to decouple the generating process if possible.  Avoid the arms race to begin with.  (Give peace a chance!)

Absent that, the best antidote is to treat evidence and signals pluralistically, which means avoiding the tendency to commit to one interpretation or another too early.  This is very hard to do within one person or even one cohesive team. It's easier to assign different "lenses" to different people or teams who then proceed with their analysis and decision recommendations independently.

Finally, the decision makers who can "pull the trigger" should seek strategy alternatives to the preference for precautionary aggressiveness ("Shoot first, ask questions later").  While decision makers may feel like this is their only choice (and it may be), there is great advantage if more flexible alternatives can be found.

The Cyber Insurance Emperor Has No Clothes

(Of course, the title is hyperbole and attention-seeking. Now that you are here, I hope you'll keep reading.)

(click to enlarge)

In the Hans Christian Anderson story, The Emperor's New Clothes, the collective delusion of the Emperor's grand clothes was burst by a young child who cried out: "But he has got nothing on!"

I don't mean that cyber insurance has no value or that it is a charade.

My main point: cyber insurance has the wrong clothes for the purposes and social value to which it aspires.

This blog post sketches the argument and evidence. I will be following up separately with more detailed and rigorous analysis (via computational modeling) that, I hope, will be publishable.

tl;dr: (switching metaphors)

As a driving force for better cyber risk management, today's cyber insurance is about as effective as eating soup with a fork.

(This is a long post. For readers who want to "cut to the chase",  you can skip to the "Cyber Insurance is a Functional Misfit" section.)

Public Statement to the Commission on Enhancing National Cybersecurity, 6-21-2016

[Submitted in writing at this meeting. An informal 5 min. version was presented during the public comment period. This statement is my own and does not represent the views or interests of my employer.]

Summary

Cyber security desperately needs institutional innovation, especially involving incentives and metrics.  Nearly every report since 2003 has included recommendations to do more R&D on incentives and metrics, but progress has been slow and inadequate.

Why?

Because we have the wrong model for research and development (R&D) on institutions.

My primary recommendation is that the Commission’s report should promote new R&D models for institutional innovation.  We can learn from examples in other fields, including sustainability, public health, financial services, and energy.

What are Institutions and Institutional Innovation?

Institutions are norms, rules, and social structures that enable society to function. Examples include marriage, consumer credit reporting and scoring, and emissions credit markets.

Cyber security[1] has institutions today, but many are inadequate, dysfunctional, or missing.  Examples:

1. overlapping “checklists + audits”; 
2. professional certifications; 
3. post-breach protection for consumers (e.g. credit monitoring); 
4. lists of “best practices” that have never been tested or validated as “best” and therefore are no better than folklore.  

There is plenty of talk about “standards”,  “information sharing”, “public-private partnerships”, and “trusted third parties”, but these remain mostly talking points and not realities.

Institutional innovation is a set of processes that either change existing institutions in fundamental ways or create new institutions.   Sometimes this happens with concerted effort by “institutional entrepreneurs”, and other times it happens through indirect and emergent mechanisms, including chance and “happy accidents”.

Institutional innovation takes a long time – typically ten to fifty years.

Institutional innovation works different from technological innovation, which we do well.  In contrast, we have poor understanding of institutional innovation, especially on how to accelerate it or achieve specific goals.

Finally, institutions and institutional innovation should not be confused with “policy”.  Changes to government policy may be an element of institutional innovation, but they do not encompass the main elements – people, processes, technology, organizations, and culture.

The Need: New Models of Innovation

Through my studies, I have come to believe that institutional innovation is much more complicated  [2] than technological innovation.   It is almost never a linear process from theory to practice with clearly defined stages.

There is no single best model for institutional innovation.  There needs to be creativity in “who leads”, “who follows”, and “when”.  The normal roles of government, academics, industry, and civil society organizations may be reversed or otherwise radically redrawn.

Techniques are different, too. It can be orchestrated as a “messy” design process [3].  Fruitful institutional innovation in cyber security might involve some of these:

• “Skunk Works”
• Rapid prototyping and pilot tests
• Proof of Concept demonstrations
• Bricolage[4]  and exaptation[5]
• Simulations or table-top exercises
• Multi-stakeholder engagement processes
• Competitions and contests
• Crowd-sourced innovation (e.g. “hackathons” and open source software development)

What all of these have in common is that they produce something that can be tested and can support learning.  They are more than talking and consensus meetings.

There are several academic fields that can contribute defining and analyzing new innovation models, including Institutional Sociology, Institutional Economics, Sociology of Innovation, Design Thinking, and the Science of Science Policy.

Role Models

To identify and test alternative innovation models, we can learn from institutional innovation successes and failures in other fields, including:

• Common resource management (sustainability)
• Epidemiology data collection and analysis (public health)
• Crash and disaster investigation and reporting (safety)
• Micro-lending and peer-to-peer lending (financial services)
• Emissions credit markets and carbon offsets (energy)
• Open software development (technology)
• Disaster recovery and response[6]  (homeland security)

In fact, there would be great benefit if there were a joint R&D initiative for institutional innovation that could apply to these other fields as well as cyber security.  Furthermore, there would be benefit making this an international effort, not just limited to the United States.

Endnotes

[1] "Cyber security" includes information security, digital privacy, digital identity, digital information property, digital civil rights, and digital homeland & national defense.

[2] For case studies and theory, see: Padgett, J. F., & Powell, W. W. (2012). The Emergence of Organizations and Markets. Princeton, NJ: Princeton University Press.

[3] Ostrom, E. (2009). Understanding Institutional Diversity. Princeton, NJ: Princeton University Press.

[4] “something constructed or created from a diverse range of available things.”

[5]  “a trait that has been co-opted for a use other than the one for which natural selection has built it.”

[6] See: Auerswald, P. E., Branscomb, L. M., Porte, T. M. L., & Michel-Kerjan, E. O. (2006). Seeds of Disaster, Roots of Response: How Private Action Can Reduce Public Vulnerability. Cambridge University Press.

Work in Progress (Cyber Security Investment Game)

Here is a video demo of the NetLogo "Cyber Security Investment Game", as a work in progress.  This run is a 6/3 multiplayer game, meaning 6 cooperative Defenders and 3 adversarial Attackers. Defenders play a 2-player game with everyone else, while Attackers only play 2-player games with Defenders. It's a complicated game, with up to 400 possible "moves" for each agent at each step, and also a dynamic game, meaning the structure of the game can change dynamically during the course of play. (Game Theory Purists will probably hate it for that reason!)

Here's a video of 1,700 time steps, with 20x speed-up.



What you see in this simulation is an ecosystem that provides a positive environment for both Defenders and Attackers. That is, they both have consistently positive payoffs (see bar graph, center left).

While the overall trends of payoffs are pretty clear (bottom graph), there is an "unruly" back and forth between attackers and defenders. Crucially, the time series of payoffs is non-stationary for all agents (see center graphs, third and fourth from the top). Simply, non-stationary means that the probability distribution for each payoff  changes over time. You can see a sample distribution of payoffs in the two histograms, center right. You'll notice them changing shape (going from skew to symmetric) and also changes in mean and standard deviation ("SD").  Non-stationarity has implications for risk estimation, as I will detail in the up-coming WEIS paper.

In terms of progress toward the goal, I would say this is not yet a model of cyber security investment. It models competitive relationships rather than not host-parasite relationships which I believe are close to the true nature of cyber security ecosystems. The good news is that I believe I know what extensions and modifications need to be made. I'll save that for a later post.

Here are some key features of this version of the model:

• Three levels of investment: 1) architecture/infrastructure; 2) capabilities; 3) practices/routines (a.k.a. "moves")
• Asymmetric number and variety of "moves" for Defenders and Attackers
• Parametric control over diversity of "moves" and investments by Defenders and Attackers

Yes, the model is getting complicated.  But I hope this richness will be rewarded when crucial results are revealed in experiments.  We shall see...

<update>

Here is the same run after 4,000 time steps.  No sign of equilibrium or stationary time series.

(Click to enlarge)

Time & Uncertainty (2nd post: "What kind of game is cyber security investment?")

Summary: Time and uncertainty are essential features of any model of the "game of cyber security".  Models that do not include them as central features are not fit for purpose.  But, yes, they do make life more difficult for modelers and their audiences. While I make the case that both are essential, I leave open the question as to what is the most parsimonious method or treatment.

What kind of game is cyber security investment? (post #1 of ?)

This is first in a series of blog posts where I think out loud as I build a paper for WEIS 2016, and also a component for my dissertation.

The focus is on "investment" broadly defined.  This means money invested in people, tools, infrastructure, processes, methods, know-how, etc.  It also means architectural commitments that shape the business, technical, legal, or social aspects of cyber security for a given person or organization.  All these investments provide the foundation for what a person or organization is able to do (i.e. their "capabilities") and the means of executing day-to-day tasks ("routines", "processes", "practices", etc.).

If cyber security investment is a strategic game between attackers and defenders, and among defenders, then what kind of game is it?

Summary

In simple terms, people tend to think of cyber security investment as being one of (at least) five types of games:

1. An optimization game, where each player finds the optimal level of spending (or investment) to minimize costs (or losses).  This view is favored by Neo-classical Economists and most Game Theorists.
2. A collective wisdom game, where the collective searching/testing activities of players leads to the emergence of a "collective wisdom" (a.k.a. "best practices") that everyone can then imitate. This view is favored by many industry consultants and policy makers.
3. A maturity game, where all players follow a developmental path from immature to mature, and both individual and collective results are improved along the way.  This view is favored by many industry consultants.
4. A carrots-and-sticks game, where players chose actions that balance rewards ("carrots") with punishments ("sticks") in the context of their other goals, resources, inclinations, habits, etc.  This view is favored by some Institutional Economists, and some researchers in Law and Public Policy.  It is also favored by many people involved in regulation/compliance/assurance. 
5. A co-evolution game, where the "landscape" of player payoffs and possible "moves" is constantly shifting and overall behavior subject to surprises and genuine novelty.  This view is favored by some researchers who employ methods or models from Complexity Science or Computational Social Science.  This view is also a favorite of hipsters and "thought leaders", though they use it as metaphor rather than as a real foundation for research or innovation.

But what kind of game is cyber security, really?  How can we know?

These questions matter because, depending on the game type, the innovation strategies will be very different:

1. If cyber security is an optimization game, then we need to focus on methods that will help each player do the optimization, and to remove disincentives for making optimal investments.
2. If cyber security is a collective wisdom game, then we need to focus on identifying the "best practices" and to promote their wide-spread adoption.
3. If cyber security is a maturity game, then we need to focus on the barriers to increasing maturity, and to methods that help each player map their path from "here" to "there" in terms of maturity.
4. If cyber security is a carrots-and-sticks game, then we need to find the right combination of carrots and sticks, and to tune their implementation.
5. Finally, if cyber security is a co-evolution game, then we need to focus on agility, rapid learning, and systemic innovation. Also, we should probably NOT do some of the strategies listed in 1) through 4), especially if they create rigidity and fragility in the co-evolutionary process, which is the opposite of what is needed.

Institutional Innovation in Contested Territory: Quantified Cyber Security and Risk

Say you are an entrepreneurial sort of person who wants to really change the world of cyber security. Problem: nobody seems to know where the game-changing innovation is going to come from.  Is it technology?  Is it economics?  Is it law and policy? Is it sociology? Maybe combination, but what? And in what sequence?

If you aim for institutional innovation, then at some point you are going to need to take sides in the great "Quant vs. Non-quant" debate:

• Can cyber security and risk be quantified? 
• If "yes", how can quantitative information be used to realize security to significantly improve outcomes?

Whether you choose Quant or Non-quant, you will need some tools and methods to advance the state of the art.  But how do you know if you are choosing the right tools, and using them well?  (Think about the difference between Numerology and Calculus as they might be applied to physics of motion.)

Whoever makes sufficient progress toward workable solutions will "win", in the sense of getting wide-spread adoption, even if the other is "better" in some objective sense (i.e. "in the long run").

I examine this innovation race in a book chapter (draft). The book will probably come out in 2016.

Abstract:

"The focus of this chapter is on how the thoughts and actions of actors coevolve when they are actively engaged in institutional innovation. Specifically: How do innovators take meaningful action when they are relatively ‘blind’ regarding most feasible or desirable paths of innovation? Our thesis is that innovators use knowledge artifacts – e.g. dictionaries, taxonomies, conceptual frameworks, formal procedures, digital information systems, tools, instruments, etc. – as cognitive and social scaffolding to support iterative refinement and development of partially developed ideas. We will use the case of institutional innovation in cyber security as a way to explore these questions in some detail, including a computational model of innovation."

Your feedback, comments, and questions would be most welcome.

The computational model used is called "Percolation Models of Innovation".  Here is the NetLogo code of the model used in the book chapter.   Below are some figures from the book chapter.

Innovation as percolation. Progress moves from bottom to top. Each column is a "technology",
and neighboring columns are closely related.  This version (S&V 2005) only models
rate of progress and distribution of "sizes", not anything about the technology or
trajectory of innovation.

A screen shot of the user interface.  Three different models can be selected (upper left).

Dissertation proposal: "Shaping Possibility Space"

At long last, I have submitted my dissertation proposal:

• "Shaping Possibility Space: Coevolution of Thinking and Doing In Organizational and Institutional Innovation"

Fair warning: it is long -- 72 pages not including Glossary and Bibliography.  Being academic work, it will not be 'light reading' for many readers.  I have done my best to be clear and direct, but the subject matter is complicated.

Spoiler alert: I don't propose to solve the Cyber Security Problem(tm) in my research.  Instead, I'm studying the process of innovation that might, eventually, lead to new solutions, especially institutional innovations.  Some readers might find this boring, irrelevant, or 'ivory tower'.

Feedback is most welcome.  If you don't have my email address, ping me on Twitter.

The defense meeting is in mid-October.

B-Sides LV slides

Here are my slides for today's B-Sides Las Vegas talk (5pm Wednesday).  I'll be demoing the B-Sides SF spreadsheet (see previous post).  A video of the talk will be available on Archive.org in a day or so.

Entry in Schneier's Eighth Movie-Plot Threat Contest

Every year, on April Fool's Day, Bruce Schneier hosts a "movie plot threat" contest on his blog.  This year's theme is "evils of encryption".  This is my third year submitting an entry (I won two years ago -- w00t!).  Here is my entry for the 8th contest (500 word limit):

My inputs to DHS on cyber economics & incentives

I'm at the 3rd day of Workshop on Economics of Information Security (WEIS) at Penn State.  The focus of this day is to provide input and ideas to the Science & Technology (S&T) Directorate in US Department of Homeland Security regarding R&D on cyber economics and incentives.

Here is the 2007 working paper I co-authored: "Incentive-based Cyber Trust -- A Call to Action".  I think many of the arguments and ideas are still relevant.  (It's long -- 27 pages -- but I think readers will be rewarded.)

Here are my slides.