Work in Progress (Cyber Security Investment Game)

Here is a video demo of the NetLogo "Cyber Security Investment Game", as a work in progress.  This run is a 6/3 multiplayer game, meaning 6 cooperative Defenders and 3 adversarial Attackers. Defenders play a 2-player game with everyone else, while Attackers only play 2-player games with Defenders. It's a complicated game, with up to 400 possible "moves" for each agent at each step, and also a dynamic game, meaning the structure of the game can change dynamically during the course of play. (Game Theory Purists will probably hate it for that reason!)

Here's a video of 1,700 time steps, with 20x speed-up.



What you see in this simulation is an ecosystem that provides a positive environment for both Defenders and Attackers. That is, they both have consistently positive payoffs (see bar graph, center left).

While the overall trends of payoffs are pretty clear (bottom graph), there is an "unruly" back and forth between attackers and defenders. Crucially, the time series of payoffs is non-stationary for all agents (see center graphs, third and fourth from the top). Simply, non-stationary means that the probability distribution for each payoff  changes over time. You can see a sample distribution of payoffs in the two histograms, center right. You'll notice them changing shape (going from skew to symmetric) and also changes in mean and standard deviation ("SD").  Non-stationarity has implications for risk estimation, as I will detail in the up-coming WEIS paper.

In terms of progress toward the goal, I would say this is not yet a model of cyber security investment. It models competitive relationships rather than not host-parasite relationships which I believe are close to the true nature of cyber security ecosystems. The good news is that I believe I know what extensions and modifications need to be made. I'll save that for a later post.

Here are some key features of this version of the model:

• Three levels of investment: 1) architecture/infrastructure; 2) capabilities; 3) practices/routines (a.k.a. "moves")
• Asymmetric number and variety of "moves" for Defenders and Attackers
• Parametric control over diversity of "moves" and investments by Defenders and Attackers

Yes, the model is getting complicated.  But I hope this richness will be rewarded when crucial results are revealed in experiments.  We shall see...

<update>

Here is the same run after 4,000 time steps.  No sign of equilibrium or stationary time series.

(Click to enlarge)

Time & Uncertainty (2nd post: "What kind of game is cyber security investment?")

Summary: Time and uncertainty are essential features of any model of the "game of cyber security".  Models that do not include them as central features are not fit for purpose.  But, yes, they do make life more difficult for modelers and their audiences. While I make the case that both are essential, I leave open the question as to what is the most parsimonious method or treatment.

What kind of game is cyber security investment? (post #1 of ?)

This is first in a series of blog posts where I think out loud as I build a paper for WEIS 2016, and also a component for my dissertation.

The focus is on "investment" broadly defined.  This means money invested in people, tools, infrastructure, processes, methods, know-how, etc.  It also means architectural commitments that shape the business, technical, legal, or social aspects of cyber security for a given person or organization.  All these investments provide the foundation for what a person or organization is able to do (i.e. their "capabilities") and the means of executing day-to-day tasks ("routines", "processes", "practices", etc.).

If cyber security investment is a strategic game between attackers and defenders, and among defenders, then what kind of game is it?

Summary

In simple terms, people tend to think of cyber security investment as being one of (at least) five types of games:

1. An optimization game, where each player finds the optimal level of spending (or investment) to minimize costs (or losses).  This view is favored by Neo-classical Economists and most Game Theorists.
2. A collective wisdom game, where the collective searching/testing activities of players leads to the emergence of a "collective wisdom" (a.k.a. "best practices") that everyone can then imitate. This view is favored by many industry consultants and policy makers.
3. A maturity game, where all players follow a developmental path from immature to mature, and both individual and collective results are improved along the way.  This view is favored by many industry consultants.
4. A carrots-and-sticks game, where players chose actions that balance rewards ("carrots") with punishments ("sticks") in the context of their other goals, resources, inclinations, habits, etc.  This view is favored by some Institutional Economists, and some researchers in Law and Public Policy.  It is also favored by many people involved in regulation/compliance/assurance. 
5. A co-evolution game, where the "landscape" of player payoffs and possible "moves" is constantly shifting and overall behavior subject to surprises and genuine novelty.  This view is favored by some researchers who employ methods or models from Complexity Science or Computational Social Science.  This view is also a favorite of hipsters and "thought leaders", though they use it as metaphor rather than as a real foundation for research or innovation.

But what kind of game is cyber security, really?  How can we know?

These questions matter because, depending on the game type, the innovation strategies will be very different:

1. If cyber security is an optimization game, then we need to focus on methods that will help each player do the optimization, and to remove disincentives for making optimal investments.
2. If cyber security is a collective wisdom game, then we need to focus on identifying the "best practices" and to promote their wide-spread adoption.
3. If cyber security is a maturity game, then we need to focus on the barriers to increasing maturity, and to methods that help each player map their path from "here" to "there" in terms of maturity.
4. If cyber security is a carrots-and-sticks game, then we need to find the right combination of carrots and sticks, and to tune their implementation.
5. Finally, if cyber security is a co-evolution game, then we need to focus on agility, rapid learning, and systemic innovation. Also, we should probably NOT do some of the strategies listed in 1) through 4), especially if they create rigidity and fragility in the co-evolutionary process, which is the opposite of what is needed.

Complex dynamics in learning complicated games (replicating Galla & Farmer 2012)

I have written a NetLogo version of the random game model of Galla & Farmer (2012) (free download).  It has been uploaded to the NetLogo community library and should appear in a day or so.  Read on if you are interested in Game Theory, esp. learning models and computational methods.

Chaotic dynamics in a complicated game. The payoffs are negatively correlated (-0.7) and memory for learning is long (alpha ≈ 0). Notice the squiggly lines in the time series plots (lower right). Each line is probability for a given move.
If the game were in equilibrium, these lines would be flat. (click for larger image)

Download the .nlogo file 
Download NetLogo (Win, Mac, Linux)

What analysis do we really need to guide vulnerability management?

This is the first of a series of posts on the topic of doing quantitative risk analysis in the face of intelligent and adaptive adversaries.  Later posts will dig into research topics like combining risk analysis with game theory, but this first post is mostly a reaction to what other people have said recently.

Rafał Łoś recently posted an article, and then followed with a guest post from Heath Nieddu, with this general theme (paraphrasing and condensing):

It's senseless and distracting to attempt to use quantitative risk analysis to make decisions about vulnerability remediation, and even for information security as a whole.  Uncertainties about the future are too great; adversaries too agile and intelligent; and the whole quant risk endeavor is too complicated.  Keep it simple and stick with what you know for sure, especially the basics.

In this post I'm going to address some of the issues and questions this skeptical view raises, but I won't attempt a point-by-point counter argument.  For the record, there are many points I disagree with, plus many ideas that I think are confused or just mis-stated.  But I think the discussion will be best served by keeping focused on the main issues.

I'm also appearing on Rafał's podcast, Down the Rabbit Hole, along with some other SIRA members.  I'll let you know when it is posted for listening.