Look Papa! I'm on the Loopcast! -- Talking complexity, simulation, black swans, randomness, resilience, and institutional innovation

If you have a spare 1 hr 40 min. *, you might want to listen my interview on the "Loopcast" podcast (below).  The host is Sina Kashefipour (@rejectionking on Twitter).

* Personally, I think I sound better at 1.5X speed, but then again I listen to most podcasts at 1.5X speed.



In this podcast, I reference the following websites and resources:

• Santa Fe Institute -- birthplace of Complexity Science.  SFI on YouTube, and "Complexity" podcast.
• Complexity Explorer -- coursework and resources for the study of complex systems,
• NetLogo -- a free and open source platform for agent-based modeling and dynamical systems.
• Ignorance and Uncertainty  -- book by Michael Smithson, out-of-print but available used
• Uncertainty and Risk: Multidisciplinary Perspectives -- editors Gabriele Bammer and Michael Smithson 

 

Does Modern Portfolio Theory (MPT) apply to cyber security risks?

Many months ago, my colleague David Severski asked on Twitter how Modern Portfolio Theory (MPT) does or does not apply to quantified cyber security risk:

I replied that I would blog on this "...soon".  Ha!  Almost four months later.  Well, better late than never.

Short answer: No, MPT doesn't apply.  Read on for explanations.

NOTE: "Cyber security risk" in this article is quantified risk -- probabilistic costs of loss events or probabilistic total costs of cyber security.  Not talking about color-coded risk, categorical risk, or ordinal scores for risk.  I don't ever talk about them, if I can help it.

Orange TRUMPeter Swans: When What You Know Ain't So

Was Donald J. Trump's political rise in 2015-2016 a "black swan" event?  "Yes" is the answer asserted by Jack Shafer this Politico article. "No" is the answer from other writers, including David Atkins in this article on the Washington Monthly Political Animal Blog.

Orange Swan

My answer is "Yes", but not in the same way that other events are Black Swans.   Orange Swans like the Trump phenomenon is fits this aphorism:

"It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so." -- attributed to Mark Twain

In other words, the signature characteristic of Orange Swans is delusion.

Rethinking "Black Swans"

As I have mentioned at the start of this series, the "Black Swan event" metaphor is a conceptual mess. (This post is sixth in the series "Think You Understand Black Swans? Think Again".)

It doesn't make sense to label any set of events as "Black Swans".  It's not the events themselves, but instead they are processes that involve generating mechanisms, our evidence about them, and our method of reasoning that make them unexpected and surprising.

Institutional Innovation in Contested Territory: Quantified Cyber Security and Risk

Say you are an entrepreneurial sort of person who wants to really change the world of cyber security. Problem: nobody seems to know where the game-changing innovation is going to come from.  Is it technology?  Is it economics?  Is it law and policy? Is it sociology? Maybe combination, but what? And in what sequence?

If you aim for institutional innovation, then at some point you are going to need to take sides in the great "Quant vs. Non-quant" debate:

• Can cyber security and risk be quantified? 
• If "yes", how can quantitative information be used to realize security to significantly improve outcomes?

Whether you choose Quant or Non-quant, you will need some tools and methods to advance the state of the art.  But how do you know if you are choosing the right tools, and using them well?  (Think about the difference between Numerology and Calculus as they might be applied to physics of motion.)

Whoever makes sufficient progress toward workable solutions will "win", in the sense of getting wide-spread adoption, even if the other is "better" in some objective sense (i.e. "in the long run").

I examine this innovation race in a book chapter (draft). The book will probably come out in 2016.

Abstract:

"The focus of this chapter is on how the thoughts and actions of actors coevolve when they are actively engaged in institutional innovation. Specifically: How do innovators take meaningful action when they are relatively ‘blind’ regarding most feasible or desirable paths of innovation? Our thesis is that innovators use knowledge artifacts – e.g. dictionaries, taxonomies, conceptual frameworks, formal procedures, digital information systems, tools, instruments, etc. – as cognitive and social scaffolding to support iterative refinement and development of partially developed ideas. We will use the case of institutional innovation in cyber security as a way to explore these questions in some detail, including a computational model of innovation."

Your feedback, comments, and questions would be most welcome.

The computational model used is called "Percolation Models of Innovation".  Here is the NetLogo code of the model used in the book chapter.   Below are some figures from the book chapter.

Innovation as percolation. Progress moves from bottom to top. Each column is a "technology",
and neighboring columns are closely related.  This version (S&V 2005) only models
rate of progress and distribution of "sizes", not anything about the technology or
trajectory of innovation.

A screen shot of the user interface.  Three different models can be selected (upper left).

B-Sides LV slides

Here are my slides for today's B-Sides Las Vegas talk (5pm Wednesday).  I'll be demoing the B-Sides SF spreadsheet (see previous post).  A video of the talk will be available on Archive.org in a day or so.

B-Sides SF Talk

Here is the demo spreadsheet I'll be using in today's B-Sides SF talk on the Thomas Scoring System (TSS):

• InfoSec Maturity Assessment (TSS) V1.xlsx

Download the spreadsheet and open in Microsoft Excel 2008 or later.  It uses conditional formatting and cell data validation, but no macros or other advanced features.  The sheets are protected to avoid data entry errors, but there is no password.

This is a realistic, fully functional implementation of the TSS applied to a general case: scoring the maturity of a company's information security capability.

Precision vs Accuracy

When ever you do any kind of measurement, it's important to understand the uncertainties associated with it.  Two characteristics of measurement that are inverse to uncertainties are 'precision' and 'accuracy' (also known as 'fidelity').  The following graphic, from this blog post, nicely demonstrate the difference between these two characteristics.

Other measurement characteristics include stability (repeatability from measurement to measurement), resolution (number of significant digits), sensitivity (ability to detect very small signals), linearity, range (from smallest valid value to largest valid value), and sampling rate (time slice or number of samples to establish a valid measurement).

S Kauffman on Emergent Possibility Spaces in Evolutionary Biology, Economics, & Tech. (great lecture)

Below is a great lecture by Stuart Kauffman on the scientific and philosophical consequences of emergent possibility spaces in evolutionary biology and evolutionary economics, including technology innovation and even cyber security. This web page has the both video and lecture notes.

The lecture is very accessible anyone who reads books or watches programs on science aimed at the general public -- especially evolution, ecology, complexity, and innovation. He does mention some mathematical topics related to Newtonian physics and also Quantum Mechanics, but you don't need to know the details of any the math to follow his argument.  He gives very simple examples for all the important points he makes.

There are very important implications on epistemology (what do we know? what can be known?), scientific methods and research programs, and the causal role of cognition, conception, and creativity in economic and technological change. This last implication is an important element in my dissertation. I'll write more on that later.

#BSidesSF Prezo: Getting a Grip on Unexpected Consequences

Here are the slides I'm presenting today at B-Sides San Francisco (4pm).  I suggest that you download it as PPTX because it is best viewed in PowerPoint so you can read the stories in the speaker notes.

Does a model and its data ever speak for themselves? No -- A reply to Turchin

This post is the first of a series to reply to Dr. Peter Turchin regarding his PNAS article (full text PDF -- free, thanks to Turchin & team), my letter to PNAS, and his PNAS letter reply.  I wrote a blog post here because I didn't think that Dr. Turchin's reply addressed my questions due to misunderstanding and I invited Dr. Turchin to engage in a colloquy via blog posts. I'm happy to say that Dr. Turchin wrote three blog posts (here, here, and here) in reply to my post, and this is my first reply.

While this post talks about interpreting simulation results, the general topic of data interpretation applies to all empirical research, and even data analysis in industry.  

Out-of-the-Blue Swans: Megatsunami, Supervolcanos, The Black Death, and Other Cataclysms

The Out-of-the-Blue Swan is out there waiting to ruin our
day, month, year, decade, or century.

This is the fifth in the series "Many Shades of Black Swans", following on the introductory post "Think You Know Black Swans? Think Again." This will be a short post because the phenomena and implications on risk management are fairly simple (at least for individual people and firms). I've seen a few people include these in their list of "Black Swans" if they want to emphasize events with massive destruction and unpredictable timing.

Disappearing Swans: Descartes' Demon -- the Ultimate in Diabolical Deception

The Disappearing Swan.  Now you see it.   Now you don't.
Descartes' Demon has fog machines,  fake signs,
and much much more to mess with your head.

This is the fourth in the series "Many Shades of Black Swans", following on the introductory post "Think You Know Black Swans? Think Again." This one is named "Disappearing" because the emphasis is on deception to the ultimate degree.

The Disappearing Swans are mostly a rhetorical fiction -- an imaginary and socially constructed entity that is treated as real for the purposes of persuasion.  They are often mentioned as reasons why we can never understand anything about any variety of Black Swan, especially those with "intelligent adversaries".  I'm including Disappearing Swans in this series mostly for completeness and to make distinctions with other, more common Swans like Red Swans.

Red Swans: Extreme Adversaries, Evolutionary Arms Races, and the Red Queen

The Red Swan of evolutionary arms races, where the
basis for competition is the innovation process itself.
As the Red Queen says: "...it takes all the running you can do,
to keep in the same place."

This is the third in the series "Many Shades of Black Swans", following on the introductory post "Think You Know Black Swans? Think Again." This one is named "Red" after the Red Queen Hypothesis in evolutionary biology, which itself draws from the Red Queen in Lewis Carroll's Through the Looking Glass (sequel to Alice in Wonderland).  But in this post I'll talk about competitive and adversarial innovation in general, including host-parasite systems that are most analogous to cyber security today.

In addition to the usual definition and explanations, I've added a postscript at the end: "Why Red Swans Are Different From Ordinary Competition and Adversarial Rivalry".

Green Swans: Virtuous Circles, Snowballs, Bandwagons, and the Rich Get Richer

The Green Swan of cumulative prosperity.
The future's so bright she's gotta wear shades.

This is the second in the series "Many Shades of Black Swans", following on the introductory post "Think You Know Black Swans? Think Again."  This one is named "Green" as an allusion to the outsized success and wealth that often arise through this process, though by no means is it only limited to material or economic gains.

Taleb includes the Internet and the Personal Computer among his prime examples of Black Swan events.  In this post I hope to convince you that these phenomena are quite different than his other examples (e.g. what I've labeled "Grey Swans") and that there is value in understanding them separately.

Grey Swans: Cascades in Large Networks and Highly Optimized/Critically Balanced Systems

A Grey Swan -- almost Black, but not quite. More narrowly defined.

This is the first of the series "Many Shades of Black Swans", following on the introductory post "Think You Know Black Swans? Think Again."

I'll define and describe each one, and maybe give some examples. Most important, each of these Shades will be defined by a mostly-unique set of 1) generating process(es); 2) evidence and beliefs; and 3) methods of reasoning and understanding.  As described in the introductory post, it's only in the interaction of these three that Black Swan phenomena arise. Each post will close with section called "How To Cope..." that, hopefully, will make it clear why this Many Shades approach is better than the all-lumped together Black Swan category.

This first one is named "Grey" because it's closest to Taleb's original concept before it got hopelessly expanded and confused.

Tutorial: How Fat-Tailed Probability Distributions Defy Common Sense and How to Handle Them

This post is related to the Grey Swans post, but is a good topic to present on it's own.

For random time series, we often ask general questions to learn something about the probability distribution we are dealing with:

1. What's average?  What's typical?
2. How much does it vary?  How wide is the "spread"?  Is it "skewed" to one side?
3. How extreme can the outcomes be?
4. How good are our estimates, given the sample size?  Do we have enough samples?

If we have a good sized sample of data, common sense tells us that "average" is somewhere in the middle of the sample values and that the "spread" and "extreme" of the sample are about the same as those of the underlying distribution.  Finally, common sense tells us that after we have good estimates, we don't need to gather any more sample data because it won't change our estimates much.

It turns out the that these common-sense answers could all be flat wrong, depending on how "fat" the tail of the distribution is.  Now that's surprising!

Think You Understand Black Swans? Think Again.

"Black Swan events" are mentioned frequently in tweets, blog posts, public speeches, news articles, and even academic articles.  It's so widespread you'd think that everyone knew what they were talking about. But I don't.

Coming soon: 23 Shades of Black Swans

I think the "Black Swan event" metaphor is a conceptual mess.

Worse, it has done more harm than good by creating confusion rather than clarity and by serving as a tool for people who unfairly denigrate probabilistic reasoning.  It's also widely misused, especially by pundits and so-called thought leaders to give the illusion that they know what they are talking about on this topic when they really don't.

But rather than just throwing rocks, in future posts I will be presenting better/clearer metaphors and explanations -- perhaps as many as 23 Shades of Black Swan.  Here are the ones I've completed so far:

1. Grey Swans: Cascades in Large Networks and Highly Optimized/Critically Balanced Systems
2. Green Swans: Virtuous Circles, Snowballs, Bandwagons, and the Rich Get Richer
3. Red Swans: Extreme Adversaries, Evolutionary Arms Races, and the Red Queen
4. Disappearing Swans: Descartes' Demon -- the Ultimate in Diabolical Deception
5. Out-of-the-Blue Swans: Megatsunami, Supervolcanos, The Black Death, and Other Cataclysms
6. Orange TRUMPeter Swans: When What You Know Ain't So
7. The Swan of No-Swan: Ambiguous Signals Tied To Cataclysmic Consequences
8. Swarm-as-Swan: Surprising Emergent Order or Aggregate Action
9. Splattered Swan: Collateral Damage, Friendly Fire, and Mis-fired Mega-systems 

In this post, I just want to make clear what is so wrong about the "Black Swan event" metaphor.

The Bayesian vs Frequentist Debate And Beyond: Empirical Bayes as an Emerging Alliance

This is one of the best articles I've ever seen on the Bayesian vs Frequentist Debate in probability and statistics, including a description of recent developments such as the Bootstrap, a computationally intensive inference process that combines Bayesian and frequentist methods.

Efron, B. (2013). A 250-year argument: Belief, behavior, and the bootstrap. Bulletin of the American Mathematical Society, 50(1): 129-146.

Many disagreements about risk analysis are rooted in differences in philosophy about the nature of probability and associated statistical analysis.  Mostly, the differences center on how to handle sparse prior information, and especially the absence of prior information. "The Bayesian/frequentist controversy centers on the use of Bayes rule in the absence of genuine prior experience."

What's great about this article is that it presents the issue and alternative approaches in a simple, direct way, including very illuminating historical context.  It also presents a very lucid description of the advantages and limitations of the two philosophies and methods.

Finally, it discusses recent developments in the arena of 'empirical Bayes' that combines the best of both methods to address inference problems in the context of Big Data.  In other words, because of Big Data and the associated problems people are trying to solve now, pragmatics matter more than philosophical correctness.  Another example of empirical Bayes is Bayesian Structural Equation Modeling that I referenced in this post.

The Rainforest of Ignorance and Uncertainty

One of the most important books I've ever read is Michael Smithson's Ignorance and Uncertainty.  It gives a tour of many varieties of ignorance and uncertainty and the many strategies that have been developed in different disciplines and professional fields.  Through this tour, it becomes very clear that uncertainty is not a single phenomena, and not even a couple, but instead is like a rainforest ecosystem of species. (My metaphor, not his.)

One vivid illustration of this is the taxonomy of ignorance and uncertainty.  Here's the original taxonomy by Smithson in Ignorance and Uncertainty:

In 2000, I modified this for a presentation I gave at a workshop at Wharton Business School on Complexity Science in Business.  Here's my taxonomy (2000 version):

Smithson and his colleagues have updated their taxonomy, which is presented as Figure 24.1 in Chapter 24 "The Nature of Uncertainty" in: Smithson, M., & Bammer, G. (2012). Uncertainty and Risk: Multidisciplinary Perspectives. Routledge.   (I can't find an on-line version of the diagram, sorry.) If you are looking for one book on the topic, I'd suggest this one.  It's well edited and presents the concepts and practical implications very clearly.

I don't think there is one definitive taxonomy, or that having a single taxonomy is essential for researchers.  I find them useful in terms of scoping my research, relating it to other research (esp. far from my field), and in selecting modeling and analysis methods that are appropriate.

Of course, there are other taxonomies and categorization schemes, including Knight's distinction between risk (i.e. uncertainty that can be quantified in probabilities) and (true) uncertainty (everything else).  Other categorization you'll see is epistemic uncertainty (i.e. uncertainty in our knowledge) and aleatory uncertainty (i.e. uncertainty that is intrinsic to reality, regardless of our knowledge of it).  The latter is also known as ontological uncertainty.  But these simple category schemes don't really capture the richness and variety.

The main point of all this is that ignorance and uncertainty come in many and varied species.  To fully embrace them (i.e. model them, analyze them, make inferences about them), you can't subsume them into a couple of categories.

[Edit:  Smithson's blog is here.  Though it hasn't been updated in two years, there's still some good stuff there, such as "Writing about 'Agnotology, Ignorance and Uncertainty'".]

Visualization Friday: Probability Gradients

I'm fascinated with varieties of uncertainty -- ways of representing it, reasoning about it, and visualizing it.  I was very tickled when I came across this blog post by Alex Krusz on the Velir blog.  He presents a neat improvement over "box and whiskers" plot for representing uncertainty or variation in data points which he calls "probability gradients".