Does a model and its data ever speak for themselves? No -- A reply to Turchin

This post is the first of a series to reply to Dr. Peter Turchin regarding his PNAS article (full text PDF -- free, thanks to Turchin & team), my letter to PNAS, and his PNAS letter reply.  I wrote a blog post here because I didn't think that Dr. Turchin's reply addressed my questions due to misunderstanding and I invited Dr. Turchin to engage in a colloquy via blog posts. I'm happy to say that Dr. Turchin wrote three blog posts (here, here, and here) in reply to my post, and this is my first reply.

While this post talks about interpreting simulation results, the general topic of data interpretation applies to all empirical research, and even data analysis in industry.  

PNAS letter & reply: You say potāto, I say potəto…

If we have mis-communicated, should we call the whole thing off?
Not just yet.  I say: once more, with FEELING!

Big news: my letter to Proceedings of the National Academies of Science (PNAS) has been published, along with the author's reply.  (pay wall)

This post includes an early draft of my letter plus some commentary.

My published letter: "Does diffusion of horse-related military technologies explain spatiotemporal patterns of social complexity 1500 BCE–AD 1500?"

The authors' reply is here.

The authors are Peter Turchin, Thomas Currie, Edward A. L. Turner, and Sergey Gavrilets.  In case you don't know him, Dr. Peter Turchin is one of the founders of this field called Cliodynamics, or the mathematical modeling of large scale, long time horizon historical dynamics.

The not-so-good-news is that the authors misunderstood my objections so their answers didn't address them.  Thus, we didn't really communicate successfully in the format of PNAS letters. Both my letter and the author's response were restricted to 500 words, and this significantly contributed to the miscommunication.

Message to PNAS editors: Your 500-word restriction on letters is anachronistic, unnecessary, and is an obstacle to productive scholarly debate. Since letters are only published online, there is no justification for the 500-word limit, which is presumably justified to save precious paper in the print version of PNAS journal. With online publication, letters should be edited to express their essential meaning without any fixed word count limit.

For copyright reasons, I can't copy verbatim either my letter or the author's response. Instead, I'll splice them together, along with my commentary on the miscommunication and more details about my objections.   My sincere desire is that the authors will respond here in the comments or elsewhere to address these (clarified) objections.

My objections focus on the authors' design and simulation choices, and not their underlying theories of social complexity.

Did chariots & cavalry drive social complexity for 3,000 yrs.? (letter to PNAS)

Battle scene decoration on the chariot of Thutmose III
(click to enlarge)

(It's holiday time, so it's probably a good time for non-risk, non-InfoSec posts.  Time for war chariots!)

tl;dr:  I wrote a letter to PNAS disputing the claims made by a famous author in a recent article.  My beef was with their modeling methods. We'll see if it gets published and if the author(s) respond.

Several pieces of good news

Sorry I haven't posted in a while.  I've been pretty busy with research work -- writing papers for conferences, mostly.  But I've got some good news to report.

Cash will be flowing as nature intended.

First, I'm starting a full-time job at a Financial Institution* with the title Security Data Analyst/Scientist, which I choose to shorten to Security Data Scientist.  This is a big deal on many levels.  One of  the best things is that their capabilities are comparatively mature and the leadership is both visionary and pragmatic.  This means that I hope to do some fairly compelling analysis drawing on some rich data sources and previous analysis rather than having to start from scratch.

(* My Twitter followers will know.)

I'm continuing my PhD program part-time, with focusing on my dissertation.  I hope to complete that in 2014.

Also, I'll continue blogging here on all the same topics.

Second, I'm very happy to say that I've had a talk accepted at the RSA Conference in February 2014, co-presenting with David Severski:

10 Dimensions of Security Performance for Agility & Rapid Learning
2/26/2014, 10:40 AM - 11:00 AM

Abstract: Information security is an innovation arms race. We need agility and rapid learning to stay ahead of adversaries. In this presentation, you'll learn about a Balanced Scorecard method called the Ten Dimensions of Cyber Security Performance. Case studies will show how this approach can dramatically improve organization learning and agility, and also to get buy-in from managers and executives. 

This is a 20 minute time slot, and there's no way that I can compress my 60 minute or 45 minute versions of "Ten Dimensions" into such a short time.  Therefore, David and I are going to cook up an extended "trailer" that conveys the basic idea of double loop learning in practice (David is doing some neat stuff that we'll try to "fly through").  In parallel, I hope to have some videos, webinar, or other media that people can go to in order to get a proper introduction and survey.

Also, I've proposed a peer-to-peer session at RSA on a related theme: "Building a Quantitative Evidence-based Security & Risk Management Program".  I should hear later in November whether it's been accepted.  It will be an hour long session and I will only be facilitating, but it should be a good time for Q&A, sharing insights, etc.

Finally, I'll be presenting a SIRA webinar "Big 'R' Risk Management - from concept to pilot implementation".  This is basically the same talk I gave at SIRAcon, but some people couldn't attend that session (we had parallel tracks) and many people couldn't attend SIRAcon at all.  I think it'll be in December, but there isn't a date set yet.

I've got some good blog posts in the works, including Game Theory Meets Risk Analysis, several more Shades of Black Swans, a review of RIPE, some philosophy, and others.   Thanks for reading and thanks for your comments, both here and in other media.

--------

One more bit of good news from a completely different domain: the book Chasing Chariots is coming soon!  Includes most of the papers presented at the First International Chariot Conference held in Cairo in December 2012.  The evolution of technology in the Late Bronze Age became an strong interest (a.k.a. compulsion) of mine a couple years ago, with particular focus on the so-called "first revolution in military affairs" -- the war chariot.  Beyond just curiosity, I'd like to do some serious research in this area, but short of getting a second PhD, the only way it's going to happen is if I can find some collaborators (after I graduate!).

Periodically, I'll post some war chariot stuff here.  Bruce S. has his squids;  I have my war chariots.