Conflict simulation, peacebuilding, and development

Daily Archives: 31/10/2012

Policy simulations and high-level participants

Mr. Punch’s Victorian Era: An illustrated chronicle of the reign of Her Majesty the Queen, Volume 2 (1888), via Google eBooks.

Two months ago a discussion began via PAXsims on simulation design when Natasha Gill (TRACK4) posted a guest contribution on “Happy endings, doomsday prophesies, and the perils of think tank simulations,” responding in part to earlier comments I had made on a Syria simulation at the Brookings Institution. Devin Ellis (ICONS Project) then offered some reflections on Natash’a’s points and the “Dance of the Simulation Designer” (to which Natasha, in return responded).

Today we offer round three of this valuable discussion, as Devin addresses some issues and questions raised by Natasha and myself, and offers more thoughts on policy simulations and high-level participants.

* * *

Policy Simulations and High-Level Participants – Continued

Devin Ellis, ICONS Project

Natasha Gill wrote a very thoughtful response to my spur-of-the-moment comments on her initial post, and I promised Rex that I would continue the dialogue on high-level participants (I think I’m going to give up and go for an acronym – HLP from here on in). I have been distracted by my day job for the past month, but I am hoping to answer Gill’s question(s) and further clarify some of the points in our discussion. I am doing a lot of think tank work right now, watching much more of it as an invited guest, and I have kept this discussion firmly at the forefront of my mind. Assessing what I thought was valuable and not valuable about what I’ve witnessed the past few weeks sharpened my thinking on this subject.

Gill asked:

My question to Ellis is this: he writes that the purpose of the exercise is to “explore possible policy reactions to a crisis” but in the same sentence admits that “no one…believes the purpose of a well designed and run wargame is to predict the ‘real future’ in a complex policy environment.

I think this means that although facilitators and participants are well aware that the details of the future can’t be known, the responses of various players to a crisis might be generally predictable in a simulation, in such a way as to be informative or useful for policy makers.

But upon reflection, if this is what Ellis meant, I’m not sure it makes sense to me. If a simulation can’t predict the future, then how much is it really telling you about possible ‘policy reactions’? And how much of what it does tell you about these actually useful (rather than merely interesting) to real policy makers?”

Gill is almost right about what I meant. I did not originally frame it this way, but I stand by the idea that a think tank simulation can tell you a great deal about possible policy reactions without having to be predictive of the one specific future – indeed the well run policy planning simulation should do exactly that.

She also, however, throws in the existential dilemma for all of us in this business: how much of what you get out of this is actually useful to policy makers vs. merely interesting? My answer to this two part question is that I believe there are a lot of circumstances under which think tanks should not do policy simulations. But if they do them it should be with HLPs, and they should not attempt to get them out of their own background and experiences. I argue that this very formula can be what gets your desired outcome: a set of possible courses of action which are useful for planning.

My answer raises two questions: 1) What does a ‘good’ think tank simulation need to deliver? 2) Why do HLPs help get you there? Here we go…

It’s a Bird, It’s a Plane, It’s the Saban Center!

The think tank simulation is a different beast from an in-house policy planning exercise for an agency or office. It is also very different from a simulation used for training. Usually the objective is not to give the participants in the exercise new insight into the issue – if they get it, bully for them, but that’s secondary. The objective is to influence current and possibly future policy makers who are almost never, themselves, participants.

In Stephen Downes-Martin’s talk this year at Connections, he emphasized one of his key questions to the client: “When do you rotate out of here?” He asks that (and we all should!) because with an in-house program you want to make sure that client is served by the time frame of your exercise and the recommendations that emerge from it. For a think tank, your recommendations often go public, and even when they don’t your client is usually an SME, not an SES/O-8-O-9er. This has implications.

As a think tank, you need to be able to defend your conclusions to:

  1. The current decider
  2. The next decider
  3. Their numerous opponents and critics.

What does this mean? The content of the simulation needs to be either non-partisan or bi-partisan in significance; it’s best not to assume any specific political agenda or personality for the actors in major democratic powers (the U.S.); the more your exercise focuses on core thematic and structural issues, the longer its shelf life and the wider its acceptance. In short, you benefit from less – not more – situational and personality specificity.

My first question to the think tank interested in running a crisis simulation is: “what do you want this exercise to tell you about policy planning options?” If the answer is “I want one exercise to reveal the hidden truth of how we are going to untangle the mess in Syria” then I will try and get them to re-think their agenda. If the question is: “I want to better understand what courses of development might face us in the event that US decision makers are confronted by scenario X,” then a good simulation might just be for you!

Admiral, if you could put down the putter for just one moment please…

On doing what we do, I could not say it better than Peter Perla:

Wargames, on the other hand, focus precisely on human behavior, particularly on human decision making. The learning that comes from wargames comes both from the experience of making decisions (playing) and from the process of understanding why those decisions are made (game analysis)… Wargames do not do very well at producing quantitative measures because they are often little more than a single realization of a complex stochastic process. Instead, the value of a wargame lies in qualitative assessments of why decisions are made.

The think tank’s sim has to work for multiple real world audiences, so the specificity of things like political leanings and psychology that would make role sheets for a simulation examining a very specific set of parameters so vital are instead counterproductive. The think tank’s sim is looking for a very particular type of outcome – it’s an exercise in identifying problems and options which can be generalized.

Rather than trying to break the HLPs out of their shells, I am in fact looking for those instinctive, experiential elements they bring. Their professional background – if not their political party or personal feelings – is usually representative of their near peers. When you analyze the thought processes they went through, how they decided what to do, and what they did, you can draw inferences about how a real world administration would likely be advised of its options. Doing so can show you both areas where they are presenting good ideas, and also areas where you can point to problems with myopic thinking, lack of creativity, and constraints on policy choices which should be addressed BEFORE the crisis happens. If a good think tank sim can do these things, it is well worth the time spent on it.

Go Ahead, Kill the Messenger

It will come as no surprise, given the proceeding, that I don’t do role sheets for HLPs in a think tank simulation. As I said, what I’m looking for are their real life assumptions, reactions, and ingrained knowledge. It would be total hubris for me to assume I know more about the gritty details of each of their specialized realms than they do, and the last thing I want is to impose a filter. Also, from a purely practical standpoint, most HLPs are not going to read all their materials if you give them more than half a dozen pages. I love the ones who will, but I plan for those who won’t.

What does this mean? Your scenario has to be superb – it has to withstand the critical glare of your participants, and it has to drive the key elements you want them to focus on in the simulation. Part of my problem with many think tank simulations and wargames is that scenarios are often not rigorously enough vetted for both realism AND their ability to elicit the responses you are most interested in examining in your exercise. Additionally, I am in 100% agreement with Gill on the deleteriousness of throwing endless, seemingly unconnected injects at your participants to stir things up (“let’s see how they’ll react to this! And now this! Etc”). Stick to your story.


Rex raised a good point: for the think-tankers to sell their conclusions, it helps to have those big names attached. Many good ideas only make the leap from academia to policy after a big name policy shop has written them up with the implicit endorsement of beltway heavy hitters. Of course it cuts both ways, and plenty of trash has been given marketing sparkle by former cabinet members. Garbage in, garbage out – at all levels. I think it’s up to the savvy consumer to look at the simulation product much the same as he or she would any other think tank product.

She Don’t Like Firefly…

Yes, I can see this would be a problem.

Review: Peterson, Playing at the World

Book review: Jon Peterson, Playing at the World: A History of Simulating Wars, People, and Fantastic Adventures from Chess to Role Playing Games (San Diego: Unreason Press, 2012). 632pp + bibliography and index. $34.95.

Despite its lengthy (and slightly misleading) subtitle, Playing at the World is really about one thing: the evolution, development, and impact of the roleplaying game Dungeons and Dragons from its earliest precursors to the late 1970s. Yet this narrow focus is, at the same time, both far-reaching and exhaustive in its treatment of the subject. Peterson masterfully tells the story of D&D in the broader context of wargaming since the 18th century, in reference to the gaming subculture and networks that shaped it, and with due attention to both the economics of the nascent RPG industry and the key personalities of the day. He does this, moreover, in exquisite detail and on a foundation of truly stunning primary research, with copious footnotes and an extensive bibliography that runs the gamut from original game design notes through to rare hobby newsletters and fanzines to historical texts and academic analyses. The result is a thoughtful, erudite, yet highly readable book that should be viewed as a seminal contribution to the history of an important gaming genre.

The book itself is divided into five main chapters, plus an epilogue. The first chapter  explores the immediate context in which D&D was born. This includes the rise of wargaming clubs, wargaming of the medieval and the fantasy settings (including the publication of D&D’s precursor rules, Chainmail, cowritten by Gary Gygax), Dave Arneson‘s Blackmoor adventure campaign of the early 1970s, and the collaboration between Gygax and Arneson that produced the three-booklet original edition of Dungeons & Dragons (1974).

In the next two chapters, Peterson shifts focus to explore the inspirations for D&D. Chapter 2 thus examines the medieval fantasy genre, and the literary influences that would shape the emergence of RPG classes, equipment, and scenarios. In Chapter 3, the book traces the historical evolution of wargaming from the late 18th century onwards, and in doing so demonstrates the emergence of game design elements and concepts that would be incorporated into D&D and similar games.

In Chapter 4, the focus shifts once again, this time to the notion of role-playing itself, and how campaign games embed players in imagined world. In large part this is told through the lens of significant wargame club campaigns, some of whose participants were (or would be) influential figures within the broader hobby. Chapter 5 then returns to continue the story begun in Chapter 1, recounting the popularization and expansions of D&D into the mid- and late-1970s.

Finally, the Epilogue to the book shows how the fantasy RPG genre would make its first transition to computer games. This is in many ways the least satisfying part of the volume, taking the reader no further than the late 1970s and Zork. Then again, by this point the page count is already up to over 600 pages of meticulous analysis. Anything more and the book might be reclassified as a two-handed weapon (doing the same d6 damage as every other weapon in original D&D—but increasing to d10 by the time of the 1975 Greyhawk supplement).

For many readers the analytical value of the book will happily complemented by the sheer gaming-geek-pleasure of being transported back to the era of early D&D, reading about the first origins of the thief character class, or seeing once more the advertisements for the 1970s Miniature Figurines “Mythical Earth” wargaming figures (a set of which is still packed away in my gaming closet—minus the hobbits, which have more recently been converted to hungry undead children for gaming the near-future zombie apocalypse). Make no mistake about it, however: Playing the World is a serious piece of cultural and intellectual history. While the author apparently chose to self-publish the volume so as to retain full editorial control (and, doubtless, to avoid an editor’s insistence that he shorten his manuscript), I worry that this might make it less likely to appear in public and university libraries where it rightly deserves to be. Readers would be well advised not only to put it on their own Christmas lists this season, but to suggest it to their local collections librarian as well.

Jon Peterson’s “Playing at the World” blog can be found at

%d bloggers like this: