PAXsims

Conflict simulation, peacebuilding, and development

Reflections on the art and science of game design

 

As noted a few weeks ago, Ed McGrady’s email comments on “10 lessons on game design for amateurs” sparked a long email conversation among participants at the recent NDU Roundtable on Strategic Gaming (including two additional lessons from Ed, bringing the total to 12). With the permission of all involved, I’ve posted a slightly-edited compendium of that discussion below.

* * *

Peter Perla (Center for Naval Analyses):

I must congratulate Ed on his rapid and articulate description of these ideas, which he had ranted on before but never in such coherent terms. I can scarcely add anything useful. I will highlight one small phrase, however: “Getting players to come to the game is basic.” The reason that I do so is that I just have gone through a prime example of what happens when one forgets this. My corollary is a take-off on the old saw about strategy and logistics: ” Amateurs talk about design, professionals talk about players.” We just had to postpone a game I got called upon to design and run because the amateurs (a DoD contractor) spent a ton of time and money developing a “rigorous” process for deriving scenarios about hypothetical futures and proposing clever “design” ideas that they put the task of securing the participants on the back burner.

In addition to rule 8, “Games are about the players,” my own take on that has always been that games take place in the minds of the players; all else exists to insert the game there and turn it on. Games are the players. They are performance art, but it’s not the designer’s performance. Sometimes the designers and facilitators and controllers forget that and get caught up in the moment and their own coordinating role, mistaking it for centrality. At that point they forget rule 8 and Ed’s stricture, ” you should never, ever, drive the game yourself directly.”

In case you’re not aware of it, Stephen Downes-Martin (with whom Ed maintains  a running banter over the art and science of gaming, one which I tend to observe from an amused sideline position) has been lobbying for a handbook for Wargaming similar to what you are describing. I have, therefore, cced him on this as well, along with Rex. (Whose presentation yesterday was outstanding, by the way—thanks Rex!)

Finally, I will be happy to help your efforts in any way I can. I haven’t yet had the time to look carefully at what you handed out yesterday, but I will and I will send you any additional comments that seem useful. AS Ed, Mike, and I discussed in the car as drove back, I think for a newly interested and totally clueless prospective game designer, the key idea to begin with is that games are stories. Everyone knows what a story is. Everyone has told them and everyone has read them. Fewer have written them. Even fewer have had success at getting others to read them, much less pay for them. It’s the same with games. Finding a balance between encouraging the newbie and warning them not to try this at home is probably the biggest challenge your effort will face. I’m not sure I know how to do that (indeed, I guess I’m sure that I don’t), but I would like to help you try.

Skip Cole (United States Institute of Peace):

These lessons are great!

I learned a new phrase recently which is constantly on my mind. It is “people learn from people they love.” It comes from a really excellent TED talk from David Brooks. I combine this in my mind with something I learned recently in a course on short story fantasy and fiction. The lesson was that people in a Brothers Grimm fairy tale never tell you that they love you, they just offer you food. (Of course, they never tell you that they hate you either. They just offer a poisoned apple!) In lesson 8, Ed emphasizes the importance of food. I don’t think this is trivial. It really is all about the player’s experience and our relationship with the player.

Our players should know (on some level) that we love them, but that we want to help them become better people. Because I am a computer programmer, I have put all of this into the logical sequence below:

  1. People learn from people they love. (David Brooks)
  2. People who love you offer you food. (Lessons from The Brothers Grimm Fairy Tales)
  3. We should offer our students food. (Stated by Ed, and follows from the above.)

I am cc’ing Julia Loughran on this because I think that this validates a theory she has long held on ‘coordination through eating and drinking together’.

One of the great advantages of working at a ‘peace’ institute is that we can use words like ‘love’ almost naturally. Love is important.

I thought that describing a strategic game as a piece of performance art was something I had come to all on my own. Shoot!

I have long loved and quoted the phrase “Amateurs talk strategy. Professionals talk logistics.”  I know that I will now quote Peter’s corollary  “Amateurs talk about design, professionals talk about players” often.

Stephen Downes-Martin (US Naval War College):

I believe the claim that “people learn from people they love” is a restricted version of a more general truth, that being “people learn from repetition or trauma”.  Love being one kind of trauma and not necessarily the best one to use when helping people learn horrible things (like the best ways to inflict lethal violence on large numbers of people trying to do the same thing to your subordinates — that’s why professionals call it a war game, not a peace game and although I don’t doubt that peace games exist I suspect their design might be different).

In the absence of repeated gaming with the same players (which the Naval War College actually did during the inter-war years) careful use of trauma is required, enough to focus the mind and make the event fresh despite the passage of time, but not so much as to make the players unwilling to play.  Willingness to play depends on trust (that the game is going to be worthwhile compared to the cost of participation), and on the stakes (the future cost in lives if one does not engage in the game).  Professional discipline by the players helps as well.

Ed McGrady (game designer, military and business consultant):

This is a strange debate.  How about “people learn from people and things they are interested in?”  Being in love certainly captures interest, so does the near presence of a raised pickaxe.  Likewise the gravity of the situation and the friendliness of the companions.  I’m not sure, however, that this helps novice game designers figure out how to design games.

I do, however, believe that one of the very unique things games bring to the issue of professional learning is emotion.  How will it feel to give the order?  What will be going through your mind when you are trying to save a million lives from a tsunami?  Do you want to feel the way you might feel if your failure to fund costs an extra 500,000 lives?

To me games are about feeling even more than intellect.  So, in that sense, love, fear, hatred, and other emotions do matter for games.  And they are very hard things to account for within the DoD (reference terrorism, Afghanistan, etc.).  Of course there is another way we deal with emotion and relationships, its called art…

Tim Wilkie (CASL, National Defense University):

Allow me to add my belated thanks for these thoughts—I am only now getting caught up on email.

I have little to add, except to cite my favorite Bernard Brodie quote, from his discussion of the U.S. decision to go to war in Korea soon after having placed Korea outside of our “defense perimeter” in the Pacific:

“President Truman and his Secretary of State had endorsed the well-known and obviously rational opposition of the military to becoming involved on mainland Asia.  When the crisis came all, including the military, immediately reversed their position.  Apparently few had asked themselves: How will we feel, and how will we in consequence respond, if there is a flagrant attack upon this state and this government that we have ourselves set up and but recently withdrawn from on the now disproved assumption that they were no longer in danger?  Will we really stand by and let them go down?

“This kind of issue arrises again and again.  What is predictable is that high government leaders will not usually ask themselves searching questions about what they will really do under a variety of circumstances, some of which will deeply engage their emotions.  They are content to accept for themselves the facile assurances they give to others.”

Stephen Downes-Martin (US Naval War College):

Tim’s point about not knowing how one would feel under real life circumstances speaks to two things.

  1. First, no matter how well put together, no matter how compelling the war game story, it is NOT realism and even if the players after the game claim they found it “realistic and forgot it was a game”, when real life comes around they CAN then tell the difference.
  2. Second, as the research shows, even reflective people have an extremely hard time predicting what decisions they would make under different information (i.e. it’s real as opposed to a compelling story).

See:

I discussed the implications of this on wargaming at the Dec 2009 NDU Round Table … see the speaker’s notes for that session (provided to NDU in 2009), specifically section 4.

Peter Perla (Center for Naval Analyses):

Stephen’s points are well founded and, as usual, on target scientifically. Ed’s position, of course, (though he can and will certainly speak for himself) is that science is not the issue. As usual, my position is, “It depends.”

If we are using games to “predict” what people will do under certain circumstances that we present to them in the game, the best we can hope for is a non-scientific (i.e., non-repeatable, difficult to disconfirm) set of insights. What I have in the past described as a pseudo-historical analysis rather than a scientific one. (And as my buddy Taleb points out, history is not all that it seems, being subject at the very least to the silent evidence issue.) What I have begun to think is that Ed’s focus on art provides a rounding out of this notion. Just as Shaara’s Killer Angels is not a scientific explanation of the whys and wherefores of Gettysburg, it is nevertheless an insightful artistic exploration of the factors that are at play in the human heart and psyche in the situation it depicts and (the key, I think) other similar situations. Would all “Lee players” in a Gettysburg game act as the original did in the original fight? Probably not. Would all actual decision makers in a real tsunami crisis act the way one particular one did during a tsunami game (even the same one)? Almost certainly not. But why do we care about that? We only care if we think that the game predicts that future action and we, therefore, use it slavishly, like the result of a physics-based simulation, to make now decisions on the basis of then predictions. “Truman will not intervene in Korea, therefore we can invade with impunity.”  “We will never need to defend Korea so we can safely rule out intervening in a war in its defense.” Oops.

If, on the other hand, we use the game to increase and widen our spectrum of understanding of all the elements of a future contingency, including the very human emotional ones, then it can help. Yes, the emotions are not real and are not likely to be the same as other people will feel in real situations. Yet just as novels tell us some version of the truth of human reality and feelings, so too the game can tell us some version of the potential future realities. Depending only on the game is as foolish as depending only on the Lanchester model. Or depending only on the Monte Carlo simulation. Or depending only past history of similar situations. We can always predict the future; what we cannot do is bound the inaccuracy or uncertainty or error bars (whichever terms floats your boat) of our prediction. Can we do things that make us feel like we have narrowed those error bars? Yes. Can we know whether that narrowing reduces the range of likelihood from 0.01–0.99 to 0.40–0.60? No. Can our feeling about it be mistaken? Can we simply be deluding ourselves? Yes. That’s life. All we can do is the best we can.

But what games can do for those who play them (at least when they are designed by insightful and knowledgeable and skillful designers) is give them that dull grey shadow of what a black future might look like and feel like. And getting as much practice as possible at making decisions in those sorts of environments can be very helpful to some of those decision makers (the best ones, I contend), especially if knowledgeable, talented, and skillful mentors and analysts help them understand and profit from those experiences. That seems to me to be the best we can hope for. And I think that’s a lot when compared to the sort of nonsense or intentionally self-deluding results of models and sims and pseudo-analyses and junk arithmetic that so many of them get today.

Rex Brynen (McGill University):

I think Peter is right here, but I would add something else: a major contribution of a simulation can also be to highlight our lack of understanding “of all the elements of a future contingency.” A game can generate useful doubts about doctrines, standard operating procedures, and our confidence about the knowability of the world. It can encourage critical analysis-not necessarily generating definitive “answers,” but helping participants to formulate the sorts of questions that they might need to ask of a future situation or unexpected challenge.

In short, a well-designed game can be a wonderful reminder of the complexity of the world, and antidote to the hubris of power and capabilities.

Stephen Downes-Martin (US Naval War College):

I believe Peter and I are in violent agreement (the other kind is no fun).  We both agree that war games cannot be used to predict (with a known accuracy) future decisions.  But that introduces some design issues, such as getting at the misinterpretations of messages, beliefs and culture.  Since beliefs are remarkably robust in the face of contradictory evidence, beliefs exhibited in a game are a better predictor of beliefs that will be held in a future real life situation than are the decisions.

Peter also talks to the purpose of a war game.  Is it to learn something about a (set of) problem(s), give the participants a “feel” for the future or for themselves, training, something else?   My professional war gaming focus is on learning something about a problem, and that shapes the way I think about professional war games.

Ed’s advice to amateurs is brilliant.  A useful exercise might be to examine the implications of a game designed by an amateur following his rules.  What would it look like, for what purposes would it be useful,  and how useful would it be for the different possible purposes that we apply professional war games?

Ed McGrady (game designer, military and business consultant):

As usual Stephen and I totally agree about the two completely different approaches we take toward gaming!

In thinking about the differences I suspect I’m coming at the problem from a very different set of goals.  My primary goal is for those who sponsor and play in the games to find out something about themselves, their organizations, and the subject matter that they had not known going in.  In order to accomplish this I have to fascinate, entertain, and interest them.  Its a rather simple (but not easy) set of goals.  The reason I focus on those goals is because I believe compelling games get at the real issues, and games cannot be compelling unless the players are invested in them (entertained).

If it is a boring scrape that they simply endure, I would feel like I had not given them a good experience, and they probably would not be back.   This matters not just because happy players/clients are repeat players/clients, but also because the stuff they learn about themselves and their organizations will stay with them and effect real change.

I believe Stephen’s goals are both more scientific and focused on developing a data-based understanding of stuff.  Something which I wouldn’t even try to explain for him, but I, of course, applaud.

In answering Stephen’s question about what a simple game would look like:  the game I had in mind when I wrote that was one that involved 5-10 players, with the players divided along natural or artificial organizational fault lines, looking at a compelling scenario that required them to compromise some of their own closely held values in order to succeed.  I would want the players to self organize, and have a simple set of deliverables on a turn or end-of-game basis.  Not a bogsat, but nothing too fancy either.  So maybe I’d add a couple of rules:

11)  Identify naturally occurring fault lines between individuals, organizations, or other relationships.  Exploit these ruthlessly in your design.  Divisions, conflict, and compromise all add drama and depth to you game and give the players much harder decisions.  In many organizational settings these kind of compromises and discussions are exactly what is required to resolve the issues you are worried about.

12)  Get the players moving.  Don’t just have the players sitting around talking (bogsat = bunch of guys sitting around a table).  Have them move to different corners of the room to discuss some strategy or plan.  Put them in separate rooms and let them walk down a hall to talk to each other.   Kinetic movement is part of learning, and games provide an ideal venue for kinesthetic learners.

Of course if you view my rules as a decision tree most attractive and mentally well balanced people would stop at rule #1.  It seems like most wargamers press onward…

Stephen Downes-Martin (US Naval War College):

Ed and I have the same goals (“find out something about themselves, their organizations and the subject matter that they had not known going in”) but very different opinions about what is important in achieving those goals.  I am deeply suspicious of the centrality that Ed places on narrative.  I do not believe narrative is synonymous with compelling or with truth finding, or that narrative is the only (or most useful) way of being compelling.  Mein Kampf was a highly compelling narrative and the Nurembutg Rallies excellent examples of players living that narrative.

How we best combine the artistic and scientific approaches is a central question, and although placing either exclusively at the center is an obvious mistake, it is also a mistake to ignore or downplay the scientific knowledge about psychology and its unavoidable effects on war game design that I referenced earlier, or to ignore the fact that compelling narrative does not carry within itself (unlike science) any indication of the truth value of its contents.

Of course, it is not unknown for both scientists and artists to abuse their skills for what they erroneously believe to be a greater truth.

Rex Brynen (McGill University):

I would suggest that there is no perfect balance between artistry and science in this process–it very much depends on who is the client group, and what it is that we’re trying to do with/for them.

If we’re wargaming naval capabilities, it quite important that we have the science right, and that the take-away that participants have bears reasonably close accord to the way we expect things would work out in the real world. In such a case, the costs of getting it wrong (“I didn’t know the Chinese had ballistic missiles that could do THAT to my carrier–they didn’t in the games…”) are potentially quite large.

Conversely, if we’re trying to stretch the imagination, develop new skills, encourage empathy for the other, highlight complexity, etc—well, in this case it is both harder to accurately model these things, and immersive “narrative” often becomes all the more important.

(Incidentally, I’m using the “scientific” category to highlight simulation approaches that strive for higher degrees of verification and validation. This is, I think, i different thing from using the scientific insights of psychology (etc) in simulation design.)

The really interesting question, perhaps is whether the “art” and “science” of gaming are at opposite ends of a spectrum, such that increasing one necessarily comes at the expense of the other. I don’t think they are (certainly we’ve all seen simulations that are both artless and unscientific!), although I suspect at a certain point pursuing one may involve trade-offs with the other.

Stephen Downes-Martin (US Naval War College):

When I discuss science in war gaming I am not referring to the science content of the game (e.g. naval science when war gaming naval capabilities for OPNAV), I am referring to sciences such as psychology that are applicable to the design of ANY game no matter what client, objective, or content.  One can argue for whatever balance of art and science one likes, one ignores the sciences, specifically psychology, at the risk of significantly reducing the value of the game’s results.  Incorporating the science of whatever one is gaming into the game is an obvious requirement often ignored by clients who prefer pixie-dust to reality when gaming their favorite funded programs and platforms.

I also think it a mistake to confuse simulation with war gaming.  In my area of professional war gaming we use computer simulations as labor saving devices for adjudication (for example), and sometimes refer to a simulation design meaning that the players are role playing in some C2 structure that mimics to some designed degree of accuracy the C2 structure in the real world.  Again, in my area of professional war gaming, we use gaming when the narrative is mostly unknown — in an objective sense.  What is the narrative for information warfare in a 2020 world of major regional conflict between nuclear armed sectarian non-state actors?  The fact that a writer can write a compelling science fiction story around that scenario does not necessarily make that story useful or in any way assist in predicting what would, or could, or might, happen.  In my area of professional war gaming we refer to structural indeterminacy and design a war game to generate the narrative.

I do not believe the art and science of war gaming are at opposite ends of a spectrum with an associated trade-off, one chooses the levels of each and the interactions between them in a holistic way.

One response to “Reflections on the art and science of game design

  1. Christopher Weuve 20/06/2011 at 11:09 am

    There’s a lot to comment on here — I’ll limit myself to only one.

    It’s possible to let reality get in the way of the design goals.

    The best example of this is a game that the Naval War College does for the Senior Enlisted Academy. (I assume they still do it.) It’s best described as a cartoon game — ships and planes move in the same time scale, air transports are consumed by being used, combat is very simplistic, et cetera. Overall, no one would mistake it for a realistic simulation.

    And that’s the point. The target audience is enlisted personnel who are being asked to look out of their comfort zones, and start thinking about the D, I, and E parts of DIME, probably for the first time in their professional lives. So, the M part of the game is intentionally cartoonish, to avoid distracting the players with stuff that they will recognize as being not quite right.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: