Walt on the Harvard Iran simulation

Following on from my earlier blog post on Harvard’s recent Iran nuclear simulation, I’ve come across additional pieces on the simulation by Gary Sick (who participated as part of the Iran team) and Stephen Walt (who participated as part of the US team). Both provide additional detail on what transpired during the exercise.

What is especially interesting from a PaxSims point-of-view, however, are Walt’s comments on how the technical design of the simulation may have shaped the validity of its outputs. Given their broader applicability to simulation design, they are worth quoting at length:

In my view, what one might call the “external validity” of the game was limited by three unrealistic features.

First, the timetable of the game was extremely compressed. In effect, we were trying to simulate a full year of negotiations in a mere six hours. Thus, each hour of the game covered two months, which meant that a team could send a message to another team and receive a reply in due course, only to discover that a month or more had passed and the original message was now effectively obsolete. More to the point, the breakneck pace of the game did not allow for any time for reflection, for the weighing of alternatives, or even the formulation of clear or novel strategies. (Each team was given about twenty-five minutes to plan its approach before the game began, and I like to think U.S. leaders do a bit better than that in real life. Heck, Obama just spent several months deciding what to do in Afghanistan). Yes, time is a precious commodity and policymakers are often forced to juggle multiple commitments, but I believe a more realistic timetable would have produced very different results.

Second, trying to simulate a complex multiparty negotiation with four or five-person “teams” was problematic, particularly when some team members (myself included), had to leave the game temporarily to teach their regular classes. This constraint required me to be absent for 90 minutes, which in terms of the game’s timetable meant that the U.S. Secretary of Defense was effectively incommunicado for three “months.” The same problem sidelined the person who played the Secretary of State for a similar period. Moreover, given that team members had no staff and thus no subordinates to give orders to, there was no one to delegate to and it was impossible to conduct continuous consultations with all of the relevant parties, even when both sides may have wanted to. What must have looked to some like Bush-era “unilateralism” was instead simply an unavoidable artifact of the game’s structure.

Third, the composition of the different teams was unavoidably slanted. The U.S., Russian, Chinese, Iranian teams were all populated with and led by Americans, while the Israeli team was made up entirely of Israelis and the EU team was composed of Europeans. To have confidence in the validity of the results, therefore, you have to assume that each of the teams actually played the way that their real-life counterparts would have. That might be true in the case of the U.S., Israeli, and European teams (though I wouldn’t assume it), but it’s obviously more of a stretch with the others.

These difficulties are not the fault of the game’s organizers, who faced obvious constraints in putting the exercise together. Ideally, such a simulation would have been played over a long week-end and covered a shorter time period, but it would have been far more difficult to assemble an equally impressive array of participants for an entire weekend. Putting together a genuinely multi-national participant list (including appropriate Iranians?), would have been even harder if not impossible.

The bottom line is that one ought to be exceedingly wary about drawing any conclusions about what this artificial exercise actually teaches. To me, its real value is not as a crude crystal ball that allows us to divine the future, or even as an analytical device that helps us identify particular barriers to resolving some thorny diplomatic problem. After all, it’s not exactly headline news to discover that resolving the Iranian nuclear issue isn’t easy, that there are certain tensions within the P5+1, or that Iran’s objectives are at odds with those of the other participants.

Rather, the potential value of such an exercise lies in forcing participants to take on different roles and see how a problem looks from a wholly different perspective. With hindsight, I wish we had mixed things up a lot more: with some Israelis on the Iran team, with real Russians, Chinese or EU citizens playing on the U.S. or Israeli side, and so forth. That might have taught us about some of the sources of misunderstanding that have made this issue so hard to resolve, whatever the actual “outcome” of the game might have been.