PAXsims

Conflict simulation, peacebuilding, and development

Monthly Archives: September 2009

the allure of immersion

An ongoing conversation on this blog focuses on the value of electronically simulated worlds as learning environments.  A comment on our first post and the lively discussion at USIP over the summer have reignited my thinking on virtual simulations as learning tools and I think it has bubbled in to something to actually say on the subject – you can be the judge.

At the USIP event, Beth Noveck suggested that avatars are valuable because they reflect back our own behavior.  Skip Cole suggested that our end goal for much of this work would be integrating realistic scenario design with virtual environments, ie. holodecks.  Often we invoke MMORPGs as examples of spaces where social networks evolve in very complex, user created environments (see Rex’s insightful discussion on COIN sims with WoW references below).  Still, I have a fundamental objection to using avatars for training.

Avatars do not and cannot convey complex human signals.  It is possible that they never will.  By design, every action by an avatar is the will of the user.

Wailing CavernsTake a simple example.  Rexio and Garus happen to bump into each other outside the wailing caverns in the Barrens and Garus starts telling Rexio a long and, frankly, quite boring story about the unusual swift zhevra he just killed.   As Garus goes on and on about the unusual hide and mane of the zhevra, Rexio, stands there in rapt attention, without interruption, and finally concludes with a resounding, “Well fought, sir!”

While Garus was rambling on and the player behind Garus, let’s call him Gary, was feverishly typing his epic tale, the player behind Rexio, let’s call him Amos, King of the Cosmos, could’ve actually been paying close attention, hanging on every word or he could’ve been surfing the web, he could’ve run to the refrigerator for a soda, he could’ve been on the phone to a friend (with the proper technology, he could be doing all three).  None of the typical signals of boredom: rolling of his luminous, heavenly blue elven eyes, constantly looking at his gnomish timepiece, tapping his chain mail clad foot, yawning, etc – would be conveyed.

This is just one rather silly example of human signalling that is vital to every day interaction but completely missing from avatar interaction.  My point is that avatars only convey what the user wants them to convey.  Additionally, I would contend that this is both a function of technology and incentives.

Technology cannot currently integrate human behavior behind the keyboard.  Sure, techies might suggest motion and bio tracking software that uses human signals to overlay behaviors on their avatars – but then, I think, users would be training themselves to avoid the biomedical conditions that lead to negative signals, not the negative signals themselves.  I’ve learned how to be still and balance on a Wii Fit – still, I don’t presume that I’ve learned how to meditate.  I think there is a very interesting analog in poker players that learn online versus those that learn in person – those that play online are very good at not conveying tells, but are often quite poor at reading other players (or tilting other players or resisting tilt or the other sorts of psychological warfare necessary for “good” play) – this is all anecdotal, of course – I’m open to evidence to the contrary.  The point is that avatars can’t convey all of the complexities of human signals that are used in conversations.

And even if they could, why would they?  In addition to the technology limitations, there are incentive problems for individual users in committing to convey negative signals.  If there are negative consequences for negative signals that would otherwise be difficult to control in real human interaction, why not write subroutines that take out yawns, eye-rolling and all the other rude stuff we humans do to each other?

Amos, King of the – err, Rex runs a giant simulation, full of human interaction where his students learn that their actions have real effects on the world, mostly as a result of how they coordinate, cooperate and communicate with each other.  Having run Carana eight or nine times now for more than 200 participants, I am firmly convinced that the main value and worth of the exercise as a teaching tool is that people learn about human interaction in complex social environments with pressing timelines and exogenous (and occasionally endogenous, participant created) shocks.   I’ve had participants ask “What could I have done differently to convince them that they were making a mistake?” and others say “I finally understood what was going on in other similar conversations I’ve had in real situations like this.”  I think this fundamental learning about not just what is said, but how it is said, is a vital component of using simulations for peacebuilding.  Avatars can’t deliver that now, I don’t know if they ever will.

Simulation & Gaming

The latest issue of Simulation & Gaming 40, 5 (October 2009) has just been published. The table of contents can be found here.

The Internet is for COIN?

JDMSA forthcoming issue of  the Journal of Defense Modeling and Simulations is devoted to modeling and simulation in counter-insurgency and irregular warfare. Among the items slated to be published there (and preposted on JDSM’s website) is an interesting piece by David Earnest on “Growing a Virtual Insurgency: Using Massively Parallel Gaming to Simulate Insurgent Behavior.”

Models and simulations of counter-insurgency warfare and irregular (COIN) operations are only as effective as their underlying models of insurgent behavior. Existing simulations of insurgencies rely upon strong assumptions that may limit their validity, and thus their use in training for COIN operations. This paper suggests an alternative approach to modeling insurgencies: using a massively parallel game architecture. Massively parallel systems exhibit surprising capacities for learning, adapting and solving complex problems, while games may stimulate individual learning. By harnessing these adaptive capabilities, the proposed massive multiplayer online first-person shooter (MMOFPS) game holds promise for a more realistic and valid simulation of the behavior of insurgencies by incorporating actual human players. Furthermore, by constructing a persistent virtual world in which human players simulate insurgents, the MMOFPS game allows researchers anddecision-makers to observe and measure the behavior of ‘meta-insurgents’, allowing for model validation. Data collection and post-game interviews of players also allow for both quantitative and ethnographic experimentation. This paper proposes a gaming architecture and evaluates the technical risks.

The article nicely highlights an issue that has been often discussed here at PaxSims, namely the embedded assumptions of simulations. The bases for these assumptions and social models are not always clear, validation is difficult (if it is even attempted), and the increasing technological sophistication of simulation makes them simultaneously more alluring and their underlying (theoretical, ideological, and even normative) presumptions perhaps even less apparent to the user.

The solution that is suggested in the article is to use massively parallel gaming—or, in the language of gamers, a counter-insurgency Massively Multiplayer Online game. In such an environment, in which large numbers of players are communicating and cooperating in smaller or larger groups to achieve goals, one finds social innovation and learning, adaptive approaches (and responses), and generally a much more fluid and dynamic social and operational environment. Indeed, one might argue that the value of such an approach even goes beyond this, to issues of group leadership, dynamics of recruitment and group loyality, virtual social norms, even identity politics—something that anyone who has ever been a member of a Warcraft guild (or other game equivalent) will readily understand.

Moreover, while Earnest’s article is aimed at the military/COIN community, there is no reason why such an approach couldn’t work—in theory at least—in the aid and development community too, with a variety of live players and AI-driven autonomous agents (“non player characters”) interacting in ways that simulate the myriad web of complex social,  economic, and political dynamics that intertwine local stakeholders. In practice, however, there may be more practical obstacles: the lack of appropriately large R&D and training budgets in aid agencies, NGO, and international organizations; the much smaller number of personnel that need to be trained; the much greater ease of training through more conventional human-moderated role-play techniques; and the desirability of maximizing the human dimension of direct communication, negotiation, and diplomacy through interactions that aren’t mediated through a computer interface.

stenstoutarmMoreover, several other issues arise. The first is the tendency of many MMO environments to encourage meta-gaming rather than realistic behaviour among players (to use a game analogy, “stand outside that cave, and the creature you need to slay will respawn in 13 minutes…”). Cooperation among players often involves shared knowledge as to how to use a a variety of in-game techniques intended to make best use of the quirks and processes of the simulation programming.

A second issue is the danger that if a COIN (or peacebuilding or development or humanitarian assistance) MMO is entirely staffed by participants from the same organization, it may replicate organizational conventional wisdom about how the “other” (insurgent, warlords, local villagers, refugees) operates rather than reproducing realistic behaviour—thereby perhaps reinforcing and perpetuating the very stereotypes it ought to be challenging. Ironically, one of the examples that the article cites as highlighting the value of technologically-facilitated collective wisdoms (“massively parallel systems of distributed knowledge”) also highlights the potential problems:

Wikipedia invites readers to correct errors – a form of selection that allows the population of Wikipedia readers to filter out inaccuracies. Today it is nearly as accurate as the Encyclopedia Britannia.

In many of the physical science and other entries, yes (and I’m a dedicated wiki-surfer and sometimes editor myself).  However, in my own area of work— Middle East politics—Wikipedia is notoriously unreliable, with waves of edits often reflecting the ideological preferences of contributors and even a degree of organized propagandizing rather than any sort of analytically-grounded set of views. Earnest does partially recognize the dangers of massively distributed groupthink, noting that “An audacious game would distribute the client widely to civilians and military personnel alike. School teachers, accountants, teenagers, and service station attendants may not make the best insurgents, but they think like civilians and probably are free of the cultures and doctrinal training of the armed services.”

It is all very interesting stuff, and it will be fascinating to see how it develops in the coming years.

And yes, for those who were wondering, the title of the post is indeed an allusion to the Avenue Q song. For those who haven’t heard it or seen the Warcraft version, you’ll find it here—with the caveat that its not entirely SFW.

SimCity meets urban COIN operations

The Institute for Creative Technologies at the University of Southern California is doing some very interesting work with software-based simulations, ranging from “virtual humans” to immersive environments to improved technologies for narratives and storytelling. One of their current projects for the US military which may also be of interest to those in the peacebuilding and development communities is UrbanSim.

UrbanSim_screen3.PNGThis UrbanSim—not to be confused with the similarly-named and completely-different UrbanSim urban planning simulation software first created in the 1990s—is intended to be training simulator that will enable military commanders to develop their skills for counter-insurgency (COIN) and stabilization missions. The primary emphasis here is not on traditional military means–although the simulator does seem to allow for so-called “kinetic” operations, and maintaining and enhancing local security is a key objective. More important, however, is the simulation’s emphasis on the complex “non-kinetic” aspects of such operations, including mentoring host country security forces, intelligence collection, information operations, providing essential services, increasing local employment, capacity-building, respecting local religious, ethnic, and other sensitivities, and so forth. Students are encouraged to learn and manage a variety of intertwined lines of effect, engage in social network analysis of their area of operations, and understand the importance of unintended and 2nd and 3rd order effects. The simulation includes in-game tutorials and a series of learning modules, as well as opportunities for post-game debriefs with instructors.

LOEThere are several interesting things here. Before discussing them, however, I should make it clear that I haven’t seen, much less tinkered with, the software—rather I’m basing my reflections wholly on presentations that the designers have made available here and here and here, so I could well be more than a little off-base.

One of the most interesting issues, of course, is the very use of these sorts of technologies for these sorts of purposes, with the military (with high demand at the moment, and large R&D budgets) leading the way. A second fascinating component is the technology of this sort of simulation, with its combination of triggered events and narratives (that is, events which may occur if certain contextual conditions are met) as well as the modeling of behavior of key actors (whether they be insurgents, tribes, sheikhs, religious leaders, neighborhoods, the local police, etc)—each represented by a series of algorithms that determine what sorts of variables shape the agent’s behavior, in turn generating a complex array of interdependent goal-seeking behaviors.

As a political scientist with more than a passing interest in insurgency and political violence, however, there are two other aspects that particularly interest me. The first is how we derive the underlying social, economic, and political models that are embedded in the simulation. To be frank, social scientists are far from a consensus on what spurs political mobilization and violence. Does economic growth and employment reduce radicalism? In many places, yes. In other places, no—indeed, it might even generate it, whether through redistribution of social resources, the growing availability of lootables, other forms of economic empowerment of challengers, grievances generated by inequitable social distribution, or simply because the population isn’t primarily motivated by economic concerns. Do tribal and religious leaders matter? In some places yes, in other places and times, less so. Is corruption bad? Yes, although in some cases quasi-corrupt neopatrimonialism may be a primary stabilizer of the political order.

Related to this is the current debate over COIN doctrine. The December 2006 release of FM 3-24, the US Army’s Counterinsurgency doctrine,  was rightly regarded as a substantial change shift to a military approach that now emphasized the importance of “armed social work,” with an emphasis on changing social and political conditions and attendant warnings that “sometimes the more force you use the less effective it is” and “sometimes some of the best weapons for counterinsurgents do not shoot” (FM 3-24, p. 1-27). The so-called “COINdinistas” have been in the doctrinal ascendency in recent years, exemplified by CENTCOM commander General David Petraeus, FM 3-24 coauthor John Nagl (now at the Center for a New American Security), and former Australian army officer (and anthropologist) David Kilcullen, all of whom have stressed the importance of population-centric COIN. (For a lively discussion of all this, check out the Abu Muqawama and Small Wars Journal blogs.)

There have been criticisms, however. Some of those criticisms relate to the question of how the emphasis on COIN may be distorting the institutional and doctrinal development of the UA armed forces; others relate to US national interests, and whether those are best served by fighting counterinsurgency wars in Afghanistan (or elsewhere). For the purposes of the discussion here, however, the most important critique is one that argues that COIN efforts, including the capacity-building and developmental aspects of them, are far more contextually dependent than a superficial reading of COIN doctrine would suggest. Simply put, what works in one case (or, for that matter, one town, village, or valley) may not work in another, because of underlying social dynamics. The ongoing debate over why violence in Iraq has declined highlights the indeterminacy of all this. Was it because of FM 3-14 type tactics, the “surge” of US combat power, the prior mistakes of al-Qaida, a realignment of Sunni leaders caused by the threat of growing Shi’ite militias, or some combination of all of these (and if so, in what measure)? There is simply no agreement, either within the military or among outside subject matter experts.

testThis problem of variability, contextuality, and indeterminacy can be illustrated by one of the apparent (beta?) review quizes included in UrbanSim, which I’ve reproduced on the right (click to expand). I think I could argue, on sound theoretical and practical grounds, that the correct answer falls somewhere between “it depends” and “none of the above,” depending on the social and political context. It is certainly not as clear as any of the four choices would suggest.

Of course, there are ways in which software-based simulations can reduce the risk of positing an over-deterministic model of how insurgencies (or peacebuilding, or conflict-sensitive development) ticks. One way would be to randomize some of the starting conditions and relationships in subtle ways, and bury clues and cues to this in the pre-game/mission briefings. In some siminsurgencies, therefore, tribal leaders might be powerful social actors; in others, increasingly marginalized representatives of a discredited old order, playing on the ignorance of outsiders in the hopes of gaining allies and regaining their power. In some cases, individuals might be motivated to join the insurgency because of lack of economic opportunities; in others, political grievances might be so powerful as to render them largely unresponsive to economic blandishments (or even alienated by them). I’m particularly wary about reducing behavior to rational utility maximization, and reminded that in experimental trials with ultimatum games that they only ever appear to be played that way by other economists (and certainly not members of my classes, who consistently produce results that affirm the importance of normative issues of fairness). I’m not saying UrbanSim excessively emphasizes the material, by the way—indeed, reading between the lines it would appear that some agents are designed to value what might be considered intangibles—but it is an intrinsic risk when you’re looking to reduce human actions to mathematical formula reproducible in software coding.

By now, I suspect that many PaxSim readers who are professionals in the aid and humanitarian assistance communities may be rolling their eyes at the notion that this sort of software-based training can ever be valuable, or can even begin to capture the complex web of social interactions and economic and political relationships which they deal with on a daily basis. It is not just an issue of budget and available tools, I suspect, that lead most people in these communities to emphasize the sorts of role-playing training simulations that we’ve often highlighted on the website (and, indeed, that Gary and I both run for different client sets in our regular jobs)—it is also an almost instinctual aversion to trying to use computer simulations for this purpose.

I must admit that I’m sympathetic to much of this cynicism, just as much as I am intrigued by the possibilities of an UrbanSim-type approach. However, that very tension provides some fascinating opportunities for learning. Software like this—reflecting as it does all of the embedded understandings of military COIN experience, and undoubtedly a long process of reviewing apparent best practices—would be an intriguing way of teaching people in the development community how folks in the military see the developmental and capacity-building aspects of peace and stabilization operations. The former might well disagree with the latter just as much after playing Urban Sim (and perhaps even more so!) At least, however, they would have a better understanding of where they are coming from. In operational environments where there are frequent and very real negative consequences of the operational cultural disjunctures between the military, multilateral agencies, and the NGO community, increasing the level of mutual understanding like this before deployments could be a very useful thing.

Ironically, therefore, UrbanSim—designed as a command training tool for US officers—could well have a valuable secondary use as a cultural awareness trainer for those in the peacebuilding community well outside of the military.

Designing Exercises for Teaching and Analysis

The latest issue of Joint Forces Quarterly 55 (4th Quarter 2009) has a short but interesting piece on designing exercises, games, and simulations. Two observations stand out.

The first concerns the importance of validation in social and political simulations.

If we are conducting an exercise to explore the contours of some ill-defined future problem, for instance, it is crucial that we be able to justify why we reach certain conclusions or how we generalize lessons learned from an exercise.

Answering the “How do I know that I know that?” question is routine in the social sciences, including in qualitative work common in political science and sociology, but not always thoroughly discussed in the exercise design and evaluation community. Nevertheless, it is crucial to a defensible analysis.

As developments in information and computer technology make more complex (and visually-appealing) simulations possible—especially in the military, where there are large R&D budgets and considerable demand for software-based training and planning solutions —I must admit to being worried about the theoretical assumptions that may (or may not) be built in. This is especially so because (to be frank) I don’t think social science yet has a very firm grip on such issues as insurgency, civil war, peace building, and political stability, much less how to model them.

A second important point raised by the article is the difference between exercises and simulations for analytical purposes, as opposed to those that primarily have a training role:

The elements of good exercise design for teaching and analysis can be somewhat different for the simple reason that the lessons to be learned are different. Analytically, what we learn from tabletop exercises usually has to do with whether the model of the problem described in the scenario introduces the right independent variables, whether others should be added, how they could be refined and their relative weight, and how differences in them might require different actions and result in different outcomes.

Exercises for teaching purposes are rooted in an assumption of the value of experiential learning, that giving participants a visceral feel for the exigencies of policy decisionmaking will be an effective way of making theoretical lessons they have learned concrete.

The first observation highlights the “garbage in, garbage out” problem once more, and hence the issues of validation raised earlier. The latter points to the importance of “feel”—something that may have as much to do with the way a simulation is staged as it does with the rules and procedures involved. In my own simulations, I have at times deliberately engineered information overload, exhaustion, and time-pressures into the process to give participants a sense of how these influence behaviour. One can go even further, and modify the physical layout of the simulation, the immediate environment, and so forth to create an additional layer of contextual effect. At the Chatham House simulation of Palestinian refugee negotiations, for example, we deliberately provided players with physical resources (dedicated team rooms, phones, printer access, interns assigned as support staff, etc.) in order to proxy the degree of institutional support each party might enjoy in real life. This meant that our refugees—played in the simulation by actual refugees—were given no room at all, and were forced to make do with whatever corners of the building they could find. One effect of this, in turn, was to place them in a marginalized position that nicely captured how often they are left out of consultative and negotiating processes on the issue.

%d bloggers like this: