Last month, RAND’s new Gaming Methods Center hosted an open house for gamers in the DC area. The event provided an opportunity for an exchange on the current state of gaming. Highlights of the discussion are summarized below. The event was a great chance to see what different folks in the community are thinking about, and I hope that similar events will occur regularly in the future!
The Gaming Methods Center is one of six new internal organizations, intended to “facilitate the development and dissemination of analytic tools and methodologies as well as employ existing ones in a collaborative and synergistic fashion across the entire RAND research and policy domain landscape.” The center’s immediate objectives include:
- Encourage the development of innovative new tools and techniques and encourage the evolution of existing forms and methods
- Encourage the use of these methods across the entire RAND research portfolio (cross-disciplinary)
- Encourage interdisciplinary cooperation on methods
The open house featured presentations from long-time RAND gamers who highlighted the history of gaming at RAND, the important role gaming can play in confronting current security challenges, and current RAND methods for both seminar style and board game/table top design games. A few highlights include:
- A discussion of past RAND gamers work, which highlighted a favorite RAND paper of mine, Crisis Games 27 Years Later
- A discussion of the evolution of the “Day After” method for seminar gaming, which I’ve used to good effect in some of my educational games
In the course of the event, there was lively debate from both RAND staff and outside participants about how gaming can best be employed to support national security decision makers. Highlights include:
Benefits of Gaming. Over the course of the day, participants offered a range of thoughts about the benefits gaming offers to the national security community. One participant describe three criteria for problems that are tractable to gaming:
- Blue or red operational concepts are not decided or not good
- Human agency is a major determinant of outcomes (adversary behavior in particular)
- Designer needs to convey a future people haven’t yet experienced
Other participants linked the practice of national security gaming to research on the power of the “urge to play” as a means of education and discovery. Still others noted that games tap into human’s need for narrative by providing an opportunity to build our own narratives in a setting that’s shaped by designers to further the right narrative.
Finally, participants highlighted the benefits that gaming offers to analysts. One individual commented on the tendency of analyst to waste too much time “worshiping the model.” Gaming encourages analyst to get to insights more quickly by forcing the analyst to work in broad strokes and model the truly important without getting tied up by the minutia. Games are also helpful when they disrupt assumptions that are built into the model. By watching what assumptions break down during the game, analysts can then go back to develop a better model.
Is gaming a scientific method or not. The art vs. science debate is an old standard in the field. In addition to a discussion of Peter Perla’s concept of games as part of the cycle of research, highlights of the discussion include the analogy of gaming methods to method acting, discussion of “the cult of spurious precision” that falsely seeks precise quantification rather than broader insight, and the importance of good design.
While this debate was interesting, I found myself agreeing with a participant who said that he was tired of hearing the same debate on the topic over the last several years. He urged the community to move forward to determine the implications of this debate for gaming practice. More work that lays out how the practical process of design and assessment would differ base on this theoretical debate seems more likely to move the field forward.
Game design standards. Some interesting commonalities in the participants’ standards for game design emerged during the discussion. Participants stressed that good game design, like good analysis of any kind, is primarily about caveating the limitations of analysis. However, right now there are not consistent standards about how such caveats are documented and communicated to fellow analysts and sponsors. Instead, there is currently a lot of responsibility on the principle investigator to communicate limitations. Some participants stated that this is best done by telling stories about “dynamics of the campaign” to senior leaders.
Relationship between gaming and other common types of modeling. Participants stressed the differences between gaming and other common types of modeling such as campaign planning. Gaming isn’t a cheap or fast way to do campaign planning and assessment, and should not be used in its place. As a result, it is critical that gamers know other methods well enough to direct sponsors to other methods that are more appropriate to answer the questions at hand.
At the same time, participants also stressed that gaming and modeling can and should work in tandem, not in oppositions. For example, games can be used as a screening tool, in order to determine what topics are worth spending the time to create in-depth, higher-resolution model. Games can also be used to test assumption that will be used to build models to minimize the risks of a faulty foundation to analysis.
Finally, participants stressed that gaming is a broad field that likely includes many related techniques. Being more aware of the strengths and weaknesses of different techniques can make us better able to pick the correct technique for the problems we are asked to analyze.
Challenges of communicating the potential and limitations of games to sponsors. Discussion highlighted the necessity of educating sponsors about what games can and cannot achieve. Many in the group stressed that right now sponsors who are new to gaming don’t understand what types of questions are appropriate to game. Furthermore, because gaming has become the main tool in the box for a number of current issues, the professional community needs to set ground rules and expectations (a point I’ve discussed at length here). Right now, there is not enough guidance available outside of the expertise of senior practitioners to help identify good games and bad games. If the field cannot develop alternative ways to ensure the quality of games, there is a real concern that “bad money will drive out good.” Participants agreed that there is a need to make sure that the interest in gaming doesn’t drive us into bad practices, not only for our own careers, but also for national security.
Additionally, unlike many other approaches to analysis, gaming is an event and a process rather than a group of methods for data analysis. Issues like the inability to replicate games, the need for space and in-person interaction, and other related challenges are hard to communicate to sponsors. In particular, stressing organizations through gaming often stresses process and procedures—space requirements, security, etc. all become major challenge.
Shortcomings of the traditional guide-style gaming education system. The current system for educating new gamers has been challenged by the noticeable generational gap in the field, where an established core of senior gamers is supported by staffs of entry-level folks (of whom there were a number in attendance, a nice change of pace from the more senior crowd that attends many gaming events). Discussion highlighted how the field’s traditional reliance on commercial board wargames may limit mentorship. The group also discussed the ways in which limited formal methods for game design make entering the field challenging.
Balancing the convenience of digital with impact of in-person events. There was a sustained discussion of whether games can be done remotely, using technology to bridge distances and allow for asynchronous games. The hope is not only will this make games cheaper, but also allow more, and more diverse player perspectives. However, the utility of such “virtual” games was contested. In general folk felt that the usefulness of virtual games depends on the purpose of the game, what types of findings are you looking for, what type of events you want to simulate, the audience, and what kind of time and interaction are available. One comment that particularly resonated with me is that games give a simulated experience that can shape behavior in the real world, so they require “intellectual if not physical proximity.” If analysts can created the intellectual proximity required in a virtual environment than these games can be successful, but often interpersonal interaction is still required.
Converting individual discoveries in the game to institutional insights. Participants discussed the necessity, and the challenges, of converting the individual experience of game designers and participants into organizational change.
Participants noted that often the group that learns the most from games in the design team, who may not be in the best position to advocate for change after the game is compete. Participants stressed the importance of game designs building up the ability to communicate game results in compelling (often narrate based) ways.
Likewise individual player experiences during games can be profound, but missed by game designs that focus on documenting group discussion and decision making. Participants suggested interviews with individual players focused on how individuals framed problems can be helpful. Likewise, focusing on understanding the thinking of individuals who deviate from the group’s “mean” opinion can be particularly valuable. To capture these perspectives, ensuring that research questions and data capture plans include a focus on individuals and small groups as well as the group as a whole is critical.