PAXsims

Conflict simulation, peacebuilding, and development

Tag Archives: Defence Research and Development Canada

DRDC: Investigating wargaming and capability based planning

p806080_A1bcover.jpg

Back in January, Murray Dixson and Fred Ma at Defence Research and Development Canada published a valuable paper entitled An Investigation into Wargaming Methods to Enhance Capability Based Planning. This is a summary of a DRDC project in which “Several wargaming methods are investigated as possible ways to enhance the Capability Based Planning (CBP) process in the areas of capability requirements identification and force planning scenario validation”

The concept behind the methodology was to begin the exploration of wargaming methods with several game styles chosen based on the knowledge and experience of the SPORT [Strategic Planning Operational Research Team] staff, and in parallel, build familiarity over the period of the work with wargaming methods through literature reviews and exposure to the wargaming community. The relevant wargaming community exists primarily amongst the five eyes allies through forums such as (but not limited to) the “Connections” wargaming conferences and the Military Operations Research Society (MORS) workshops and conferences [16][17].

The knowledge gained from the literature and the wargaming community influenced possible adjustments to the methods chosen as the explorations proceeded. There was an implicit feedback loop in place where the knowledge and experience gained from each game iteration was used to decide on the next steps of the testing which involved either running more iterations of the game or choosing another game type.

Ideally, the methodology was intended to include these elements:

  1. Try out several game styles to see if any had any particular advantages for CBP in terms of the issues observed and discussed in the previous section
  2. Run several iterations of each game style to generate some (limited) statistics from which informative or indicative trends might emerge
  3. Run multiple game iterations with different groups of players to try and obtain a broader range of inputs and experiences with playing the games
  4. Compare the wargame outputs to those from the JCPTs [Joint Capability Planning Team] and draw conclusions about whether the wargaming methods tested showed any potential benefit for addressing the issues discussed earlier with CBP

The games used in the study included ISIS Crisis and several other matrix games; the Rapid Campaign Assessment Toolset (RCAT, developed by the UK Defence Science and Technology Laboratory and Cranfield University); and PAXsims’ very own AFTERSHOCK: A Humanitarian Crisis Game. It should be noted, of course, that  these were not necessarily designed to support Capability Based Planning—rather, DRDC was playtesting them for more basic insight as to whether a particular sort of game approach might contribute to capability requirements identification and force planning scenario validation.

DRDCCBP.png

The study’s conclusions were as follows:

In this work, we explored several wargaming techniques as a way to mitigate some of the concerns and observations originating from previous cycles of CBP. There was concern that the capability analyses in Phase 2 could become too narrowly focussed and that the planning scenarios may not be adequately validated before the JCPT capability analysis. The properties of table top wargames lend themselves to addressing problems like this because they immerse players in the scenario in a way not possible by simply reading a scenario narrative from a document. The resulting experiential learning provides a richer experience for the player, which translates into a more holistically encompassing view and understanding of the key elements of the scenario.

We therefore investigated the potential of tabletop wargames to enhance the identification of capabilities under CBP and to act as a scenario validation mechanism. We developed a hypothesis that wargaming would enhance capability identification through the broader understanding of the scenario and also through a better understanding of the driving factors therein.

The number of game styles and trials was limited, as was the range of backgrounds of the players that were able to participate. Nonetheless, three styles of game (a traditional tabletop game (RCAT), matrix games and a commercial game) were examined, and nine trials spread across them were run.

From the limited data, we saw clear indications that playing the games would be an effective way of checking the consistency and believability of a scenario, thus providing a validation mechanism.

The data indicated that by playing the games, players felt that they had a better sense of the key drivers of the scenarios. We interpreted that finding to be an indication that wargaming, through the concept of experiential learning, could provide JCPT members with a broader understanding of an FDS which logically should reduce the risk of them becoming too narrowly focussed in their capability assessments. However, the results were assessed as inconclusive with respect to the application of wargaming to directly identifying capability needs. A key contributor to this result was that we were not able to test a game that was properly designed for that purpose, as there wasn’t enough time and people to do a best-practices “design-test-run” game development cycle.

Although the time and personnel resource constraints did not allow for testing additional game styles or running more iterations, it was clear that the game styles tested are feasible and reasonable choices for future use, given the levels of resources that are available now and that are expected in the future. Similarly, it was not possible to conduct a comparison of the utility of the wargaming method with that of the current CBP model, which employs the OPP; hence, this too remains an open question.

At the level of granularity of strategic planning, tabletop wargaming has potential as a supporting element of scenario design and qualitative analysis. It brings together the knowledge embedded in the game design with the knowledge of participants (players, facilitators/adjudicators, control personnel) in order to explore plausible courses of events resulting from decisions made from the perspectives of the various roles. It can reveal nuances that are not obvious from scenario descriptions alone, and can illuminate relevant and driving factors associated with a scenario. It is not a method to statistically characterise selectively defined quantifications for a scenario.

To summarise the findings above, although the data from our trials is rather limited, we believe that there is evidence to support the hypothesis that wargaming can improve scenario validation. However, we judge the data to be inconclusive on whether it would improve capability identification.

Organising wargames always requires time and people resources. The amount is dictated by the scale and complexity of the game. The present work demonstrated that games suitable for the tasks in this project are feasible within our resource constraints. However, obtaining players on an ongoing basis, new or not, can present a challenge for which senior leadership support will be needed if it is to be met. Despite these challenges, a reasonable selection of game styles and scenarios was explored in this work and we had success (albeit limited) with bringing in players with diverse backgrounds.

A strength of wargaming is that game styles can be highly varied to suit different purposes, e.g., exploring plausible outcomes, scenario validation, familiarising planners with driving factors and training. In our trials, wargaming illuminated the dynamics that drive a scenario, uncovered logical inconsistencies (applicable to scenario validation), and sometimes revealed an unanticipated course of events. Applying wargaming to scenario validation should improve scenario design, while the insights into the driving factors would inform JCPT analyses, e.g., formulation of CoAs, required capabilities, MoC assessments, and capability delivery options.

In the end, this work revealed that none of the game designs trialled were optimally suited for explicit identification of capabilities. In the designs trialled, capability requirements were mostly implied by player CoAs and would have to be explicitly identified via a post hoc analysis. Although the Baseline matrix game did explicitly ask players about capability requirements, the ones identified were, in our view, not significantly different from those observed from a JCPT analysis; hence, the game design would need changes to guide players toward more innovative ideas.

We note that capabilities at tiers 2 and 3 in the JCF are defined to be generic, technology-independent, and thus enduring. Hence, it may be uncommon to identify wholly new capability requirements at that level, though any that are identified would certainly be informative. We expect that these would likely arise from an unanticipated course of events and/or CoAs. In contrast to tiers 2 and 3 capabilities, we noticed that players often thought in terms of, and identified, finer grain capabilities more readily (i.e., those tied to specific technologies, platforms, or assets), along with circumstantial factors that determine what capabilities make sense, at least in part.

Based on the limitations experienced in executing our wargames, it is possible that identification of CoAs and capabilities can be improved by mixing players from diverse backgrounds to enhance synergy, e.g., including relevant geopolitical expertise, operational experience, and futurist backgrounds. More open-ended games, e.g., Matrix style, have the flexibility to leverage the greater knowledge and ideas. In view of the challenges in obtaining players, this could be treated as an ideal to capitalise on, if it can be realised, and if it makes sense for the particular CBP phase in which a wargame is conducted.

France Info reports on ISIS Crisis

IMG_1311First it was Vice News, and now it is France Info reporting on the ISIS Crisis games conducted by Defence Research and Development Canada last year.

Les hauts-gradés de l’armée canadienne ont pris l’habitude de tester leur stratégie via… des jeux de société. Le dernier en date s’inspire de l’actualité au Moyen-Orient.

La crise de Daech” se joue avec six équipes: Daech, le gouvernement irakien, le gouvernement régional du Kurdistan, les milices sunnites, l’Iran et les États-Unis. En début de chaque partie, les factions ou puissances étrangères élaborent un plan d’action et des stratégies. Elles peuvent aussi passer des accords secrets, sans que leurs adversaires ne soient au courant. C’est un professeur de science politique, Rex Brynen, enseignant à l’Université Mc Gill à Montréal qui a eu l’idée de ce jeu, en collaboration avec un commandant de l’armée britannique. Ils l’ont proposé cet automne à un centre de formation militaire aux Etats-Unis.

Ce jeu n’a pas d’application concrète sur le terrain et les théâtres d’opération. C’est plutôt une façon pour les militaires et les stratèges de réfléchir à des actions possibles, d’élaborer des scénarios, exactement comme ils le font déjà lors d’une réunion classique. Mais ces outils ludiques ne sont pas vraiment nouveaux. L’armée prussienne utilisait un jeu baptisé Kriegsspiel (Le Jeu de la Guerre) pour planifier ses campagnes militaires. Et il semble qu’Henry Kissinger, le célèbre secrétaire d’état américain, adorait jouer à Diplomatie, qui consiste à monter des alliances pour conquérir l’Europe.

Click the link above for the full audio report. This time AFTERSHOCK gets a shout-out too!

For a discussion of what the games were really about, see my earlier blog post on the subject, as well as the actual DRDC report that sparked all the interest.

For more in the ISIS Crisis game, have a look at these links.

h/t Tracy McNicoll‎  

 

AFTERSHOCK in Ottawa

IMG_2382.JPG

Today I had an enjoyable day running two games of AFTERSHOCK for colleagues at Defence Research and Development Canada. The first game involved four DRDC/DND analysts playing, while second game included one from DRDC, a very experienced humanitarian aid worker, and two staff from Global Affairs Canada. Both teams eked out a narrow win in the closing turns of the game, with the second group scoring a little higher (in large part because of better coordination).

IMG_2381.JPG

Although AFTERSHOCK was designed as an educational game for university students, military personnel, trainee humanitarians, and junior diplomats and aid officials, the purpose here was to assess whether it might offer a differing perspective on humanitarian assistance and disaster relief operations than the one generally adopted in military planning scenarios for capability-based planning. In particular, the game highlights not the hardware of platforms and assets, but rather the software of coordination and inter-agency synergies.

In any case I think everyone found the game enjoyable, and certainly the fictional, earthquake-afflicted population of Carana was grateful for their help.

I’ll be running one more game of AFTERSHOCK in Ottawa this weekend, in the very different setting of the CanGames gaming convention. Come and join us in the Sunday 2pm slot!

 

 

(Matrix) wargaming to support strategic planning

Last year Murray Dixson, Michel Couillard, Thierry Gongora, and Paul Massel of Defence Research and Development Canada wrote a paper on “Wargaming to Support Strategic Planning” which describes DRDC’s study of matrix games as a tool to explore the Force Development Scenario Set used by the Canadian Armed Forces as part of their capability-based planning process:

The Canadian Armed Forces (CAF) capability based planning process uses a set of force planning scenarios to assess different options for the capability requirements of future forces. A good understanding of the key drivers of the scenario is important so that the subject matter experts can more fully understand and identify the capabilities required for success in it. A project is underway to investigate whether this capability identification can be enhanced through the use of various wargaming techniques. The Matrix game methodology is one that has been chosen for this research and was used in a recent series of research games. An ISIS conflict scenario was used as an explorative tool in all the games which were played out using several combinations of player types. Each iteration of the game was analysed using a set of metrics to help determine the utility of the games for the force planning application. The results are provided in this paper.

Readers of PAXsims will already know something about this, based on Ben Taylor’s thoughtful piece on serious matrix games, our game at the University of Ottawa, and our various other posts about the ISIS Crisis game that was used as a testbed for the study.

The study concludes:

As a result of these experiments a number of useful observations were obtained concerning the intricacies of organising and conducting a wargame; the value of participating in a wargame from the players’ perspective; and the potential applicability of augmenting Canada’s capability assessment efforts with one or more wargames. In terms of conducting a wargame, valuable experience in understanding the importance of the rules and structure of the game; of the principles and limits of keeping players involved in the game; and of the nature and key role that the GM or adjudicator plays in the conduct of a successful game. From the players’ perspective new players gained a greater understanding of the Matrix wargaming methodology, and more experienced gamers gained a greater appreciation of the many layers of complexity and dynamics that characterise this regional conflict. Finally, in terms of the relevance of Matrix wargaming methods to supporting Canada’s capability assessment effort, this experiment was limited by the nature of the game itself. The ISIS Matrix game is a replication of a complex, multiplayer, geo political situation. As such, it was observed to be a useful platform for introducing some of the region’s complexities to the assembled players. This would seem to have similar promise if this methodology were to be applied to one or more of Canada’s defence planning scenarios, but this clearly resides in the realm of future work.

I think Murray and the team are right that ISIS Crisis is a game heavily skewed towards political-military dynamics—in their test games, kinetic actions only accounted for slightly more than half of all player moves. Moreover, because military actions are dealt with at high level of generalization and abstraction, ISIS Crisis may not be very useful at teasing out questions of capability.

fig4fig5

However, that is in large part a function of the scenario design: a better test of the matrix game method for capability-based planning would probably focus on military activities more narrowly, with units on the map representing clearly-defined assets rather than indicators of relative combat power, and a more rigorous time scale for player actions.

On the other hand, as DRDC’s RCAT playtest suggested, some of DND’s current Force Development Scenarios probably hinge far more on political and other non-kinetic actions than is intended. Political-military matrix games as useful for pretesting and refining planning scenarios, and could certainly be used to generate vignettes that could then be explored in greater detail through a capability-based matrix game, another type of wargame, or other forms of analysis.

The DRDC report also offers some interesting insight into the challenges of game adjudication (in the MAGIC 1 playtest they describe, where I was double-hatted as both facilitator and subject matter expert, left an impression among some of heavy-handed adjudication), compressed vs extended playtime, the ease of learning the rules, and other issues. It is very helpful reading for those considering using matrix games as an accessible method for wargaming complex problems.

fig11fig12.png

Connections North AAR

Join_the_Team_RCAF

On February 22, a small group came together in Ottawa in what will hopefully be the first of many “Connections North” interdisciplinary wargaming meetings. The miniconference—organized at the very last minute by Defence Research and Development Canada and PAXsims to take advantage of a visit to Canada by the one and only Jim Wallman—attracted eleven participants with expertise in wargaming, operations research, medical and humanitarian simulation, virtual simulation and training, higher education, and game design.

Following introductions and introductory remarks, Murray Dixson (DRDC) made a presentation on the MAGIC (“Matrix Games for Improvement of CBP”) project. This has involved a series of trials of the matrix game method to explore how it might be used to enhance capability-based planning at the Department of National Defence. To date they’ve run several games of the ISIS Crisis scenario, and found that the approach could be helpful for scenario testing and validation, although less so for identifying specific capability gaps. Analysis suggests that subject matter experts and non-SMEs take similar lengths of time to make a game move, and that the ratio of political, military, economic, social, infrastructure, information, and cultural (PMESII-C) actions was broadly similar across games. Both the presentation and the subsequent discussion highlighted the importance of game facilitation skills, and the risk that facilitators/adjudicators might insert their own views into the game. It was noted that the game materials also potentially cued players into making certain types of moves—a map with military assets displayed, for example, tends to encourage military actions. Game participants expressed some frustration at the difficulty of pursuing a coherent long-term strategy in ISIS Crisis. While this is partly a function of the sequential turn sequence, it likely has even more to do with the nature of the scenario, with its multiple conflicting actors and objectives. Future trials will likely involve a different scenario, thus allowing analysts to determine what game dynamics may be scenario-specific.

Paul Massel (DRDC) then made a presentation on a recent playtest of the Rapid Campaign Analysis Toolset (RCAT), which is being assessed by DRDC as a possible mechanism for stress-tresting defence planning scenarios, as well as a campaign planning tool. Given that the Canadian military is most likely to deploy as a “plug-and-play” component of much larger coalition efforts, other tools (such as the Peace Support Operations Model) have proven less useful for this task. As a member of the playtest group (and the nefarious Red commander), I had been impressed by RCAT’s flexibility and adaptability. Interestingly it sometimes uses a matrix game-like approach to resolving contextual circumstances and other issues that lie outside the normal rules, including those at the “fuzzy edge” of wargaming (ie, non-kinetic dynamics).

Next, I offered a slightly revised version of the presentation I made a few weeks ago at RAND on “Gaming the Semi-Cooperative: Peace Operations, HADR, and Beyond.” My key argument here was that extrinsic rewards and victory conditions were not the sole method to elicit semi-cooperative behaviour among players in a game. Player psychology is also important, and players can be influenced in a variety ways so as to introduce friction and tension in otherwise cooperative games, or to encourage a degree of cautious cooperation in otherwise largely adversarial ones.

Jim Wallman (Past Perspectives) offered some insightful remarks of his own on “wargaming for insight.” The key elements of such gaming, he argued, were the developmental or analytical requirements; relevant scenarios/vignettes; adoption of an appropriate game type and structure; resolution of the inevitable tensions between the time needed, the time available, and the time actually spent on a wargame; skilled facilitation; effective recording and reporting of the game; and post-game analysis.

With regard to the former he emphasized the need to define the scope of the investigation, and clarity as what was to be stress-tested or compared. He warned against the “kitchen sink” syndrome where sponsors attempt to encumber a wargame with too many components or questions. He warned against using off-the-shelf scenarios, or reusing those from different games, as they were rarely as effective as those that had been purpose-designed. Indeed it was important to recognize that insights were usually scenario-dependant. Game design and implementation should reduce the temptation and ability of players to “fight the white (cell)” by arguing against the scenario, rules, or adjudication. Game structures could be open or closed, rigid or free, and manual, digital, or a synthesis of both. Throughout he stressed that wargaming, while a very useful analytical tool, was not always the best tool to explore a particular issue—and that game designers needed to be clear about this.

A group lunch at Nandos was then followed by a general discussion of gaming issues, followed by a demonstration session of AFTERSHOCK: A Humanitarian Crisis Game. Here the players found the first days of the earthquake extremely challenging, with their collective (relief point) score approaching -20. However, by the second week of the disaster their position had improved considerably due to increasingly effective interagency coordination. The HADR Task Force and the government of Carana took the lead in repairing the port and airport, thus enabling a greater volume of relief supplies to flow into the country. Social unrest proved to be very limited, and was dealt with through community mediation and the occasional government security operation. The UN and NGO teams had begun to repair critical infrastructure, and thereby support a transition from emergency relief to early, sustainable recovery. We had to stop the game before it was finished, but the players seemed well on their way to a successful humanitarian assistance operation.

20160222_140637[1].jpg

All-in-all, I thought it was a very successful day. There seemed to be considerable support among participants for making Connections North a somewhat larger annual event that brings together an interdisciplinary group of those involved in the serious application of national security gaming methods in Canada. In the coming months the next steps will be to determine the best time of year for this, expand the network of serious gamers who could be involved, and to find an institution (ideally in the Ottawa area) to host such an event. If you’re interested in becoming involved, drop me a line.

%d bloggers like this: