DRDC: Investigating wargaming and capability based planning

Back in January, Murray Dixson and Fred Ma at Defence Research and Development Canada published a valuable paper entitled An Investigation into Wargaming Methods to Enhance Capability Based Planning. This is a summary of a DRDC project in which “Several wargaming methods are investigated as possible ways to enhance the Capability Based Planning (CBP) process in the areas of capability requirements identification and force planning scenario validation”

The concept behind the methodology was to begin the exploration of wargaming methods with several game styles chosen based on the knowledge and experience of the SPORT [Strategic Planning Operational Research Team] staff, and in parallel, build familiarity over the period of the work with wargaming methods through literature reviews and exposure to the wargaming community. The relevant wargaming community exists primarily amongst the five eyes allies through forums such as (but not limited to) the “Connections” wargaming conferences and the Military Operations Research Society (MORS) workshops and conferences [16][17].

The knowledge gained from the literature and the wargaming community influenced possible adjustments to the methods chosen as the explorations proceeded. There was an implicit feedback loop in place where the knowledge and experience gained from each game iteration was used to decide on the next steps of the testing which involved either running more iterations of the game or choosing another game type.

Ideally, the methodology was intended to include these elements:

1. Try out several game styles to see if any had any particular advantages for CBP in terms of the issues observed and discussed in the previous section
2. Run several iterations of each game style to generate some (limited) statistics from which informative or indicative trends might emerge
3. Run multiple game iterations with different groups of players to try and obtain a broader range of inputs and experiences with playing the games
4. Compare the wargame outputs to those from the JCPTs [Joint Capability Planning Team] and draw conclusions about whether the wargaming methods tested showed any potential benefit for addressing the issues discussed earlier with CBP

The games used in the study included ISIS Crisis and several other matrix games; the Rapid Campaign Assessment Toolset (RCAT, developed by the UK Defence Science and Technology Laboratory and Cranfield University); and PAXsims’ very own AFTERSHOCK: A Humanitarian Crisis Game. It should be noted, of course, that  these were not necessarily designed to support Capability Based Planning—rather, DRDC was playtesting them for more basic insight as to whether a particular sort of game approach might contribute to capability requirements identification and force planning scenario validation.

The study’s conclusions were as follows:

In this work, we explored several wargaming techniques as a way to mitigate some of the concerns and observations originating from previous cycles of CBP. There was concern that the capability analyses in Phase 2 could become too narrowly focussed and that the planning scenarios may not be adequately validated before the JCPT capability analysis. The properties of table top wargames lend themselves to addressing problems like this because they immerse players in the scenario in a way not possible by simply reading a scenario narrative from a document. The resulting experiential learning provides a richer experience for the player, which translates into a more holistically encompassing view and understanding of the key elements of the scenario.

We therefore investigated the potential of tabletop wargames to enhance the identification of capabilities under CBP and to act as a scenario validation mechanism. We developed a hypothesis that wargaming would enhance capability identification through the broader understanding of the scenario and also through a better understanding of the driving factors therein.

The number of game styles and trials was limited, as was the range of backgrounds of the players that were able to participate. Nonetheless, three styles of game (a traditional tabletop game (RCAT), matrix games and a commercial game) were examined, and nine trials spread across them were run.

From the limited data, we saw clear indications that playing the games would be an effective way of checking the consistency and believability of a scenario, thus providing a validation mechanism.

The data indicated that by playing the games, players felt that they had a better sense of the key drivers of the scenarios. We interpreted that finding to be an indication that wargaming, through the concept of experiential learning, could provide JCPT members with a broader understanding of an FDS which logically should reduce the risk of them becoming too narrowly focussed in their capability assessments. However, the results were assessed as inconclusive with respect to the application of wargaming to directly identifying capability needs. A key contributor to this result was that we were not able to test a game that was properly designed for that purpose, as there wasn’t enough time and people to do a best-practices “design-test-run” game development cycle.

Although the time and personnel resource constraints did not allow for testing additional game styles or running more iterations, it was clear that the game styles tested are feasible and reasonable choices for future use, given the levels of resources that are available now and that are expected in the future. Similarly, it was not possible to conduct a comparison of the utility of the wargaming method with that of the current CBP model, which employs the OPP; hence, this too remains an open question.

At the level of granularity of strategic planning, tabletop wargaming has potential as a supporting element of scenario design and qualitative analysis. It brings together the knowledge embedded in the game design with the knowledge of participants (players, facilitators/adjudicators, control personnel) in order to explore plausible courses of events resulting from decisions made from the perspectives of the various roles. It can reveal nuances that are not obvious from scenario descriptions alone, and can illuminate relevant and driving factors associated with a scenario. It is not a method to statistically characterise selectively defined quantifications for a scenario.

To summarise the findings above, although the data from our trials is rather limited, we believe that there is evidence to support the hypothesis that wargaming can improve scenario validation. However, we judge the data to be inconclusive on whether it would improve capability identification.

Organising wargames always requires time and people resources. The amount is dictated by the scale and complexity of the game. The present work demonstrated that games suitable for the tasks in this project are feasible within our resource constraints. However, obtaining players on an ongoing basis, new or not, can present a challenge for which senior leadership support will be needed if it is to be met. Despite these challenges, a reasonable selection of game styles and scenarios was explored in this work and we had success (albeit limited) with bringing in players with diverse backgrounds.

A strength of wargaming is that game styles can be highly varied to suit different purposes, e.g., exploring plausible outcomes, scenario validation, familiarising planners with driving factors and training. In our trials, wargaming illuminated the dynamics that drive a scenario, uncovered logical inconsistencies (applicable to scenario validation), and sometimes revealed an unanticipated course of events. Applying wargaming to scenario validation should improve scenario design, while the insights into the driving factors would inform JCPT analyses, e.g., formulation of CoAs, required capabilities, MoC assessments, and capability delivery options.

In the end, this work revealed that none of the game designs trialled were optimally suited for explicit identification of capabilities. In the designs trialled, capability requirements were mostly implied by player CoAs and would have to be explicitly identified via a post hoc analysis. Although the Baseline matrix game did explicitly ask players about capability requirements, the ones identified were, in our view, not significantly different from those observed from a JCPT analysis; hence, the game design would need changes to guide players toward more innovative ideas.

We note that capabilities at tiers 2 and 3 in the JCF are defined to be generic, technology-independent, and thus enduring. Hence, it may be uncommon to identify wholly new capability requirements at that level, though any that are identified would certainly be informative. We expect that these would likely arise from an unanticipated course of events and/or CoAs. In contrast to tiers 2 and 3 capabilities, we noticed that players often thought in terms of, and identified, finer grain capabilities more readily (i.e., those tied to specific technologies, platforms, or assets), along with circumstantial factors that determine what capabilities make sense, at least in part.

Based on the limitations experienced in executing our wargames, it is possible that identification of CoAs and capabilities can be improved by mixing players from diverse backgrounds to enhance synergy, e.g., including relevant geopolitical expertise, operational experience, and futurist backgrounds. More open-ended games, e.g., Matrix style, have the flexibility to leverage the greater knowledge and ideas. In view of the challenges in obtaining players, this could be treated as an ideal to capitalise on, if it can be realised, and if it makes sense for the particular CBP phase in which a wargame is conducted.