Banks: The Methodological Machinery of Wargaming

David Banks (King’s College London) has just published an important piece on “The Methodological Machinery of Wargaming: A Path toward Discovering Wargaming’s Epistemological Foundations” in International Studies Review 26, 1 (March 2024). How is it we know what we know about wargaming, and how do we understand the claims of knowledge or insight that wargaming purports to offer?

This paper proposes a comprehensive research program for determining the epistemological foundations of analytic wargaming. Wargaming has been used in military, government, and private sectors for decades, with tens of millions of dollars spent annually on it. In light of the changing strategic circumstances of the twenty-first century, it has only become more popular. However, the epistemological foundations of the method are poorly understood. Many professional wargamers contend that wargaming is an “art” and thus unable to be systemically evaluated. Recent work by a small coterie of international relations scholars has contended that wargaming can be reconciled with social science, typically by evaluating wargaming according to experimental standards. However, this solution strips wargames of most of their unique features and cannot explain why some of the most prominent wargames in history produced meaningful results. In this paper, I argue that in the attempt to better understand wargaming’s epistemology, scholars should begin by recognizing the prominent features of wargames and research each of these to determine if and how wargames produce rigorous knowledge. In making this argument, I identify five distinct “methodological machineries” of wargaming—the recurring processes through which wargames may produce knowledge—that distinguish wargaming from other social science methods: (i) they are representative, (ii) they feature consequential decisions made by human players, (iii) they are adjudicated, (iv) they are immersive, and (v) they are bespoke designs. I show how each of these machineries offers potential opportunities and dangers in the production of knowledge through the method of wargaming. In outlining these distinct features, I offer a clear and viable research program for epistemologists of wargaming.

The full article is at the link above. It’s a terrific piece—indeed, my only quibble is that it perhaps dispenses a little too quickly with mixed methods and research triangulation (Perla’s “cycle of research”), which provides a possible source of validation even when the “epistemological foundations” are imperfectly understood.

“Wargaming doesn’t work”

Earlier this month the Washington Post published a lengthy, two-part article on planning and execution of Ukraine’s 2023 offensive (here and here). In these they explore differences between the US and Ukraine over how narrow the primary focus of the planned offensive should be, issues regarding weapons supply and training, the challenge of breaching defensive positions and fortifications, and the effect of UAVs, mines, plentiful ATGMs, air power, and other weapons systems on the modern battlefield.

The articles also address wargaming. According to the Washington Post,

The sequence of eight high-level tabletop exercises formed the backbone for the U.S.-enabled effort to hone a viable, detailed campaign plan, and to determine what Western nations would need to provide to give it the means to succeed.

“We brought all the allies and partners together and really squeezed them hard to get additional mechanized vehicles,” a senior U.S. defense official said.

During the simulations, each of which lasted several days, participants were designated to play the part either of Russian forces — whose capabilities and behavior were informed by Ukrainian and alliedintelligence — or Ukrainian troops and commanders, whose performance was bound by the reality that they would be facing serious constraints in manpower and ammunition.

The planners ran the exercises using specialized war-gaming softwareand Excel spreadsheets — and, sometimes, simply by moving pieces around on a map. The simulations included smaller component exercises that each focused on a particular element of the fight — offensive operations or logistics. The conclusions were then fed back into the evolving campaign plan.

Top officials including Gen. Mark A. Milley, then chairman of the U.S. Joint Chiefs of Staff, and Col. Gen. Oleksandr Syrsky, commander of Ukrainian ground forces, attended several of the simulations and were briefed on the results. …

Ukrainian officials hoped the offensive could re-create the success of the fall of 2022, when they recovered parts of the Kharkiv region in the northeast and the city of Kherson in the south in a campaign that surprised even Ukraine’s biggest backers. Again, their focus would be in more than one place.

But Western officials said the war games affirmed their assessment that Ukraine would be best served by concentrating its forces on a single strategic objective — a massed attack through Russian-held areas to the Sea of Azov, severing the Kremlin’s land route from Russia to Crimea, a critical supply line. …

The rehearsals gave the United States the opportunity to say at several points to the Ukrainians, “I know you really, really, really want to do this, but it’s not going to work,” one former U.S. official said.

At the end of the day, though, it would be Zelensky, Zaluzhny and other Ukrainian leaders who would make the decision, the former official noted.

Officials tried to assign probabilities to different scenarios, including a Russian capitulation — deemed a “really low likelihood” — or a major Ukrainian setback that would create an opening for a major Russian counterattack — also a slim probability.

“Then what you’ve got is the reality in the middle, with degrees of success,” a British official said.

The most optimistic scenario for cutting the land bridge was 60 to 90 days. The exercises also predicted a difficult and bloody fight, with losses of soldiers and equipment as high as 30 to 40 percent, according to U.S. officials.

The numbers “can be sobering,” the senior U.S. defense official said. “But they never are as high as predicted, because we know we have to do things to make sure we don’t.”

U.S. officials also believed that more Ukrainian troops would ultimately be killed if Kyiv failed to mount a decisive assault and the conflict became a drawn-out war of attrition.

But they acknowledged the delicacy of suggesting a strategy that would entail significant losses, no matter the final figure.

“It was easy for us to tell them in a tabletop exercise, ‘Okay, you’ve just got to focus on one place and push really hard,’” a senior U.S. official said. “They were going to lose a lot of people and they were going to lose a lot of the equipment.”

Those choices, the senior official said, become “much harder on the battlefield.”

The Washington Post goes on to note Ukrainian dismay with wargaming as a military planning tool.

On that, a senior Ukrainian military official agreed. War-gaming “doesn’t work,” the official said in retrospect, in part because of the new technology that was transforming the battlefield. Ukrainian soldiers were fighting a war unlike anything NATO forces had experienced: a large conventional conflict, with World War I-style trenches overlaid by omnipresent drones and other futuristic tools — and without the air superiority the U.S. military has had in every modern conflict it has fought.

“All these methods … you can take them neatly and throw them away, you know?” the senior Ukrainian said of the war-game scenarios. “And throw them away because it doesn’t work like that now.”

There are several important take-aways here.

Wargaming can fail—and fail badly—when either data or embedded models are wrong. Perhaps, as some Ukrainians suggest, the wargame failed to take account for the dramatic increase in ISR capabilities that tactical drones can provide, or their use as a weapon system? Or perhaps game assumptions about breaching operations were based on less dense defences, a less skilled defender, or a better-trained and better-equipped attacker? Or perhaps the problem was that Russia’s real world response differed from the way RED responded in games and simulations?

Wargamers like to say that wargames aren’t predictions, and it is certainly true that no wargame nor series of wargames can fully address all assumptions and all choices, and hence explore all of a given problem space—even when the underlying data and models are correct. However, in many ways this caveat is also an evasion. Wargames are often asked to anticipate likely outcomes or responses. That is indeed a form of prediction, even if it is highly contingent and rests on sometimes shaky foundations. And military decision-makers may treat wargames as a sort of crystal ball, regardless of whatever caveats are attached.

The fact that the offensive didn’t unfold as the wargames suggested also may have nothing to do with the wargames—which might have been excellent—but rather divergence between the plan that was gamed and the plan that was executed. We have no way of knowing, for example, whether Ukraine would have been more successful if it had concentrated its forces in a single major thrust as the US and UK apparently preferred. (It should be noted that Ukraine’s multi-prong approach was also driven by political concerns—which a wargame may well not have addressed.) After all, it’s hardly an uncommon human response for a user to blame their tool for disappointing results.

Finally, it is possible that all of these factors, and others beside, may have been at work. Professional wargaming is full of historical lore about the successes and failures of major wargames, recounted in presentations, conversations, and conferences: the gaming of the Schlieffen Plan (1914), Pearl Harbour (1941), and Midway (1942), the interwar wargaming of the US Naval War College, the work of the Western Approaches Tactical Unit during WWII, or the problems encountered by Millennium Challenge (2002). In almost all cases, the reasons for success or failure were far more complex than the lore suggests, or the success or failure was much less clear and absolute than makes for a good story.

Rarely does either uncritical (war)gaming evangelism or knee-jerk methodological cynicism illuminate what happened, why, or what we can do better. Instead, what is required is a long and detailed examination (internal, and inevitably classified) review that accounts for complexity and multi-causality. One hopes that the Ukraine wargames of 2022 will receive just such a thoughtful, sober, and constructive review. There is likely much to learn.

Ethics compliance and wargaming

The following item was written for PAXsims by Ivanka Barzashka. Dr. Barzashka is the CEO and co-founder of Strand Analytica, a British-American tech startup dedicated to powering the emerging science of wargaming through technology for national security and defence applications. She was a founding director of the Wargaming Network at King’s College London, serving as its co-director (2018-2019) with Professor Philip Sabin, director (2019-2020) and managing director (2020-2022) working with Dr David Banks as academic director. 

---

A recent Freedom of Information Act (FOIA) request revealed a surprising lack of scientific and ethical oversight over analytical wargames, which inform UK defence decisions. According to the MOD, only a single study was submitted for Ministry of Defence Research Ethics Committee (MODREC) review over the past five years.  

This is not a uniquely British problem. The 2023 King’s College London survey of over 140 wargame designers across 19 countries suggests that 80 percent bypassed ethics reviews of their analytical wargames, disregarding norms for research studies involving human participants. 

MOD Ethics Requirements 

The MOD mandates MODREC ethics review for all studies that meet three criteria: (1) are MOD-funded or involve MOD-funded staff or participants, (2) are research defined as “the attempt to derive generalisable or transferable new knowledge to answer or refine relevant questions with scientifically sound methods”, and (3) gather data from human participants. 

According to these criteria, all analytical wargames that are sponsored by the MOD or involve MOD participants should undergo ethics review. Low risk wargaming studies still require the MOD sponsor to submit a MODREC application, but this will trigger a “proportionate review process allows for a rapid (10 working day) turn-around,” as opposed to a full committee review.  

All applications go through a two-step process. First, they undergo a scientific review “to ensure that the research is properly designed and planned” and ensure the “quality research will also be safe, ethical, legal and feasible.” This assessment is done by a Scientific Assessment Committee. If this review is successful, the application proceeds to an ethics review during a full meeting of MODREC, except where there is “minimal risk, minimal burden and minimal intrusion for research participants” and “minimal risk to researchers”. In these circumstances, applications are reviewed “by a sub-committee rather than at a full meeting of MODREC”. 

However, the FOIA response suggests most MOD analytical wargaming studies fail to comply with these guidelines. 

The Need for Ethics Compliance 

Analytical wargaming should adhere to established academic integrity and ethics standards. These principles include mitigating risks, ensuring informed consent and privacy, and promoting participant welfare. Independent oversight provides accountability for these protections. Institutional research ethics committees help assess and address these risks. 

The single 2021 application released under our FOIA request provides a valuable example of MODREC’s rigorous scrutiny of an observational wargaming study. Before granting an approval, the committee asked the researcher to improve their protocols around participant exclusion criteria, the consent process, data usage policies, and risk mitigations. This enhanced protections for the study participants. 

Why Has Wargaming Lacked Oversight?  

For one, authoritative guidelines, like NATO’s wargaming handbook released in September 2023, and the UK’s 2017 variant, do not mention ethics requirements. Additionally, government institutions that sponsor these games often do not mandate formal compliance with research ethics standards. Furthermore, ethical reviews can be time-consuming because they require extensive documentation and lengthy approval processes, which can conflict with the pressing timelines of policy decisions. 

Conclusion 

Strengthening protections for human participants must remain a top priority. As universities and militaries conduct more wargaming research, robust, independent ethics review will be essential to uphold integrity and minimize risks. The current oversight gap highlights the pressing need to reinforce MODREC and expand review of defence wargaming studies.

20231023-foi2023_11650_mod_response_letter-v3_redactedDownload

Tweedy and McKechnie: Assessing wargaming at the company level

At The Maneuverist blog of the Warfighting Society, Matthew Tweedy and Taylor McKechnie have written a very important piece that seeks to empirically assess the value of relatively simple wargaming as a training tool.

With the extensive focus, resources, and discussions dedicated to wargames in the US military, the DOD should prioritize determining wargaming’s tangible impact on training and readiness. An experiment conducted by the 2d Battalion, 2d Marine Regiment (2/2), in December 2019 offers an insightful case study.

Two-Two conducted the experiment at the Camp Schwab Beachhead Club in Okinawa, Japan. It marked the culmination of seven months of preparation across various countries and combatant commands. It attracted the attention of senior officers in divisions, Marine Expeditionary Forces, and Headquarters Marine Corps. A group of 37 Marines, spanning lance corporal to sergeant, and representing military occupational specialties (MOS) from combat arms, communications, intelligence, and logistics, played in a March Madness-style tournament. The purpose was to determine the best Memoir ’44 player in the battalion and to make an empirically inspired assessment of board games as training tools.

…

Thirty-six Marines and one Sailor played in the March Madness-style tournament. (The odd number of participants resulted in an early round bye and a latter-round semi-final “play-in.”) Matches consisted of three games. To maintain tempo, each game lasted no more than 30 minutes. The bulk of collected data comes from the early rounds of the tournament (more players, more matches, more games). To control for variables, all participants played the same scenario during each round.

Players and observers recorded match data. The players completed data sheets and used stop-watches to record turn time, turn action, and game summaries. Observers evaluated performance in real-time (not post-game) using performance indicators like risk assessment, enemy analysis, and terrain use, concluding each evaluation with written comments. In addition, all participants completed a survey about their opinions of using games – both video and board games – as professional military education (PME) tools. These efforts resulted in over 14,000 collected data points.

…

We found that board games provide a decision-rich experience. On average, each player made roughly 42 tactical decisions an hour. We consider a “decision” an active game action, such as moving, attacking, or using a Command Card. Our tournament lasted 12 hours, and most players made between 68-204 decisions. Half of the players were eliminated in 1.5 hours and made 42-135 decisions. Several players who made it to the latter rounds made over 500 tactical decisions.

The most notable difference between winners and losers was their tempo of decisions. As the median turn times in Figure 1 show, victors unambiguously out-cycled their opponents. The faster players moved on to later rounds, and the fastest advanced to the final rounds.

…

While our participants played the game, we assessed their tactical decision-making ability in six areas: gap analysis, enemy analysis, use of key terrain, understanding purpose, ability to sequence actions, and risk analysis. Results are shown in Figure 2. The assessment criteria are derived from two concepts familiar to all Marines: the framework of METT-TC (mission, enemy, terrain, troops available, time, and civilian considerations) and the OODA (observe, orient, decide, act) Loop.

Figure 2 shows a clearly detectable and statistically significant difference in decision-making trends between victors and losers. As the tournament progressed, the significance of a single determining factor for victory diminished. We assert this shift can be attributed to the proficiency of the later-round competitors in employing the OODA cycle and METT-TC analysis. But notably, the skill of discerning enemy surfaces and gaps retained statistical significance until the fourth round.

A takeaway from Figure 2 is that personnel who do not grasp these concepts can be easily identified in a short period (60 minutes or less). Additionally, the fact that these concepts were less significant in later rounds does not indicate the game is a poor teaching tool. Rather, it suggests these concepts can be trained using the game and that small failures and setbacks become more significant as the competition increases. For instance, although we judged our later round participants as generally astute at applying the OODA/METT-TC concepts, small slip-ups came with high costs since their competitors were also good at applying those same concepts.

Finally, while certain assessment feedback downplayed the importance of a competitor’s risk assessment ability, an examination of Figure 2 unequivocally demonstrates that an inadequate risk assessment guarantees defeat.

As Yuna Wong once noted, there are striking few robust empirical studies that seek to assess whether (and how) wargaming actually works. This is a rare exception, and a substantial contribution to the broader professional literature.

Williamson: Generative AI, wargaming, and PME

The following was written by Kevin Williamson. He is a wargame SME at Marine Corps University under Tim Barrick and has been involved in Professional Military Education for the past year. His background includes time served as a Logistician in the US Army and as a Gunner’s Mate in the USN Reserves. Kevin Williamson also volunteers his free time to help run USA Fight Club and recently became part of DSET 2024’s International Committee.

---

Generative AI has taken the world by storm in the past year and wargaming in the Marine Corps has a place for it. This report is the product of an idea that Large Language Models, if developed properly can augment the wargaming industry and Professional Military Education. The LLM used in the report is going public towards the end of this summer and has been developed specifically for DoD wargaming applications. The report is based on a test conducted using the software Command: Professional Edition as the feedback tool for the Large Language Model and is a proof of concept that Generative AI can assist in educating our warfighters on a wide variety of topics not specific to their job.

1690153589342Download

UK MoD: Influence Wargaming Handbook

The UK Ministry of Defence (Development, Concepts and Doctrine Centre) Influence Wargaming Handbook is now available.

The Handbook is divided into four chapters and an annex:

For sponsors

Chapter 1 – Why wargame influence? This chapter describes influence and wargaming. It explains why influence is important, and why wargaming is particularly suited to exploring and representing influence.

Chapter 2 – Sponsoring influence wargames. This chapter outlines the various factors that influence wargame sponsors must consider. It then highlights key risks associated with wargaming influence and suggests how to manage and mitigate these risks.

For practitioners

Chapter 3 – Challenges to wargaming influence. This chapter outlines the challenges facing influence wargaming practitioners. These are explained by illustrating the differences between conventional and influence wargames.

Chapter 4 – Addressing the challenges to wargaming
influence. This chapter suggests how the challenges raised in Chapter 3 might be addressed.

Annex A – Case studies. Annex A presents recent case studies that illustrate how wargaming methods and techniques have been applied to influence-related defence and security problems.

Return to Portsdown West

The following post has been cleared for release by the Defence Science and Technology Laboratory.

AI (DALL-E2) interpretation of my return to Dstl Portsdown West.

In early May I was fortunate to spend four days at the Defence Wargaming Centre of the UK Defence Science Technology Laboratory (Dstl), at Portsdown West near Portsmouth. I had been on similar visits before (in 2016, 2017, and 2018) but this had been followed by a hiatus due to COVID and other factors. It was good to be back, and I’m very grateful to everyone at Dstl who organized and supported the visit.

On the Tuesday and Wednesday I offered a series of lectures on “Mishaps and Minefields in Professional Wargaming,” which examined common mistakes we make and why we make them. These were primarily intended for newer analysts, although several more experienced wargamers participated and contributed to our wide-ranging discussion.

• Mishaps and Minefields I: So You Think You Need a Wargame?
• Mishaps and Minefields II: Game Development
• Mishaps and Minefields III: Participants and Resources
• Mishaps and Minefields IV: Game Control, Adjudication, and Facilitation
• Mishaps and Minefields V: Data Collection and Analysis
• Mishaps and Minefields: Mea Culpa

The final session involved me recounting decisions I now regret and mistakes I’ve made personally while designing and running serious games. Critical reflection is important, after all!

The slides for all of these sessions are below. Like the image at the top of this report, most of the artwork for the slides was generated by AI (and some of it is quite amusing).

dstl-2023-days-1-2Download

The Thursday and Friday of my visit involved more informal discussions of key topics, such as adjudication and end-to-end analysis. There were also playtest sessions of two games being developed as part of Dstl’s EAD (Explore, Anticipate, and Develop) Project. The first was a full-featured modular grand strategic game system that can be adapted to a variety of questions and scenarios.

The second was a much simpler “strategic game in a box” (Contested) to introduce the possibilities of strategic gaming (in much the same way that the Dstl-sponsored Matrix Game Construction Kit was designed to help jump-start matrix gaming in organizations). Both are very promising, and the latter in particular benefits from a very intutive game system that presents few barrier to adoption and play.

On a related note, in 2021 Dstl published a lengthy (165 page) paper on How Can Dstl Expand Our National Security Gaming Toolset To Generate More Meaningful And Reliable Insights? (DSTL/PUB131779 1.4) which provides some really thoughtful discussion of the methodological challenges in strategic gaming. Dstl has now cleared this report for public release, so I’ve just shared it as a separate post on PAXsims.

We also had a session on the design of We Are Coming, Nineveh!, followed by three simultaneous games. The latter went very well, with everyone quickly learning the game system. Daesh seemed to have the best of it in all three wargames, perhaps because the ISF was overly cautiously a little slow to find and fix the enemy—who, after all, were playing for time before they inevitably lost control of West Mosul. I was pleased to learn that the game is to be used in the Defence Academy of the UK to teach about modern urban warfare

Friday afternoon involved a lengthy (but highly enjoyable) drive up to Liverpool, where the Western Approaches HQ Museum hosted Dstl, Royal Navy, and other personnel for a convoy wargame based on the work of the famed Western Approaches Tactical Unit during World War Two. Kit Barry, who organized the event, has already written about the game preparations and the outcome. As captain of fictional Type-VII U-boat U2, I was quite pleased with my result: three merchantmen sunk, followed by a stealthy exit (despite one corvette doggedly trying to find me with ASDIC).

Overall, I had a terrific time—it was both professionally rewarding and very fun. There has been quite a few changes since my earlier visits: the establishment of the DWC, more facilities, and a very substantial growth in the number of Dstl analysts now supporting defence wargaming acriss the UK Ministry of Defence. The United States, of course, remains the preeminent wargaming “superpower” in NATO. Indeed, the budget for the new US Marine wargaming center alone likely exceeds the wargaming resources of most other NATO members combined. However, the UK has clearly consolidated its position as a leader in the field, as evidenced not only by Dstl’s expanding activities but also by the 2017 publication by the Development, Concepts and Doctrine Centre of the Defence Wargaming Handbook as well as the forthcoming Wargaming Influence Handbook. It has developed this capacity, moreover, from a position of greater resource scarcity relative to the US. In that sense I think it is sometimes better attuned to the challenges faced by small and medium-sized NATO militaries. It is also geographically closer to most of them, and the annual Connections UK wargaming conference always has strong representation from other European countries.

I hope that Dstl and others across the UK Ministry of Defence will continue to leverage these strengths to play a leading role in mentoring, supporting, and partnering with wargaming initiatives by allies and partners. They have a a great deal to contribute.

It should also be added that Dstl has been an early and avid supporter of the Derby House Principles on diversity and inclusion in professional wargaming. The effects of this can be increasingly seen in their team of analysts, how they approach their work, and the powerful synergies that arise from harnessing multiple experiences and perspectives. Here too, others have much to learn from their example.

Dstl: Expanding national security gaming to generate more meaningful and reliable insights

The following report has been cleared for release by the Defence Science and Technology Laboratory .

---

In May 2021, the UK Defence Science and Technology Laboratory published a lengthy (165pp) report entitled How Can Dstl Expand Our National Security Gaming Toolset To Generate More Meaningful And Reliable Insights? This addresses a broad range of related issues, including the experiential value of games, identifying genuine insights (as opposed to artifacts of the game design), and post-game analysis.

• Section 1 – Introduction
• Section 2 – How Is An ‘Analytical Game’ Defined?
• Section 3 – How Can We Develop Creating Knowledge Games That Are More Analytical?
• Section 4 – How Can We Conduct More Analytical Games Within TheConstraints Of Engaging Very Senior Players?
• Section 5 – How Can We Encourage More Representative Red Cell ResponsesTo Blue Cell Actions?
• Section 6 – Proof of Concept Escalation Dynamics Game and Concept of Analysis
• Section 7 – Conclusions and Recommendations
• Section 8 – Closing Summary

This report has now been approved for general public release, and can be found in its entirety below (DSTL/PUB131779 1.4).

28052021-dstl_pub131779-how_can_dstl_expand_our_national_security_gaming_toolset_to-generate_more_meaningful_and_valid_insights-v1_4_-oDownload

Wnorowski: Wargaming Practitioners Guide

The Doctrine and Training Centre of the Polish Armed Forces has just published a very useful Wargaming Practitioner’s Guide, written by Mirosław Wnorowski. The English version is shared below. The book explores:

• The essence and objectives of wargames (including definitions, benefits and limits, history, and a link to the key elements and dilemmas of game theory).
• The use of wargames (in the armed forces, as an element of planning, and a classification of types).
• The key elements (participants, scenario, adjudication, data collection).
• Game tools (space, time, actors and the interaction between them).
• The process of preparing and executing a wargame (including particular attention to seminar and matrix games).

Excellent work, Mirosław!

bwargaming_en-online_print_offDownload

Majnemer and Meibauer: Fictitious country names affect experimental results

Jacklyn Majnemer (MIT) and Gustav Meibauer (Radboud University Nijmegen) have published a very interesting article in International Studies Quarterly, 7, 1 (March 2023) exploring whether fictitious country names in survey vignettes affect experimental results. The answer: yes they do.

Using fictitious country names in hypothetical scenarios is widespread in experimental international relations research. We survey sixty-four peer-reviewed articles to find that it is justified by reference to necessary “neutralization” compared to real-world scenarios. However, this neutralization effect has not been independently tested. Indeed, psychology and toponymy scholarship suggest that names entail implicit cues that can inadvertently bias survey results. We use a survey experiment to test neutralization and naming effects. We find not only limited evidence for neutralization, but also little evidence for systematic naming effects. Instead, we find that respondents were often more willing to support using force against fictitious countries than even adversarial real-world countries. Real-world associations may provide a “deterrent” effect not captured by hypothetical scenarios with fictitious country names. In turn, fictionalization may decrease the stakes as experienced by respondents. Researchers should therefore carefully explain rationales for and expected effects of fictitious country names, and test their fictitious names independently.

In Table 2 below you can see that respondents were more willing to use military force against “Celesta,” “Drakhar,” or “Minalo” than they were either a friendly real country (Canada) or a hostile one (Iran).

The research here focuses on survey responses, not serious game play. However the findings may have some interesting implications for strategic-level wargames using fictional country names, which may be more prone to escalation than similar games using real countries.

Interestingly, the authors also suggest that the more “real” a country sounds, the less fictionalization effects are evident:

Our results suggest that the more clearly fictitious a country name, the easier to condone attacking it—fictionality and its perceived costlessness can therefore embolden respondents to provide more aggressive responses.

These results point to the relevance of perceived realistic-ness: the more “real” a country name sounds to respondents, the weaker the fictionalization effect. In particular, there seems to be a deterrent effect associated with realistic-ness, for example, of being able to imagine more easily the consequences associated with attacking Iran, especially bar any additional information that “fills out” the scenario. 

The explanation they suggest for this is deterrence: respondents are better able to imagine the costs of an attack when the survey question asks about a real country rather than a fictional one. However, there may also be an empathy factor here—it’s easier to imagine killing and maiming actual Iranians or Canadians than it is “Minalans,” “Brakharis,” or “Celestians.”

In professional wargames, it is sometime necessary to use fictionalized countries, usually because of political sensitivities. In experimental games there may also be a desire to exert better control of key variables than is possible using a real-life settings. Both reasons apply, for example, to a recent series of NATO experimental wargames that examined Intermediate Force Capabilities in a fictional conflict between the Illyrian Federal Republic and Hypatia (the latter backed by Organization for Collective Security).

An unclassified NATO STO SAS wargame in 2022. You’ll note the Illyrian Federal Republic operations orders (OP IRKALLAN FREEDOM), in a conflict that seems rather reminiscent of a real one in some ways, but set in the northern Aegean.

If Majnemer and Meibauer’s findings do indeed expand beyond international relations survey research to wargaming, there are several implications. One is the need to provide game participants with a rich and realistic fictional environment and to work hard to promote narrative engagement. Another is the need to caveat experimental findings, especially as they relate to use-of-force decisions but possibly other things as well, such as risk aversion or casualty sensitivity more broadly.

Heath: Wargames can’t tell us how to deter a Chinese attack on Taiwan—but different games might

Having posted a link earlier today to the launch of a major CSIS report on wargaming a potential Chinese invasion of Taiwan, now is a good time to also flag this thought-provoking piece by Timothy Heath (RAND) a few days ago at the Lawfare blog. In it, Heath argues that kinetic wargames focused on military operations are useful, but they aren’t optimized to tell us how we might stumble into war—or avoid it.

He states (emphasis added):

Wargames that simulate combat between the United States and China near Taiwan can provide useful insight about potential military challenges. However, analysts should be wary of repurposing the same games to explore political questions such as those related to deterrence, escalation control, alliance politics, and war prevention or termination. Asymmetries in the information requirements for political versus military topics make it exceedingly difficult to design games to explore both in a rigorous manner. Paradoxically, the deliberate falsification of facts in peacetime offers the best hope of painting a more vivid and convincing portrait of a situation that would actually confront policymakers in wartime.

Wargames featuring conflict between China, the U.S., and Taiwan have taken the Washington, D.C., area by storm in the past two years. The U.S. military has held classified wargames on the topic. The Center for Strategic and International Studies (CSIS) held 22 iterations of such a scenario, and other think tanks such as the Center for a New American Security (CNAS), CNA, and RAND have held their own wargames on the topic as well. The appeal of wargames is not hard to figure out: They provide a vivid and dynamic simulation of armed conflict. The China-Taiwan war scenario is especially appealing because the U.S. and China are locked in a rivalry and are also equipped with large, advanced, and powerful militaries. What would happen if the two fought is an inherently fascinating question. The U.S. military advantage is fading, and China’s military is growing stronger. But how the two might fare against each other in combat is unclear. Wargames offer the possibility of exploring such critically important topics, whether as part of a research design or as critical context for creative discussion.

The results of the games have generated several key findings. The most obvious and compelling lesson is that combat between U.S. and Chinese military forces would probably be immensely destructive. In the CSIS game, the U.S. lost 200 aircraft, 20 warships, and two aircraft carriers. Attacks on cyber and space infrastructure are not uncommon. U.S. missiles may strike China’s homeland. Both sides might escalate to the threat, or even use, of nuclear missiles, as happened in at least one CNAS game. Analysts have also noted the military implications for operational topics such as the importance of massing forces, adequate munition stores, and the vulnerability of surface ships on the modern battlefield.

But for many, these lessons are not enough. The frightening results of such simulations naturally raise deeper questions of a fundamentally political nature, such as: How can such a war be avoided? If it can’t be avoided, how can escalation in such a war be controlled? What can the U.S. do to deter China from attacking Taiwan? How long can Taiwan successfully hold out against such an attack? Which allies will support the U.S. in such a war? These political considerations permeate the news accounts of the wargames. In the CSIS event, for example, participants debated whether the pre-positioning of marines on Taiwan prior to war would be “too provocative” or not. Players also debated whether China should attack Japan or not. Military decisions regarding escalation also carried significant political considerations, which may or may not have been debated at the game. In one game, for example, players for the U.S. sideauthorized missile strikes on Chinese ports.

Political decisions on the initiation or escalation of war are immensely important. Yet they are also extremely difficult to answer owing in part to the dearth of reliable data. After all, a U.S.-China war remains, thankfully, completely hypothetical. That leaves virtually no firsthand information with which one can answer such questions. Wargames, and the scenarios that underpin them, have sometimes been used to explore such questions. Since they incorporate many facts about relevant combatants, wargames offer the possibility of exploring political as well as military dimensions of war through a structured, analytic method.

Yet analysts should be wary of trying to use wargames designed for military questions to analyze political questions. The analysis of political topics has fundamentally different information requirements than those for military ones. Wargames that support analysis of military decisions do not necessarily support analysis of political decisions in the same situation, and vice versa.

He further argues that efforts to game future crises by tweaking the status quo are inherently problematic:

One way to get around this problem is to incorporate as much of the current world situation as possible into the game scenario and make only those changes needed to introduce conflict. A game designer could create a scenario that depicts U.S.-China relations largely as they exist today and then inject some crisis near Taiwan to begin the war. This is, in fact, the most commonly used method to build “realistic” scenarios for wargames. But a scenario set in wartime that hews to facts as they exist in peacetime introduces a serious analytic error.

The problem is that, by definition, many factors in a peacetime situation favor peace—factors that can be numerous and diffuse. A scenario based on a contemporary, nonhostile relationship between two countries implies many incentives to avoid hostilities. A main reason why the U.S. and China have not gone to war over Taiwan, after all, is because they have many compelling reasons to favor peace. What exactly about the current situation favors peace remains in debate, but candidates include mutual economic interdependence, the presence of nuclear weapons, relatively modestthreat perceptions, and involvement in shared multilateral institutions. Injecting a “trigger event” such as a crisis related to Taiwan does not resolve the structural incentives for peace. Instead, it merely creates an artificial and unconvincing driver of war. Scenarios that aim to explore political topics in wartime but share considerable continuity with peacetime situations are thus inherently contradictory—they depict a situation with as many structural incentives for peace as one that favors war. This contradiction helps explain why so many wargame scenarios strike participants as implausible and unbelievable.

His answer is to build models of crisis escalation that build on historical examples:

A better approach to wargames would be to model the political assumptions for a hypothetical wargame on the experiences of countries that have actually gone to war. As mentioned earlier in this piece, the deliberate falsification of facts in peacetime offers a good model for what might actually happen in wartime and how policymakers would likely react. After all, the most realistic and relevant facts that confront decision-makers in a war are not those that typify situations in peacetime, but those that typify situations in wartime. The very act of envisioning a war situation that does not exist requires the imaginative visualization of a world radically different from a peacetime status quo.

For such historical data to be useful, it should be as rigorous and scientifically derived as possible. The best resource for scenario designers that aim to replicate realistic and relevant facts and incentives for political decisions lies in the historical experience of countries in analogous situations. 

This, of course, is what many international relations scholars do: attempt to create generalized and testable hypotheses from historical data. There are, I think, a number of challenges to this approach too. After all, generalizations are simply tendencies and not iron laws of causality, specific contexts matter, and historical analogies are often misleading because of very different circumstances.

However, good IR scholarship can offer insight into is what sort of factors (political, economic, and otherwise) might shape escalation decisions, and we can then try to model those much as we might model the factors that shape combat outcomes. Certainly the social science here is far from settled or definitive, but the mere process of constructing models forces us to make explicit our assumptions about the way the world (or an adversary) works for further discussion, research, and refinement.

In general we know that subject matter experts are not necessarily very good predictors of the future (in fact, they’re quite poor at it), in part because a tendency to be cognitively over-attached to favoured paradigms. We also know that intelligence communities often outperform other forecasters, not so much because of access to classified material (although that can be a factor) but also because recruitment tends to prioritize cognitive characteristics associated with better forecasting performance, and because a well-developed analytical process emphasizes training and methodologies to check cognitive biases while encouraging constructive challenges to assumptions and interpretations. As an academic who has worked as an intelligence analyst (and assessed the predictive accuracy of other analysts), these skills are NOT ones they teach in political science (or international relations or security studies) graduate school. My impression is that they are even less present in most PME programmes.

Heath ends his piece with an important warning about the dangers of hubris and the value of humility:

Even with such improvements, however, humility about what we can achieve is required. Re-creating hypothetical war situations based on the experiences of past wars will be imperfect at best and carry their own flawed assumptions. Carrying out different iterations with slightly different assumptions could help mitigate some of these limitations. Yet even in the most optimal case, we can at best aspire to craft a crude simulacra of the incentives and factors leaders might confront in a hypothetical situation that will carry all sorts of unimaginable complexities. Given the stakes involved, even an imperfect and partial approach offers a potentially significant improvement over current methodologies for defense planners, analysts, and decision-makers alike who seek to explore political questions in wartime.

On this I think we can all agree.

Bae: A tale of many kinds of wargames (for many kinds of purposes)

Yesterday we posted a link to Jon Compton’s War on the Rocks piece “A Tale of Two Wargames.” Today, Sebastian Bae (CNA) adds some thoughts of his own in a series of Twitter posts.

Let's talk #professional wargaming. Buckle in. 🔥

First, there are some valid underlying points in this article. Small-scale multi-method, iterative analytical #wargames are great. No doubt. I even agree that we should do more of them. 1/https://t.co/Y8Nt8TvLAh

— Sebastian Bae (@SebastianBae) September 24, 2022

He correctly notes that wargame sponsors may have rather different objectives, and that there are many different game approaches depending on those purposes. As a result, he suggests, “not every game will need the kind of approach outlined in this article.” He goes on to add “this binary between multi-method games and ‘event wargames’ is misleading. Some sponsors need event wargames for LOTS of people because they want to socialize an idea or get stakeholders together. Or have a specific timeline. There is no ‘ideal’ perfect form for a wargame.”

He concludes:

Ultimately, a designer must make sure to align the design to the problem at hand. Avoid being the hammer 🔨 that believes everything is a nail. Wargaming is an AWESOME tool/method, but there is always room for improvement. END.

— Sebastian Bae (@SebastianBae) September 24, 2022

You can read the full thread on Twitter.

Henåker: Decision-making style and victory in battle

Comparative Strategy has just published a piece by Lars Henåker (Swedish Defence University) entitled “Decision-making style and victory in battle—Is there a relation?” In it he reports on a series of experimental wargames which examined the relationship between general decision-making styles and tactical victory:

Can decision-making styles impact victory and defeat in armed conflicts? To answer the question of whether decision-making styles are linked to the victories and defeats of individual tacticians, this study utilizes five general decision-making styles: Rational, Intuitive, Dependent, Avoidant and Spontaneous. The aim of this study is to examine whether one or several of the general decision-making styles (GDMS) have an impact on tactical outcomes in wargames. A total of 104 officers and academics participated in the study. The study’s foremost conclusion is that the Dependent style is significantly connected to defeat in the wargame’s dueling set up.

The participants were 104 officers from the Swedish Defence University in Stockholm and in the Swedish Armed Forces (Skövde Garrison), ranging in rank from Lieutenant to Colonel. The study found little relationship between decision-making styles and wargame outcomes except in the case of the “dependent” style.

The Dependent decision-making style is typified by individuals who seek advice and guidance from others prior to making important decisions. This style adversely impacts the capacity for innovative behavior and creativity for the same reason as the Rational decision-making style. The Dependent decision style is also affiliated with a reduced ability to complete a thought process (e.g., a decision-making process) without being distracted by irrelevant thoughts. Individuals with a Dependent decision style tend to desire to solve quandaries rather than avoid them, although they also have a tendency to doubt their own ability to find a solution.¹⁰ A study by Alacreu-Crespo et al. pos- ited that the Dependent decision style is strongly associated with the need for emo- tional and instrumental support. The Dependent decision style encompasses individuals with socially open and constructive natures, as well as passive and anxious individuals.¹¹

The author goes on to conclude:

One reasonable interpretation is that an individual with a Dependent decision-making style requires more tactics at their disposal and more time to make good decisions. If the individual’s decision-making style is regarded partly as acquired and habitual behavior, and identified when an individual is confronted with a decision situation, we can assume that practical training would reduce a tactician’s need for time and external support. Furthermore, studies should be conducted on how a group of tacticians would manage against another group of tacticians in the corresponding circumstances. It seems reasonable to suggest that decision-style tests be used as a tool for increased self-awareness among military officers, although it is probably too soon to use decision-style tests as a recruit- ment tool.

Finally, we can now pose the question: what practical benefits can we derive from the insight that the Dependent decision-making style adversely impacts the outcome of a dynamic, complex and high-pace environment? The simple answer is that tacticians with a Dependent decision-making style should not have first-call responsibility for making quick decisions during battle, or there would be a risk that decisions are made too slowly in relation to an opponent. However, the study does not indicate whether tacticians with a Dependent decision-making style will function positively or negatively as a member of the group, e.g., staff member, under extreme stress with incomplete decision data.

MWI: Why gamers will win the next war

At the Modern War Institute, Nick Moran and Arnel P. David argue that gamers will win the next war.

A storm is brewing. Thousands of gamers are working to upend traditional models of training, education, and analysis in government and defense. This grassroots movement has developed across several countries, under a joint venture—Fight Club International—within which civilian and military gamers are experimenting with commercial technologies to demonstrate what they can do for national security challenges. But while technology is at the core of this initiative, its more fundamental purpose is to change culture—no easy feat in military organizations, with their characteristic deep sense of history and layers of entrenched bureaucracy.

A common obstacle to introducing transformational technology is the imagination of the user—or, put differently, the willingness of the user to be genuinely imaginative. Early testing with Fight Club, in a constructive simulation called Combat Mission, showed that civilian gamers with no military training outperformed military officers with years of experience. The military gamers were constrained in their thinking and clung dogmatically to doctrine. They discovered, to their frustration, that their speed of decision-making was lacking against gamers with greater intuition and skill.

The piece is an enthusiastic care for greater inclusion of wargames in professional military education—a point with which all of us at PAXsims would agree.

On a methodological note, however, one needs to be careful not to put excessive emphasis on civilian gamers beating non-gaming officers in wargames. Certainly, games test tactical analysis and insight. However, they also test familiarity with interface, rules/algorithms, and other quirks of the simulation. No matter how engaging the graphics, they’re usually quite different from actual command. Indeed, as Sherry Turkle and her colleagues pointed out more than a decade ago, as simulations become more realistic-looking there’s a risk we overlook the important ways in which they depart from reality. I know that some recent experimental work has been done on diversity in wargaming, which among other things assessed the strategic performance of “gamers” as opposed to neophytes and subject matter experts—as soon as that report is available, we’ll share it here at PAXsims.

KWN: Wojtowicz on evaluating effectiveness in wargames

The next public lecture of the King’s Wargaming Network will take place on June 1:

The Wargaming Network is pleased to announce the third lecture in our 2021-2022 public lectures series on wargaming. The theme for this year is evaluating and assessing the impact of wargaming on individuals and organizations and will feature speakers who have made important new contributions to wargaming assessment. The lecture will take place online on 01 June, 17:00-18:30 BST. Please register for the lecture here to receive the log in details for the online event. 

Natalia Wojtowicz will showcase different methods of evaluating effectiveness of wargames, compiled from academic, industrial and governmental sector. A comparison of common and distinct factors will be analyzed to connect the effects with structure of the wargame. The question of objectivity of results will be explored based on recent experiments on adjudication. This presentation will be focused on identifying next steps in measuring and evaluating wargames.

Natalia Wojtowicz is a lecturer at the Hague University of Applied Sciences in the Safety and Security Management Programme. She teaches about wargaming, game design, and digital skills. Her research includes effectiveness of wargaming, new methods and experimental implementation. Previously she worked at the NATO Civil-Military Cooperation Center of Excellence, leading the Wargaming, Modelling and Simulation project focused on introducing civilian population into training and education. Later she designed 14 new wargames implemented across NATO. Currently she is researching adjudication in wargaming and testing an upcoming game about uprising in Belarus. You can follow her [on Twitter] at @Wojtowicz_N

Please register for the lecture here to receive the information for attending this online event on 01 June 2022.