Return to Portsdown West

The following post has been cleared for release by the Defence Science and Technology Laboratory.

AI (DALL-E2) interpretation of my return to Dstl Portsdown West.

In early May I was fortunate to spend four days at the Defence Wargaming Centre of the UK Defence Science Technology Laboratory (Dstl), at Portsdown West near Portsmouth. I had been on similar visits before (in 2016, 2017, and 2018) but this had been followed by a hiatus due to COVID and other factors. It was good to be back, and I’m very grateful to everyone at Dstl who organized and supported the visit.

On the Tuesday and Wednesday I offered a series of lectures on “Mishaps and Minefields in Professional Wargaming,” which examined common mistakes we make and why we make them. These were primarily intended for newer analysts, although several more experienced wargamers participated and contributed to our wide-ranging discussion.

• Mishaps and Minefields I: So You Think You Need a Wargame?
• Mishaps and Minefields II: Game Development
• Mishaps and Minefields III: Participants and Resources
• Mishaps and Minefields IV: Game Control, Adjudication, and Facilitation
• Mishaps and Minefields V: Data Collection and Analysis
• Mishaps and Minefields: Mea Culpa

The final session involved me recounting decisions I now regret and mistakes I’ve made personally while designing and running serious games. Critical reflection is important, after all!

The slides for all of these sessions are below. Like the image at the top of this report, most of the artwork for the slides was generated by AI (and some of it is quite amusing).

dstl-2023-days-1-2Download

The Thursday and Friday of my visit involved more informal discussions of key topics, such as adjudication and end-to-end analysis. There were also playtest sessions of two games being developed as part of Dstl’s EAD (Explore, Anticipate, and Develop) Project. The first was a full-featured modular grand strategic game system that can be adapted to a variety of questions and scenarios.

The second was a much simpler “strategic game in a box” (Contested) to introduce the possibilities of strategic gaming (in much the same way that the Dstl-sponsored Matrix Game Construction Kit was designed to help jump-start matrix gaming in organizations). Both are very promising, and the latter in particular benefits from a very intutive game system that presents few barrier to adoption and play.

On a related note, in 2021 Dstl published a lengthy (165 page) paper on How Can Dstl Expand Our National Security Gaming Toolset To Generate More Meaningful And Reliable Insights? (DSTL/PUB131779 1.4) which provides some really thoughtful discussion of the methodological challenges in strategic gaming. Dstl has now cleared this report for public release, so I’ve just shared it as a separate post on PAXsims.

We also had a session on the design of We Are Coming, Nineveh!, followed by three simultaneous games. The latter went very well, with everyone quickly learning the game system. Daesh seemed to have the best of it in all three wargames, perhaps because the ISF was overly cautiously a little slow to find and fix the enemy—who, after all, were playing for time before they inevitably lost control of West Mosul. I was pleased to learn that the game is to be used in the Defence Academy of the UK to teach about modern urban warfare

Friday afternoon involved a lengthy (but highly enjoyable) drive up to Liverpool, where the Western Approaches HQ Museum hosted Dstl, Royal Navy, and other personnel for a convoy wargame based on the work of the famed Western Approaches Tactical Unit during World War Two. Kit Barry, who organized the event, has already written about the game preparations and the outcome. As captain of fictional Type-VII U-boat U2, I was quite pleased with my result: three merchantmen sunk, followed by a stealthy exit (despite one corvette doggedly trying to find me with ASDIC).

Overall, I had a terrific time—it was both professionally rewarding and very fun. There has been quite a few changes since my earlier visits: the establishment of the DWC, more facilities, and a very substantial growth in the number of Dstl analysts now supporting defence wargaming acriss the UK Ministry of Defence. The United States, of course, remains the preeminent wargaming “superpower” in NATO. Indeed, the budget for the new US Marine wargaming center alone likely exceeds the wargaming resources of most other NATO members combined. However, the UK has clearly consolidated its position as a leader in the field, as evidenced not only by Dstl’s expanding activities but also by the 2017 publication by the Development, Concepts and Doctrine Centre of the Defence Wargaming Handbook as well as the forthcoming Wargaming Influence Handbook. It has developed this capacity, moreover, from a position of greater resource scarcity relative to the US. In that sense I think it is sometimes better attuned to the challenges faced by small and medium-sized NATO militaries. It is also geographically closer to most of them, and the annual Connections UK wargaming conference always has strong representation from other European countries.

I hope that Dstl and others across the UK Ministry of Defence will continue to leverage these strengths to play a leading role in mentoring, supporting, and partnering with wargaming initiatives by allies and partners. They have a a great deal to contribute.

It should also be added that Dstl has been an early and avid supporter of the Derby House Principles on diversity and inclusion in professional wargaming. The effects of this can be increasingly seen in their team of analysts, how they approach their work, and the powerful synergies that arise from harnessing multiple experiences and perspectives. Here too, others have much to learn from their example.

Dstl: Expanding national security gaming to generate more meaningful and reliable insights

The following report has been cleared for release by the Defence Science and Technology Laboratory .

---

In May 2021, the UK Defence Science and Technology Laboratory published a lengthy (165pp) report entitled How Can Dstl Expand Our National Security Gaming Toolset To Generate More Meaningful And Reliable Insights? This addresses a broad range of related issues, including the experiential value of games, identifying genuine insights (as opposed to artifacts of the game design), and post-game analysis.

• Section 1 – Introduction
• Section 2 – How Is An ‘Analytical Game’ Defined?
• Section 3 – How Can We Develop Creating Knowledge Games That Are More Analytical?
• Section 4 – How Can We Conduct More Analytical Games Within TheConstraints Of Engaging Very Senior Players?
• Section 5 – How Can We Encourage More Representative Red Cell ResponsesTo Blue Cell Actions?
• Section 6 – Proof of Concept Escalation Dynamics Game and Concept of Analysis
• Section 7 – Conclusions and Recommendations
• Section 8 – Closing Summary

This report has now been approved for general public release, and can be found in its entirety below (DSTL/PUB131779 1.4).

28052021-dstl_pub131779-how_can_dstl_expand_our_national_security_gaming_toolset_to-generate_more_meaningful_and_valid_insights-v1_4_-oDownload

Wnorowski: Wargaming Practitioners Guide

The Doctrine and Training Centre of the Polish Armed Forces has just published a very useful Wargaming Practitioner’s Guide, written by Mirosław Wnorowski. The English version is shared below. The book explores:

• The essence and objectives of wargames (including definitions, benefits and limits, history, and a link to the key elements and dilemmas of game theory).
• The use of wargames (in the armed forces, as an element of planning, and a classification of types).
• The key elements (participants, scenario, adjudication, data collection).
• Game tools (space, time, actors and the interaction between them).
• The process of preparing and executing a wargame (including particular attention to seminar and matrix games).

Excellent work, Mirosław!

bwargaming_en-online_print_offDownload

Majnemer and Meibauer: Fictitious country names affect experimental results

Jacklyn Majnemer (MIT) and Gustav Meibauer (Radboud University Nijmegen) have published a very interesting article in International Studies Quarterly, 7, 1 (March 2023) exploring whether fictitious country names in survey vignettes affect experimental results. The answer: yes they do.

Using fictitious country names in hypothetical scenarios is widespread in experimental international relations research. We survey sixty-four peer-reviewed articles to find that it is justified by reference to necessary “neutralization” compared to real-world scenarios. However, this neutralization effect has not been independently tested. Indeed, psychology and toponymy scholarship suggest that names entail implicit cues that can inadvertently bias survey results. We use a survey experiment to test neutralization and naming effects. We find not only limited evidence for neutralization, but also little evidence for systematic naming effects. Instead, we find that respondents were often more willing to support using force against fictitious countries than even adversarial real-world countries. Real-world associations may provide a “deterrent” effect not captured by hypothetical scenarios with fictitious country names. In turn, fictionalization may decrease the stakes as experienced by respondents. Researchers should therefore carefully explain rationales for and expected effects of fictitious country names, and test their fictitious names independently.

In Table 2 below you can see that respondents were more willing to use military force against “Celesta,” “Drakhar,” or “Minalo” than they were either a friendly real country (Canada) or a hostile one (Iran).

The research here focuses on survey responses, not serious game play. However the findings may have some interesting implications for strategic-level wargames using fictional country names, which may be more prone to escalation than similar games using real countries.

Interestingly, the authors also suggest that the more “real” a country sounds, the less fictionalization effects are evident:

Our results suggest that the more clearly fictitious a country name, the easier to condone attacking it—fictionality and its perceived costlessness can therefore embolden respondents to provide more aggressive responses.

These results point to the relevance of perceived realistic-ness: the more “real” a country name sounds to respondents, the weaker the fictionalization effect. In particular, there seems to be a deterrent effect associated with realistic-ness, for example, of being able to imagine more easily the consequences associated with attacking Iran, especially bar any additional information that “fills out” the scenario. 

The explanation they suggest for this is deterrence: respondents are better able to imagine the costs of an attack when the survey question asks about a real country rather than a fictional one. However, there may also be an empathy factor here—it’s easier to imagine killing and maiming actual Iranians or Canadians than it is “Minalans,” “Brakharis,” or “Celestians.”

In professional wargames, it is sometime necessary to use fictionalized countries, usually because of political sensitivities. In experimental games there may also be a desire to exert better control of key variables than is possible using a real-life settings. Both reasons apply, for example, to a recent series of NATO experimental wargames that examined Intermediate Force Capabilities in a fictional conflict between the Illyrian Federal Republic and Hypatia (the latter backed by Organization for Collective Security).

An unclassified NATO STO SAS wargame in 2022. You’ll note the Illyrian Federal Republic operations orders (OP IRKALLAN FREEDOM), in a conflict that seems rather reminiscent of a real one in some ways, but set in the northern Aegean.

If Majnemer and Meibauer’s findings do indeed expand beyond international relations survey research to wargaming, there are several implications. One is the need to provide game participants with a rich and realistic fictional environment and to work hard to promote narrative engagement. Another is the need to caveat experimental findings, especially as they relate to use-of-force decisions but possibly other things as well, such as risk aversion or casualty sensitivity more broadly.

Heath: Wargames can’t tell us how to deter a Chinese attack on Taiwan—but different games might

Having posted a link earlier today to the launch of a major CSIS report on wargaming a potential Chinese invasion of Taiwan, now is a good time to also flag this thought-provoking piece by Timothy Heath (RAND) a few days ago at the Lawfare blog. In it, Heath argues that kinetic wargames focused on military operations are useful, but they aren’t optimized to tell us how we might stumble into war—or avoid it.

He states (emphasis added):

Wargames that simulate combat between the United States and China near Taiwan can provide useful insight about potential military challenges. However, analysts should be wary of repurposing the same games to explore political questions such as those related to deterrence, escalation control, alliance politics, and war prevention or termination. Asymmetries in the information requirements for political versus military topics make it exceedingly difficult to design games to explore both in a rigorous manner. Paradoxically, the deliberate falsification of facts in peacetime offers the best hope of painting a more vivid and convincing portrait of a situation that would actually confront policymakers in wartime.

Wargames featuring conflict between China, the U.S., and Taiwan have taken the Washington, D.C., area by storm in the past two years. The U.S. military has held classified wargames on the topic. The Center for Strategic and International Studies (CSIS) held 22 iterations of such a scenario, and other think tanks such as the Center for a New American Security (CNAS), CNA, and RAND have held their own wargames on the topic as well. The appeal of wargames is not hard to figure out: They provide a vivid and dynamic simulation of armed conflict. The China-Taiwan war scenario is especially appealing because the U.S. and China are locked in a rivalry and are also equipped with large, advanced, and powerful militaries. What would happen if the two fought is an inherently fascinating question. The U.S. military advantage is fading, and China’s military is growing stronger. But how the two might fare against each other in combat is unclear. Wargames offer the possibility of exploring such critically important topics, whether as part of a research design or as critical context for creative discussion.

The results of the games have generated several key findings. The most obvious and compelling lesson is that combat between U.S. and Chinese military forces would probably be immensely destructive. In the CSIS game, the U.S. lost 200 aircraft, 20 warships, and two aircraft carriers. Attacks on cyber and space infrastructure are not uncommon. U.S. missiles may strike China’s homeland. Both sides might escalate to the threat, or even use, of nuclear missiles, as happened in at least one CNAS game. Analysts have also noted the military implications for operational topics such as the importance of massing forces, adequate munition stores, and the vulnerability of surface ships on the modern battlefield.

But for many, these lessons are not enough. The frightening results of such simulations naturally raise deeper questions of a fundamentally political nature, such as: How can such a war be avoided? If it can’t be avoided, how can escalation in such a war be controlled? What can the U.S. do to deter China from attacking Taiwan? How long can Taiwan successfully hold out against such an attack? Which allies will support the U.S. in such a war? These political considerations permeate the news accounts of the wargames. In the CSIS event, for example, participants debated whether the pre-positioning of marines on Taiwan prior to war would be “too provocative” or not. Players also debated whether China should attack Japan or not. Military decisions regarding escalation also carried significant political considerations, which may or may not have been debated at the game. In one game, for example, players for the U.S. sideauthorized missile strikes on Chinese ports.

Political decisions on the initiation or escalation of war are immensely important. Yet they are also extremely difficult to answer owing in part to the dearth of reliable data. After all, a U.S.-China war remains, thankfully, completely hypothetical. That leaves virtually no firsthand information with which one can answer such questions. Wargames, and the scenarios that underpin them, have sometimes been used to explore such questions. Since they incorporate many facts about relevant combatants, wargames offer the possibility of exploring political as well as military dimensions of war through a structured, analytic method.

Yet analysts should be wary of trying to use wargames designed for military questions to analyze political questions. The analysis of political topics has fundamentally different information requirements than those for military ones. Wargames that support analysis of military decisions do not necessarily support analysis of political decisions in the same situation, and vice versa.

He further argues that efforts to game future crises by tweaking the status quo are inherently problematic:

One way to get around this problem is to incorporate as much of the current world situation as possible into the game scenario and make only those changes needed to introduce conflict. A game designer could create a scenario that depicts U.S.-China relations largely as they exist today and then inject some crisis near Taiwan to begin the war. This is, in fact, the most commonly used method to build “realistic” scenarios for wargames. But a scenario set in wartime that hews to facts as they exist in peacetime introduces a serious analytic error.

The problem is that, by definition, many factors in a peacetime situation favor peace—factors that can be numerous and diffuse. A scenario based on a contemporary, nonhostile relationship between two countries implies many incentives to avoid hostilities. A main reason why the U.S. and China have not gone to war over Taiwan, after all, is because they have many compelling reasons to favor peace. What exactly about the current situation favors peace remains in debate, but candidates include mutual economic interdependence, the presence of nuclear weapons, relatively modestthreat perceptions, and involvement in shared multilateral institutions. Injecting a “trigger event” such as a crisis related to Taiwan does not resolve the structural incentives for peace. Instead, it merely creates an artificial and unconvincing driver of war. Scenarios that aim to explore political topics in wartime but share considerable continuity with peacetime situations are thus inherently contradictory—they depict a situation with as many structural incentives for peace as one that favors war. This contradiction helps explain why so many wargame scenarios strike participants as implausible and unbelievable.

His answer is to build models of crisis escalation that build on historical examples:

A better approach to wargames would be to model the political assumptions for a hypothetical wargame on the experiences of countries that have actually gone to war. As mentioned earlier in this piece, the deliberate falsification of facts in peacetime offers a good model for what might actually happen in wartime and how policymakers would likely react. After all, the most realistic and relevant facts that confront decision-makers in a war are not those that typify situations in peacetime, but those that typify situations in wartime. The very act of envisioning a war situation that does not exist requires the imaginative visualization of a world radically different from a peacetime status quo.

For such historical data to be useful, it should be as rigorous and scientifically derived as possible. The best resource for scenario designers that aim to replicate realistic and relevant facts and incentives for political decisions lies in the historical experience of countries in analogous situations. 

This, of course, is what many international relations scholars do: attempt to create generalized and testable hypotheses from historical data. There are, I think, a number of challenges to this approach too. After all, generalizations are simply tendencies and not iron laws of causality, specific contexts matter, and historical analogies are often misleading because of very different circumstances.

However, good IR scholarship can offer insight into is what sort of factors (political, economic, and otherwise) might shape escalation decisions, and we can then try to model those much as we might model the factors that shape combat outcomes. Certainly the social science here is far from settled or definitive, but the mere process of constructing models forces us to make explicit our assumptions about the way the world (or an adversary) works for further discussion, research, and refinement.

In general we know that subject matter experts are not necessarily very good predictors of the future (in fact, they’re quite poor at it), in part because a tendency to be cognitively over-attached to favoured paradigms. We also know that intelligence communities often outperform other forecasters, not so much because of access to classified material (although that can be a factor) but also because recruitment tends to prioritize cognitive characteristics associated with better forecasting performance, and because a well-developed analytical process emphasizes training and methodologies to check cognitive biases while encouraging constructive challenges to assumptions and interpretations. As an academic who has worked as an intelligence analyst (and assessed the predictive accuracy of other analysts), these skills are NOT ones they teach in political science (or international relations or security studies) graduate school. My impression is that they are even less present in most PME programmes.

Heath ends his piece with an important warning about the dangers of hubris and the value of humility:

Even with such improvements, however, humility about what we can achieve is required. Re-creating hypothetical war situations based on the experiences of past wars will be imperfect at best and carry their own flawed assumptions. Carrying out different iterations with slightly different assumptions could help mitigate some of these limitations. Yet even in the most optimal case, we can at best aspire to craft a crude simulacra of the incentives and factors leaders might confront in a hypothetical situation that will carry all sorts of unimaginable complexities. Given the stakes involved, even an imperfect and partial approach offers a potentially significant improvement over current methodologies for defense planners, analysts, and decision-makers alike who seek to explore political questions in wartime.

On this I think we can all agree.

Bae: A tale of many kinds of wargames (for many kinds of purposes)

Yesterday we posted a link to Jon Compton’s War on the Rocks piece “A Tale of Two Wargames.” Today, Sebastian Bae (CNA) adds some thoughts of his own in a series of Twitter posts.

Let's talk #professional wargaming. Buckle in. 🔥

First, there are some valid underlying points in this article. Small-scale multi-method, iterative analytical #wargames are great. No doubt. I even agree that we should do more of them. 1/https://t.co/Y8Nt8TvLAh

— Sebastian Bae (@SebastianBae) September 24, 2022

He correctly notes that wargame sponsors may have rather different objectives, and that there are many different game approaches depending on those purposes. As a result, he suggests, “not every game will need the kind of approach outlined in this article.” He goes on to add “this binary between multi-method games and ‘event wargames’ is misleading. Some sponsors need event wargames for LOTS of people because they want to socialize an idea or get stakeholders together. Or have a specific timeline. There is no ‘ideal’ perfect form for a wargame.”

He concludes:

Ultimately, a designer must make sure to align the design to the problem at hand. Avoid being the hammer 🔨 that believes everything is a nail. Wargaming is an AWESOME tool/method, but there is always room for improvement. END.

— Sebastian Bae (@SebastianBae) September 24, 2022

You can read the full thread on Twitter.

Henåker: Decision-making style and victory in battle

Comparative Strategy has just published a piece by Lars Henåker (Swedish Defence University) entitled “Decision-making style and victory in battle—Is there a relation?” In it he reports on a series of experimental wargames which examined the relationship between general decision-making styles and tactical victory:

Can decision-making styles impact victory and defeat in armed conflicts? To answer the question of whether decision-making styles are linked to the victories and defeats of individual tacticians, this study utilizes five general decision-making styles: Rational, Intuitive, Dependent, Avoidant and Spontaneous. The aim of this study is to examine whether one or several of the general decision-making styles (GDMS) have an impact on tactical outcomes in wargames. A total of 104 officers and academics participated in the study. The study’s foremost conclusion is that the Dependent style is significantly connected to defeat in the wargame’s dueling set up.

The participants were 104 officers from the Swedish Defence University in Stockholm and in the Swedish Armed Forces (Skövde Garrison), ranging in rank from Lieutenant to Colonel. The study found little relationship between decision-making styles and wargame outcomes except in the case of the “dependent” style.

The Dependent decision-making style is typified by individuals who seek advice and guidance from others prior to making important decisions. This style adversely impacts the capacity for innovative behavior and creativity for the same reason as the Rational decision-making style. The Dependent decision style is also affiliated with a reduced ability to complete a thought process (e.g., a decision-making process) without being distracted by irrelevant thoughts. Individuals with a Dependent decision style tend to desire to solve quandaries rather than avoid them, although they also have a tendency to doubt their own ability to find a solution.¹⁰ A study by Alacreu-Crespo et al. pos- ited that the Dependent decision style is strongly associated with the need for emo- tional and instrumental support. The Dependent decision style encompasses individuals with socially open and constructive natures, as well as passive and anxious individuals.¹¹

The author goes on to conclude:

One reasonable interpretation is that an individual with a Dependent decision-making style requires more tactics at their disposal and more time to make good decisions. If the individual’s decision-making style is regarded partly as acquired and habitual behavior, and identified when an individual is confronted with a decision situation, we can assume that practical training would reduce a tactician’s need for time and external support. Furthermore, studies should be conducted on how a group of tacticians would manage against another group of tacticians in the corresponding circumstances. It seems reasonable to suggest that decision-style tests be used as a tool for increased self-awareness among military officers, although it is probably too soon to use decision-style tests as a recruit- ment tool.

Finally, we can now pose the question: what practical benefits can we derive from the insight that the Dependent decision-making style adversely impacts the outcome of a dynamic, complex and high-pace environment? The simple answer is that tacticians with a Dependent decision-making style should not have first-call responsibility for making quick decisions during battle, or there would be a risk that decisions are made too slowly in relation to an opponent. However, the study does not indicate whether tacticians with a Dependent decision-making style will function positively or negatively as a member of the group, e.g., staff member, under extreme stress with incomplete decision data.

MWI: Why gamers will win the next war

At the Modern War Institute, Nick Moran and Arnel P. David argue that gamers will win the next war.

A storm is brewing. Thousands of gamers are working to upend traditional models of training, education, and analysis in government and defense. This grassroots movement has developed across several countries, under a joint venture—Fight Club International—within which civilian and military gamers are experimenting with commercial technologies to demonstrate what they can do for national security challenges. But while technology is at the core of this initiative, its more fundamental purpose is to change culture—no easy feat in military organizations, with their characteristic deep sense of history and layers of entrenched bureaucracy.

A common obstacle to introducing transformational technology is the imagination of the user—or, put differently, the willingness of the user to be genuinely imaginative. Early testing with Fight Club, in a constructive simulation called Combat Mission, showed that civilian gamers with no military training outperformed military officers with years of experience. The military gamers were constrained in their thinking and clung dogmatically to doctrine. They discovered, to their frustration, that their speed of decision-making was lacking against gamers with greater intuition and skill.

The piece is an enthusiastic care for greater inclusion of wargames in professional military education—a point with which all of us at PAXsims would agree.

On a methodological note, however, one needs to be careful not to put excessive emphasis on civilian gamers beating non-gaming officers in wargames. Certainly, games test tactical analysis and insight. However, they also test familiarity with interface, rules/algorithms, and other quirks of the simulation. No matter how engaging the graphics, they’re usually quite different from actual command. Indeed, as Sherry Turkle and her colleagues pointed out more than a decade ago, as simulations become more realistic-looking there’s a risk we overlook the important ways in which they depart from reality. I know that some recent experimental work has been done on diversity in wargaming, which among other things assessed the strategic performance of “gamers” as opposed to neophytes and subject matter experts—as soon as that report is available, we’ll share it here at PAXsims.

KWN: Wojtowicz on evaluating effectiveness in wargames

The next public lecture of the King’s Wargaming Network will take place on June 1:

The Wargaming Network is pleased to announce the third lecture in our 2021-2022 public lectures series on wargaming. The theme for this year is evaluating and assessing the impact of wargaming on individuals and organizations and will feature speakers who have made important new contributions to wargaming assessment. The lecture will take place online on 01 June, 17:00-18:30 BST. Please register for the lecture here to receive the log in details for the online event. 

Natalia Wojtowicz will showcase different methods of evaluating effectiveness of wargames, compiled from academic, industrial and governmental sector. A comparison of common and distinct factors will be analyzed to connect the effects with structure of the wargame. The question of objectivity of results will be explored based on recent experiments on adjudication. This presentation will be focused on identifying next steps in measuring and evaluating wargames.

Natalia Wojtowicz is a lecturer at the Hague University of Applied Sciences in the Safety and Security Management Programme. She teaches about wargaming, game design, and digital skills. Her research includes effectiveness of wargaming, new methods and experimental implementation. Previously she worked at the NATO Civil-Military Cooperation Center of Excellence, leading the Wargaming, Modelling and Simulation project focused on introducing civilian population into training and education. Later she designed 14 new wargames implemented across NATO. Currently she is researching adjudication in wargaming and testing an upcoming game about uprising in Belarus. You can follow her [on Twitter] at @Wojtowicz_N

Please register for the lecture here to receive the information for attending this online event on 01 June 2022. 

Domaingue: Cultivating breakthrough thinking in serious games

Tabletop exercise – Wikimedia Commons.

This post was written for PAXsims by Robert Domaingue. Before retiring from the U.S. State Department, Robert Domaingue was the lead Conflict Game Designer in the Bureau of Conflict and Stabilization Operations.  He now works with local organizations to utilize serious games for solving community problems.

---

Serious games are used to provide insights into complex problems.  They help decision makers and staff test assumptions, examine strategies, and determine deficiencies in planning.  Many different government departments, businesses, and organizations utilize serious games to provide a safe environment to learn from failure.  These organizations can improve the design of their serious games by incorporating principles from experiential learning.

Experiential learning focuses on the learning that emerges from concrete experience and the reflection and application of that experience.  A frequently cited model of the experiential learning cycle comes from David Kolb’s 1984 book Experiential Learning.  He proposes a cycle that begins with Experiencing, → moves to Reflecting, → to Generalizing, → to Testing, → and starting over again with new Experiencing.  The learner proceeds through all steps in order to make sense of the experience and apply the insights.  There are similar earlier models from John Dewey: Observation, → Knowledge, → Judgement, → more Observation; and Kurt Lewin: Concrete Experience, → Observation and Reflection, → Formation of Abstract Concepts and Generalizations, → Testing Implications of Concepts in New Situations, → and continuing the cycle with new Concrete Experience.  All of these models highlight the importance of reflecting on the nature of the experience for general learning to occur.  Furthermore, John Dewey felt that experiencing something served as a linking process between action and thought.  But not all experiences lead to learning.  In his 1938 book Experience and Education, Dewey wrote “Any experience is miseducation that has the effect of arresting or distorting the growth of further experience (p25).”  This is an important point I will build on.

One problem with these models is with the very first step of identifying the experience.  It implies that we actually understand that we are having an experience – that we “see it”.  E.M. Forster said that the only books that influence us are the ones we are ready for.  Likewise, we may only see what we already know.  We may not identify the experience because we don’t have the awareness to make sense of the experience, or our prior conceptual models block us from seeing the experience as it is.

“The blinders of our categories prevent us from seeing what is there.”

A way to highlight this act of “not seeing” what is in front of us is to explore two psychological experiments that examine “functional fixedness.”  Functional fixedness refers to not seeing the potential novel uses of something because of your narrow prior category of the object in your mind.  The classic “candle experiment” gave subjects a candle, a box of thumb tacks, a bulletin board, and asked them to attach the candle to the bulletin board.  Most people tried to use the thumb tacks to stick the candle to the board, which doesn’t work very well.  A second group of subjects was given the same instructions and materials, with one small change.  This time the tacks were removed from the box and placed next to the empty box.  While it was a small change, it was large enough for people to see a new way of solving the problem.  Subjects in the second group saw that if they tacked the box to the bulletin board they could then place the candle inside the box.  When the box was full of tacks the subjects’ functional fixedness prevented them from seeing other uses for the box.

Another experiment involved giving subjects a problem to solve in which a length of string could be used in the solution of the problem.  The string was hung from a nail on the wall, and most people figured out to use the string as part of the solution to the problem.  Other groups were given the same instructions, but this time the same string on the same nail was used to hold a picture.  In this case no one thought to use the string to solve the problem.  The subjects’ functional fixedness on the string as part of the picture prevented them from seeing it as a resource to solve the problem.  The blinders of our categories prevent us from seeing what is there.

How do we overcome “not seeing”, and what is the impact on serious game design?  When utilizing the experiential learning models to guide our game design we should change the first step to “Identifying the Experience”.  People do not necessarily understand the nature of the experience that they are expected to reflect upon and draw lessons from.  The game designer must have a clear idea of the nature of the experience that the game will be providing to the participants.  This does not mean that the experience needs to be clearly delineated for the players at the start of the game.  It can be, but there are times when ambiguity and uncertainty are valuable features of the game.  In these cases, the nature of the experience needs to be highlighted in the debriefing session at the end of the game.  Here the facilitator can direct attention to how the players made sense of the experience and what they got out of exploring that experience.  It is very important to look at the assumptions that players operated under as to what were viable and nonviable approaches to solving the problem.

Players bring prior experience and preconceived ideas with them to the game, and a novel experience that challenges pre-held beliefs may not even be seen.  The game designer must be aware of the dangers of misinterpreting the nature of the experience by the players.  It could easily lead to learning the wrong thing from the experience.  If, however, the facilitator with the help of the players can identify examples of functional fixedness that occurred when approaching the problem, then they have identified fruitful topics to develop additional games around.  These new iterative games could provide breakthrough thinking for approaching the problem.  The insights are so valuable that the game facilitator must be continually searching for opportunities to explore them when they arise.  Spending the time designing and playing serious games can be enormously useful for organizations if sufficient attention is given to framing the experience and guiding the learning that results from exploring that experience.

Robert Domaingue

Sepinsky and Bae: Wargaming is about the process, not the result

In Foreign Policy magazine, Jeremy Sepinsky and Sebatian Bae discuss the use, utility, and limits of wargaming as an analytic tool, using as an example a potential Chinese invasion of Taiwan.

This is the kind of narrative most people imagine when they think of military war games—scenes in the bowels of the Pentagon, units fighting digitally on electronic maps, commanders pondering their next step in a fast-moving crisis. Victory in the simulation, so the popular imagination goes, shows how to win a real-life conflict. Defeat in a war game, on the other hand, is an acknowledgement that any actual conflict will likely be lost.

Contrary to the popular imagination, however, this is not how war games work. Rarely is a war game designed to predict the future or develop a single definitive strategy. Instead, a war game helps military planners and analysts explore and understand a complex problem, regardless of the outcome. Win or lose, the purpose isn’t to define a strategy for the U.S. military but to help it better understand the capabilities it has, what it can already do, and what it needs.

Whether it’s Taiwan or any other potential conflict, the scenario is rarely the focus of the war games we at CNA design for the U.S. Defense Department. Instead, war games are about better understanding how the U.S. military can build deterrence, what technology gaps could hobble its forces, how an adversary’s capabilities might evolve in response to U.S. capabilities, and how all that might impact what Washington should invest in today. Fundamentally, war games strive to explore and distill the fundamental nature of the problem itself—which rarely leads to definitive scenarios or solutions.

In fact, using war games to craft a clear-cut strategy is impossible. Done right, war games are a plausible method of providing a brief and limited glimpse into a possible future—a single future in a multiverse of possibilities. Trying to imitate victory in a war game, on the other hand, means trying to align both sides’ future decisions in a complex conflict with the scenario that played out during the game. Obviously, these decisions are numerous and mostly beyond one’s control.

…

What worked in a single war game has limited utility—it worked against a specific adversary making a specific set of decisions using a specific set of game rules that may or may not accurately reflect the world. Failure, on the other hand, doesn’t require the game to be a perfect simulation. We often hear complaints from players that our war game rules make the adversary “10 feet tall.” But it is better to stress U.S. forces more than to give the adversary too little credit and not stress U.S. forces enough. Stressing the capabilities of the U.S. forces to their breaking point from all sides allows analysts and researchers to identify vulnerabilities and what might be needed to fix them.

They conclude:

So, in a war game, pay no attention to who won or lost. War-gaming is about the process, not the result—and analyzing that process is what will allow the U.S. military to turn losing into winning.

You can read the full article at the link above. For more on wargaming Taiwan, see Drew Marriott’s 2021 summary of recent Taiwan wargames here at PAXsims.

Diversity, inclusion, and national security

At War on the Rocks, Caesar Nafrada and Joseph Caddell have written an excellent piece on the value of diversity and inclusion in national security analysis, focusing on the role that racism and stereotypes played in leaving the United States vulnerable to the Pearl Harbour attack:

“I never thought those little yellow sons-of-bitches could pull off such an attack, so far from Japan.” So confessed Adm. Husband Kimmel, former U.S. Pacific Fleet commander, to a member of the congressional Pearl Harbor investigation. Our 2021 reaction to Kimmel’s words is to focus on their racist invective. Yet, there’s an even more obvious problem with Kimmel’s statement. He was simply wrong, and he should have known better. Adm. Kimmel, Gen. Walter Short (the U.S. Army Hawaiian Department commander), and other military leaders on Oahu fundamentally underestimated and misunderstood the threat of Japanese carrier aviation to their detriment, and to the detriment of national security. Their underestimation and misunderstanding was deeply grounded in racial prejudice and went unchallenged in an environment of ethnocentric groupthink. Even in acknowledging his mistake, Kimmel’s words suggest he remained angrily defiant that the attack materialized the way it did — as if it were unfair that reality did not conform to his prejudices. Ethnocentric and racist attitudes and actions are objectionable for their harmful effects on the groups they malign, of course, but we should never forget that they are fundamentally stupid because they are factually incorrect. Prejudice literally means passing judgement prior to possessing adequate information. For national security professionals, prejudice is dangerous. Prejudice is fatal. The prejudicial unwillingness of Kimmel, Short, and others to posture adequately against a potential Japanese aerial attack was fatal 80 years ago today. Pearl Harbor is a concrete example that demonstrates how devastating ethnocentric bias brought on by largely homogenous institutions can be.

They go on to note:

The details of the warning failure at Pearl Harbor illustrate how toxic ethnocentrism, the byproduct of a homogenous workforce, taints analysis and decision-making in various ways. A lack of diversity fosters devastating shared blind spots, skewing the foundations upon which every process is built. Without diversity, some flawed beliefs go unchallenged. Pearl Harbor demonstrates the dangerous results of unchallenged ethnocentric assumptions. Pervasive ethnocentrism and racism lead to disastrous outcomes when they supplant real evidence or lead one to underestimate a foe. These dynamics do not merely reflect the prevailing racial attitudes of the American military of the 1940s. They illustrate how a lack of diversity and inclusion in the national security workforce could have lethal consequences today.

The private sector has no shortage of industry research that demonstrates how a lack of diversity and inclusion negatively impacts organizational performance. National security organizations are similarly vulnerable. Since national security leaders have made the argument that diversity and inclusion can strengthen their organizations, extrapolations from these industry findings should be further explored for their applicability.

While organizations lacking diversity risk prejudicial blind spots, teams comprised of people from diverse backgrounds are more likely to mitigate this bias thanks to multiple perspectives drawn from personal experiences. This is especially true in the realm of international affairs. Ethnocentrism can damage analytical tradecraft through groupthink, mirror-imaging, and the misreading of cultural norms and behaviors. …

National security decision-makers must be willing to incorporate more varied sources into their assessments to better avoid strategic surprises from peer competitors. Improved diversity and inclusion can drive better consideration of possible (if unlikely) threats by inhibiting groupthink and reducing cognitive biases, resulting in greater objectivity. Conversely, research shows that homogenous groups are less able to recognize the value of contrary information, leading to poor decision-making. Diversity and inclusion increase the integration of broader perspectives, reducing shared blind spots. Inclusion fosters new paradigms and heuristics that may have been absent within an otherwise homogeneous organization or team. Studies of racially diverse groups show that social differences generate teams more likely to anticipate differences of opinion, driving them to integrate multiple perspectives while building consensus.

The article isn’t about wargaming, but it’s a piece every serious game designer should read.

PAXsims is a proud supporter of the Derby House Principles on diversity and inclusion in professional wargaming. If your organization would like to join the impressive list of cosponsors, drop us an email.

Mason: Designing public safety games and tabletop exercises

At the LECMgt blog, Roger Mason discusses designing public safety games and tabletop exercises.

Public Safety organizations employ tabletop style exercises to train their personnel. These exercises provide a flexible and cost-effective system to simulate a variety of critical incidents. This article will discuss how to design tabletop exercises and the next level of simulation complexity, wargames. I will discuss the uses of these exercises and the common and unique characteristics of each.

We will explore five steps for designing a game or exercise and how to validate the design. Some people maintain that tabletop exercises are simple to design because they appear to be simple. There may be more to designing a tabletop exercise or wargame than some people believe.

It’s a very useful overview of the process and considerations involved and well worth a read.

Barzashka: Do academic standards for research excellence apply to professional wargaming?

The following item has been written for PAXsims by Ivanka Barzashka (Managing Director of the Wargaming Network at the School of Security Studies at King’s College London), based on her recent presentation at the Connections US 2021 professional wargaming conference. This article expresses her personal views and does not necessarily reflect an institutional position. 

---

Wargaming has long practiced as a professional enterprise but is only emerging as an academic discipline. Civilian universities play an important role in bringing established standards of academic excellence to the theory and practice of wargaming for both research and education. 

Ensuring excellence in analytical wargames is especially important as governments increasingly looking to wargames to innovate and inform decisions. The pool of analytical wargame providers is rapidly expanding beyond the well-established expert circles. 

To achieve and demonstrate research excellence, analytical wargames need to follow established research integrity and ethics principles. Researchers, institutions and funders can then ensure these principles are implemented in practice, and action is taken when behaviours fall short. 

Research Integrity & Ethics in Analytical Wargaming 

What are these fundamental principles and how do they apply to wargames used for research? I answered this question in a recent presentation at the US Connections Professional Wargaming Conference from an academic perspective informed by King’s College London policies. I also highlighted some challenges facing scholars who wargame. You can listen to the talk here and view my slides below. 

The key takeaway: while wargaming scholarship is progressing, there is still a way to go. To properly meet research integrity standards, we need more fundamental research on wargaming, more educational opportunities in wargaming theory, methods and practice, and appropriate publication outlets. It is impossible to follow “disciplinary standard and norms” when scholars do not know or agree what these are. It is difficult to demonstrate rigour in “using appropriate research methods” when scientifically-sound analytical wargaming methods are only beginning to emerge in the open literature and are being applied for the first time for scholarly inquiry. Academics who strive for “transparency and open communication” still have issues publishing wargame findings in reputable journals.  

Expectations for Scholars vs Professional Wargamers 

But to what extent do research integrity and ethics requirements for analytical wargaming differ for academics versus professional wargamers? To advance this discussion, I offer three propositions.  

Most, but Not All, Academic Research Integrity Principles Apply to Professional Wargaming 

First, while the general principles for research excellence are fairly standard, a major difference between academic and professional wargaming is the expectation for transparency and open communication. For example, analysts who use wargames to support research for government clients are not expected to make their methods and findings available to others. In contrast, scholars are required to publish research and are promoted on the number of publications. 

Responding to Research Misconduct and Questionable Research Practices 

Second, the extent to which research integrity principles are applied in practice differs significantly among institutions and sponsors. This includes taking appropriate measures when there is evidence of research misconduct or questionable research practices.  

Research misconduct, which includes falsification, fabrication, plagiarism and misrepresentation, is a potentially fireable offence at a university. But could professional wargamers lose their jobs over poor game design or inadequate analysis of gameplay data?  

Looking at King’s definitions for categories of misconduct, many common wargaming practices would raise red flags in academe. Here are some examples: 

Falsification includes “inappropriate manipulation and/or selection of a research process.” According to this definition, creative injects by a control team that affect or determine outcomes of player decisions, but do not clearly link to research objectives and protocols, would raise questions.   

Misrepresentation includes “suppressing relevant results or data, or knowingly, recklessly or by gross negligence representing flawed interpretation of data.” Cherry-picking insights from a plenary discussion, while ignoring gameplay data, would get wargame analysts in trouble in this category. 

Another issue is plagiarism. Not acknowledging other people’s “ideas, intellectual property or work (written or otherwise)” in wargame design would be especially problematic in an academic setting but is common practice in the gaming community. 

Major Research Ethics Risks 

My third proposition concerns research ethics. The ethical issues that arise from the application of a particular analytical wargaming method that collects data from human subjects are mostly the same – regardless of whether the principal investigator works for a university or a government agency. However, the likelihood and consequence of ethical risks materialising will differ significantly in different settings. 

Scholars applying for research ethics review of an analytical wargaming process are most worried about preserving anonymity of research participants and ensuring the confidentiality of personal data. This risk arises because wargames are conducted in group settings and require support from large research teams (e.g. rapporteurs and facilitators). 

However, scholars can effectively manage these risks by carefully applying best practices, such as minimisation of directly or indirectly identifiable personal data, pseudo-anonymisation, access limitation, data separation and retention policies. These risks can be further reduced by careful recruitment and training of game staff. (At King’s, we spend 6 months selecting and training our wargame rapporteurs.) 

For professional wargamers, the major ethical risk is the conflict of interest between them and their sponsor. Stephen Downs-Martin describes the issue well in this article. Research ethics problems deepen when lines of responsibility and accountability are not clearly defined, and when the research process is not (or cannot) be made transparent. Mitigating these risks requires clear communication between a wargame provider and their sponsor but doing so might not be in the self-interest of the parties involved. 

Other ethical risks will be just as big, regardless of setting. For example, risks of harm to individuals could result from using wargames to investigate topics that could trigger stress or violence. If a principal investigator uses deception, including not fully informing participants of the purpose of the wargame, this also raises ethics concerns. (Thanks to Rex Brynen for highlighting these points.) 

Are Scholars Better Positioned to Ensure Research Excellence in Wargaming?

Ensuring and demonstrating research excellence in wargaming requires understanding, applying, and enforcing integrity and ethics principles. These principles are well established, but expectations differ in academic and professional wargaming settings. Professional wargamers face greater ethical risks than scholars who wargame, and these risks cannot be easily mitigated.   

Overall, scholars at universities are better positioned to ensure research excellence in wargaming than their professional wargaming colleagues. This does not mean professional wargamers are less interested in honesty, rigour, transparency, or ethics. Quite on the contrary – the wargaming community of practice is conscious of these risks and limitations, and the topic of this year’s US Connections conference is testimony to this fact. But there are powerful institutional incentives that influence research integrity and ethics in practice, which cannot be wished away. 

If people who are professional wargamers want to effectively demonstrate research excellence in wargaming, they should consider a sabbatical to spend some time at a university. 

Ivanka Barzashka

barzashka-ethics-integrity-cxusa21Download

Building a climate change megagame (Part 3)

The following series of articles was written for PAXsims by Ola Leifler, Magnus Persson, and Ola Uhrqvist. You can read Parts 1 and 2 here and here.

---

Concluding thoughts

One of the first impressions was that we were rather overwhelmed by the experience, which is one of the reasons this blog post, long overdue and way too long, did not materialize until at least one academic period had transpired after the main CCM event. However, now that we have gathered our thoughts a bit, we realized that we have probably learned a great number of things so far. For instance:

1.     Reasons for creating a megagame on climate change and social transformation

There are many types of games that relate to climate change and negotiations, but few that we feel concern the types of negotiations, dilemmas and interactions that may be common for professionals in companies and citizens in local regions facing the prospect of societal change. One of us, Ola Uhrqvist, had previous experience developing a game about city planning to take both climate adaptation into account— but there, few negotiations were conducted as the game was primarily a single-player web application. 

In the literature on learning for a sustainable development, engagement and various pedagogical forms is stressed as key to ensure that learners experience first-hand the dilemmas and difficulties they need to overcome. Furthermore, we noticed that when we pitched the idea of a “Climate Change Megagame”, it immediately piqued people’s interest in a way that acted as an icebreaker and helped us to engage rather diverse groups in conversations. Even though there were practical issues with every single version of the game we have tried, the concept itself has been intriguing enough to make people joining as players or contribute as control team and even contributing to game development. However, to understand exactly which difficulties to subject players, and what type of realistic situations to simulate, has proved to be almost as elusive as real societal transformation.

2.     The eternal challenge of playable realism

Serious games always needs to balance between relevance and playability. The activities players engage in, and the type of experience they have, must be of relevance whether it is “realistic” or not. We learned that some types of realism, such as players getting bogged down by managing their daily lives, may not be helpful in ensuring that the resulting experience is relevant to the end goal of understanding dilemmas and options for societal transformation. We wanted the game to offer interesting challenges without directing players too much with respect to what they would want to do. As designers, we can include mechanisms that reflect aspects of reality such as economic capital being vital for investments in infrastructure, say, without going so far as to say that without a growing economy, people would starve to death. We wanted to provide enough context and feedback mechanisms to stimulate discussions and make different visions apparent, without constricting players in such a way that their room for creative discussions and maneuvering would be artificially restricted. 

A golden rule for how to ensure players understand the rules well enough to be comfortable about breaking them and understanding just how much freedom they have to negotiate freely probably don’t exist but we understand much better now than before what would count as interesting and relevant challenges compared to “realistic” ones. In our experience minimalism of game mechanics is desirable in order to let participants focus on the content. 

3.     Recruiting and maintaining a committed and diverse design team

Including more people from the early playtests in game design and discussions made it apparent that it was difficult to ensure equal commitment among all when the game concept changed quite a lot, partly as a result of feedback. Also, we wanted to be open to suggestions about how different groups could contribute to the project, which placed high demands on participants to express clearly what they wanted to contribute to and what they expected. Some of the early contributors who provided invaluable feedback on the game and made it much better in the end still did not feel comfortable joining at the end as the game changed quite a lot between playtests. Though it was necessary to make the changes, it became difficult for all members of the design team to keep up with the ideas for changes that the core group brought forward, especially as we became limited to digital meetings during the pandemic. The take home lesson is the value of a clear aim, participants roles and modes of decision making and communication is increasingly important in a dynamic, explorative project.

4.     Going digital

Going digital opened up new opportunities for players from around the world to join and it greatly simplified our ability to collect data on how the game progressed, but also introduced a whole host of new issues. We spent quite some time even after the core game mechanics and graphical elements had been decided to ensure that the digital platform (Miro) could handle all graphical components and the 50 players with decent latency. Therefore, some graphical optimizations were required before the main event took place. For instance, components were merged into bitmaps instead of hundreds of separate graphics components. The communications channel (Discord) was set up very professionally by our Megagame colleague Darren Green from Crisis Games in the UK and that enabled players to have both private and public spaces for communications. Even with such a setup though, some players felt lost between all the channels and the Miro board. Having a technical setup and preparation before the main event, just focusing on the technical aspects of the game would probably have helped some participants who were struggling.

The main event was hosted at a venue where we broadcast everything live from a studio over Vimeo. This worked rather well as a compromise between having only an internal event and only having a studio with professional talking heads but having dual roles as hosts for both the game and the “show” was hard to manage. It would have been better to have studio hosts who could have focused on being hosts. Then again, a digital event that plays out through discussions on Discord and board changes on Miro might not offer enough continuous action for a continuous live show.

5.     The importance of good debriefing

The main event was intended to let people experience and reason about the needs for mitigation and adaptation, as in the needs for making changes to our societies that will reduce emissions versus the needs to adapt to climate change we cannot avoid. The primary aim of the debriefing was to capture the perceptions of these potentially conflicting needs, but it became apparent that the participants were mostly preoccupied with thoughts about the game mechanics, graphical elements and direct experiences. A debriefing is very important for a proper learning experience, and for us, the fact that people became preoccupied with the mechanics and graphical elements indicated that these were in fact the objects they thought mostly in terms of directly afterwards. Maybe the game was too heavy on mechanics since it became hard to talk about abstract things such as mitigation and adaptation in direct connection to having played. It would probably have been easier to first address game-specific issues and then later broaden the horizon to comprise the real world.

6.     Future development

The project had until this point been run exclusively on a small amount of seed money for a pedagogical project and a lot of personal commitment. We realized that continued work with this require us to leverage our initial experiences and gain access to proper funding for work that could significantly expand on what we have been doing. The game itself is not a goal, it is not even a product that may be finished but at best a way to help us think better, as designers and players, about what a sustainable society may be like. With some luck, we may have a chance to build on all we have learned and enable others to learn as we have about how to move constructively towards a societal transformation to sustainability.

---

Ola Leifler is a senior lecturer in software engineering at Linköping University who, over the last ten years and upon learning more about the state of the world and the effects of how we educate, has formed a strong interest in learning for a sustainable development. With a special interest in boardgames, role-playing games and simulations, he now explores how they can be harnessed to promote more constructive thinking about global challenges.

Magnus Persson is a translator and academic proofreader with an interest in board game development who has been serious about games for as long as he can remember and only in recent years came into contact with the megagame genre and the concept of serious games. 

Ola Uhrqvist is a teacher and researcher in the field of Environmental and Sustainability Education with a special interest in using serious games as a tool to enhance engagement in and understanding of complex issues, such as environmental and social change.