PAXsims

Conflict simulation, peacebuilding, and development

Daily Archives: 25/06/2021

Barzashka: Do academic standards for research excellence apply to professional wargaming?

The following item has been written for PAXsims by Ivanka Barzashka (Managing Director of the Wargaming Network at the School of Security Studies at King’s College London), based on her recent presentation at the Connections US 2021 professional wargaming conference. This article expresses her personal views and does not necessarily reflect an institutional position. 


Wargaming has long practiced as a professional enterprise but is only emerging as an academic discipline. Civilian universities play an important role in bringing established standards of academic excellence to the theory and practice of wargaming for both research and education. 

Ensuring excellence in analytical wargames is especially important as governments increasingly looking to wargames to innovate and inform decisions. The pool of analytical wargame providers is rapidly expanding beyond the well-established expert circles. 

To achieve and demonstrate research excellence, analytical wargames need to follow established research integrity and ethics principles. Researchers, institutions and funders can then ensure these principles are implemented in practice, and action is taken when behaviours fall short. 

Research Integrity & Ethics in Analytical Wargaming 

What are these fundamental principles and how do they apply to wargames used for research? I answered this question in a recent presentation at the US Connections Professional Wargaming Conference from an academic perspective informed by King’s College London policies. I also highlighted some challenges facing scholars who wargame. You can listen to the talk here and view my slides below. 

The key takeaway: while wargaming scholarship is progressing, there is still a way to go. To properly meet research integrity standards, we need more fundamental research on wargaming, more educational opportunities in wargaming theory, methods and practice, and appropriate publication outlets. It is impossible to follow “disciplinary standard and norms” when scholars do not know or agree what these are. It is difficult to demonstrate rigour in “using appropriate research methods” when scientifically-sound analytical wargaming methods are only beginning to emerge in the open literature and are being applied for the first time for scholarly inquiry. Academics who strive for “transparency and open communication” still have issues publishing wargame findings in reputable journals.  

Expectations for Scholars vs Professional Wargamers 

But to what extent do research integrity and ethics requirements for analytical wargaming differ for academics versus professional wargamers? To advance this discussion, I offer three propositions.  

Most, but Not All, Academic Research Integrity Principles Apply to Professional Wargaming 

First, while the general principles for research excellence are fairly standard, a major difference between academic and professional wargaming is the expectation for transparency and open communication. For example, analysts who use wargames to support research for government clients are not expected to make their methods and findings available to others. In contrast, scholars are required to publish research and are promoted on the number of publications. 

Responding to Research Misconduct and Questionable Research Practices 

Second, the extent to which research integrity principles are applied in practice differs significantly among institutions and sponsors. This includes taking appropriate measures when there is evidence of research misconduct or questionable research practices.  

Research misconduct, which includes falsification, fabrication, plagiarism and misrepresentation, is a potentially fireable offence at a university. But could professional wargamers lose their jobs over poor game design or inadequate analysis of gameplay data?  

Looking at King’s definitions for categories of misconduct, many common wargaming practices would raise red flags in academe. Here are some examples: 

Falsification includes “inappropriate manipulation and/or selection of a research process.” According to this definition, creative injects by a control team that affect or determine outcomes of player decisions, but do not clearly link to research objectives and protocols, would raise questions.   

Misrepresentation includes “suppressing relevant results or data, or knowingly, recklessly or by gross negligence representing flawed interpretation of data.” Cherry-picking insights from a plenary discussion, while ignoring gameplay data, would get wargame analysts in trouble in this category. 

Another issue is plagiarism. Not acknowledging other people’s “ideas, intellectual property or work (written or otherwise)” in wargame design would be especially problematic in an academic setting but is common practice in the gaming community. 

Major Research Ethics Risks 

My third proposition concerns research ethics. The ethical issues that arise from the application of a particular analytical wargaming method that collects data from human subjects are mostly the same – regardless of whether the principal investigator works for a university or a government agency. However, the likelihood and consequence of ethical risks materialising will differ significantly in different settings. 

Scholars applying for research ethics review of an analytical wargaming process are most worried about preserving anonymity of research participants and ensuring the confidentiality of personal data. This risk arises because wargames are conducted in group settings and require support from large research teams (e.g. rapporteurs and facilitators). 

However, scholars can effectively manage these risks by carefully applying best practices, such as minimisation of directly or indirectly identifiable personal data, pseudo-anonymisation, access limitation, data separation and retention policies. These risks can be further reduced by careful recruitment and training of game staff. (At King’s, we spend 6 months selecting and training our wargame rapporteurs.) 

For professional wargamers, the major ethical risk is the conflict of interest between them and their sponsor. Stephen Downs-Martin describes the issue well in this article. Research ethics problems deepen when lines of responsibility and accountability are not clearly defined, and when the research process is not (or cannot) be made transparent. Mitigating these risks requires clear communication between a wargame provider and their sponsor but doing so might not be in the self-interest of the parties involved. 

Other ethical risks will be just as big, regardless of setting. For example, risks of harm to individuals could result from using wargames to investigate topics that could trigger stress or violence. If a principal investigator uses deception, including not fully informing participants of the purpose of the wargame, this also raises ethics concerns. (Thanks to Rex Brynen for highlighting these points.) 

Are Scholars Better Positioned to Ensure Research Excellence in Wargaming?

Ensuring and demonstrating research excellence in wargaming requires understanding, applying, and enforcing integrity and ethics principles. These principles are well established, but expectations differ in academic and professional wargaming settings. Professional wargamers face greater ethical risks than scholars who wargame, and these risks cannot be easily mitigated.   

Overall, scholars at universities are better positioned to ensure research excellence in wargaming than their professional wargaming colleagues. This does not mean professional wargamers are less interested in honesty, rigour, transparency, or ethics. Quite on the contrary – the wargaming community of practice is conscious of these risks and limitations, and the topic of this year’s US Connections conference is testimony to this fact. But there are powerful institutional incentives that influence research integrity and ethics in practice, which cannot be wished away. 

If people who are professional wargamers want to effectively demonstrate research excellence in wargaming, they should consider a sabbatical to spend some time at a university. 

Ivanka Barzashka

%d bloggers like this: