PAXsims is pleased to present a selection of recently-published items on simulation and serious gaming. (OK, we’re a bit behind on this, but we’ve been busy!)
Some of these may not address conflict, peacebuilding, or development issues at all, but have been included because of the broader perspective they offer on games-based education or analysis. Others might address “gaming-adjacent” issues such as group dynamics and decision-making, assessment, forecasting, or related topics. Items previously featured on PAXsims are not included. However, if you have published something recently and we haven’t yet included it, let us know.
Articles may be gated/paywalled and not accessible without subscription access to the publication in which they appear.
This capstone project aimed to enhance the Center for Army Analysis’s (CAA) Strategic Competition and Crisis (SC2) wargame to improve the strategic planning skills of Global Combatant Command (GCCs) staffs. The client, CAA, developed the SC2 wargame two years ago “to capture the discussion associated with ‘Competition,’ and how it supports the Army’s Global Strategic Framework” (Engelmann & Kearney, 2021). The objective of the game is to improve one’s global reputation score through actions impacting Diplomatic, Military, and Economic influence. Last year, this capstone created a model to output perspective-based, dynamic reputation scores based on credibility, capabilities, and stability metrics (McConville, 2021). This year, this project focused on integrating the dynamic reputation model metrics to enhance the SC2 wargame by contextualizing game actions with aggregated data on strategic objectives, increasing realistic factors within the negotiation process, and improving the player’s narrative experience through architecture that automates game mechanics and incorporates a robust game user interface.
This paper analyzes the importance of digital game-based learning within the higher military education system. The advantages of implementing digital wargaming in the learning system are presented, including its risk-free and cost-effective nature, its ability to allow “what-if” scenarios, facilitate knowledge transfer, skill acquisition, attitudinal/behavioral change, time-saving benefits, and the ability to understand the operational environment or test new technologies. The combination of these specific advantages, along with the motivation derived from the enjoyment of the game, should be the focus of military education academics and practitioners. Additionally, the paper highlights the requirements that the chosen game must meet, such as complexity, realism, balance, and providing opportunities for learning and developing skills that are tracked through data collection and subsequent feedback. By integrating digital games and using the principle of “play, have fun and learn”, the learning process in higher military education can effectively improve the development of essential tactical skills that are difficult to achieve through other traditional teaching methods.
Recent years have seen increased interest in the professional use of wargames, and wargames are a potential tool to enable a better understanding of past urban conflicts and to plan for future urban security. Whilst access to professional wargames are limited, hobby wargames have been identified as useful and closely related areas to study. Previous work has identified around 214 manual hobby wargames that deal with urban conflict, but only 5 battles are covered by 5 or more wargames, and so provide a reasonable sample for comparative reviews. The Battle of Hue battle had many of the hallmarks of a modern urban battle, with both symmetric and asymmetric opposition, combined arms, a civilian and media presence, and the use of innovative technology. This article examines how 6 different wargame designers have approached the Battle of Hue, and how their design choices relate to the key characteristic of the Battle of Hue. The article also identifies where the principal deficiencies are. The article concludes by considering the issues highlighted by these games that wargaming has in representing urban conflict, and how these could be addressed in order to make wargaming a more useful tool to model urban conflict and security.
The paper proposes an approach to improving player engagement and learning outcomes in seminar-style games, based on emphasizing player agency, meaningful choices, and uncertainty. As a case study, it introduces a game focused on trust in cybersecurity, whose design incorporates resource management, hidden agendas for each player, and a shared failure condition.
We propose an integration framework for a wargame prototype using the modern game engine – Unreal Engine. This wargame functions as a simulation tool for strategy training, strategy testing, and simulating enemy forces like warships, aircraft, and weapons. Existing wargames come in proprietary and free versions. The former is often expensive and exclusive due to security reasons, while developing a functional wargame is complex, requiring various technological components such as a physics system and artificial intelligence (AI). To overcome these challenges, we propose a rapid prototype using Unreal Engine. This approach leverages advanced technology and ensures the prototype is ready for future upgrades when new versions of the game engine are released. We evaluate the prototype’s system capabilities and expert assessments.
Joseph Chedzoy, D’Andre Tobias, Joseph Miller, Max Hwang, and Trent Geisler, “Reinforcement Learning in Cyber Wargaming Defense,” Proceedings of the Annual General Donald R. Keith Memorial Conference, May 2023.
In recent decades the necessity for cyber security has grown for both private companies as well as government agencies. This growth is the result of increasing ability for organizations to mount cyber-attacks. As a response, organizations have been developing cyber defense artificial intelligence (AI), which greatly improves cyber-security capabilities. This ne- cessitates not only the development of cyber-attack, defense, and vulnerability frameworks to simulate a realistic environment, but also methods with which to train the AI. Further, the number and variety of networks necessitates a framework with which AI can be quickly and cost-effectively trained. This paper will explore how our team has worked to develop an efficient and comprehensive framework under which a variety of AI can be trained to fulfill the need for cyber resiliency
The “new” Great Game suggests that, like the imperial competition of the past, we are witnessing a trans-imperial moment whereby Japan and China are vying for hegemony in East Asia. This is a new moment because East Asia, unlike Europe, has never had two co-existing superpowers. The prospect of a new imperial competition is complicated by the still-present American military power and the non-statist arena, especially in popular culture, where the imperial games are played out. Using two popular anti-Japan videogames, Glorious Mission Online (2013) and The Invisible Guardian (2019) as case studies, I argue these games are symptomatic of the relations between warfare and game in general. I then outline the trend in game development that subverts conventional wargames. Finally, I speculate on alternative game design over the disputed territories in the Southern China Sea that prioritizes ecology over human conflict and development.
Serious games have long been used in domains like defense, management, finance, and environmental protection to improve plans and procedures. In the aftermath of the COVID-19 pandemic, public health and emergency management organizations are beginning to use such games to enhance their preparedness and readiness activities. In this paper, we present a Knowledge Acquisition Analytical Game (K2AG) focused on understanding and providing training for command, control, coordination, and communication (C3C) functions during an infectious disease outbreak. Unlike traditional game-based exercises, which target strategic, operational and tactical decision making, the K2AG games focus on the cognitive level at which decision-making under uncertainty takes place. Specifically, the C3C Game collects data reflecting the cognitive processes by which players gain situational awareness, make decisions, and take actions. The C3C Game was created through a community-centered design process and leverages methods from human factor engineering, including hierarchical task analysis. This paper describes the game, presents results from a pilot exercise conducting with public health and emergency response decision makers from a large US metropolitan area, and discusses the potential for such games to improve pandemic preparedness and resilience.
Reinforcement Learning (RL) has become a promising technique to deal with the tough decision modeling problem in the wargaming field. However, to deploy current RL algorithms requires reward-engineering scenario by scenario, which is laborious for massive wargaming scenarios. To tackle this issue, this paper proposes an improved RL method, curiosity-tuned experience replay (CTER), which allows the RL-driven decision model to achieve a relatively effective policy under the sparse reward. CTER uses the curiosity mechanism to regulate the three critical procedures during learning with experience replay: the exploration, storage, and revisitation of the experiences. Based on the prediction-based curiosity, CTER generates an intrinsic reward to fill the sparse reward space, and further provides an adaptive exploration strategy to collect more informative experiences. Moreover, CTER develops a novel prioritized replay and memory updating mechanism to reuse experiences more efficiently. Through the systematic evaluation and comparison on typical game tasks and wargaming tasks, CTER shows its effectiveness and generalization in different scenarios without reward-engineering. Especially, the policy performance of CTER-based RL with the sparse reward is almost equivalent to that of ordinary RL with dense engineered rewards. Our work may offer a relatively universal approach for wargaming decision modeling, which can free the RL-based decision modelers from the laborious reward-engineering.
Wargames and tabletop exercises (TTXs) have existed for centuries in military planning contexts but have seen relatively limited use as methods of inquiry within the human factors community of practice. We argue that that the use of these tools is not commensurate with their value, as they are effective and efficient means to conduct naturalistic inquiry. We demonstrate the value of wargames and TTXs by presenting two case studies of their use in different domains, highlighting how their outputs were used to achieve research objectives and to glean insights about human cognition in the context of work. We also discuss the similarity of wargames and TTXs to other methods of inquiry, and suggest future areas for research.
In mid-2022, after the opening months of the Russian invasion of Ukraine, a group of RAND Corporation researchers conducted an internal series of discussions to examine the recent events of the Russia-Ukraine War. All the participants had been involved in the design and execution of tabletop exercise (TTX) wargames involving Russia during the past eight years, mostly centered on the challenges facing a North Atlantic Treaty Organization defense of the Baltic states. The focus of their deliberations was understanding the reasons for the similarities and differences between how Russian forces had fared in those games and the relatively poor real-world performance of the Russian armed forces in Ukraine. This report summarizes the unclassified conclusions of the discussions that followed a comprehensive review of wargaming methods and analysis used. The authors also identify implications for future wargaming and analysis of potential conflicts involving major power adversaries. This is not an in-depth research report about Russian military capabilities or the war in Ukraine but rather the proceedings of an internal set of discussions from experienced wargamers reflecting on recent events.
Although numerous plausible scenarios can be envisaged for wargaming ADF contributions to coalition operations in the western Pacific, there are nevertheless limits to our knowledge based on the immaturity of rapidly evolving US operating concepts; or else they exist only in the classified realm. Based on what we do know, however, and the geopolitical context and rationales for policy, strategy and operations established in other Issue Briefs in this series it is possible to establish different representative scenarios that may be useful for wargaming ADF contributions.
Thus, other Issue Briefs have described how elements of strategy, concepts such as deterrence, identification of types of operations needed to support strategy, and consequent capability development needs can contribute to Australia’s defence planning. While having a viable ends, ways and means articulation of strategy, and the resources to execute it are critical to the ADF’s success, developing a viable coalition “theory of victory” is also essential.
The grid model is the core component of strategy campaign wargame environment model and influences other behavior models. Building multi-level and multi-resolution grid models is the basis of extending strategy campaign wargame, supporting fine-grained environment models and key tactical operations. In this paper, the modeling approach and models of multi-level and multi-resolution equal longitude and latitude division for strategy campaign wargame are put forward based on the military requirements and technical requirements. Firstly, grid map is divided by using the equal latitude and longitude quadrangle. Then the coordinate system cluster of equal latitude and longitude grid and related elements are defined. Finally, an example is given. By comparing with single-resolution hexagon grid modeling approach, the advantages are analyzed. The modeling approach can support multiple-resolution of strategy campaign wargame, and meet the requirements of running efficiency and environment model for strategy campaign wargame.
The paper reviews the literature on simulations-based teaching in the discipline of international relations and associated social science fields, tracing the development of frequently-used simulations platforms over the last half century. It will then examine the application of the Council on Foreign Relations Model Diplomacy program in three courses, International Crisis Simulations, Political Development and Politics of Asia at a small liberal arts college in the South from 2018 to 2019, and consider the effectiveness of simulations-based teaching in achieving desired learning outcomes, such as critical and analytical thinking, oral and written communication, and collaboration. Finally, the paper will provide practical steps and suggestions for the integration of Model Diplomacy and other simulations into an array of Asian studies courses.
Australia’s Defence Strategic Review (DSR) has been completed, with the final report handed to Government (Prime Minister of Australia, 2023). In considering priorities for Defence capability and assessing force structure, posture and preparedness, the DSR is likely to recommend difficult choices for a relatively small Defence Force which must live within necessarily-limited means. Much like in war itself, Defence’s ability to adapt to DSR outcomes will be crucial. Indeed, former defence official Paul Dibb has warned Defence requires ‘serious historical and cultural change’ to implement the DSR’s recommendations (Hurst, 2023). Such change could be painful, especially if each service views the DSR’s recommendations through the lens of a zero-sum game, with winners and losers. Doing so not only misses the point of the review itself, but diverts our attention away from an important—arguably unique—opportunity offered by the DSR: using it as a wargame to practise the kind of adaptability we will need to fight the next war. Australia’s current strategic circumstances, and their trajectory, mean we cannot afford to miss such an opportunity. This article discusses how the Royal Australian Air Force (RAAF) can use the DSR as a wargame to test its adaptability in a joint environment.
Wargaming is a pre-practice activity to study national security and competition, military conflict and war, crisis management, and other major strategic issues. An intelligent wargaming system needs the ability of artificial intelligence technology. This paper briefly summarizes the research progress of intelligent game, the evolution of wargaming, intelligent wargaming, and strategic gaming methods. From the perspective of game theory, it analyzes the game problem model for intelligent wargaming and sorts out the application mode of intelligent wargaming and the organization mode of a strategic game. An intelligent wargaming service-oriented architecture based on cloud native is proposed. Pre-training method for future decision-making cloud is explored for distributed confrontation at cloud edge, and a meta-game analysis framework for strategic wargaming is constructed for cognition of crisis events. It is expected to provide a scientific and effective reference for the research on the theory and tools of intelligent wargaming and offer theoretical and technical support for the new generation of intelligent cloud-edge cooperative command and control.
With rapid urbanization, the importance of urban warfare is increasing, and it is also required to reflect the characteristics of cities in wargame models. However, in the military’s wargame models, the urbanization factor was calculated and used without theoretical basis. In this study, we investigate techniques for estimating the urbanization factor using Fractal dimension theory. The urbanization factor we propose can suggest a logical and valid representative value when used in conjunction with Agent Based Model and other methodologies.
How do people make their decisions? Searching for the answer in the relevant literature, we can find that decisions are based either on rationality or intuition. Rational thinking is mainly observed in situations characterized by certainty (in terms of data or the consequences of decisions), while heuristic intuitive methods are mainly observed in situations of uncertainty. Training for the enhancement of decision making skills usually employs problem-based activities which mainly focus either only on rationality or only on intuition. However, problems in real life cannot always be solved with the contribution of only one way of thinking. In a decision making process often rationality works up to an extent and then intuition will lead to the final decision. For this reason, we designed and developed a game-based learning activity that enhances both rational and intuitive decision making skills. More specifically, we created a decision scenario in a virtual environment in which participants were provided with uncertainty-based information in their decision making process. As they tried to follow a rational decision making process, most of them realized that based on the given information they were confused and they had to decide intuitively at the end. This experiential learning activity was a tickler for the participants to decide under uncertainty and trust their intuition.
Wargames often include fog of war, i.e. hidden information, and multi-action turns, where each turn requires making multiple, sequential action choices. These properties provide unique challenges for Artificial Intelligence agents. Extensions to Monte-Carlo Tree Search (MCTS) allow it to perform well in games with hidden information as well as multi-action games. However, these extensions do not specifically consider both properties simultaneously nor how information-gaining actions could improve agent performance. Information-gaining actions are important in multi-action turns where initial actions can reveal state information, thus improving later action decisions. This paper presents enhancements to MCTS that add an information gain incentive and a risk determinization to balance locating opponent pieces while minimizing exposure to enemy fire. The information gain incentive and risk functions are implemented in Perfect Information-MCTS (PIMCTS) and Information Set-MCTS (ISMCTS) and evaluated on the multi-action hidden information game TUBSTAP. Results show that these additions improve performance over the baseline algorithms and against a Cheating MCTS implementation.
Hao Liu, Yanyan Ding, Haimin Hu, Hongyan Ou, and Junfei Wang, “Thinking of War Game Lab Construction,” International Conference on Man-Machine-Environment System Engineering, 2023.
Actively innovating and developing simulated training means is an effective way to enhance the confrontational, strategic, and actual combat nature of combat training. War game has broad application prospects in the fields of military education and training and war simulation, and wargaming is the origin and important development direction of modern combat simulation. Starting from the significance of the construction of the war game lab of the air defense troops, this paper puts forward the construction objectives, functions and specific contents of the war game lab, and discusses the relevant issues that should be grasped in the construction of the war game lab, which is of great significance for promoting the transformation of our army’s training mode from mechanization to informatization, and further improving the quality and efficiency of combat training.
In tactical wargames, such as Warhammer 40K, two or more players control asymmetrical armies that include multiple units of different types and strengths. In these type of games, unit are assigned point costs, which are used to ensure that all players will control armies of similar strength. Players are provided with a total budget of points they can spend to purchase units that will be part of their army lists. Calculating the point value of individual units is a tedious manual process, which often requires long play-testing sessions and iterations of adjustments. In this paper, we propose an automated way of predicting these point costs using a linear regression approach. We use a multi- unit, turn-based, non-balanced game that has three asymmetric armies. We use Monte Carlo Tree Search agents to simulate the players, using different heuristics to emulate playing strategies. We present six different variants of our unit-point prediction algorithm, and we show how our best variant is able to almost reduce the unbalanced nature of the game by half.
Wargames and wargaming have been used by military planners since the Middle Ages. Games have been employed to model evolving strategies and emerging technologies. There has been an absence of research regarding wargaming non-traditional and uncertain domains such as disinformation warfare. The problem is further complicated by the unfamiliarity of the military leaders confronting disinformation warfare. This chapter demonstrates how to develop wargames to evaluate the strategies and tactics of disinformation warfare. The key is developing games that employ familiar operational elements such as terrain, maneuver, and combat in the design. This design method should include some adjunct forms of validity such as construct, content, and face validation. Aspects of a disinformation wargame can also be employed in workshops and tabletop exercises. This research indicates that disinformation warfare can be understood and strategies to employ it or counter it can be developed using wargames.
The study to be presented is about observing the war and crisis games research domain regarding the Chinese BRI (Belt and Road Initiative). In this respect, it will be necessary to detect the trends and uses of wargaming as a tool to train and predict, considering the state of the art of discipline. Given the transitions and developments of Chinese projection in the international system, it is intended to qualify how the arrangements under development collaborate with the project to expand the influence of the Belt and Road Initiative. Likewise, there is the need to segment the principles and models in progress for a possible exploratory typification derived from the study. For this purpose, we use primary and secondary sources related to the government structure and the intelligentsia system, primarily from think tanks and universities. As such, the understanding is that observing the characteristics in progress can allow for estimating the synergies or gaps between the model of influence and expansion in progress and the domain of wargaming developments. Then, we highlight that the wargaming knowledge is part of building upon the national security agenda to uphold Chinese international participation from the constitution of the Chinese national identity. Furthermore, we identify ways wargaming could help with the challenges and opportunities in the literature on BRI. We note that alongside the wargaming national security perspective, the technical training apparatus marks an influential trend for Chinese defense development by the West.
Scholars have been using the wrong card games to analyze Carl von Clausewitz’s analogies in On War, which has led to errors in understanding his ideas. This article identifies the games Clausewitz discusses, allowing for a more accurate interpretation of his original meaning for the study of war. Since Clausewitz’s ideas underpin strategy development within service education systems, it is critical his ideas are fully understood in context.
The military has developed wargames to explore their doctrines and train operational concepts. Wargames are often designed to teach educational objectives and share and produce knowledge. The success of a wargame design depends on how it achieves these objectives. Players are the main actors in wargames and often group into teams. The final result of the wargame depends on the collaboration and interaction between the members of each team. Therefore, this work aims to understand how collaboration occurs in wargames. Moreover, we observed the activities and processes during a large-scale wargame to gather how the players collaborate. A better understanding of collaboration in wargames may provide insights for game designers to improve the wargames design process to consider how the players collaborate.
The recent literature has inquired how media, official discourses, popular culture, or even sports and toys, can shape political imaginary into thinking that military intervention is the only relevant course of action. In this endeavour, the role of wargaming in justifying further militarisation remains largely understudied. By building upon the RAND Baltic 2014–2015 wargame and the subsequent NATO decision to deploy troops in the Baltics and Poland, I propose that certain wargames can legitimise the use of force in three ways: by reflecting the security community’s concerns in the storyline of the game, by designing the game in such a manner that preparation for war becomes the only well-founded means of tackling the issues posed by the game, and by enhancing its circulation in the defence field, notably by presenting the wargame as having the same level of credibility as science. Drawing upon assemblage theory, I propose that wargaming encompasses more than the individual experiences of its players. It encompasses the extensive sociotechnical assemblage of practices, technologies, networks of actors, and resulting emergent properties that together can amplify the conditions of possibility favourable to military deployment.
How do emerging technologies affect nuclear stability? In this paper, we use a quasi-experimental cyber-nuclear wargame with 580 players to explore three hypotheses about emerging technologies and nuclear stability: (1) technological uncertainty leads to preemption and escalation; (2) technological uncertainty leads to restraint; and (3) technological certainty leads to escalation through aggressive counterforce campaigns. The wargames suggest that uncertainty and fear about cyber vulnerabilities create no immediate incentives for preemptive nuclear use. The greater danger to strategic stabil- ity lies in how overconfidence in cyber exploits incentivizes more aggressive counterforce campaigns and, secondarily, how vulnerabilities encourage predelegation or automation. Both of these effects suggest worrisome relationships between cyber exploits and inadvertent nuclear use on one hand and cyber vulnerabilities and accidental nuclear use on the other hand. Together, these findings reveal the complicated relationship between pathways to escalation and strategic stability, highlighting the role that confidence and perhaps- misplaced certainty—versus uncertainty and fear—play in strategic stability.
War-Gaming is recognized as a valuable tool for commanders, leaders, and managers. Well-executed War-Games have delivered significant competitive advantages in numerous conflicts. The war-game confirmed the commanders’ knowledge of weapon systems and performance, as well as the time and space necessary to carry out battlefield maneuvers. One of the primary missions of each army on the battlefield is weapon target assignment. The weapon target assignment (WTA) is a critical problem to command to be solved in battlefield decisions. In a WTA problem, we should assign available weapons to determined targets appropriately to optimize the performance criteria. This study discusses a problem in relation to allocating and scheduling in WTA considering the mobility weapons and mobility targets. Bi-level linear programming problem is defined so that each level independently optimizes its own objective functions but is influenced by actions taken by another unit. To solve the under studied problem, three famous meta-heuristic algorithms including simulated annealing (SA), genetic algorithm (GA) and grey wolf optimizer (GWO) methods are proposed. Since the performance of meta-heuristic algorithms depends on setting the parameters, the Taguchi method has been used statistically for set parameters of the developed Algorithms. Performance evaluation of the presented algorithms is conducted through numerical experiments involving randomly generated test problems. To compare the results of proposed meta-heuristic algorithms, ANOVA and Tukey tests were used. The Computational results have shown that proposed GWO algorithm worked better than the SA and GA algorithms.
There are multiple arguments for and against wargames. Many scientists do not recognise the science in wargames. It is suggested that there is not enough literature relating to wargaming, for there to be any large-scale research into wargames. This is primarily because scientists often refuse to publish results, thus creating a vicious cycle where research is not published because there is not enough research being published. This ultimately deters researchers from studying wargaming in any serious fashion. Owing to this limitation, published work on the results, and protocols of wargames are scarce in scholarly research. Wargaming has considerably less academic focus with a fragmented and practical focus on design and benchmarking. This is surprising given the long history of wargaming (dating back to the early 1600’s), when compared to the relatively recent history of other domains such as software engineering. To better understand the current state of research into wargaming in reference to benchmarking and validation, a scoping review (SR) was conducted. The scholarly research into wargaming reveals papers on general modelling, conflict modelling, influence modelling, evaluation of wargames, analytical tools, use of AI in wargame design, evaluation of predictive modelling in wargames, improving command and control through wargaming, and cost-benefit analysis for decision making. The initial analysis of the coverage of wargaming research, together with the limited number of papers found, indicate that there is a distinct lack of academic research into wargaming. Additionally, there is a wide variety of areas that are interested in the wargaming field, however, with no universal method of analysis or benchmarking, this limits the reproducibility of results, and the ability to judge the overall effectiveness of wargaming efforts. Wargame designers need to be able to assess wargame components to validate, compare, and predict the effects on gameplay and for decision-makers to draw conclusions with more confidence.
To assist the U.S. Department of Defense and other stakeholders addressing the subsiding interest in and knowledge of a continued nuclear threat, RAND Corporation game designers and subject-area experts developed A Strange Game, named in reference to the 1983 techno-thriller movie, Wargames, which presents thermonuclear war as a conflict that no one can win. A Strange Game, however, very much concerns the present day and the future in that it gives players an opportunity to explore the problems of near- or longer-term future intra-war deterrence where the taboo against the use of nuclear weapons has broken down.
This report provides the scientific and theoretical background that informed the creation of the A Strange Game prototype. This report also offers relevant context that went into game design. Therefore, it should be useful to those considering adopting the game, game masters who will want to conduct fully informed gaming sessions, and players wanting a first-hand account of the diplomatic and military concepts used in the game. The information presented in this report can also assist other game designers as they work toward the development of further games that immerse participants in environments designed to improve decisionmaking before, during, and after conflict.
Wargames are essential simulators for various war scenarios. However, the increasing pace of warfare has rendered traditional wargame decision-making methods inadequate. To address this challenge, wargame-assisted decision-making methods that leverage artificial intelligence techniques, notably reinforcement learning, have emerged as a promising solution. The current wargame environment is beset by a large decision space and sparse rewards, presenting obstacles to optimizing decision-making methods. To overcome these hurdles, a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) based wargame decision-making method is presented. The Partially Observable Markov Decision Process (POMDP), joint action-value function, and the Gumbel-Softmax estimator are applied to optimize MADDPG in order to adapt to the wargame environment. Furthermore, a wargame decision-making method based on the improved MADDPG algorithm is proposed. Using supervised learning in the proposed approach, the training efficiency is improved and the space for manipulation before the reinforcement learning phase is reduced. In addition, a policy gradient estimator is incorporated to reduce the action space and to obtain the global optimal solution. Furthermore, an additional reward function is designed to address the sparse reward problem. The experimental results demonstrate that our proposed wargame decision-making method outperforms the pre-optimization algorithm and other algorithms based on the AC framework in the wargame environment. Our approach offers a promising solution to the challenging problem of decision-making in wargame scenarios, particularly given the increasing speed and complexity of modern warfare.
The Secretary of the Navy disperses Navy forces in a deliberate manner to support Department of Defense (DoD) guidance, policy, and budget. The current strategic, laydown, and dispersal (SLD) process is labor intensive, time intensive, and less capable of becoming agile for considering competing alternative plans. SLD could benefit from the implementation of artificial intelligence. We introduced a relatively new methodology to address these questions which was recently derived from an earlier Office of Naval Research funded project that combined deep analytics of machine learning, optimization, and wargames. This methodology is entitled LAILOW which encompasses Leverage AI to Learn, Optimize, and Wargame (LAILOW). In this paper, we developed a stand-alone set of pseudo data that mimicked the actual, classified data so that experimental excursions could be performed safely. We show LAILOW produces a score from a wargame-like scenario for every available ship that might be moved. The score for each ship increases as fewer resources (e.g., lower cost) are required to fulfill an SLD plan requirement to move that ship to a new homeport. This produces a mathematical model that enables the immediate comparison between competing or alternate ship movement scenarios that might be chosen instead. We envision a more integrated, coherent, and large-scale deep analytics effort leveraging methods that link to existing real data sources to more easily enable the direct comparisons of potential scenarios of platform movement considered through the SLD process. The resulting product could facilitate decision makers’ ability to learn, document, and track the reasons for complex decision making of each SLD process and identify potential improvements and efficiencies for force development and force generation.
PAXsims 