PAXsims

Conflict simulation, peacebuilding, and development

Category Archives: simulation and gaming history

Wargaming, bomber escorts, and the P-51 Mustang

P51.jpeg

War is Boring has just published an excellent piece by James Perry Stevenson and Pierre Sprey on the P-51 Mustang fighter aircraft, highlighting the many doctrinal and bureaucratic battles that shaped its development during WWII. Among the points that are made: many early US army aviation wargames and exercises were designed to validate the flawed concept that fast bombers required no fighter escort, while later wargames and exercises that pointed to the vulnerabilities of unescorted bombers were ignored or reinterpreted.

Between World War I and World War II, bombers began flying higher and faster than existing obsolete biplane fighters. Still, the U.S. Army Air Corps’ bomber generals refused to foresee that enemy fighters might prevent the lumbering aircraft from always getting through to the target.

These officers even ran field exercises designed to support their premises of bomber invincibility. U.S. Army Lt. Col. Henry “Hap” Arnold, a leading bomber advocate who would eventually become chief of the service’s Air Corps, was particularly determined to prove this point.

“Exercises held in 1931 seem to reinforce the idea that fast bombers could fare well on their own,” military historian Dr. Tami Davis Biddle wrote in Rhetoric and Reality in Air Warfare. “Arnold reached this conclusion, as did the umpires, one of whom proclaimed: ‘[I]t is impossible for fighters to intercept bombers and therefore it is inconsistent with the employment of air force to develop fighters.’”

This rigid mindset became embedded in the Army’s air power strategy, its budget battles and its endless barrages of air power propaganda.
Within just a few years, however, fighter war games and actual air combat abroad provided ample evidence the Army Air Corps brass was committed to the wrong conclusion.

“[I]n 1933, … squadrons intercepted 55 percent of enemy day-formations as they flew toward the target, … another 26 percent as they left it; [and] 67 percent of individual night raiders were intercepted,” Biddle noted. “But what might have seemed clear defensive victories were not perceived as such: proponents of strategic bombing refused to grasp the devastating bomber attrition forecast by these exercise outcomes.”

“When assessing results, the bomber advocates created both formal rules and cognitive filters [to insure] they would see what they expected to see: the primacy of the aerial offensive waged by determined bombers,” she added. “The rules under which the exercise were run gave advantages to bombers, and umpire rulings explained away unexpected, [inconvenient] results.”

The piece highlights that wargames themselves take place in a broader institutional and doctrinal context, in which sponsors and participants may be anxious to have them serve other agendas. Of course, good wargame design can reduce some of these problems of bias and cognitive filtering. However, the design alone is not enough: serious gamers also need to think to about the broader processes within which their games are embedded.

Engle: A short history of matrix games

There has been growing interest in matrix gaming in recent years, and it is a topic that we have covered extensively here at PAXsims. A few days ago Bob Cordery also posted a fascinating account of the early development of matrix gaming in the UK at his Wargaming Miscellany blog.

Today we are very pleased to present a piece by the inventor of the matrix gaming approach, himself Chris Engle.


 

A Short History of Matrix Gaming

Interest is rising in matrix games, and along with it some questions and confusion about the history of the idea. Here is an account of my part in the project. I draw it from my published games and articles, personal journals, and my recollections of anecdotes.

I invented the idea of matrix gaming in January 1988, just after finishing my Masters degree in Social Work. I was visiting a philosophy graduate student friend in Bloomington Indiana. We were discussing the idea of how to roleplay entire countries. He wanted to do it with a set of numbered statistics. I proposed using words. This grew out of my work as a psychotherapist. My practice has always included a strong use of narrative and teaching allegories (especially Sufi teaching stories which eventually lead to my conversion to Islam). My friend thought the idea unworkable so we agreed to work on the problem from our different approaches. Matrix games then grew out of an interesting question. How can you run a game with words rather than numbers?

The answer is two fold. First how to describe the world using words and second how to put that verbal picture into motion. The picture of the world is the matrix of matrix games. I started off using literal matrixes of short phrases that described various institutions and ideas. Together with scenario information (maps, character descriptions, and opening events) each player forms their own mental matrix of the world, a gestalt. The matrix of the world changes by additions to the narrative. Each turn players make arguments about what they want to have happen next.

This was a brand new idea in 1988 but I had the idea that it was good and that if I was willing to do the footwork it could spread. All it required was dedication and a willingness to stick with the message. I set a goal of talking about it and to keep on talking till someone asked me why I was saying the obvious. It took years before that happened.

I wrote the first article on matrix games in 1988. “Verbal analysis wargaming” appeared in Nugget 44 (the newsletter of Wargame Developments). It earned the Editor’s Award for most original game idea of 1988. I set a goal of telling one hundred people about the idea over the next year. I did this by writing more articles and running games at conventions in the Midwest USA. Over the next couple of years I got encouragement from some game community luminaries such as Frank Chadwick, then of Game Designers Workshop. Steve Jackson, of Steve Jackson Games, told me you would need a Masters degree in philosophy to play the game. Which I knew this was wrong because I had already had mentally handicapped people play it.

From 1989 to 1994 I published the Experimental Game Group newsletter. It was mentioned in Simulation and Gaming. I used it to work out rules and test them in yearly play by mail games. Early games included a replay of the events of the fall of Communism in 1989, which predicted the refusal of the Russian army to back the communist party and the secession of Russia from the Soviet Union before they happened. The Peninsular Campaign in 1809, the French Revolution and an Agatha Christie murder mystery followed.

My second goal was to have one hundred people play a matrix game and to tell one thousand people about it over three years. I wrote around sixty articles in gaming magazines like The Midwest Wargamer’s Association Newsletter, Lone Warrior (the journal of the Solo Wargamer’s Association,) PW Review and various Historical Miniature Game Society newsletters on top of publishing EGG. I ran matrix games at Midwestern and Near South gaming conventions including Gen Con and talked about them to anyone who would listen. I viewed gaming as a market of ideas so at some point I needed to produce an actual product. I did this in 1992. “Campaign in a Day” presented game rules, military campaign scenarios, and a miniatures game and was the basis of the game later adopted by the British army in the mid 90’s.

I corresponded widely and Peter Suber of Earlham College recognized matrix games as Nomic games in 1994. Paddy Griffin recognized them as Mugger games.

In England other writers including Bob Cordery and Tim Price started developing their own matrix games in 1990. These are important games but I am not the one to best describe them. After 1994 our two trains of development diverged when I stopped writing articles and began work on developing commercial games.

Dave Arneson, co-creator of Dungeons and Dragons, mentored and encouraged me to learn the business starting in 1995. He later introduced me to someone as “This is Chris Engle. He makes weird games.” I continued to build up my contacts in the game industry and learned about business and the mechanical process of game production. I published Dark Portals my first professional game in 1998. I followed this with a series of role play game like books from 1999 to 2005. After that I put out board game versions between 2006 to 2011 and eventually card games 2012 to 2014. Historically about half of all my players have been women. Unfortunately none of my products were commercial successes. I closed Hamster Press in 2015 and began work on an archive of all my game writings. I’ve got several interesting books from that and am looking for a publisher. Any suggestions would be appreciated.

Right now I’m working on a professional matrix game book that spells out an intellectual argument for the game approach and includes chapters on many ways to apply it. One new idea is to do iterative matrix games. I see them as a cheap way to collect a body of mineable data from a potentially large body of players. It can tap the wisdom of crowds in a completely new way. I would like to have a lot of co-authors in this project, pulling on many people’s experiences. Over the years, matrix game have been used by the British and Australian armies for military planning and reorganization, in education to teach history and creative writing, by myself in psychotherapy and by the French army to teach English. I’m aware of academics using it to explore literary criticism and the nature of being European. Some people are beginning to use them in business consulting.

Matrix games started as an idea. With work they grew into articles and published games. Now they are wide spread and looks like they will be useful to a growing body of users. My nearly thirty years of experience boils down to a few simple rules. Start with a problem. Pick a scene and say what happens. Others can add to that or change it. This overwrites what was said before. Anyone can ask you to roll to see if the action doesn’t happen. When the problem is solved the game ends.

Chris Engle 

Hanson on “Improving Operational Wargaming: It’s All Fun and Games Until Someone Loses a War”

i-think-you-should_sharris.jpg

Lt Col Matthew E. Hanson (USAF) recently submitted a monograph entitled “Improving Operational Wargaming: It’s All Fun and Games Until Someone Loses a War” for the School of Advanced Military Studies, United States Army Command and General Staff College. I am grateful for his permission to post a copy here (pdf).

In the monograph he explores how the theory and practice of wargaming often diverge, with negative consequences. He further argues that current US military wargaming doctrine does not sufficiently address this problem.

In 2015, Deputy Secretary of Defense Robert Work committed the Department of Defense (DOD) to overhaul its approach to wargaming in order to reinvigorate innovation across the DOD, including a five-year target to use wargames to improve operational planning. This monograph explores the causes of wargaming failures and proposes recommendations for successful wargames. Does doctrine provide sufficient guidance, striking the appropriate balance between prescriptive and descriptive guidance? This monograph postulates that wargaming theory—including game element analysis and wargame pathologies—provides an excellent rubric for creating and evaluating wargames and wargaming doctrine, that doctrine and practice diverge from wargame theory, and that current doctrine does not provide sufficient guidance. The theory—history—doctrine approach of this monograph is intended for military planners, doctrine authors, and wargaming professionals.

Wargames are a useful tool to assess plans as directed in operational planning processes; however, commanders and staffs should neither equate wargame victory with wargame success, nor consider either as “validation” of a given plan. There are ten elements of wargame design: objectives, scenario, database, models, rules and procedures, infrastructure, participants, analysis, culture and environment, and audiences. These elements provide a framework for creating wargames, and analyzing wargames and their failure modes (known as pathologies).

By evaluating Japan’s Midway campaign plan through the theories of game element analysis and wargame pathologies, this monograph creates greater understanding of those theories and provides recommendations for doctrine. Pathologies exhibited by Japanese planners include those related to wargame objectives, scenario, database, model, participants, and culture; genuine testing of the Operation MI plan appears to have been impossible. Wargame officials twice rejected inconvenient outcomes, undermining the credibility of the game, creating lasting controversy, and preventing meaningful analysis.

Current operational planning doctrine lacks sufficient detail on how to design and conduct wargames, neglecting the diverse needs of planning staffs. At present, doctrine diverges from wargame theory in its contents and by its omissions. Improving doctrine would capitalize on these insights and potentially avert an otherwise foreseeable military catastrophe.

In the absence of updated joint and service doctrine, operational planners will lack the descriptive—yet detailed—instruction necessary to ensure useful and valid operational planning wargames. Doctrine authors should include the lessons of game element analysis, wargame pathologies, and other sources into joint and service doctrine to assist operational planners in creating wargames that are theoretically sound and operationally insightful.

Lt Col Hanson is interested in constructive feedback from PAXsims readers, and especially comments that address the following points:

  • New sources (particularly primary sources) on Midway that would enable stronger correlation to the battle outcome and/or the pathologies framework
  • Similar sources for a secondary case study such as Tannenberg or Barbarossa.
  • Additional evidence/proof for the efficacy of wargames in testing and strengthening operational plans.  How does a commander and his planning staff know that wargaming will improve their planning outcomes?  Can I improve from my general recommendations to improve wargame doctrine to more specific practices and techniques relevant to the operational planner?

Comments can be left in the comments section.

MORS wargaming news

Vol 49 N2 .jpg

The latest issue of the Military Operations Research Society magazine Phalanx (June 2016) contains an article by Michael Garrambone (InfoScitex Corporation), Lee Ann Rutledge (Air Force Resesearch Lab), and Trena Covington Lilly (Johns Hopkins University/APL) on “Wargaming at MORS for Another 50.”

MORS has been involved in military wargaming for most of its existence. There were wargaming working groups in the symposia of the early 1970s, and various members of the operations research community have made many presentations on gaming through the years.

MORSwargaming.png

The same issue also contains an announcement of the MORS special workshop on wargaming to be held in the fall:

MORS will hold a special workshop on wargaming in support of the Department of Defense on October 24–27 at the National Defense University in Washington, DC. The Fall 2016 Wargaming meeting will be the second recent MORS meeting on wargaming and is in response to the continued interest in wargaming from senior levels in the Department of Defense (DoD). It will serve as a venue for the services and others to share wargaming best practices and wargaming insights that have impacted service programs. It will also focus on how wargaming and other forms of analysis should best complement each other. This meeting will have portions at the SECRET/NOFORN level, as well as some unclassified sessions. Unclassified tutorials will be held October 24.

This workshop will focus on wargame execution and will provide senior officials leading the wargaming efforts within DoD a forum to provide guidance and answer questions. The workshop will showcase how wargames have been, are being, and will be employed in analytic processes within the department. During the workshop, working groups will discuss wargaming design, methods, and best practices, and provide hands on training for participants.

For details of last year’s MORS special meeting on wargaming, see my report for PAXsims. Information on the MORS Wargaming Community of Practice can be found here.

If the Cold War went hot in Asia

 

PournelleF2Sept14.jpg

Wargaming at the US NWC (1987).

Last month in the National Interest—and reprinted today in War is Boring—Robert Farley reviews the late Cold war series of Global wargames at the Naval War College, and what they had to say about a potential US-Soviet clash in Asia:

Nearly every analyst during the Cold War agreed that, if Moscow and Washington could keep the nukes from flying, the Central Front in Europe would prove decisive in war between the United States and the Soviet Union. The NATO alliance protected the Western European allies of the United States from Soviet aggression, while the Warsaw Pact provided the USSR with its own buffer against Germany.

But when the Cold War really went hot, the fighting took place in Asia. In Korea and Vietnam, the Soviet Union waged proxy struggles against the United States, and both sides used every tool available to control the destiny of China. However, while few believed that the Pacific theater would determine the victor of World War III, both the United States and Soviet Union needed to prepare for the eventuality of war there.

Scholars have devoted far less attention to the planning of World War III in East Asia than to the European theater. The two classic novels of the Third World War (Tom Clancy’s Red Storm Rising and John Hackett’s The Third World War) rarely touched on developments in Asia. However, in the 1970s and 1980s, the Naval War College traced the potential course of war in East Asia as part of a series of global war games. These games lend a great deal of insight into the key actors in the conflict, and how the decisive battles of a Second Pacific War might have played out.

Both the Soviets and the Americans had options in Asia. The strategic environment was far more fluid than in Europe, allowing a variety of different choices to disrupt and destabilize the opponent. This made the course of war far less predictable. At its (nonnuclear) worst, war could have raged across Asia on multiple fronts, from Korea to Japan to the Sino-Soviet border. At its best, the combatants might have observed an uneasy quiet, at least until it became necessary to outflank a stalemate in the West. But as was the case in Europe, everyone concerned is fortunate that tensions never led to open combat.

For more on wargaming (or, as they would have it, war gaming) at the US Naval War College, see their website. This includes unclassified reports from some of the more recent Global series games.

Matrix games at the US Army War College

USAWC.jpgThe following piece was contributed by Colonel Jerry Hall and Lieutenant Colonel Joseph Chretien of the Strategic Simulations Division (SSD), Center for Strategic Leadership, U.S. Army War College.


Dr. Rex Brynen of McGill University in Montreal, Canada recently delivered a presentation on “Conflict Simulation and Gaming in the Classroom” at the Royal Military College of Canada in Kingston, Ontario. During the workshop, Dr. Brynen introduced us to Matrix Games. He also moderated “ISIS Crisis,” a Matrix Game on the rise of ISIS set in the summer of 2014. Matrix Games have the potential to enhance experiential education in both wargaming and Professional Military Education (PME).

A Matrix Game is a low-overhead, facilitated, multi-player, argument-based game where players propose actions, weigh arguments and counter-arguments, and a die roll decides success or failure. Matrix Games typically last 2-3 hours and require a scenario with map and counters, a facilitator/umpire, a subject matter expert, and 4-6 players or teams of players. Matrix Games can be created on any topic, however the focus of this article is on strategic geopolitical crisis Matrix Games.

Chris Engle created Matrix Games in the late 1980s. He wanted to develop a game system in which it was possible for a player to role-play an entire country, but that did not have extensive rules, unit counters and combat results tables (like most wargames).[1] He based his system on roleplaying games, using a free-play framework where players propose actions, state their desired effect, and then posit arguments in support of why they believe the proposed action will succeed (other players may offer counter-arguments). Initially his games included a matrix of cue words, although over time the matrix was dropped, but the name stuck.[2] For additional information on Matrix Games, as well as free Matrix Games, see:

ISIScrisismap

Subsequently, the Strategic Simulations Division at the Army War College hosted its first Matrix Game demonstration session on December 10, 2015 for staff members of the Center for Strategic Leadership. The purpose of the demonstration was to provide an overview of Matrix Games and their potential for use as an additional wargaming method. The War College hosts several strategic wargames a year, using the two-sided seminar format. In ISIS Crisis, the participants represent one of six sides: the United States, Iran, Iraqi Government, Sunni minority, Iraqi Kurds, and ISIS. Prior to the game, each team was provided team-specific background information, objectives, and a special rules card explaining rules unique to each side.

AUTHOR’S NOTE: The purpose of the ISIS Crisis demonstration described below was to inform staff members on the Matrix Game methodology, not to formulate policy or strategy. Player actions do not reflect official policy of the Department of the Army, Department of Defense, or the U.S. Government.

The game began with a strategy and diplomacy phase, during which each team developed its strategy and conducted diplomatic negotiations with other the teams. For some teams, the negotiation session was instrumental in brokering deals that would significantly shape the subsequent gameplay. For others, the negotiation phase provided a sense of where they stood politically with other teams.

AWCbriefing

The pre-game overview brief.

At the end of the strategy and diplomacy phase, each team announced the results of any negotiations (if they chose to). The United States team used the opportunity to announce a “four point” strategy for defeating ISIS. The final point of the strategy was support for a confederation system of government in Iraq, rather than continuing to support the Shia-dominated “unity” government. This announcement both surprised and immediately impacted the other players, especially the Iraqi, Iranian, Sunni minority and Kurdish teams.

The US team’s policy announcement set the tone for the game. The US built on its policy announcement by conducting a strategic information operations campaign to discredit ISIS and reduce its ability to recruit foreign fighters. Following a successful ISIS attack into Kurdish controlled Hasakah province and a successful Kurdish counter-offensive into Mosul, the US team deployed a significant aid package to the Kurds, in the form of air support, advisors, equipment and funding. Iraq interpreted the US policy statement and its direct support of the Kurds as destabilizing and sought to conduct reforms to increase minority representation in Parliament and its Ministries. The reform movement failed however, and the predominantly Shia Iraqi government faced the situation of a US-backed and resurgent Kurdish minority, combined with a now disenchanted Sunni minority leaning toward ISIS. The Iraqi Government responded by publicly appealing for military support. Iran responded to the call by announcing it would deploy ground forces into Iraq to help combat ISIS (the Iraqi and Iranian teams struck this secret deal during the diplomacy phase, unknown to the facilitator and the other players).

maporientation

Map orientation.

The US continued its diplomatic efforts to defeat ISIS by approaching the Gulf Cooperation Council (GCC) countries and asking them for help in cutting off funding for ISIS, then announced their own version of the 2014 Iranian Nuclear Deal, with a caveat that allowed Iran to deploy into Iraq to help fight ISIS. This deal effectively divided Iraq into a three-party state by driving the Sunni minority toward ISIS and further reinforcing Kurdish autonomy. ISIS and the Sunni minority successfully took Tikrit, then Fallujah, while Iranian forces deployed into Najaf, Karbalah and Samarra. ISIS then successfully conducted a covert operation in Samarra, destroying several Sunni mosques with explosives, and blaming Iranian forces. This event further strengthened the fissures between the Iraqi Government, the Sunnis and the Kurds. The Iraqi government attempted to gloss over the situation by conducting a “One Iraq” strategic communication campaign, but it did not reflect reality on the ground and was ignored by the other players. The game ended with Iraq in control of its Shia regions with significant Iranian ground forces, ISIS in control of the Sunni regions, including Tikrit and Fallujah, and the US-backed Kurds firmly in control of the Kurdish region. The new US policy announcement and the clever Iranian deals with the US and Iraq effectively created a three party Iraq. A by turn summary of all player actions as recorded is at the end of this article.

Iranteam

Iranian team deliberations.

The after action review with the players, who were a mix of wargaming and research analysis experts, yielded several insights. Collectively the players thought that Matrix Games could be most beneficial before or during (or even in place of) the prevalent two-sided seminar wargaming method. They felt that the Matrix Game method better promoted participation and engagement among the players. The analysts felt that Matrix Games provided more quantitative data to collect due to the increased interaction, as well as more qualitative data in the form of the supporting arguments and the die rolls. All players thought that Matrix Games would be best for current or future (potential) conflicts to avoid participant knowledge of historical scenarios. They did acknowledge that historical scenarios could be used for Matrix Games to gain insights and understanding into why actors behaved as they did in historical conflicts.

Finally, ISIS Crisis demonstrated the potential utility of Matrix Games in policy and strategy formulation. National Security practitioners could conduct multiple iterations of a Matrix Game, testing a different policy or strategy approach in each one, to gain insights into how the various parties may react. For example, had this game been a test of a “confederation Iraq” policy, the US team would likely discard that policy course of action due to the implications vis-à-vis Iran, the Sunnis and the Kurds.

Since this ISIS Crisis demonstration, we briefed the War College Commandant and began to design our own Matrix Games. We plan to provide the War College faculty training on the use of Matrix Games as another tool in their instructor “toolkit” and look forward to providing future strategic leaders an additional experiential education experience during their time here at Carlisle Barracks.

ISIS Crisis actions by turn summary:

Turns 0-1

  • Turn 0 (Diplomacy Round): US announced new “4 Point” Policy to defeat ISIS; final point was support for an Iraqi Confederation Government
  • US: Global IO Campaign to discredit ISIS (success)
  • Iran: Negotiate covert SOF advisors and equipment to Syria (success)
  • ISIS: Conquer Hasakah Province from Kurds (success; doubles*)
  • ISIS Free Move: Counter US IO Campaign based on taking Hasakah (success)
  • Iraq: Expand minority representation across minsitries (fail)
  • Sunni: Propose law for proportional minority representation in Parliament (fail)
  • Kurds: Conquer Mosul from ISIS (success)

*ISIS Crisis special rule: when any player rolls doubles on two six-sided dice, ISIS receives a bonus action related to the roll.

Turn 2

  • US: Deploy forces in support of Kurds (Drones, SOF, Air, Equipment) (success)
  • Iran: Move SOF (via air) and equipment (via sea) to Syria (fail; moved but detected and attributed to Iran)
  • ISIS: Retake Mosul from Kurds (fail)
  • Iraq: Open request for ground forces in support of fight against ISIS (no roll; Iran agrees to help)
  • Sunni: Conduct uprising in Tikrit: phase 1 build militia (success)
  • Kurds: Retake Hasakah Province from ISIS (fail; doubles)
  • ISIS Free Move: Provide support to Sunnis for Tikrit uprising (success)*

*Umpire mistake, not related to failed roll!

Turn 3

  • US: Soft diplomacy to GCC to stop flow of money to ISIS (success)
  • Iran: Deploy ground forces to Iraq: Najaf and Karbala (success)
  • ISIS: Conquer Tikrit with Sunni militia support (success)
  • Iraq: Conduct anti-ISIS IO campaign based on “one Iraq” (fail)
  • Sunni: Re-propose law for proportional minority representation in Parliament (success)
  • Kurds: Retake Hasakah Province from ISIS (success)

Turn 4

  • US: Announced Iranian nuclear deal in exchange for Iranian help against ISIS (success)
  • Iran: Deploy additional ground forces to Iraq: Samara (success; doubles)
  • ISIS Free Move: Blows up several mosques in Samara; Iran blamed (success)
  • ISIS: Regional recruiting campaign (success; doubles)
  • ISIS Free Move: Conquer Fallujah from Iraq (success)
  • Iraq: Coordinate for Combined Iraqi-Iranian assault to retake Fallujah from ISIS (fail; Iraq attacks alone)
  • Sunni: Appeal to US for support (no roll)
  • Kurds: Recuit/deploy additional Peshmerga into Kirkuk Provice (fail)

[1]Matrix Games: The Origins of Matrix Games,” Wargame Developments,  (accessed January 27, 2016).

[2] John Curry and Tim Price, Matrix Games for Modern Wargaming (Bristol, UK: The History of Wargaming Project, 2014), 7.


The views expressed in this article are those of the authors and do not necessarily reflect the official policy of the Department of the Army, Department of Defense, or the U.S. Government.

COL Jerry Hall is an Army Simulations Officer and the Director of the Strategic Simulations Division, Center for Strategic Leadership, US Army War College. He can be reached at jerry.a.hall.mil@mail.mil

LTC Joseph Chretien is an Army Simulations Officer assigned to the Strategic Simulations Division. He can be reached at joseph.c.chretien.mil@mail.mil

2015 in review

2016-2015-new-year.jpeg

2016 is now upon us, and so PAXsims would like to wish all of our regular readers—as well as those who may have accidentally found themselves here while looking for the PaxSim advanced aviation passenger and  baggage simulation tool—a very happy new year. May all your conflicts be merely simulated!

It’s an appropriate time too to review some statistics for PAXsims in 2015:

PAXsims had 52,343 visitors (and 94,152 views) in 2015, up from 44,611 visitors the years before. Since the blog was established in 2008 we’ve now had well over 378,000 views—which is certainly more than any of my traditional academic writings have been read! In addition, 243 people subscribe to blog updates via email or wordpress.

Our visitors have come from an impressive 178 countries and territories, with the United States accounting for almost half of these:

  1. US: 45.9%
  2. Canada: 9.3%
  3. UK: 8.4%
  4. Germany: 3.5%
  5. Netherlands: 3.3%
  6. France: 2.7%
  7. Australia: 2.3%
  8. Italy: 1.5%
  9. Spain: 1.4%
  10. Brazil: 1.2%

We even had visitors this past year from North Korea, South Sudan, and Bhutan.

This year we’ve topped the one thousand mark for total number of posts on the blog, reaching 1,004. Our top ten items posted in 2015 were:

  1. Boardgames and the indirect surveillance state
  2. AFTERSHOCK
  3. Revisiting the “ISIS Crisis”
  4. Teaching professional wargaming
  5. Zones of Control
  6. ISIS Crisis at the Royal Military Academy Sandhurst
  7. ISIS Crisis at MIGS
  8. Updated ISIS Crisis materials
  9. Simulating the Syrian refugee crisis in Lebanon
  10. Wargaming and innovation

Another very popular item was Gaming the crisis in the Ukraine, first posted in March 2014 but updated regularly since then.

In addition to various search engines, our most common referrers were Facebook, Reddit, Twitter, Grognard.com, BoardGameGeek, and ConsimWorld.

Finally, let me thank my fellow PAXsims editors (Gary Milante, Ellie Bartels, Devin Ellis), our research associates (Nikola Adamus, Corinne Goldberger, Ryan Kuhns, Nick LaLone, and Christian Palmer), and all those who have contributed to the blog this year. Without them there would be much less to read.

Onwards into 2016!

 

 

Playtesting RCAT

1280px-NDHQ.jpg

Last week I was invited to participate in a demonstration and playtest of the Rapid Campaign Analysis Toolset in Ottawa. RCAT has been developed by the (UK) Defence Science and Technology Laboratory and Cranfield University, and is intended as flexible, low-overhead wargaming system for military planning and analysis.You’ll find more on RCAT here and here (from Connections UK 2013), here (Falklands war operational commanders test, via the LBS blog), and here (in conjunction with a digital simulation, again from LBS).

DSTL.pngDefence Research and Development Canada are interested in seeing whether RCAT might be used to help refine the scenarios used for capability-based planning within the Department of National Defence. These scenarios aren’t based on current events, nor are they meant to represent actual planned operations. Instead they are intended to be broadly representative of the sorts of missions that the Canadian Armed Forces might be called upon to perform. They are thus intended to provide the Joint Capability Planning Team with plausible problems that might be  addressed by military means, enabling the identification and validation of various military capabilities.banner.jpg

To this end, the visiting RCAT team (Colin Marston of Dstl, Jeremy D. Smith from Cranfield University,and Graham Longley-Brown of LBS) had developed a version of RCAT that addressed an existing force development scenario—specifically, a hybrid warfare scenario that explored the ability of Canadian forces to operate as part of a larger coalition in a complex conflict environment running the gamut from high intensity combat to later stabilization operations.

RCATdesign.jpg

RCAT design process

 

I headed up the Red Team, and proceeded to throw every plausible curve I could think of at both the Blue and Green players and the RCAT system itself. The sessions were very much a participatory seminar on the game’s design, as we discussed how RCAT modelled various kinetic and non-kinetic effects, how the system might be modified, and the extent to which it might offer insight into scenario design and capability issues. To this end, we gamed a few turns of everything from major campaign moves (days/weeks/months), through to tactical/operational vignettes (hours)—the former including one major surprise by me, and the latter including a very successful urban operation and airborne insertion by my opponents.

RCATsequence.jpg

RCAT turn sequence (with apologies for the creases).

 

What impressions did I draw from all this?

I was impressed with RCAT. It is flexible and easy to understand, and can be easily modified (even during a game) to address issues and needs as they arise. The military outcomes all seemed highly plausible.  I thought the combat components worked better than the stabilization model, but then again the scenario was a challenging one. Moreover the political, social, and economic dynamics of stabilization are, in my view, much more complicated and much less well understood than the art and science of conventional military operations.

RCAT’s design lends itself to both training and analytical use—and possibly both at once. Many professional wargamers would suggest that analytical and training games are quite different things, and one should design a game to serve either one purpose or the other. I certainly accept that a game’s experimental design might be compromised by training requirements, and vice-versa. However, I do think there are cases where one can get two (simulated) bangs for one (very real) buck. Because of its elegant design it is easy to imagine RCAT being run as part of professional military education, while analysts use player behaviours to explore research questions of interest.

Game design and playtest sessions can themselves generate useful experimental data. The usual practice with many analytical wargames is the develop the game, playtest it to identify shortcomings, and refine the design. Having done this, the final wargame is conducted—and only then is data systematically recorded regarding the research question being examined. However, our RCAT discussions, although intended simply as introduction and game development sessions, themselves produced substantive findings relating to both scenario development and future Canadian Forces capability requirements. This suggests that we need to think about more systematically identifying insights generated by game design processes.

Scenario designers need to think seriously about politics. There were a few times in the force development scenario we were using where politically-appropriate behaviour by scenario actors threatened to compromise the ability of the scenario to fully explore the intended research questions. While RCAT is certainly not a role-playing or negotiation game, the adversarial (and coalition) nature of game play did force players to think critically about their interests and motivations.

Game facilitation skills matter—a lot. The RCAT team knew exactly when to play the rules-as-written, and when to tweak the system on the fly to best model the unfolding situation. They also had the wisdom and experience to keep the game flowing despite potential distractions (including incessant comments and suggestions from me!)—and, conversely, also knew when to slow things down to allow for a deeper-dive or extended discussion.

Such facilitation skills are not necessarily intrinsic to all wargamers. Indeed, if anything they’re more common among role-playing gamers, especially experienced dungeon/gamemasters, than among “grognard” conflict simulationists. That, however, is a PAXsims post for another day.

Towards a history of professional wargaming – Kriegsspiel-related research at Julius-Maximilians-Universität Würzburg

Dr. Jorit Wintjes kindly provided this summary for PAXsims of research that they are undertaking at at Julius-Maximilians-Universität Würzburg on the history of early Kriegsspiel and related topics.

* * *

JMUW

When a few years ago the Digital Humanities (DH) program was established at Würzburg University, the decision was made to push the boundaries of what was then mainly (though of course not exclusively) a text-orientated subject in order to include a wider variety of humanities-related topics. As a result, students of the DH BA and MA programs at Würzburg come into contact with topics as diverse as digital cartography, GIS referencing of archaeological data, advanced image procession technology used by papyrologists – and conflict simulation.

In the history department, we ancient historians had already before the establishment of the DH program experimented with the employment of political and conflict simulation in class. Results were promising, so when it was our turn to contribute to the fledgling DH program, we decided to concentrate on simulations. As a consequence, we offer at the moment two courses on simulations are now an integral part of the Würzburg DH programs and are also open to interested History students.

More on the Würzburg DH program can be found here. For a very brief overview over the ancient history contribution see here.

Currently, we offer a general introduction to conflict simulation, usually ending in an actual war game, as well as an advanced course concentrating on specific issues like wargames-related statistics, issues of computer modelling, KI etc.

While we initially concentrated on the establishment of the teaching program, contributing also to the research profile of the program seemed rather logical as the next step. For that – and for the foreseeable future – we decided to establish a research group on the history of professional wargaming. This is currently still very much in its infancy, and we are still a long way from our initial goal, that is providing a definite history of professional wargaming for the 1812/24 to 1914 period, yet there has already been some progress.

So far, we have focussed on three different issues – first, the “prehistory” of the Kriegsspiel. This is usually understood to include pre-Reiswitz wargames only for completeness’ sake, yet a closer look at Reiswitz not only shows that the connection between Reiswitz and his predecessors is much deeper than usually acknowledged (after all, he calls his game “a Helwig-type game”!) but also that far from being the lone genius who was suddenly struck by inspiration, designing wargames was in fact something of a fashion in the late 18th and very early 19th century – evidently, the Kriegsspiel is not the result of the “new spirit” of the post-1806 reforms in Prussia. Also, research in the very early history of the Kriegsspiel as turned up evidence for the use of the term Kriegsspiel for a simulative military game already in the latter half of the 16th century – which may mean that there is some need for rethinking about what happened before the two Silesian Barons.

The first output – planned output, I should say – of this research is an annotated edition of Reiswitz’ 1812 Kriegsspiel. As its introduction is highly valuable for anyone researching the history of professional wargaming in general and the Kriegsspiel in particular, we felt there is enough justification to produce such an edition – and the way things happen, it was supposed to appear in September, but will probably only see the light early next year.

The second issue currently occupying much of our mind – partly because it has direct relevance to how we use rulesets in class – is the issue of the “Free Kriegsspiel” in Germany. The orthodoxy here is that “traditional” Kriegsspiel all but came to an end in the early 1870s, with everybody “seeing the light” after Meckel’s and Verdy du Vernois’ publications. While this may well have been the case, one should be aware of the fact that this interpretation rests to some extent on an enormously influential publication by Konstantin von Altrock – who was about as ardent a supporter of the “Free Kriegsspiel” as they come, so one might take his testimony with a small amount of the proverbial salt. In fact, our ongoing attempt at locating and collecting all surviving pre-1914 rulesets has shown that even after 1880 rulesets appeared that heavily depended on the use of dice – and were apparently pretty successful.

Finally, we have started to take a somewhat “digital” approach to the rules, inspired originally by the statement of Georg Heinrich von Reisswitz to have based his data on the weapons testing data published by Scharnhorst in 1813. The general idea here is to analyse the statistics behind the rulesets, comparing them both to each other and to “real life” data. It is interesting to note, for example, that in the 1824 rules, artillery firing canister is pretty deadly, going beyond the data published by Scharnhorst (well, Reisswitz was an artillery officer after all…). This was noted and corrected in the 1828 supplement, and during the following decades Prussian and German rulesets often make mention of corrections in the tables accompanying the rulesets. Given the technological progress during the 19th century it would be interesting to see how they incorporated both technological and tactical advances into the rules.

As I said, it is all still very much in its infancy. Given however that the general circumstances for doing wargames research are currently very favourable here at Würzburg, we hope to produce at least some interesting results in the future.

PD Dr. Jorit Wintjes 

Thomas Schelling on POLMIL wargames

schellingNobel Prize winner Thomas Schelling—who played an important role in the early development of political-military (POLMIL) wargames during the late 1950s and early 1960s, while at RAND—delivered the keynote address at the Connections 2014 conference in July 2014. The audio of his presentation is now available as a podcast here (the audio quality is uneven for the first 20s, but then improves).

After you’ve listened to his presentation, it is well worth reading Crisis Games 27 Years Later: Plus c’est déjà vu, a RAND reproduction of a lengthy 1964 exchange of internal communications between Robert Levine, Thomas Schelling, and William Jones on the strengths and weaknesses of crisis games as an experiential and analytical tool. Levine is skeptical and cautious, while Schelling (as in his talk) argues they have considerable value when used properly.

h/t Yuna Wong

Simulations miscellany, 1 March 2014

miscellany

Some recent items on serious games and conflict simulations that may be of interest to our readers:

* * *

gri_970031_6_344545ds_horiz

At Slate, Rebecca Onion discusses a 1940s board game for French kids that taught tactics for successful colonialism. You’ll find additional detail at the blog of the Getty Research Institute:

Made in France at the outbreak of World War II, the game sought to educate children about the colonial world supporting the French economy. With tokens printed in vivid colors to represent places and natural resources in regions colonized by the French, from North Africa to Oceania to southeast Asia, this game encapsulated the mighty business opportunities that lay ahead for adventurous explorers willing to embark for faraway colonial lands.

As described in the rules at the center of the board, the underlying purpose of the game was to admire, through play, the greatness of the French colonial undertaking. The colonization of a land was symbolically achieved first by hoisting the French flag on its soil, then by the establishment of a hospital, a school, and ultimately a harbor. But the ultimate win was to export the rich natural resources of the colonies back to France by boat. Images on the game provide a vivid picture of the vast variety of resources, including animals, plants, and minerals, that the colonies provided to France from all around the globe.

* * *

Todd Mason and  Mariana Zafeirakopoulos are hoping to organize a Connections conference for wargaming professionals in Australia, following the model of the successful US and now UK versions. For further information, see the announcement at Wargaming Connection.

2game_whole

* * *

At his Sources and Methods blog, Kris Wheaton uses computer modelling to simulate the possible future flow of Syrian refugees in Turkey.

Sim_1_onMap

* * *

The University of Minnesota will be holding their 2014 “Disaster Camp” on 5-7 September 2014:

Disaster Camp is an overnight experience that allows participants to see firsthand what it might feel like to be on the ground during a humanitarian crisis situation. Participants learn how to provide leadership in such a scenario and to maximize the effectiveness of humanitarian program interventions .

Further details can be found here.

* * *

The December/January newsletter of the US Department of Defense Modelling & Simulation Coordination Office is now available. (Well, it was available last month, but we’ve been slow in posting the news!)

* * *

rttp-header

The Reacting to the Past project has a series of forthcoming conferences and workshops:

We now look forward to our spring series of conferences and workshops. Registration is open for a Regional Conference at Gustavus Adolphus College (St. Peter, MN | March 14-16), which will feature Defining a Nation: India on the Eve of Independence, 1945 and Mexico in Revolution, 1911-1920 (game under review). The priority registration deadline is February 24, 2014.

Program details are also available for the Fourteenth Annual Faculty Institute at Barnard College (New York, NY | June 5-8).  We invite individual faculty and/or teams to share their experiences with RTTP at the institute by submitting a concurrent session proposal. Proposals can reflect the variety of ways in which individual instructors or programs are using RTTP games to achieve outcomes for their institutions, to engage faculty, to rejuvenate teaching and learning, or to develop/revise new curricular programs. The proposal deadline is March 31, 2014.

Faculty interested in game design are encouraged to participate in the annual Game Development Conference (GDC) at Simpson College(Indianola, IA | July 17-19); please refer to the GDC call for proposals for details on how to apply for a play-testing or presentation slot. The“Reacting to the Past” Editorial Board also seeks nominations for a new member, whose term will begin in 2014-15. The nomination deadline is April 1, 2014.

D&D historian Jon Peterson on Reddit today

On this, the 40th anniversary of D&D, you can ask D&D historian Jon Peterson anything on Reddit—courtesy, appropriately enough, of Gygax Magazine:

anniversary_AMA

Happy Birthday, D&D!

Expy-birthdayAccording to Jon Peterson—author of the seminal history of Dungeons & Dragons, Playing at the World—the best guess as to the birthdate of the role-playing game is tomorrow:

Many sources, including Playing at the World, assign to Dungeons & Dragons an initial release in January 1974. Our best evidence comes from contemporary notices like the one above, a letter written by Gary Gygax late in 1973 that foretells the imminent release of the game. Now, with the fortieth anniversary nearly upon us, a burning question arises: when exactly should we celebrate? While there is no shortage of anecdotal accounts describing when, and to whom, the first copy of the game was sold, there is little concrete evidence to indicate any particular birthday….

Jon being Jon, he of course then goes on to provide a detailed chronology of the genesis and publication of the game, concluding:

For all the reasons listed above, it’s probably impossible to narrow in on one date and say with any certainty that this is when the game was released. But if we need to celebrate somewhere in the neighborhood of late January, then the last Sunday of the month (this year, the 26th) seems like the best candidate. As the El Conquistador advertisement above notes, Sunday was the day when Gary invited the world to drop by his house, at 1:30 PM, to have a first experience of Dungeons & Dragons. Since it’s a weekend, many of us can clear our schedules to revisit some classic tabletop. So this coming January 26th, 2014, do take the time to celebrate the birth of Dungeons & Dragons and role-playing games.

I was a serious wargamer before I was a D&D player, refighting World War II before I ever first fought Orcs or a relieved a nobleman of a heavy coin purse. However, I can also claim to have been a fairly early convert—we in the “Lymington & District Wargames Club” started playing around 1977 or so, using the original three booklet “white box” edition of the game. I have played every edition since (and a great many other RPGs as well).

While my overriding reason for playing D&D has always been the sheer fun of it, the game has also had implications for my professional career. It probably contributed to my self-confidence in groups, and a certain degree of organizational skill. It absolutely contributed to my professional work on serious games and simulations in far too many ways for me to count. My Brynania conflict simulation is, in many ways, a giant, week-long 120-player game of D&D set in the context of a country emerging from civil wars—but with UN agencies, NGOs, insurgents, and governments instead of wizards, fighters, and rogues. I have also borrowed from D&D game techniques to help organize a workshop for Libyan rebels, facilitate my civilian role-play contribution to a special forces irregular warfare exercise,  run a planning exercise for a UN humanitarian agency, and do a great many other things that lacked both dungeons and dragons. Playing D&D, and even more so, organizing and running a campaign, tells you a great deal about the importance of narrative, engagement, immersion, and group dynamics in a successful serious game or simulation. One of my current gaming group, who designs money-laundering and corruption simulations for the World Bank and anti-corruption agencies, similarly got his start in professional simulation design as a player and dungeon-master.

For more on the 40th anniversary of the game, see also articles in The Guardian and SalonAnd, in the meantime, a very happy birthday to D&D  from a trio of famed halfling rogues—Arnold Schaeffer, Arnold Wurzel, and Arnold Brandyken—as well as  Finius T. Stormfroth, the pirate warlord!

 

h/t Excellent image above by AvatarArt, via Dungeon Mastering blog.

 

simulations miscellany, 5 March 2013

flightsimulatorSome recent items on games, serious games, and conflict simulations that may be of interest to PAXsims readers:

From Clipboard

The 2013 Canadian Disaster and Humanitarian Training Program will be held in Toronto on 8-19 May 2013. The course will make extensive use of simulation methods, notably in the simulation/field exercise component:

The Simulation Exercise part of the program is designed to simulate a complex humanitarian emergency that involves understanding cultural context, war, natural disaster and forced migration of the local population in addition to other challenges injected to add stress to the participants. Participants are in the simulation for 72 hours, working in multidisciplinary teams to perform a series of assessments on the fictional populations. Teams must find ways to solve dynamic and complicated problems including security incidents, disease outbreaks, child soldiers, environmental shocks, limited resources, supply issues and populations on the move. Participants apply their skills in all areas specific to humanitarian response including: health, water and sanitation, food, shelter and protection. They also use principles of triage and of humanitarian action, coordinate the emergency, run meetings, apply globally-recognized standards to meet shelter, water, sanitation and nutritional needs, enumerate populations and calculate important health indicators that translate into numbers needed to treat and the dollars needed to do so. They draw on knowledge in international humanitarian law, negotiation, population sampling, information management, and crisis mapping along with other technologies specifically used in humanitarian emergencies. Teams establish their own compound, eat military rations, draft situation reports and evacuation plans, respond to militia strikes and kidnapping, practice landmine safety and provide media interviews on camera. At the end of the simulation teams submit and present a final proposal for their intended project to assist the affected population to the UN and other donors. Individual performance is assessed using and CCHT-created competency-based evaluation tool.

The deadline for application is March 7.

From Clipboard

The UN Staff College will offer a course on Advancing Training Skills and Simulations Development in Turin on 9-12 April 2013:

During this course, participants will be exposed to theoretical lessons, coaching sessions, demonstrations, group-work and presentation tasks enabling them to successfully design, develop and deliver effective security training programs. In order to allow participants to monitor the improvement of their training skills and capacities, their performances will be assessed during the course through pre-established criteria, which will contribute to the final evaluation of the competencies and abilities acquired during the course.

From Clipboard

The United State Institute of Peace, as usual, offers a number of forthcoming courses on peacebuilding and conflict resolution that make use simulation methods.

From Clipboard

For those putting together humanitarian and disaster relief simulations, there are a great many useful resources to be found at the website of the Emergency Capacity-Building (ECB) project. You’ll find an overview of their work on simulation-based training here, and links to various resource materials here.

From Clipboard

ActionAid is looking for an international programme manager, to lead their “preparedness and emergency response work, managing a team for emergency response as and when necessary.” While primary emphasis is on applicants with extensive field and management experience in insecure environments, including emergencies and conflict situations, part of the job also involves ensuring that “simulation exercises take place with country programmes and ensure that country based preparedness plans are in place.” Applications close on March 11—details here.

There are also several United Nations positions currently advertised for which familiarity with simulation methods would be an asset. Search at UN Jobs for more details.

From Clipboard

At Playing the World, Jon Peterson discusses how gaming got its (polyhedral) dice.

From Clipboard

Allan B. Calhamer, the postal worker who invented the influential boardgame Diplomacy, died on February 25 at age 81

From Clipboard

At Slate, Farhad Manjoo reviews the new edition of SimCity, and likes what he finds—especially the shift in the game engine from macro-modellling to agent-based modelling of urban processes.

From Clipboard

h/t last two items, Jacob Levy

Beitrag zum Kriegsspiel (1876)

While serving as a lecturer at the Prussian military academy, then Colonel Julius von Verdy du Vernois proposed a revision of the earlier Kriegsspiel developed by the von Reiswitzs a half century earlier. In particular, he called for doing away with the increasing cumbersome system of charts and tables used in the adjudication of wargames, and adopting in their place a more fluid and flexible system whereby umpires would make determinations of outcome based on their experience. These ideas were published in a small book in 1876 entitled Beitrag  zum Kriegsspiel [Contribution to the Wargame]. As the author notes in the Preface:

The utility of the War Game is universally acknowledged at the present day. Nevertheless, cases are often met with in which attempts to practise it have been very speedily abandoned. As excellent books which contain instructions for carrying out the Game, and by means of which it has been practised for many years with great advantage, are easily accessible, this state of things is remarkable, and calls for closer investigation. Now, when I have inquired into the reason, I have, in most cases, received the answer, “We have no one here who knows how to conduct the Game properly.”

So far as relates to the solution of the military problems which occur in the course of the Game, this answer cannot be always considered satisfactory. For what the Game especially requires is a knowledge of the capabilities and fighting power of all arms, as well as of their principal accepted formations. Now, the elementary training of an officer ought to have laid the foundations of these acquirements; and even when it has not sufficed to do so, the very playing of the Game would develop more fully such knowledge as might be already possessed. Again, as every officer, as soon as he attains field rank, is liable to be called upon to command bodies composed of all three arms, he ought, of course, to prepare himself beforehand for such duties; and in the War Game he will find an opportunity for solving the questions which arise in connection with them.

But even if the previous knowledge of the young officer should prove insufficient; even if the decisions of the Umpire should be questionable—such a state of things arises in very many military exercitations, even when carried out with troops on actual ground.

To these causes the neglect of the War Game cannot be to any great extent referred; the source of that neglect must be sought for elsewhere, and, so far as my experience goes, I have found it to lie chiefly in the purely technical part of the conduct of the Game, the novice failing to understand the Rules, or the use of the Dice and the Tables of Losses.

It is, indeed, only by severe toil and a great expenditure of time that any one who has not learned the Game by actual practice can, through unassisted study of the books of instruction, so thoroughly master the subject as to be competent to undertake the conduct of an exercise of this kind. So it comes about that there are assuredly in the smaller garrisons many officers who should be especially fitted, from their position, to take the matter in hand who utterly shrink from doing so.

I in no way underrate the service which the received books of Instruction, with their Rules, Dice, and Tables of Losses have rendered, and will

render in the future. It consists chiefly in this, that the Umpire finds in the Rules, fixed principles by which the limits of the capacity and fighting power of the troops are defined, that the effective use of weapons gains full credit by means of the Tables of Losses, and that the dice, which give to chance its due influence, provide an apparent security against partiality in the decisions.

The question, however, arises, whether the Game might not be made even more useful than it is if the difficulties of execution and the expenditure of time, which the above-mentioned devices involve, could be avoided. Experience shows that this question must be answered in the affirmative.  (Translation by J.R. MacDonnell)

Verdy du Vernois’s call for a more easily-playable “free” kriegsspiel was taken up by others, and his work was soon translated into other languages. In London, an English translation by J.R. MacDonnell was published under the title The Tactical Wargame in 1884. In the US, another translation by Captain Eben Swift was published as A Simplified Wargame in 1897. The trend towards simplified rules was also partly reflected in the UK War Office’s Rules for the Conduct of the War-Game (1884).

 * * *

All three of these books are freely available online, via Google eBooks or the US Combined Arms Research Library Digital Library (click the images to access each one).

%d bloggers like this: