Conflict simulation, peacebuilding, and development

CFP: CUNY Games Festival 2016


The City University of New York will be holding a one day games festival on 22 January 2016, focussing on the role of games in higher education:

The CUNY Games Festival is a one day conference on game-based pedagogies in higher education. Participants include faculty, researchers, graduate and undergraduate students, and game designers. Both CUNY and non-CUNY participation is welcome.

With the growing maturity of game-based learning in higher education comes a new set of questions. The focus has shifted from whether games are appropriate for higher education to how games can be best used to bring real pedagogical benefits and encourage student-centered education.

This conference seeks to address current topics in game-based learning for higher education. Consequently, all submissions must address the guidelines established by the Conference Theme and Rubric. Topics may include, but are not limited to:

  • What problems need to be addressed in higher education, and how might game-based learning address those problems?
  • How might game-based learning mediate these problems if other solutions are present?
  • How do these solutions generalize across learners, disciplines, and campuses?
  • What possibilities are there for educators to not only create/implement games for learning but also to utilize/alter/subvert commercial games for learning?
  • What differences do we see in digital versus non-digital game-based learning? What affordances or barriers are inherent in each?

The deadline for paper submission proposals is October 15. You’ll find more information here.

h/t Ben Foldy 

Conflict simulation and gaming at RMCC

Later this month I’ll be speaking about conflict simulation and gaming in the classroom at the Royal Military College of Canada in Kingston. While the sessions aren’t open to the public, if you’re an educator or military professional in the Kingston area and would like to take part, email me and I’ll see if it can be arranged.


USAID saves Carana!


Today (thanks to Matheu Schwenk) I had an opportunity to run a  demonstration game of AFTERSHOCK: A Humanitarian Crisis Game with some staff from USAID’s Office for US Foreign Disaster Assistance.

Yet again, of course, the poor country of Carana was struck by another devastating earthquake—something which seems to happen with alarming regularity. However, OFDA’s experienced staff were quick to respond!

USAID prepositions life-saving banana as they prepare to respond to the disaster.

USAID staff pre-position a life-saving banana as they prepare to respond to the disaster.

The teams assessed what critical supplies were needed in what districts of the capital. Transportation bottlenecks at the port and airport were promptly addressed by the Humanitarian Assistance and Disaster Relief Task Force (HADR-TF), which arrived, fixed things, brought lots of stuff, airlifted supplies for the NGOs… and then declared “Mission Accomplished,” departing Carana with much military fanfare.


HADR-TF on the march!

The government of Carana mobilized government workers and local civil society, with the President issuing a series of inspiration public statements (albeit several weeks into the crisis). The Minister of the Interior was quick to address incipient social unrest. The UN and NGOs helped to support a smooth transition from relief to development as the emergency phase was followed by a prolonged period of early recovery. The UN was particularly effective at responding to an outbreak of cholera in the slums of District 3.

Following the session we discussed the ways in which games could be used as educational and training tools, and the particular adaptability of manual gaming formats to a wide variety of needs. I certainly enjoyed myself, and I hope everyone else did too.

Report: MORS special meeting on professional gaming

PGW IconThe recent Military Operations Research Society (MORS) special meeting on professional gaming set itself the following task:

The meeting will produce initial content for a Professional Gaming Practitioner’s Handbook and bring together members of the community of practice to consider best practices, design, existing applications and appropriate analytic methodologies in an effort to codify the fundamentals of game design and analysis. The meeting is designed for information exchange and participant exposure to professional practice.

The main conference started off on Monday with several plenary presentations.

George Akst (Senior Analyst, Marine Corps Combat Development Command) highlighted the value of wargaming as a midpoint between large exercises and operations research analysis. Wargames are, he suggested, are generally a single/deterministic (n=1) approach to a stochastic problem, illuminating one possible plausible scenario. He pointed to the value of wargaming in identifying capability gaps, developing doctrine, and experiential training and learning. They can also help narrow the scope of problems for subsequent (OR) analysis. He also noted weaknesses in many wargames: analytical follow-through, adjudication rules and procedures. Operations analysis can address some of these shortcomings by bringing additional analytical rigour, including sensitivity analysis. Analysts should be integrated into the process early to make the combination of wargaming and OR analysis most effective. I thought it was a useful presentation on the strengths and weaknesses of analytic gaming, although I would prefer to see as gaming as a tool in the analytical toolset, rather than something performed by the gaming tribe to which OR analysts must somehow relate.

Robbin Beall (Head, Campaign Analysis and Modeling at Assessment Division, Office of the Deputy Chief of Naval Operations) stressed the growing appreciation of wargaming as a tool to help identify innovative approaches (and to weed out those that may be less useful). She also identified three areas of concern:

  1. More attention need to be devoted to innovation in gaming and game design. Designers need to be smarter, effectively pinning players into the puzzle they are expected to address. A cadre of good game designers needs to be fostered within DoD. This year’s US Navy Title X game is an example of greater innovation, with a traditionally large and monolithic game being broken into a series of smaller, more highly focused games.
  2. Adjudication remains a challenge. If a game has weak adjudication, the game fails and participants leave with a wrong impression.
  3. Despite the current renewed emphasis on wargaming, it needs to be remembered that wargaming is only one tool in the analytical toolbox. Games are not always the best approach.

E.B. Vandiver III, former Director of the Center for Army Analysis, delivered the keynote address. He focused many of his comments on CAA experience and some of the problems of wargaming. He summarized some of the most common objections to wargaming: it is too subjective, too qualitative, they aren’t repeatable, they learning effects overwhelm functional effects, and they are too time-consuming and resource intensive.

He also discussed the development of a training wargame at CAA to train junior analysts with little or no background in military history or the military decision-making process. The initial version was too complex, so they designed a new, simpler, faster, and more strategic game—but never ran it, because they were tasked to develop a front end analysis for a new Korea operations plan (building in part on the prior game development).

He also discussed using games in 2006-07 to analyze the security force requirements of the Iraqi government in the context of the US withdrawal from that country. The questions asked required specific answers: force size, deployment, and so forth. A computer-assisted, open player game was developed, based on research, COIN doctrine, data on violent acts, and assumptions vetted by the sponsor. The processes highlighted the value of participation from the actual theatre, that there was a need to address speed, efficiency, and errors in the game process; and that the game really required a precursor training game. The game was later refined and used for drawdown risk assessment and a range of other questions. It was even modified top examine Afghan drawdown risks.

Overall he suggested that many or most of the shortcomings of gaming could be mitigated.


Given the abject failures of the Iraqi security forces in recent years, it would have been useful to more fully discuss why the games Vandiver described fell so short of anticipating these shortcomings. I would argue that the poor performance of the ISF has more to do with leadership, patronage, corruption, morale, sectarian polarization, and internal politics and similar social and political “intangibles” than it has with capabilities, deployment, or formal organization. I asked Vandiver about this, and he responded that the games had not explored political context and effects, or even force motivation and morale. To my mind, that’s rather like wargaming without physical terrain or weather or visibility effects, and underscores Beall’s point about the need for more innovative approaches to gaming. (As one colleague noted in a side-comment, hobby wargame rules have long addressed morale issues, so it isn’t exactly an impossible challenge.)

Mark Gallagher (Studies and Analyses, Assessments and Lessons Learned/A9, USAF) emphasized the human-in-the-loop, adversarial character of wargaming, and suggested it ought to be seen as part of analytics. He distinguished it from military exercises. He argued that the “single output” nature of wargaming didn’t necessary limit its usefulness. The particular decision points in games can be examined, for example.

Bill Lademan (Director, Wargaming Division, Marine Corps Warfighting Laboratory) made a very lively presentation in which highlighted in the ways “next generation” wargaming could be done better. He noted that the Marine Corps will be developing a state-of-the art wargaming center (adjacent to their intel facilities, to enable access to TS/SCI SAP material). He stressed the need to “wargame at the speed of thought,” rather than being handcuffed by developing powerpoint presentations. Knowledge needs to be “maneuvered, not managed,” in a way that effectively addresses problems. He emphasized human intellectual input into wargaming, and was critical of relying on computers. Analysis, he suggested, was all about the decomposition of a problem and striving for precision. Wargaming, he argued, was about many variables, and what emerges from their dynamic interaction with player decision-making: “what comes out the soup” of a complex situation.

Addressing the purposes of the MORS special meeting, Lademan expressed the view that wargaming should not be preoccupied with achieving rigour and repeatability, in which the manipulation of data becomes a substitute for wisdom.

He pointed at length to the Vietnam War, where “analysis replaced strategy” and failed to appreciate that the war was not about kill ratios but rather about winning over people. Later, the bodycount became the analytical measure of success. The disagreed that wargaming ‘was an analytical methodology,” arguing it was an assessment methodology. Analysis, he argued, was about exact conclusions, while wargaming (and assessment) is about a judgment. “Assessment is to analysis the way philosophy is to science,” he suggested. He disagreed that wargaming struggled for acceptance. Wargaming, he argued “isn’t broken”—and efforts to fix it with greater supposed analytical rigour would be counterproductive, since it could squeeze out creativity. Because of human learning and cognition, wargames are never truly repeatable. They’ll also never be rigorous, and models will never be adequate.

It was a powerful and stimulating presentation, and a credit to the bluntness for which the Marine Corps is known. It was also something of a riposte to the earlier OR-focused comments by Akst and Beall. In a dialectic sense I suppose the clash of views of useful. Or, perhaps, Lademan’s comments only reinforced the view in some of the OR community that gaming is a sort of nebulous witchcraft claiming insight into the free will and the human soul. Towards the end of his comments he offered more conciliatory comments, calling for a productive union of wargaming and analysis where each offset the weaknesses of the other.

I enjoyed it a great deal, but I’m concerned by the absolutism of some of it. Analysis and gaming, I think, have very fuzzy boundaries. Drawing sharp dichotomies between quantitative and qualitative analysis is counterproductive. Sharpening tribal divisions around these issues only contributes to hedgehogism.

Wargaming for analysis within DoD

Wargaming for analysis within DoD (from the back of the room).

After lunch an OSD panel discussion examined “wargaming for analysis within DoD.” The session was chaired by Mike Ottenberg, and featured the quad chairs for the current initiative on wargaming and innovation: COL Mark Gorak (OSD CAPE), Jacob Heim (OSD Policy), COL Neil Fitzpatrick (JS J8 SAGD), and CDR Phil Pournelle (ONA). Among the many points that were made:

  • Small games are often more useful than large, complex ones for some purposes. (Phil)
  • “Blackbox” modeling and simulation can alienate players from a game, as they are unable to understand cause-effect connections. (Phil)
  • The professional military education process needs to develop wargaming experience and skills. (Phil)
  • The style and design of a game needs to be selected according to the problem to be addressed. (Phil)
  • In the past, the senior leadership hasn’t necessarily seen their questions and priorities being addressed by wargaming. (Mark)
  • Given the natural biases of the services and the combatant commands, how can gaming help to develop DoD-wide perspective and priorities? (Mark)
  • Wargaming needs to explore left and right boundaries of issues. (Mark)
  • When inputting data into the new (OSD CAPE-managed) wargame repository, please put greater thought into significant key insights generated. (Mark)
  • SAGD is high demand for POL-MIL games, for clients such as the NSC. (Neil)
  • The next DoD-wide wargaming summit is scheduled for early November.
  • There appears to be less capacity within D0D for multi-move, adjudicated games with an active Red side. (Jake)
  • The DEPSECDEF is especially interested in games that offer insight into key challenges as well as programmatic issues. (Jake)
  • How wargaming initiatives by the services will be integrated with DoD-wide efforts? (Mike) The first step is dialogue, and trying to understand what is going on out there, and who is doing what. The DoD wargame repository is one important element of this. (Neil) Most current games address the “big five” potential adversaries—it is important to examine how to develop insights and synergies across those wargames. While resisting the temptation to impose a single set of metrics to assess the utility of games, there is a need to think about how to differentiate a good game from a bad one. (Mark)
  • How should best practices be recognized, taught, and promulgated within DoD? (Mike) We don’t want to be in a situation where a formal standard is established and wargamers are “certified.” Instead we need to invest in analysts and PME. (Phil) We also need to also educate sponsors. (Jake)
  • One audience comment challenged the anti-quantitative thrust of Lademan’s earlier comments. Phil warned about the false precisions of much M&S work.

Following these plenary sessions, we broke into several working groups:

My hotel roommate (and former fellow UVic wargaming club alumnus) Brian Train and I served as cochairs/facilitators for WG8, along with Joe Saur (Georgia Tech Research Institute), Eric Greenburg (JHUAPL) and Clyde Smithson (JHUAPL). This was primarily intended as a continuation of yesterday’s training course intended to further develop wargame design skills by actually designing a wargame.

Working Group 8 at work.

On Wednesday we started the day with another plenary address by Robbin Beall, this time on “challenges in design and execution of wargames.” She identified several of these:

  • insufficient initial research
    • lack of situation awareness of similar or complimentary supporting analytic efforts
    • lack of authoritative data on capabilities
  • flawed game design
    • overly broad scope leading to shallow conclusions
    • lack of innovation
    • failure to focus players
  • unsound basis for adjudication
    • linked to shortcomings in initial research
    • conclusions reached on anecdotal evidence
    • lack of qualified subject matter experts
    • overly simplistic wargaming tools
  • team/player issues
    • player fear of failure
    • game too short for team cohesion
    • one or two strong personalities dominate game play
  • no objective post-game evaluation of game effectiveness

I thought her points were excellent. Her final point is a particularly important one: far too much of the serious games community—wargamers included—are far too willing to assess the value of the game based on anecdotes, game enjoyment, or the designers’ own confirmatory self-evaluation.

She also discussed the synergies between wargaming and quantitative analysis. Game objectives, she suggested, should be set to utilize the particular strengths of wargaming. Game designs should focus players on a limited number of dilemmas unresolved by previous analysis. Game execution should inform players with what is known from previous analytic efforts. Game adjudication should mine what goes on in the analysis world to support adjudication decisions. Finally game lessons should be determined and new ideas should be incorporated into quantitative analysis.

The implicit thrust of some of her comments seemed to be to frame wargames as adjunct to quantitative operations research, addressing those areas that quantitative analysis could not easily answer. I would tend to view things rather more broadly, arguing for qualitative research as a more equal partner, and also suggesting that findings are most robust when they are triangulated by variety of methods. Indeed, much of the discussion at MORS has struck me as akin to the quantitative vs qualitative skirmishes that afflicted political science a decade ago, but which in that case have now largely been superseded by widespread appreciation of mixed and plural methods.

Interestingly, the one wargame she did praise—a simple, apparently largely conceptual game that modeled a basic guns/butter or kinetic/nonkinetic tradeoff—doesn’t appear to have been particularly rigorous by OR standards or anchored in research and data, but rather intended as a spur to discussion and reflection.

Much of the rest of the day was spend with the working groups. In WG8 the participants divided into two groups:

  • One group (Marcus Tregenza, Shawn Zackey, Christophe McCray, and Bob Turner, aided by WG cochair Eric Greenburg) set about designing a naval platform and technology acquisition game.
  • The other (Stacie Pettyjohn, David Maxwell, John Montonye, Stephen Mackey, aided by Brian Train and I) decided to develop a game that would explore how ISIL balances its strategic options, and how it might respond to coalition efforts in Iraq and elsewhere.

In the afternoon there was also a games expo. PAXsims had a display booth, featuring AFTERSHOCK, ISIS Crisis, and various other simulations I’ve run at McGill and elsewhere.


The MORS game expo.


AFTERSHOCK, with Brian Train in the background displaying his many game creations.

I also had an extensive discussion with Stephen Ho (Dstl) about the potential impact of llamas on modern special forces operations. This is clearly an area ripe for exploration through good game design.


Special forces combat llamas, one of the great military topics yet to be gamed.

The final afternoon we all reassembled to hear brief-backs from each of the working groups.

  • WG2 (wargame objectives). The group emphasized the importance of initial problem identification and research design. They also addressed educational wargame objectives. There was substantial discussion of the relationship with game sponsors, and what to do about sponsors who don’t know what they want, who want a game to address a non-gameable problem, or who are inclined to micro-manage. A long list of “wargame pathologies” were identified.
  • WG3/7 (game design, development, and execution). They reviewed the typical game design process: concept development; research; identification of game elements; building components, prototyping, and initial rules concepts; playtesting; finalizing components… and only after all this, the game itself. Feedback and critical evaluation needs to be continuous throughout all steps of the process. They also noted that analysis must be persistent through the design process, and that execution considerations need to be incorporated into the design and development process.
  • WG4 (data collection, analysis, and tools). The group ran through a revised and abbreviated version of the ZEFRA wargame, which was used to spur a discussion of data collection and analysis issues. Among other issues, they argued that the analysis team needed to specify player requirements (qualifications and other characteristics); the value of a GICOD (“good idea cut-off date”); to consider constraints, limitations, and assumptions; and the need to regularly review collection during the event. The game design and game analysis teams should be an integrated part of an overall project team, rather than entirely separate. The Data Collection and Management Plan (DCMP) should specify what data will be collected on each issue and sub-issue of interest; where in the scenario the data might be generated; what methods and tools will be used; and when during the game such collection needs to occur. If data looks wonky, corrective action should be taken sooner rather than later. A list of tools that can assist in data collection and analysis can be found here.
  • WG 5 (adjudication). The group reiterated Robbin Beall’s point that adjudicators should “do no harm.” Adjudicators need to be facilitators too, need to communicate with players to reduce frustration. Adjudication issues are often intimately tied to game design. Adjudicators need to be well trained and prepared, aware of the dangers of overtasking. Players must feel their choices make a difference. Overall the WG suggested it was hard to identify universal best practices.
  • WG 6 (aligning games with other studies). This group explored how to best integrate various decision analysis methods with wargames, especially in the context of various “wicked problems.”

As for our own WG 8 (quick turn-around game design) presentation, Bob Leonhart presented some overall impressions from our collective game design efforts. He noted how time pressures sparked considerable energy and enthusiasm. He also underscored how much one learns about a topic from designing, and not simply, a game. The two game design groups then presented their games.

In Buying Victory: Budget Battle Wargame the Red and Blue sides invest in hulls and various (surface, subsurface, and air) technologies. Mature technologies are cheaper and safer investments, while potentially more effective future technologies involve more time, risk, and resources. The combat power of the Red and Blue fleets then confront each other three times during the game to determine the overall winner.

Buying Victory: Budget Battle Wargame

Buying Victory: Budget Battle Wargame

Countering ISIL: The Board Game pitted ISIL against a US-led coalition. ISIL has six possible lines of kinetic operation:

  • In the Anbar/Fallujah/Ramadi area, against the Iraq security forces
  • In the Baji/Takrit/Samarra area, also against the Iraqi security forces
  • In the Irbil/Kirkuk area, against the (Iraqi) Kurdish Regional Government
  • In northeastern Syria/Kobane/Hassakeh, against the Syrian Kurds
  • In Aleppo province, against various rival Syrian armed groups
  • Around Damascus, against the Syrian Army.

It also has two possible non-kinetic lines of operation:

  • Building governance, which generates resources and helps to respond to potential governance challenges in the events deck.
  • Building prestige, which helps attract recruits and facilitates international terrorism.

The design was partly inspired by the solitaire States of Siege games by Victory Points Games, although in our case the system is adversarial and provides players with a far more complex set of options and constraints.

Players start the game by selecting several game objectives from a list of possible options. Subsequent game play is very straightforward, consisting of four sequential phases:

  • Event phase, in which an event card is drawn. This may present the players with challenges or opportunities, or otherwise affect game play.
  • Resource phase. ISIL gain recruits and resources from control of territory, effective governance of the “caliphate,” and prestige which attracts supporters and donations. The coalition gains material resources at a steady rate, but political capital is only slowly replaced.
  • Card selection phase. The players select five cards to play from a large deck of possible options. Most have a cost associated with their play: resources and/or recruits for ISIL cards; resources and/or political capital for the coalition.
  • Card play phase. The players take turns playing their cards, each of which has an effect (and possibly an associated die score to succeed). Some cards may be played to augment the effects of other cards, or complicate those of an opponent. Still others represent key decisions or investments in major initiatives, which are prerequisites to the play of other cards in the future. When both players have run out of cards or choose to pass, the next turn begins.

We took a video of one turn of game play, which you’ll find below. Overall I thought it was an excellent design, and if time allows I may put some further work into developing it, in conjunction with other members of the team.

The MORS special meeting on professional gaming ended with thoughts from the synthesis group, members of which had floated from WG to WG during the meeting. With regard to WG8, the synthesis group (and in particular, Richard Phares, who had been the primary synthesis group spy in our midst) quite rightly pointed to the differences between wargame design for serious purpose versus wargaming for fun. They also noted that game designs are living things that can evolve over time, the key linkage between research and game mechanics, and the critical value of repeated playtesting.

Overall I thought it was an excellent conference. While I regretted missing out on the discussions in the various working groups, I very much enjoyed the design work in WG8, and certainly benefitted from the excellent plenary sessions. MORS—and even more so chief organizer Scott Simpkins (JHUAPL)—did outstanding work. I look forward to the wargaming handbook that should eventually emerge from this effort.

V​eiled Ambition​: A Simulation of Iranian-American Relations in the Middle East

Strategic Crisis Simulations is a student organization affiliated with the Elliott School of International Affairs at the George Washington University in Washington, DC. They kindly provided PAXsims with this summary of their recent “Veiled Ambition” simulation.

Ryan Kuhns assisted with this report.

* * *


Scenario Overview

Veiled Ambition​is a simulation that examines the complex relationships in the Middle East in four distinct areas: Iraq, Syria, Yemen, and Lebanon. In each of these countries, writers constructed plotlines which asked the participants to consider the question of Shi’a and Iranian influence throughout the Middle East, and the shifting posture of the United States in relation to that influence.

In Yemen, participants were challenged to deal with a humanitarian disaster when they were informed that disease was rampant throughout IDP camps in the nation. In addition, as the Saudi Arabian coalition continued its air campaign against the al-Houthi rebels, participants grappled with difficult questions of human rights when bombs struck civilian targets instead of military ones. For the military and intelligence communities, HUMINT reports were given to participants that indicated a potential link between some members of the al-Houthi rebellion and Iran. It remained up to participants whether they wanted to pursue those ties on the international level, or simply deal with the al-Houthi’s as non-state actors.

In Iraq, ISIL continued to make moderate advances throughout the simulation (see timeframe below). The main plotline focused on increased political involvement and action by Shi’a groups such as Asaib Ahl al-Haq (AAH). Power shortages shook Baghdad and citizens took to the streets, rioting and calling for jobs, power, and better governance as AAH increased their involvement and their message. Participants examined how to handle (and if it was their responsibility to handle) the internal domestic problems facing Iraq, and to what extent the United States could continue to be involved in the fight against ISIL.

In Syria, the story unfolded on two primary fronts. First, participants were contacted by representatives of a group calling themselves the Southern Front, who were willing to act as proxies for U.S. interests in the Syrian Civil War. It was up to participants to decide how involved they wanted to be with this potential partner. In the north, the Kurds continued to advance and Turkey landed a number of airstrikes on Turkish positions, prompting participants to open diplomatic channels to Turkey and ideally negotiate protection for the YPG, since the Kurds are one of the most effective fighting forces involved in the Syrian Civil War, and they typically align with U.S. interests. Finally, participants dealt with the presence of Russian influence and assistance in Syria: driven more by current events than anything else, participants struggled with an unfriendly United Nations Security Council and more support for the Assad regime than they had initially believed was present.

In Lebanon, participants examined the mounting refugee crisis and were tasked to work with the Lebanese government to support the humanitarian situation. In addition, Hezbollah is currently one of the most effective actors in the region, and also a prominent Iranian proxy. Participants were challenged to re-examine their conceptions of Iranian influence throughout the region as they sought stability and safety across Lebanon, rather than mounting economic and political crisis.

Game Format


How long did the simulation take? How much simulated time was passing?

  • The simulation was not projected into the future; it began on September 19, 2015
  • The simulation took 4 hours and 45 minutes of real time, and approximately 2 months of simulated time (September 19​– November 19​, 2015)
  • Participants were given a rough timeframe (i.e. 1 hour ~ 2 weeks) but no strict timeline was adhered to

P​articipant Positions

What roles were represented? How many participants were in each office? How many participants attended the simulation?

There were 111 total participants:

  • White House (​10)
    • Executive Office of the President (​4)
    • National Security Council (​6)
  • Defense Community (​35)
    • Office of the Secretary of Defense (​5)
      • Office of the Joint Staff (​7)
      • U.S. Central Command (​7)
      • U.S. Joint Special Operations Command (​6)
      • Office of the Under Secretary of Defense for Policy (​5)
      • Office of the Assistant Secretary of Defense for Special Operations and Low Intensity Conflict (​5)
  • Diplomatic Community (​35)
    • Office of the Secretary of State (​4)
      • Bureau of Political Affairs (​4)
      • Bureau of Near Eastern Affairs (​5)
      • Bureau of Civilian Security, Democracy, and Humanitarian Assistance (​4)
      • Bureau of Conflict and Stabilization Operations (​6)
      • Bureau of International Organization Affairs (​3)
      • Bureau of Political-Military Affairs (​5)
      • Bureau of Population, Refugees, and Migration (​4)
  • Intelligence Community (​31)
    • Office of the Director of National Intelligence (​5)
    • Office of the Director of the Central Intelligence Agency (​3)
      • Central Intelligence Agency National Clandestine Service Office of Middle East Operations (​4)
      • Central Intelligence Agency Office of Near Eastern and South Asian Analysis (​7)
    • Office of the Under Secretary of Defense for Intelligence (​4)
      • Department of Defense Joint Intelligence Taskforce (​8)

Participants were placed in positions, within offices, within organizations. Represented above is the organizational and office structure, but not the individual positions. For example, a participant may be the Deputy Assistant Secretary of State for Overseas Operations in the Bureau of Conflict and Stabilization Operations in the Diplomatic Community. In total for this simulation, 173 participants registered to attend and 111 participated on September 19​. 137 positions were prepared for the simulation.


Overall, the game ran exceptionally well. In Iraq, participants worked to build a regional deal to firm up electrical infrastructure. In addition, the U.S. continued to counter ISIL efforts to take territory and attack the Iraqi government.


In Syria, the participants worked with local forces to counter increasing Russian influence and support for the Assad regime. In addition, the U.S. government successfully rescued a colonel in the Syrian Air Force who wished to defect to the United States, and prevented a member of the U.S. Embassy in Ankara from defecting to ISIL in Syria.

In Lebanon, participants worked to alleviate the Syrian refugee crisis: they set up camps near the Syrian border, and provided humanitarian support when Lebanon faced a water shortage. In addition, participants examined the complex relationship between the internationally recognized government of Lebanon, and Hezbollah, which does much of the actual capacity building and support work within the nation.

In Yemen, the work was twofold. On one front, participants worked with Saudi Arabia to alleviate the Saudi naval blockade in order to bring in humanitarian supplies. On the other, the U.S. government worked to set up intelligence networks in order to determine if Iran was supplying the al-Houthi rebels with arms and other forms of military support. Participants also briefly dealt with a diplomatic crisis when a Kuwaiti pilot, flying in the Saudi coalition, was shot down by rebels and was subsequently captured by al-Qaeda fighters.

At the beginning of the simulation, the participants requested a set of existing presidential authorities within which they could operate. The mentor acting as President responded with the following:

The President has approved:

  • Continuation of the air campaign to destroy and degrade ISIL
  • Continuation of arms transfers to our allies in the region
  • Continuation of advising and training programs
  • Continuation of consultations, outreach, and meetings with foreign governments and non­state entities

Prior Presidential approval is required for:

  • Any ground operations
  • Additional deployments of personnel
  • Expenditures exceeding $50 mil
  • Any major change of relations with Iran, Syria, or Russia

The National Security Council and Executive Office of the President briefed the President at the end of the simulation on actions they had taken and policy recommendations for further action within the region. Participants appeared to be most concerned with the instability in Syria as a potential cause for future action by the U.S.



We are still processing our exit evaluations, so currently we can only comment on preliminary observations and anecdotal evidence. In this simulation there was an excellent adherence to chain of command and bureaucratic government structure, something Strategic Crisis Simulations has tried to encourage in our participants for a long time. One of our goals is to effectively model government, as efficient or inefficient as it may be. Many participants, especially those in roles such as Joint Special Operations Command, commented that they had plenty of time to prepare operations and plans, but it was difficult to work up the chain of command to actually get approval. In our view, this is a feature of the simulation, not a bug.

In addition, participants in this simulation struggled to distinguish between the operational-level challenges of crisis management and mitigation and the larger, strategic implications for actions within the region. Due to the timeframe the simulation was set in (over the course of two months), it was a delicate balancing act which some participant offices achieved more effectively than others.

Overall, V​eiled Ambition​was one of our most successful simulations to date. It was not only our largest simulation, but very diverse (with nearly equal male/female representation and participants ranging from freshmen to graduate students and young professionals). This confluence of experiences and skill sets allowed the participants to work together extremely effectively to challenge one of the most difficult geopolitical problems we face today: how to begin to tackle the myriad of problems currently facing the Middle East.

MORS workshop on professional gaming: the course


The MORS special meeting on Professional Gaming started yesterday in Fairfax, VA with a day-long introductory course on wargaming. As was the case at recent Connections conferences too, many or most in the audience were already fairly experienced.

PGW IconMike Garrambone introduced the course, and offered a brief introduction to the various topics that would be covered:

  • Introduction to wargames and technology
  • Fundamentals of wargames
  • Characteristics of wargames
  • Wargames and technology processes
  • Special topics (scenario, preparation, game, seminar)

There was far more substantial content to the day than I can adequately summarize, so this blog post should be seen as little more than an overview. I’ll post a link to the presentation slides when and if they become available.

Mike identified seven elements of a wargame, expanding on each in his presentation:

  1. Objectives
  2. Scenario
  3. Players and sides
  4. Database
  5. Models
  6. Rules, procedures, and umpires
  7. Game analysis

He noted that wargaming is a tool for gaining insights, a source for questions, an aid to practical decision-making, a way to organize technical facts in useful ways, a technique to explore feasibility and implications, and a method of communicating ideas in vivid ways.


Bob Leonhard (Johns Hopkins University) discussed some key fundamentals. Quite rightly, he started with the centrality of purpose and the tensions and miscommunication that can arise between game designers on the one hand controllers and clients/sponsors on the other. He summarized the strengths of wargames, but also addresses the limitations, weaknesses, and dangers too. These points included:

  • wargames diverge from reality
  • wargames don’t convey the battlefield “fear of death”
  • (large, complex) wargames are not inexpensive
  • wargames should not be use to conclusively prove/disprove
  • wargames may hide their models, assumptions, and limits

Mike then returned with a discussion of the wargame-exercise-experimentation environment, the roles of participants, and some of the challenges of red teaming. After lunch, he then expanded on scenario development and game preparation. This included discussion of US Department of Defense structure and its representation in player roles and games.


Finally, participants played a sample (and somewhat simplified) game of Wings of War, both as an introduction to wargaming and as a spur to discussion of wargame execution and analysis. Sadly, my SPAD VII was set alight early in the dogfight, and took heavy fire damage before being shot down in a later burst. Subsequent discussion focused on analysis of aircraft capabilities and assessment of the game system. Certainly the game highlighted for me how much I would have liked my aircraft to have self-sealing fuel tanks and a fire suppression system!


Today the main workshop begins, and runs through to Thursday. I’ll be participating in Working Group 8, which is a rapid prototyping session in which participants will conceptualize and design a game on a contemporary military or political-military topic. I’ll post a report to PAXsims when it is all over

Simulation and gaming miscellany, 27 September 2015


In a few hours I’ll be headed to Fairfax, VA for the Military Operations Research Society (MORS) Professional Gaming Workshop. Several other PAXsims contributors will be there too, and I’ll be running a demonstration game or two of AFTERSHOCK as well (email me for details).

Before I leave, however, here are a few recent items on conflict simulation and serious (and not-so-serious) gaming that may be of interest to PAXsims readers. Corinne Goldberger contributed to this latest edition.


According to a piece by Julia Ioffe in Foreign Policy magazine, a series of Pentagon wargames has highlighted the serious military challenge that NATO and the United States would face in confronting any Russian attack on the Baltic states:

In June 2014, a month after he had left his force-planning job at the Pentagon, the Air Force asked [then Deputy Assistant Secretary of Defense for force development] Ochmanek for advice on Russia’s neighborhood ahead of Obama’s September visit to Tallinn, Estonia. At the same time, the Army had approached another of Ochmanek’s colleagues at Rand, and the two teamed up to run a thought exercise called a “table top,” a sort of war game between two teams: the red team (Russia) and the blue team (NATO). The scenario was similar to the one that played out in Crimea and eastern Ukraine: increasing Russian political pressure on Estonia and Latvia (two NATO countries that share borders with Russia and have sizable Russian-speaking minorities), followed by the appearance of provocateurs, demonstrations, and the seizure of government buildings. “Our question was: Would NATO be able to defend those countries?” Ochmanek recalls.

The results were dispiriting. Given the recent reductions in the defense budgets of NATO member countries and American pullback from the region, Ochmanek says the blue team was outnumbered 2-to-1 in terms of manpower, even if all the U.S. and NATO troops stationed in Europe were dispatched to the Baltics — including the 82nd Airborne, which is supposed to be ready to go on 24 hours’ notice and is based at Fort Bragg, North Carolina.

“We just don’t have those forces in Europe,” Ochmanek explains. Then there’s the fact that the Russians have the world’s best surface-to-air missiles and are not afraid to use heavy artillery.

After eight hours of gaming out various scenarios, the blue team went home depressed. “The conclusion,” Ochmanek says, “was that we are unable to defend the Baltics.”

Ochmanek decided to run the game on a second day. The teams played the game again, this time working on the assumption that the United States and NATO had already started making positive changes to their force posture in Europe. Would anything be different? The conclusion was slightly more upbeat, but not by much. “We can defend the capitals, we can present Russia with problems, and we can take away the prospect of a coup de main,” Ochmanek says. “But the dynamic remains the same.” Even without taking into account the recent U.S. defense cuts, due to sequestration, and the Pentagon’s plan to downsize the Army by 40,000 troops, the logistics of distance were still daunting. U.S. battalions would still take anywhere from one to two months to mobilize and make it across the Atlantic, and the Russians, Ochmanek notes, “can do a lot of damage in that time.”

Ochmanek has run the two-day table-top exercise eight times now, including at the Pentagon and at Ramstein Air Base, in Germany, with active-duty military officers. “We played it 16 different times with eight different teams,” Ochmanek says, “always with the same conclusion.”

The Defense Department has factored the results of the exercise into its planning, says the senior defense official, “to better understand a situation that few of us have thought about in detail for a number of years.” When asked about Ochmanek’s conclusions, the official expressed confidence that, eventually, NATO would claw the territory back. “In the end, I have no doubt that NATO will prevail and that we will restore the territorial integrity of any NATO member,” the official said. “I cannot guarantee that it will be easy or without great risk. My job is to ensure that we can reduce that risk.”


RCAT-Falklands-at-Connections-UK-2015-Games-Fair-compressedAt the LBS blog, Graham Longley-Brown offers an analysis of two recent wargames of the Falklands campaign fought using RCAT (the Rapid Campaign Analysis Toolset):

This is not a story about HMS Invincible and HMS Hermes; rather it’s a tale of a carrier being sunk in one RCAT wargame and staying afloat in another. This, and other less obvious variances arising from two separate plays of RCAT: Falklands, highlight a number of interesting observations and insights.

Unusually, we ran two back-to-back RCAT: Falklands wargames with different players at Connections UK 2015, simulating the period between the landings at San Carlos on 21 May 1982 and (broadly) the attack on Goose Green. Although the tactics adopted by both sets of Argentinian players were almost identical, the outcomes, while credible in both cases, were dramatically different.

The two games briefly described above (many details have been omitted) might be considered approximate ‘best case’ and ‘worst case’ outcomes. The key decision was to move the CVBG nearer the Islands, but the ensuing outcome was the result of dice rolls.

Imagine if the games had been played at Ascension Island in April 1982. As Julian Thompson and Michael Clapp said at the end of the RCAT OCT: “We liked [the manual simulation] very much and wish we had such a system in Ascension with Fieldhouse, Moore, Trant, Curtiss, Woodward, Comd 5 Bde and us sitting around the map table thrashing through possible courses of action and, hopefully, agreeing a thoroughly well-considered plan.”

One obvious dilemma/trade-off dramatically illustrated was whether to keep the CVBG well off to the east or move it closer to the Falklands to increase CAP coverage over the AOA. Sandy Woodward said that he “was the only man who could have lost the war in an afternoon” (by losing a carrier), and protecting the carriers was paramount. Why, then, were Thompson and Clapp assured that air superiority would be established and maintained over the Islands?

A few of the ‘so what’ questions that should have arisen from such a back-to-back (or more) playing at Ascension, as occurred at Connections UK, are:

  • What will be the effect of losing a carrier? Shades of Midway!
  • Are Exocet targets randomly determined?
  • Will air superiority over the islands be assured? If not, so what?
  • How effective is the ‘picket ship’ tactic (could the T42/22 combo have been envisaged before the shooting war started)?
  • Will the Argentine pilots have time to target specific ships or will attacks be random?
  • How many ships are we likely to lose, best case, worst case and most likely?
  • How can Argentine Special Forces attacks against the AOA, and logistic supplies in particular, be prevented?
  • Can the Argentine land forces launch an immediate counter-attack against the AOA?
  • Do we need to defeat the Argentine positions at Goose Green? If so, what forces will be required? See the OCT blog for the modelling of Goose Green and the operational commanders’ reaction to that.

It’s rare that a course of action can be played through back-to-back like this. The fact that two very different, but still credible, outcomes resulted from facing similar Argentine tactics reinforces the utility of rapid manual simulation. These wargames took 2 ½ hours each and concentrated on a critical aspect of the campaign; a full play through of the entire campaign takes a day. ‘So what’ questions arising can be examined in detail after the wargame, using reach-back to SMEs if necessary. Ideally, the answers would then become inputs to another series of rapid wargames.

Finally, and on another tack, it’s worth reiterating Cdre Clapp’s comment at the end of the RCAT OCT: “I feel that I’ve been properly de-briefed for the first time in 33 years.”

All these potentially significant outcomes can be achieved in just a few hours by rapid manual simulations.


Lancaster County, PA recently used a simulation to help prepare personnel for dealing with refugees:

“I think when people have a real idea of what someone has been through, it helps them respond better,” said Mary LeVasseur, Lancaster General Health manager of community health.

That’s why about 35 employees of Community Services Group spent Monday pretending to be refugees, working their way through an informative simulation.

“Our services are being called upon to respond to people with language barriers that are a challenge to our established practices,” said Susan Blue, president and CEO of Mountville-based Community Services Group. “ And the need continues to grow. Many of our staff are inspired to try to help CSG expand our definition of community to meet the many needs of these new people.”

So the employees visualized being persecuted. They were assigned to crowded “refugee camps.” They carried rice on their heads. And they ran into reams of red tape.

Jessica Knapp of Lutheran Refugee Services, who ran the simulation in collaboration with staff and volunteers from Franklin & Marshall College, Church World Service and Lancaster County Refugee Coalition, said the simulation was “frustrating on purpose.”

It worked; at the end, participants described the experience as overwhelming, eye-opening, uncomfortable. They said it helped them understand why refugees might be reluctant to divulge problems, scared to send their children to school or slow to trust people.



In Denmark, a mock game trailer has been used to satirize (or provoke discussion) about the current migrant crisis in Europe:

The Danish late night talk show “Natholdet” (“The Night Shift”) has posted a suggested trailer of a new board game entitled the “Refugee Game”, which has at least one hard copy. It challenges the players to block migrants as they attempt to enter “happy little Denmark”; and left the audience at a loss: how do you respond to such a teaser?

The trailer features a family of four, two adults and two children, playing the game, trying to block what apparently look like Middle Eastern refugees from entering into their country

The game has a playground with a map of Denmark and its neighboring countries, a set of cards, blocking fences, a boat with refuges, the figures of refugees (also mostly families with children), and the figures of four Danish politicians.

The politicians are Inger Støjberg, Danish Minister of Integration; Lars Løkke Rasmussen, the leader of the center-right liberal party Venstre, Kristian Thulesen Dahl, the leader of the Danish People’s Party and his fellow party member Martin Henriksen.

It, however, remains unclear whether the idea behind the game is sarcasm or criticism of the government in its policy towards the refugee crisis.

You’ll find the video here.



As of late, Adam Elkus has been musing even more than usual about the challenges of computational exploration of strategy. Since he keeps offering new thoughts on the subject, I’ll just direct you to his columns here.



The Active Learning in Political Science blog is always worth reading. Recent contributions include the sad news that the excellent Inside the Haiti Earthquake simulation (which I’ve often used in courses myself) is no longer online; a very critical review of the political science boardgame Agenda; and a discussion of simulations: what are they good for?


The Red Team Journal discusses the value of adversarial interaction—and hence wargaming–in effective red teaming:

Since 1997 I’ve called for more and better red teaming. I’ll continue to do so, but I also believe we must red team the practice itself. In this spirit, I offer a cautionary argument in three parts:

  1. Reciprocal action is the essence of conflict and competition.
  2. Much if not most red teaming inadequately addresses reciprocal action.
  3. Wargaming better exercises the dialogue of reciprocal action.

“But,” you protest, “red teaming is the practice of introducing reciprocal action into decision making!” And I agree, at least in theory. When the client uses the output from the red team to enhance the dialogue, they are to some degree accounting for reciprocal action, certainly more than when they fail to consider the adversary at all.

Risk-and-Opportunity-450Too often, however, clients view the vulnerabilities red teams identify as the penultimate phase of the game. All the clients need to do then is fix the vulnerabilities, and they win, right?—game over! (If only the adversary would agree to play by these artificial rules.) This kind of thinking perpetuates a static, defensive, and short-term mindset, which, as we know, can set a client up for a long-term fall.

In the security domain, you might call the blue team the client’s trusted fixer, someone who collaboratively works with the client to address the red team’s recommendations. From a wargamer’s perspective, however, this is all somewhat counterintuitive. Yes, the blue team should defend its position against risks generated by the adversary (represented by the red team), but it should also reciprocate against the adversary and generate risks for them as well. By definition, however, security blue teams perform only the first function, and in so doing help perpetuate a defensive mindset.

In reality, an organization should address both risks and opportunities, and this is precisely what a blue team typically does during a wargame. In fact, a blue team that seeks only to address risks will fail, as will the real-world organization that does the same. In this sense, then, a wargame blue team is much more expansive and realistic than a security blue team….

You can read the whole thing here.


David Vallat (University of Lyon 1) recently pointed me towards a paper he and several colleagues wrote on using serious games to leverage knowledge management. You’ll find it here. See also his blog post on fun learning.


At VICE earlier this month, Giaco Furino discussed how Dungeons & Dragons went mainstream.

Such is my geekiness, it comes as a shock that anyone ever considered it anything but normal….



If you’ve recently had trouble ordering AFTERSHOCK: A Humanitarian Crisis Game, it is now available again via The Game Crafter. (The publisher had run short of 10mm supply cubes, so we’ve swapped them for a slightly smaller 8mm version.)

Paul Vebber on Fleet Battle School

This week’s MORS Community of Practice talk featured Paul Vebber from the Naval Undersea Warfare Center, presenting on a game sandbox tool, “Fleet Battle School”. Vebber has shared quite a bit about the development process over time, and has posted many material in conjunction with this talk, so this is a good project to look at for folk interested in how digital game development actually happens in DoD.

Vebber started with a general information on gaming and game design he uses for audiences that are less familiar with gaming. While not the focus of the talk, I wanted to highlight two of his graphics, both of which provide some useful synthesis of recent debates on the nature of gaming. The first integrates the “cycle of research” with some of our recent discussion about what type of logic is used in games to generate new knowledge.

Vebber--cycle of research

The second discusses the relationship between OR analysis, gaming, and the level of problem, which is a common concern in my work and the broader field.

My interest in broad methods issues aside, Vebber’s presentation was focused on a overview of the current state of the Fleet Battle School game sandbox.

The core goal of the project was to design a capability game to determine what point does the change in capability cause changes in player decision making. As a result, the platform should have the ability to support very interesting sensitivity analysis about the intersection of combat effectiveness and decision making that is often elided or ignored in current gaming.

Right now the focus of the Fleet Battle School game is weapons and sensor capabilities of different platforms (though features like speed and fuel use are also built into the current rules). The game instantiates relative differences in capability rather than trying to mimic specific current capabilities. While you can input values that mimic specific real world platforms, that isn’t really the focus of the project. Again the focus here is how the ratio between different platforms’ capabilities impacts player decision making. As a result, the adjudication model was built to focus on plausible results, rather than claiming any type of predictive power.

Vebber-Gaming and OAFleet Battle School is a digital platform for naval operations planning game built on krigspiel principles (that is rigid rules for physical movement and combat, in contrast to a seminar table top game that would use looser rules focused on organization and political decision making). The system allows the game designer to edit the map terrain, platform capabilities, order of battle, and rules (though some programming skills are needed for really deep changes here) within the game platform.

The game also allows for multiple “levels” of players on both the blue and red team, so that the gap between commanders and line officers can be included in the game. The commander can set a daily intent, individual “officers” can then set more specific orders which the commander then approves and submits. C2 is largely handled outside of the game platform in order to accommodate different networks, but it will require that C2 be documented outside of the platform.

The system can then either auto-adjudicate or allow an umpire to override outcomes either in all cases or only in less probable die rolls. There are also some nuanced setting to represent friction that can also allow penalization of bad leadership and declining capabilities over long deployments.

That’s all is a pretty superficial description of the platform and its capabilities, but if all this sounds interesting, I would urge folks to check out the Wargaming Connections blog where Paul has posted more materials in association with the launch of the beta.

One point that I think is critical to highlight is the way Vebber’s development experience also allowed the project to avoid many common pitfalls with computer game development in the government. This has been a long development process with lots of paper playtesting, use of off the shelf products for some game functions where appropriate, and “good enough” graphics until end-users articular clear priorities. Having been involved in computer game development from within the government, I’ve seen how a lot of these can go wrong, so in many ways I think Fleet Battle School is a great case study about how to do this kind of development. Vebber’s regular and detailed updated on the development process should be a reference to anyone attempting this kind of project in the future.

MORS Professional Gaming Workshop, 28/29 September – 1 October 2015


There is still time to register for the Military Operations Research Society’s forthcoming Professional Gaming Workshop in Fairfax, VA:

A three day workshop (29 Sept. – 1 Oct.) plus optional 1 day course (28 September) on professional gaming as an analytic practice.  The meeting will produce initial content for a Professional Gaming Practitioner’s Handbookand bring together members of the community of practice to consider best practices, design, existing applications and appropriate analytic methodologies in an effort to codify the fundamentals of game design and analysis. The meeting is designed for information exchange and participant exposure to professional practice. There is no intention to conduct a game or for attendees to participate in game play.

Working Groups Include:

1. Event Synthesis

2. Objective Development

3. Game Design & Development

4. Data Collection, Analysis Methods and Tools

5. Adjudication procedures

6. Aligning Games with larger studies and methods

7. Professional Game Execution

8. Quick-Turn Design – continuation of hands-on training course

I’ll be one of the facilitators for WG8, as will be sometimes PAXsims contributor Brian Train. There will even be an opportunities to play both AFTERSHOCK and ISIS Crisis!

Fred Cameron (Operational Analytics) has created an excellent website for the MORS meeting, with information on the key steps of wargame design, implementation, and analysis. You’ll find it here.

Simulation & Gaming, April 2015


The latest issue of Simulation & Gaming 46, 2 (April 2015) is now available. This is a symposium issue devoted to Theory to Practice in Simulation.

Simulation and gaming miscellany, 18 September 2015


Some recent items on conflict simulation and serious (and not-so-serious) gaming that may be of interest to PAXsims readers. Christian Palmer assisted with this latest edition.


A recent article in the Harvard Business Review suggests “Games Can Make You a Better Strategist.”

Play has long infused the language of business: we talk of players, moves, end games, play books and so on. And now we hear often about the “gamification” of work—using elements of competition, feedback and point scoring to better engage employees and even track performance. Even so, actual games are still taboo in most organizations—the stereotype of the work-avoiding employee cracking new high scores in Minesweeper has given gaming a bad name. And the corporate executive playing games to improve his or her strategy-making skills is still rare. This is unfortunate. We think that games have an important place in cultivating good strategists, and that now more than ever games can give executives an edge over their competition.


Further evidence that board games are (re)entering the cultural mainstream: The Independent features an article on the classic game Diplomacy.

Diplomacy is a board game in which players compete to achieve world domination, taking on the role of various countries in the lead-up to the First World War. The board is a map of Europe; each player (England, France, Germany, Austria-Hungary, Italy, Turkey, and Russia) starts off with a certain number of territories, or supply centres, and the winner is the first to expand to 18. It’s a game of intrigue and aggression, which has achieved cult status, complete with tournaments, postal games and dedicated webzines. Henry Kissinger apparently used to play it as practise for the real thing and, inverting this, I’ve started paying more attention to real life diplomatic flashpoints – the negotiations over a Greek bailout, say.

A distinctive feature of the game, as conceived by its creator, Allan B Calhamer – a Harvard educated postman – is that there is no element of chance. Unlike with Risk, there are no dice; an attack succeeds based solely on whether the units attacking a territory outnumber those defending it. Because you are simultaneously trying to expand into other players’ territories and defend your own, you need to strike deals with your fellow Great Powers….


Earlier this month, the Three Moves Ahead podcast featured an interview with game designer Brian Train:

September 3, 2015 This week Bruce sits down with prolific game designer Brian Train. Brian’s games span a wide range of conflicts and scenarios, including Greek civil war, fighting in Algeria, and Special Forces actions in Vietnam. His predilection for asymmetric warfare and non-traditional combat modeling have brought his games to the attention of US military simulation experts and wargame fans throughout the world.



Although the game isn’t new—it was first released in 2014—all the current attention to illegal immigration in the current US Republican primary race makes this an apt time to mention The Migrant Trail.

“THE MIGRANT TRAIL  presents a first-person journey through Arizona’s desert borderlands.  Play as  an undocumented immigrant attempting to cross the Arizona desert and/or a border patrol agent attempting to secure the border.  Playing the game offers an alternative  platform to further engage conversation, investigation and inquiry, into the themes and questions raised by the documentary.Migrant Mode Intro

Every year an unknown number of migrants cross through the harsh Sonoran desert from Mexico into Arizona.   They pay $1500-$2500 to join a crossing party, that is led by for hire guides referred to as Coyotes.   If one cannot keep up, twists ankle or runs out of water, he or she is left behind and many die.  On average, the remains of 200 dead migrants are found each year.  It’s not known how many are never found.Border Patrol Mode Intro

Every day U.S. Border Patrol agents patrol the Sonoran Desert along the Arizona-Mexico border. Their job is to apprehend undocumented border crossers, provide first aid to the injured, and locate the remains of dead migrants.

The game was produced to accompany release of the film The Undocumented.


The (Minneapolis) Star Tribune features a report on a recent humanitarian crisis simulation conducted by the University of Minnesota:

The refugee camp that sprung up in Minnesota last weekend was much like others in conflicts across the globe. Exhausted refugees cried out for food. Camp doctors struggled to aid the sick. Soldiers toting M16s tried to keep peace.

But this camp had one big difference. The roughly 170 people in its drama were volunteer actors in an elaborate “humanitarian crisis simulation” that sprawled across woods and fields at a Boy Scout camp near Cannon Falls, Minn. It is a weekend class offered by the University of Minnesota, with help from a half-dozen nonprofit organizations and the Minnesota National Guard — whose soldiers act as not-so-friendly foreign government troops.

One of a handful of such hands-on training camps in the nation, it is designed to give individuals considering humanitarian aid work a realistic look at the complexities ahead. Given the migrant and refugee crisis exploding in Europe, it is timely instruction.


The American Political Science Association has issued a call for papers for its 2016 Teaching and Learning Conference, to be held in Portland OR on February 12-14. Submissions are due by September 25.

The theme for the 13th Annual APSA Teaching and Learning Conference is, “Rethinking the Way We Teach: High-Impact Methods in the Classroom,” which focuses on gaining a greater understanding of high-impact practices and innovative methodologies in the political science classroom. Panels and workshops will offer a forum to share pedagogical techniques and discuss trends in political science education.

As in the past, simulations and games will be among the main conference tracks:

Simulations and games can immerse students in an environment that enables them to experience the decision-making processes of real-world political actors. Examples include in-person and online role-play scenarios like the Model European Union and ICONS, off-the-shelf board games, Reacting to the Past, and exercises that model subjects like poverty, institutions of government, and ethnic conflict. Papers in this track will examine topics such as the effects of gamification of course content on student motivation and engagement, cognitive and affective outcomes from simulations and games in comparison to other teaching techniques, and the contexts in which the use of simulations and games makes sense for the instructor.

Click the image below for more information.


Frost on Game Facilitation

SAGDEarlier this month I was able to tune into the MORS Community of Practice for a talk by Adam Frost from the Studies, Analysis, and Gaming Division of the Joint Staff (J8) on game facilitation. Frost’s office is responsible for many high level policy games (including for the Chairman as mentioned in this article), so the talk was given from the perspective of running games for very high level policy makers at the strategic level. That said, much of Frost’s advice resonated with the training I’ve received, as well as my own practical experience, and I believe his advice is broadly applicable.

Frost intelligently started the session by facilitating a short discussion on what makes for good and bad facilitation—no mean feat when over half the participants were on a phone line and he had no list of participants! The group identified some patterns in poor facilitation, including:

  • losing control of the room or letting someone else take over
  • asking the wrong questions
  • having an agenda or obvious point of view that biases the outcome of discussion
  • having a facilitator unwilling to adjust to reality in the room
  • “dead air” or obvious gaps in the discussion that drain energy

Similarly, signs of good facilitation include:

  • asking the right questions
  • successful synthesis, particularly that which leads participants to novel conclusions
  • player immersion, indicated by signs like players’ willingness to play past the deadline
  • energy and pacing
  • redirecting conflict and emotion into productive discussion

One point that came up early in discussion was what the division should be between a game and a facilitated discussion. To my mind, facilitation is a skill that is needed to lead all kinds of discussions, from meetings to strategy sessions to games. It ensures that the conversation produces the desired end products, and that everyone leaves the room more or less alive. The structure of the discussion that the facilitator adheres to and the end results are all that is different.

One overarching comment was that good facilitators aren’t remembered; they fade into the background of the discussion, which is what folks remember on leaving the room. This notion came up in many ways over the course of discussion, and I think it is a particularly helpful reminder for inexperienced facilitators. The goal of facilitation is not to make an impression on senior leaders, teach, or to show off how much you know—rather it is to collect information from others so it can be processed by the group. As a result, many of the tips and tricks for facilitation mentioned by Frost and other participants depended on not having an ego and being willing to say dumb things in order to move the conversation forward. As a result, in many cases it can be better for the facilitator not to be an expert so they don’t have a reputation to maintain.

This also means that age need not be a barrier to being a facilitator, I’ve seen young facilitators who were very effective because they allowed the participants to teach them over the course of the discussion. However, it takes a very particular mindset to let go of your own ego, and that mindset takes time and practice to achieve.

Frost then moved into a more formal briefing. He first discussed the difference between the roles of the game director and game facilitator, summarized in the table below:

Game Director Game Facilitator
Primary Responsibility Ensures the game meets the purpose and objectives laid out by the sponsor or client Ensures that the discussion in the room is productive
Team Roles Team lead Supports director; should be a part of the design team from as early in the development process as possible
Game Design— Process and Rules “What questions do I need to answer?”

Determines best method/mechanics of the game to meet purpose and objectives defined in conversations with the sponsor

“How do I ask the game director’s questions of the participants?”

Determines how the flow of conversation can best meet the director’s vision, including what questions need to be answered at each stage of the game in order to move forward

Game Design— Roles Determines the right types of roles to include in the game, based on the purpose and objectives of the game Determines who are the right participants, matching individuals with designed roles; knows the background of the participants and anticipates how it will shape behavior during the game
Game Room Setup Ensures game materials are available and correct, including slides Ensures layout of room will support the discussion, including location of different teams, seating chart, etc.
During the Game Makes sure the game is meeting objectives; makes changes to the design to account for unexpected problems that are preventing the game from achieving objectives Ensures the game “goes”—that participants buy into the game process, and that discussion stays moving and on track with game director’s vision
Adjudication Designs the adjudication process Ensures participants understand and don’t fight the adjudication process—that is, “sells” it to participants

This division also comes with an interesting, and not necessarily intuitive, division of responsibility/credit for the game’s outcome. The room generally agreed that good facilitation may be able to save a badly-designed game, and bad facilitation can sink a well-designed game. That may make the facilitator more responsible for the game’s success than the director, even though they are rarely the project lead. Because of this, game directors can get lazy and dependent on the skills of a good facilitator to ensure the success of a game. One specific example of this is the development of the questions to be answered at each step of the game. These should be the purview of the director, but too often fall to the facilitator.

Frost also noted that responsibility for data capture (whether through note-taking or other means of recording game events) should be a third role. I can speak from experience: thinking that you can capture notes while facilitating will be a lost cause. You just cannot listen to the speaking participant, think about the next question(s) you will ask, and kept the session objectives and schedule in mind at the same time you are trying to keep a written transcript of the discussion!

Frost then delved more deeply into the characteristics of a good facilitator. This included:

  • Facilitation is not command. You are guiding, not telling. (Relatedly, facilitation is not teaching in the normal sense, though it does share some similarities with Socratic-style questioning.) This means that it can be an uncomfortable role for senior leaders, particularly from the military, where leaders are used to a very different role.
  • Facilitators need not be senior. In fact, Frost mentioned several times that he has found his relative youth to be an asset because it allows him to “play ignorant” and draw better explanations from participants. I’ve had similar experiences, both with age and with being an “outside” civilian who can ask for more details about military tactics and strategy. That said, Frost also highlighted ways in which his facilitation style differs from more senior facilitators, including needing to stand in order to hold the room. Time to practice and experiment is critical to figuring out what will, and will not, work for you!
  • As a facilitator, your opinion does not matter. The goal of a game is not to affirm that you are right; it is to bring the group to consensus on a decision, and you need to have the mental flexibility to let the discussion go where the participants want. Most of the community of practice agreed that floating an idea as the facilitator can be a way to bring missing ideas into discussion, but whether the group runs with, or abandons, the idea is up to them. Another way to think about this is that the facilitator can be a foil for participants’ ideas, but shouldn’t be an advocate for any idea themselves. Likewise, in some cases, you can set up questions for particular people based on a general idea of what they will say, but, as Frost put it, “if I know exactly what the person is going to say the game is going to feel scripted.” That said, particularly when participants have specific, narrow expertise or are introverts, it can be a good way to get folks engaged in the discussion.
  • Have a strong opening. In the first few minutes, you need to establish why participants are in the game; set expectations about how the game will proceed (purpose and schedule of the game); and provide the minimal critical information participants need to set the scene for initial game discussion. This information is usually in the game read-ahead, but it’s a safe bet participants haven’t read or remembered it. That said, overviews of the scenario should be short (1 slide not 13), as more details take time from discussion.
  • Anticipate participants’ tendencies. This includes common patterns of behavior (military participants won’t contradict a senior officer; policy and intel participants want to get a lay of the room before they speak) and how to manage them (ask senior leaders to speak later in the game; let people know what question you are going to ask them before you put them on the spot). Relatedly, as a facilitator, you want to make the participants look good by setting them up for success, so it is critical to let folks know you are going to call on them and give them time to think about the question.
  • Always have a question to ask to keep conversation from dying out, but don’t use it unless it has died. This question should be short, and have a question mark at the end. You talking as the facilitator both takes time away from participant discussion, and can make you look uncertain. Anyone who has ever been to a DC think tank event knows what this looks like!
  • End on time. This requires that the facilitator pay attention to the game, and as some sessions run long, develop a plan to make up for lost time. This is particularly important the more senior your participants are.
  • The facilitator sets the room’s energy level. As a facilitator, you need to seem enthusiastic and motivated, or you will lose participants. A trick I learned is to think about the volume of your voice, size of gestures, and general energy level when you are chatting with someone at home, and contrast it with the way you talk in a large, noisy room like a bar. When facilitating, you want your energy to be right in the middle—definitely more than a quiet conversation, but not so much that you seem like a crazy person!

The discussion concluded with a brief discussion of how to gain skills as a facilitator. The group generally agrees that facilitation is a practical skill that is best learned by doing. However, a few folks (myself included) spoke in favor of formal coursework on facilitation. In my experience, facilitation training often feels silly at the time, but can be critical to learning good basic practice. I left the discussion believing that coursework can be helpful in getting from bad to competent — but to get good, you need talent, mentorship, and practice.

Saur on Teaching Gaming

Joe Saur gave a good talk on teaching gaming at the MORS Community of Practice. I’ve been remiss in not posting my notes before now, particularly because teaching gaming is a subject near and dear to my heart.


Saur’s presentation focused on his experience teaching 70-plus students across the military, many who lead organizations that use wargames for analysis and training. One point that Saur highlighted was that even though his students had extensive operational experience and are quite likely to be game sponsors, very few had previously seen a wargame. This is a critical point to consider as the community thinks more about how to best communicate of methods and results to our sponsors. It really reinforces the need to spend more time and energy thinking about how we as gamers to educate sponsors and stakeholders. While Saur was working within one of the military school houses, we are going to need more approaches in the long run to get a broader understanding of the benefits and uses of gaming.

Saur noted that there are not many syllabi for wargaming classes. He was able to reference a UK wargaming and combat modeling class, but that was largely focused on the math required for combat and campaign modeling with participation in a staff game. As a result, this course provided limited guidance on how to teach gaming.

In building his syllabus, Saur aimed to teach mechanics that staff officers can actually use. His goal was to expose students to a range of games as a starting point to support student development of operations game. As a result, he tended to focus on concrete mechanisms like dice, hex grids, miniatures, and cards drawn from hobby gaming, with only limited coverage of less structure techniques like matrix and seminar games.

One point that I found particularly interesting is that during student discussions, they hypothesized that as the average member of the force has less combat experience moving forward (or their combat tour is further in the past), rigid adjudication will become more critical. Students argued that free adjudication relies on operational experience.

Not surprisingly, I’m fairly skeptical of this claim, particularly in the case of operational and strategic games. Most of the strong game designers I know are civilian analysts, because members of the military are rotated through positions too quickly to gain mastery. Furthermore, rigid systems of adjudication rarely survive analytical games intact, as players almost always seem to do something not anticipated in the game rules. As a result, even highly formalized rules will often require impromptu adjudication calls. Finally, I’m fairly skeptical of rigid adjudication’s ability to capture interpersonal social and political dynamics that strongly impact strategic and operational outcomes. Limiting ourselves to rigid rule sets cuts off from gaming many of the complex, unstructured problems that games are best suited to examine.

The presentation concluded with a selection of the games built by the students. These covered an impressive range of topics and game design approaches. In part, the approach seemed particularly impressive because Saur instructed the students to tie the games they designed to their follow-on posting. As a result, the games were designed to be practical and helpful, rather than academic in nature. I’ll be interested to see if any of the students follow up with notes about how deploying the game in their new posts goes!

Connections UK 2015: Day 3 AAR


The third and final day at this year’s Connections UK wargaming conference started with a panel discussion on wargaming best practice. David England, (Niteworks) talked about gaming, experimentation, and force development; Jeremy Smith (Cranfield) presented on validation and verification of a manual simulation, specifically the Rapid Campaign Analysis Toolset (RCAT); I talked about political-military (pol-mil) gaming; and Capt Ed Farren (British Army) made an excellent presentation on wargaming and officer training.

In my own presentation I made a series of points:

1. The very first step in political-military wargaming is to decide what you are doing and why. Is the game intended to generate ideas, stress-test existing ideas, horizon-scan for possible future developments, or as an experiential and learning exercise?

2. Is a game really the best approach to the problem?

3. Is a big or complex game the best solution to the problem, or can it be addressed as effectively with simpler gaming techniques, like matrix games?

With regard to all of these observation I made the point that we should adopt a “toolkit” approach in which game design components are matched to purpose, rather than promoting a one-size-fits-all approach. I also argued for “responsible games evangelism” in which we addressed the weaknesses as well as the strengths of game techniques.

4. Everyone doing political military wargames ought to read Robert Levine, Thomas Schelling, and William Jones, Crisis Games 27 Years Later: Plus c’est Déjà Vu. RAND Corporation Report P-7719 (1991).

5. How you structure participation, and who is represented in the game, dramatically affects many game outcomes. In many pol-mil games it is difficult to know who to incorporate and how to incorporate them.

6. Idiosyncratic factors can have heavy influence on game play. Partly this is a positive thing, reflecting the ways in which games can generate emotional and personal engagement by players (a point made at the conference by ED McGrady, Peter Perla, Graham Longley-Brown, and others). However, it may also mean that personalities—unconstrained by institutions and politics in quite the same way as real world policy-makers—distort game outputs.

7. Considerable attention must be given as to how to capture and record game play, debrief players, encourage reflections, and carry ideas and findings forward into the policy process.

The subsequent audience discussion of the panel presentations addressed such issues as the value of commercial off the shelf wargames in officer training; the impact of professional subcultures on the behaviour of game participants; and the relationship between learning styles and games-based learning.

The third day of Connections UK 2015 had obligatory diagrams too--this one by Philip Sabin.

The third day of Connections UK 2015 had obligatory diagrams too–this one by Philip Sabin.

After the coffee-break we came back for a panel on the synergies between hobby and professional wargaming. Phil Sabin explored the contemporary use of historical wargames, highlighting their value both in educational and academic settings and well as within the national security community. He placed particular emphasis on how wargaming and learning about conflict simulation helps (future) analysts to better understand and model current and future conflict. At the same time, he also noted that many contemporary hobby games are too complex or too unrealistic to be readily used for analytical or training purposes. John Curry (History of Wargaming Project) also looked at professional use of recreational wargames. He cited a number of examples of games that have been used by both hobbyists and the military, or which might be. He, like Phil, highlighted some of the limitations of commercial games too (accuracy, complexity, duration, process, topic, umpire roll-back). In response to a question about how well commercial wargames explored hybrid warfare, Phil made some excellent points about the way in which rules and victory conditions could incentivize players to engage in realistic behaviours, or be used to assess otherwise incommensurable costs and rewards.

The last part of the conference was devoted to a “skinning the cat” session on wargaming for innovation, in which participants were divided into groups, and each asked to select a topic and develop game ideas around that theme. The potential topics were:

  • conflict in the Ukraine
  • an election
  • mercenaries/private military corporations
  • national power cut
  • influence operations
  • a terrorist attack against a school
  • urban warfare
  • refugees and migration

My group chose the last three of these, and I facilitated the design discussions for Thomas Crook: A Game of Human Trafficking. In our game proposal—intended as a professional red-teaming resource for those dealing with forced displacement and migrations issues—a half dozen or so players would assume the role of smugglers, and would compete to move migrants to Europe using a variety of possible routes and techniques. It was a great session with excellent input from everyone, and the emerging game design seemed very sound indeed.

* * *

And so it all finally came to an end. Overall, this year’s Connections UK was outstanding—indeed, the best ever. I very much look forward to next year’s meeting.

Connections UK 2015: Day 2 AAR


Today is first day of the main programme of the 2015 Connections UK wargaming conference, and the second day of the event.

Frans Berkhout (Executive Dean of the Faculty of Social Science and Public Policy, King’s College London) welcomed the participants. He highlighted the broad and growing interest in the social sciences in simulation and the development of “synthetic worlds” for experimentation and exploration.


The largest Connections UK conference ever.

The first panel of the day addressed global wargaming developments. Peter Perla (CNA) discussed developments in the US in the wake of renewed interest in wargaming by the Department of Defense. He suggested that for wargaming, “it was the best of times, it was the worst of times.” He examined four recent memos on wargaming by former US Secretary of Defense Chuck Hagel, by US Deputy Secretary of Defense Robert Work, on implementing the initiative, and by the Secretary of the Navy Ray Mabus. He welcomed the renewed attention, but warned that we needed to capitalize the moment or lose a golden opportunity for many years. He expressed concern that everything would now be called a “wargame” as everyone jumped on the wargaming bandwagon with little attention to quality. Peter also expressed concern at the idea of “systematizing” wargaming and the development of a wargame repository. Discussing the Navy’s efforts in this area, he noted some of the potential problems and sources of resistance. On a very positive note he observed that the Deputy Secretary of Fefense appears to have picked up on the point—strongly made at the recent Connections US conference—that wargaming needs to be integrated into curriculum of professional military education from an early point in officers’ careers..

Matt Caffrey (USAF) highlighted how the original US-based Connections conference has gone global, with annual conferences now being held in the UK, Australia, and the Netherlands too. He started by making the general case for wargaming, then discussed the ways in which the Connections conferences could support both the general advancement of wargaming and skills development by new wargamers.

PAXsims’ own Devin Ellis (ICONS Project) discussed wargaming with the Chinese, drawing upon his experience with a decade of political-military crisis management gaming initiatives with China. He noted that the People’s Liberation Army has placed increasing emphasis on wargaming. PLA gaming tends to be kinetic, more operational than strategic, and rather doctrinal. Chinese “blue teamers” who might play the American side in a PLA pol-mil games often poorly understand US military doctrine and foreign policy. Chinese game participants tend to place a great deal of emphasis on establishing guiding principles. They appear strongly committed to not taking the first shot. Moral judgments are often applied to pragmatic behaviours, although this seems to be changing. Legal standards are important, but selectively interpreted. Chinese players tend to be suspicious of US intentions, and misunderstand US alliance relationships and force posture. Chinese interagency processes and knowledge are limited. Overall Devin suggested that while the Chinese seem increasingly committed to improving the quality of their wargaming, the learning curve is steep and there are many institutional obstacles to be overcome.

This panel was followed by a games fair briefing, in which quick three-minute overviews were provided of each of the fourteen games on display today.

After lunch attention turned to UK wargaming developments. Rob Solly, the Division Head for Defence and Security Analysis (DSA) at Dstl, discussed putting wargaming back at the heart of analysis. He suggested that the renewed interest in wargaming was due to the nature of human-centric, complex nature of contemporary problems which are less amenable to conventional analysis; because it is sometimes better to help a client learn about themselves, rather than simply being taught; and because divergent thinking mechanisms are needed to help open the minds of decision-makers. Dstl’s wargaming skills into a new wargaming hub at DSA amid a growing appetite for wargaming across the UK defence and security establishment.


The new commandant of the Royal Military Academy Sandhurst talked about the need to educate young officers for future uncertainties, while also training them for the enduring aspects of combat. Staff colleges place a great deal of emphasis on planning, but not enough in exploring execution, adaptation, and adversarial competition. Currently, wargaming at Sandhurst largely consists of some COA (course of action) (quasi)wargaming, TEWTs (tactical exercises without troops), and BOGSAT-type discussions. However, there has been little or no substantial, adversarial wargaming. They have newly introduced a map-based post-TEWT kriegsspiel, and there may be other places where they can introduce wargaming too. While there is a wargaming club at Sandhurst, there hasn’t been widespread participation from cadets.

The map for Tom Mouat's tactical kriegsspiel design for RMAS.

The map for Tom Mouat’s tactical kriegsspiel design for RMAS.

In chairing the session, Gen Andrew Sharpe (retd) suggested that Army officers would be more willing to wargame if there were senior signals that it was a good thing to do. He also noted that often large exercises or games are sufficiently rare that no one particular wants to face a creative enemy that might defeat them. He stressed that there needed to be more wargaming to see if a plan will work, as opposed to confirm that it will work.

The first games fair session was a busy one. One of the drawbacks of demonstrating a game, however, is that you have no opportunity to examine the other games on display. There were a great many that looked very interesting, with a broad range of topics, approaches, and game mechanics in evidence.

Participants get ready for the game fair.

Participants get ready for the game fair.

Part of David Vassallo's extremely impressive HOSPEX field hospital simulation game, first developed in Philip Sabin's MA class in conflict simulation at KCL.

Part of David Vassallo’s extremely impressive HOSPEX field hospital simulation game, first developed in Philip Sabin’s MA class in conflict simulation at KCL.

Kestrel's Hover, an air assault game developed for 16 Air Assault Brigade by Dstl.

Kestrel’s Hover, an air assault game developed for 16 Air Assault Brigade by Dstl.

Phil Sabin's CHACR Camberley Kriegsspiel.

Phil Sabin’s CHACR Camberley Kriegsspiel.

AFTERSHOCK set up and ready to play.

AFTERSHOCK set up and ready to play.

I did get a chance to play a few turns of The Great Crossing, Jim Wallman’s simple but elegant game of refugee flows and economic migration. Fa theced with a growing flood of migrants, noble country of Silvania (that would be me) worked out an understanding with several other regional countries on managing the flow, including offering asylum to refugees and some integration to economic migrants. Others, however, pursued more of a beggar-thy-neighbour strategy of blocking refugees and trying to push them towards the borders of other countries. Sadly, some refugees were even lost at sea.


The Great Crossing.

I also ran a game of AFTERSHOCK. The players did well, despite some periodic tensions between them—indeed, at one point the UN and host government threatened to hold a press conference denouncing the NGOs unless they cooperated on mobilizing donor support.

AFTERSHOCK underway.

AFTERSHOCK underway. The stress and horror of the disaster can be seen etched into their very souls.

A keynote address by ED McGrady followed on why wargaming works. He emphasized the importance of both narrative and play, stressing the “art” of gaming. Rationalists, he suggested, are uncomfortable with game play since it creates new, imaginary worlds. Games, he suggested, are indeed different and special territory, allowing us to explore the non-rational aspects of behavior (in war or otherwise) as well as unanticipated associations and unexpected narratives. For games to work they need to be grounded in rationalist behavior, but they become irrational once the game starts. More research was needed, he suggested, on how the play element of games affects individual and collective decision-making in serious games.

After dinner there was a second games fair session—and a second demonstration game of AFTERSHOCK. This one was fairly close, with the players suffering heavy losses in the first week of the disaster. However very effective coordination helped them to achieve considerable improvement thereafter, resulting in a comfortable collective victory by the end of the game.


Get every new post delivered to your Inbox.

Join 240 other followers

%d bloggers like this: