PAXsims

Conflict simulation, peacebuilding, and development

Workshop on history and games (Glasgow School of Art)

The Glasgow School of Art (Digital Design Studio) will be holding a workshop on history and games on 29 September 2016:

The main goal of this workshop is to give a state-of-the-art picture of Serious Games in Education, in particular in the learning domain of history, and to identify further opportunities of using digital or analogue games as a teaching tool in this domain, but also more widely. This workshop aims to reach out to various stakeholders and experts in education, game design, game development, and systems development. The format of the workshop will be: short, overview-style presentations and game demos to start with, followed by activity and discussion sessions in game design and serious mod.

This workshop is part of a longer-term effort in the development of a game engine, the JominiEngine as a practical teaching tool in the domain of history education. We hope to build a community of interested partners out of this workshop and solicit input for the further development of the engine and for the setting of priorities….

You’ll find full details here.

h/t Philip Sabin

Connections NL 2016 report

nl

The following report was provided for PAXsims by Hans Steensma, Bas Kreuger, Swen Stoop, and Anja van der Hulst.


 

waterline.png

Defending the Netherlands.

After a marvellous Connections UK, Connections NL was also exciting and fun. For the third time we got together in a fortress of the New Dutch Water Line. This Line, together with the Amsterdam Defence Line, is a 19th century defence system with a circumference of 215 kilometre, encompassing the cities of Amsterdam and Utrecht. It protects the western part of our country, with our harbours and the seat of government. This massive system of fortifications is formed by at least 105 fortresses, 6 fortified towns and two castles. The strength of the defence is in its ability to inundate large tracts of land between fortifications. A very Dutch experience indeed.

Connections NL has a broad scope and includes members from the business community, the crisis management, and education communities. Consequently we had quite a diverse group of attendants.

fortress

Matt Caffrey was our guest of honour and he did a great Wargaming 101 session and Q&A sessions afterwards. Mark Stoop showed more of his scenario based policy gaming for very senior leadership, we reported on the current developments in wargaming and we did a lot of hands-on gaming with our 55 attendees.

caffrey

Our students eagerly listening to Matt Caffrey discuss wargaming.

In the hands-on sessions, there was a special presentation by a team of Marine lieutenants (ex midshipmen). They told us the harrowing story behind the wargame Matruska that they created and hosted this spring at the naval academy. When they started designing this game, they had no experience at all with wargaming and within a month they created a modern crisis game that was remotely based on the Cuban crisis, with a total communication black-out that confronted leadership at the naval base in Den Helder with some really nasty decisions that might have had substantial political repercussions. They also showed us how perceptions can be deceiving. A good grasp of reality, sound decision making and excellent command guidance helped the players avoid ultimate disaster: going to war over a jealous husband.

matt2

Hands-on with Matt at the TNO Game Lab.

The second day was a more intimate hands-on session at the Dutch Defence research facility of TNO. We thoroughly enjoyed playing AFTERSHOCK, Command Modern Air Naval Operations, a Port Safety and Security game under development, and the re-design of a refugee game made by Jim Wallman. The redesign effort was oriented at highlighting the influence and importance of ethics in the resolution of the refugee crisis.

Since we started with the try-out in 2014, and the real first Connections NL in 2015, the Netherlands has also been infected by the US and UK surge of enthusiasm for wargaming. We see many interesting developments within wargaming in the Netherlands. The military schools are (re) introducing wargaming and we see a fair amount of spin-off to the business community. With the education community following at a distance, games are slowly gaining traction.

IMG_1747.jpeg

Playing AFTERSHOCK.

Next year we will again host the seminar in one of those awesome fortresses, and it will be your chance to visit and be part of them. As part of our maritime trading heritage our second native language is English. So even though Connections NL is oriented at awakening wargaming in the Netherlands, we welcome guests from abroad and make them feel welcome.

Although Connections NL is a lot smaller and less seasoned than Connections UK or US, it might still be interesting to an international audience, precisely for our trading culture, inviting participants from business, government and education as well as the military. Come over next year and help us build the broader base for the employment of wargaming.

For more information on Connections NL visit our website. Also recommended is a good report by a distinguished participant from Belgium.

NL2.png

Matt’s “Wargaming 101” summarized.

Review: Paddy Griffith’s Counter-Insurgency Wargames

John Curry, ed. Paddy Griffith’s Counter Insurgency Wargames (History of Wargaming Project, 2016). 91pp. £12.95pb

 

Griffith.jpegPaddy Griffith—military historian, lecturer at the Royal Military Academy at Sandhurst, hobbyist, and founding member of Wargame Developments— was an influential figure in the evolution of British wargaming. In this volume, John Curry and his History of Wargaming Project have collected together materials from two counterinsurgency (COIN) simulations that Griffith developed in the late 1970s, as well as the outline of the main components of a live action exercise. Prolific COIN wargame designer Brian Train provides a Foreword to the collection, placing the wargames in the broader context of developments in counter-insurgency doctrine and practice.

If the first game, LONGREACH VILLAGE (1980), looks rather like a fictionalized British Army and Royal Ulster Constabulary hunting for an IRA active service unit in a Northern Ireland border village—well, that’s hardly surprising, for such was the counterinsurgency challenge that would be faced by young British officers at the time. Today,  when most insurgencies and COIN wargames alike involve underdeveloped and failed states, it may all seem of marginal relevance. After all, this is not a setting where there are major impediments of poverty, language, or cultural understanding. Instead the background materials outline the milkman’s daily routine, the opening hours of the pubs, banks, and shops, and details of the local farmers’ market. However, in doing so the game provides an outstanding example of the sort pattern-of-life analysis that underpins local intelligence collection and tactical patrolling in almost all peacekeeping, COIN, counter-terrorism, and stabilization operations. This is something that—with the notable exception of Jim Wallman’s BARWICK GREEN game—is almost completely absent from modern wargames on the topic , which focus instead on either local armed clashes or larger-scale operational and strategic issues. Is Mr. X acting suspiciously, or is he they simply eccentric? Is a meeting in the pub a benign collection of friends, or a plot in progress? Where can you best position an OP to observe civilian (and possible insurgent) activity without being spotted? Where should vehicle checkpoints be established? What sorts of information should you be collecting? Who might be hoarding precursors for IEDs and other weapons, and how would you know?

Griffithmap.jpg

The border village of Longreach.

The wargame is largely played by having the Security Force and Red Cell players allocate personnel to missions, schedule their various activities, and plot their locations or routes, with the umpire then adjudicating the outcomes. The book also contains some brief suggestions for resolving some activities on the tabletop. Supporting materials include a map of the village, background information on the villagers, a list of daily routine activities, as well as the assets available to the Security Forces and Red Cell.

The second COIN exercise is SUMMER IN ORANGELAND, which envisages possible terrorist activity by the “People’s Liberation Army” in the fictional town of Dodgem-on-Sea. Any resemblance here to IRA cells (or perhaps 1970s era leftist terrorists) operating in the mainland UK would not be coincidental. In this case the primary government actor is the local police force which, in addition to dealing with a possible terrorist cell, also has to cope with a busy schedule of other challenges: planning and security for the summer carnival, a football final, a concert, gold bullion shipments, and even a royal visit. The terrorists—some of whom have decidedly Irish surnames—must plan and execute a plot before they are discovered. In typical Paddy Griffith fashion, there are a few curveballs and eccentricities to keep the players on their toes.

The final exercise, GREEN HACKLE, is a series of live-action tactical vignettes to be carried out over three days by approximately 250 Sandhurst cadets operating in a mock-up village training area. The book contains a list of scripted events, plus some photographs.

Altogether, this slim volume provides fascinating insight into British counterinsurgency training in the 1970s and early 1980s. Moreover, the first two games highlight key challenges of tactical intelligence and analysis that remain highly relevant to contemporary COIN, counter-terrorism, peacekeeping, and stabilization operations. They are easily adapted or modified for classroom use, or could provide the inspiration for similar sorts of wargames set in other, rather different, political and cultural contexts.

This is NOT a Drill!

First – Applause for the Connections UK crowd – it was a very nice week in London, and I will be posting on that front shortly.

But in the meantime… Over the weekend, I read the pretty well done Politico piece on the “missing” hours on 9/11 during which the presidential retinue was being hop-scotched around the country on Air Force One. I was struck by the following extract. Early on, they head to Barkesdale AFB to get fuel and try and figure out what’s going on. As it happens, the 8th Air Force is in the middle of dialing-in to GLOBAL GUARDIAN – the annual STRATCOM exercise – leading to the following moment:

 “Lt. Gen. Tom Keck, commander, Barksdale Air Force Base, Shreveport, La.: I was the commander of the 8th Air Force. We were in the midst of this big annual exercise called GLOBAL GUARDIAN. They loaded all the bombers, put the submarines out to sea, put the ICBMs at nearly 100 percent. It was routine, you did it every year.
A captain tapped me on the shoulder and said, “Sir, we just had an aircraft hit the World Trade Center.” I started to correct him, saying, “When you have an exercise input you have to start by saying, ‘I have an exercise input.’ That way it doesn’t get confused with the real world.” Then he just pointed me to the TV screens in the command center. You could see smoke pouring out of the building. Like everyone else in aviation that day, I thought, “How in a clear-and-a-million day could someone hit the World Trade Center?”

I had forgotten that the GLOBAL GUARDIAN exercise that year was actually investigated by the 9/11 Commission for whether it had impact the military response to the attacks detrimentally. The conclusion was ultimately that the heightened exercise readiness may have actually helped response. Go read the 2001 GLOBAL GUARDIAN scenarios – no spoilers in this post – and think about 2001 vs. 2017.

Of course 9/11 is the salient event of the contemporary, western national security narrative – but as practitioners we have to think about the same kinds of things in our daily lives. To whit, a conversation I had with a colleague and collaborator a couple of years ago:

“Me: hey [colleague], what’s up?

Colleague: Hey, did you send me some scenario materials, like draft injects?

Me: Oh, yeah, I did, a few hours ago, why?

Colleague: are they fake versions of acquisitions documents for an ISR program, with, uh, FAKE SAP markings on them?

Me: Yeah! …oh

Colleague: So, you didn’t write: “Exercise Purposes Only” on them, and now there are some guys taking all the computer hardware out of my office, and I have to see my security officer in an hour…

Me: [pause] Sorry.”

Mistakes happen in both directions. Remember to check-in!

Simulation & Gaming, October 2016

sgbar

The latest issue of Simulation & Gaming 47, 5 (October 2016) is now available. The issue is devoted to the topic of “service design games.”

Editorial
Symposium Articles
Case Example
Articles
News & Notes

IMPACT: A Foresight Game is now on Kickstarter

903b86be3c0983d8349c9e8ce93f7f34_original.JPG

IMPACT: A Foresight Game, which we recently discussed here at PAXsims, has now launched a Kickstarter campaign.

Connections UK 2016 AAR

connections-uk

The main conference component of the 2016 Connections UK professional wargaming conference started on Wednesday, with a record 160 participants registered—the largest Connections UK meeting to date, and the second largest Connections ever.

In the opening session Graham Longley-Brown highlighted the renewed interest in wargaming across the UK military and in the public realm. Among the many things he pointed to were the Sandhurst Kriegsspiel, the various Connections conferences, and the recent publication Zones of Control, edited by Pat Harrigan and Matthew Kirschenbaum. Having done so, however, he raised the question of how best to institutionalize wargaming and build further capacity—the question at the heart of this year’s conference. Phil Sabin echoed these points, welcomed participants on behalf of King’s College London and the Department of War Studies.

14242368_10102698829815347_8885893567444341423_o.jpg

The first panel, chaired by ED McGrady (CNA), explored the psychology of successful wargames. Graham Longley-Brown (LBS Consultancy) started off by addressing high-engagement wargames. Wargaming, he argued, was about people, the decisions they take, and the resulting story-living experience. He went on to identify the various components that can contribute to this, including the need to keep players in a game flow that avoids both boredom and excess frustration, and inside the “bubble” of narrative engagement and suspension of disbelief.

Next, Nick Hare (Aleph Insights) offered an excellent presentation on creativity in ludic decision-making. He noted that arriving at a solution involves a mix of both creative and critical (System I and System II) thinking— a cognitive process that we may not be well aware of when it is underway. “Analysis paralysis” tends to occur in the transition between initial assessment and deeper understanding. He noted that much of the literature on analysis focuses on shortcomings in critical thinking (such as failure to fully use information because of cognitive bias) rather than weaknesses in creative thinking (“failures of imagination”). He identified a number of games that encourage creative thinking (Pictionary, Dixit, Codenames, Diplomacy, A Distant Plain, D&D, Megagames). He suggested that (game system) complexity and legibility (maps, clear game concepts, theme) contribute to ludic encouragement of creative thinking.

Finally, the ever-cunning Stephen Downes-Martin (US NWC) talked about wargaming to deceive the sponsor. Wargame sponsors, he argued, often have an interest in deceptive wargames that validate preconceived notions. Wargamers therefore need to learn how to deceive sponsors as a way of inoculating wargaming against sponsor manipulation. This can be done at the game development stage to directly deceive sponsors, or deceiving players during game play in order to deceive the sponsor and other stakeholders. Stress, over-confidence, and career pressures can be exploited to directly manipulate sponsors. Loopholes can be designed into a game that players will exploit in order to generate the desired findings. Having identified how wargames can be deceptive, Stephen then moved on to ways of dealing with all this in order to safeguard game quality: game peer review; monitoring player stress; engaging the sponsor; punishing but learning from cheating; matching game flow to level of war being gamed; identifying and monitoring ambiguous game rules and procedures; and rotating game roles.

I asked about the dangers of too much creative thinking, whereby powerful narrative engagement leads players to forgo critical thinking about real-world feasibility. (This, for example, was a serious problem in my view with the Jane McGonical/World Bank EVOKE social entrepreneurship game.) Nick responded by stressing that the game model needs to root players in the plausible. Graham noted that he more often encounters the challenge of too little engagement rather than too much, but that the game controller should play a role in nudging players back to game objectives. Another questioner asked about the perennial issue of using dice in games, and the discomfort many military participants have with this. All panel members defended the integration of uncertainty through stochastic process. A member of the audience also asked about how one might deal with a situation where a sponsors insists on a scenario that is unwinnable (but isn’t intended to be).

After a coffee break, we were given a brief one minute overview of each of the many games that would be available for play during the later demonstration periods.

After lunch, we returned to discuss non-combat wargames, with the session chaired by Anja van der Hulst. Russell King started us off with an NHS emergency planning exercise. This began with a video announcing a plane crash onto the M-1 near Kegworth, Leicestershire. We were then presented with a series of challenges. Russell used this to discuss the approach he uses, which he sees as facilitated peer review rather than a game with a game control adjudicating outcomes. He noted that it can be hard to get senior people together to participate: they are busy, they are experienced, and they may view a major disaster as a remote possibility compared to day-to-day challenges. However, it is important and necessary that they prepare—indeed, in the UK, this is a professional and legal requirement. If the simulation is snappy, fun, respectful of professional expertise, and tailored to organizational needs it is easier to secure engagement.

IMG_3320.JPG

Russell King talks us through a brief emergency planning exercise.

Mark Stoop discussed scenario-based policy discussion. Scenarios4Summits designs scenarios for senior (ministerial, head of government) audiences. This presents particular problems: senior leaders and their staff don’t like to be surprised, there are multiple political sensitivities, and the scenario needs to be very up-to-date.

They work with professional actors and high-quality video, using realistic scenarios, detailed scripts, and considerable fact-checking. The scenarios are intended to act as a prompt to discussion. He stressed the importance of audience acceptance.

Michael Lee discussed one approach to wargaming wide area persistent messaging in information operations. They did this by identifying technologies and approaches, grouping them into categories, and then developing comprehensive sets. The cells were given a chance to develop hypothetical capability sets, which were then tested against a scenario. The sets were then refined, and tested against new scenarios. Actual game play involved technology and platform cards: each Blue cell was allowed to select from a pool of these. The local population groups were profiled for literacy, ICT access, population density, and so forth. Set refinement and multiple scenarios encouraged innovative combinations.

IMG_332.jpg

WAPM cards on display during the games fair.

Much of the subsequent discussion addressed similarities and differences between the approaches presented by the speakers, as well elements of commonality (or differences) with traditional wargaming. One question regarding summit scenarios involved the tension between making scenarios simple enough for senior leaders yet complex enough for staff with subject matter expertise. Mark noted that while the vignettes were really designed for senior (ministerial) participants, the scenarios nonetheless provided an opportunity for technical staff to more fully inform seniors on the intricacies of the challenge presented. It occurred to me that a related problem might be that of senior participants posturing: that is, behaving in ways intended to impress counterparts, rather than more cynically pursuing national or political interests.

The conference then moved to the first of two “games fair” sessions. In part years I’ve been running games, leaving little chance to see what other games were on display. This time I opted not to sign up for any so that I could tour the demonstrations. Unfortunately I didn’t manage to get pictures of them all.

IMG_3323.JPG

Camberley Kriegsspiel (Andrew Sharpe and Ivor Gardiner)

IMG_3332.JPG

Dilemma Analysis (Michael Young)

IMG_5089.JPG

TacCyber Wargame (Roke, photo by Tom Mouat)

IMG_5084.JPG

Urban Ops (Sebastien de Peyret, photo by Tom Mouat)

IMG_5094.JPG

Cyber wargame (Andreas Haggman, photo by Tom Mouat)

IMG_5098.JPG

Bellum Baltica matrix wargame (Johan Elg, photo by Tom Mouat)

IMG_3338.JPG

Teamwork BG (Swen Stoop)

IMG_3340.jpg

Wargame 2020 (Jim Wallman)

I’ll admit that I ended up volunteering myself to join Jim Wallman’s Wargame 2020 as part of the Red Team rebels holding Folkestone against a Blue attack. I think we did well in blunting the enemy’s offensive.

aerialphotofolkestoneflag

IMG_3342.JPG

Our MRL and artillery batteries, after they flattened Blue’s rear echelon support units and just before they were hammered by Blue counter-battery fire. Clearly the crews should have practised their shoot-and-scoot.

The final plenary session of the day was my own keynote address on Ten (Not Entirely Randomly-Generated) Reflections on the Social Science of Wargaming. I won’t summarize what I had to say, since you can find the slides here, and the video below.

slide01

There was a second opportunity to play games after dinner. I ended up running a hastily-organized game of AFTERSHOCK for a couple of officers from UK Standing Joint Force Headquarters and a few conference attendees.

Thursday morning started of with a panel on computer simulation and technology, chaired by Éric Jacobin. Dave Robson and Samantha Black (NSC) presented on technology in support of wargaming, focusing on professional military educational wargaming. Dave argued that technology allows a platform to include communication tools, extensive information, and a common operational picture. Samantha argued that computerized simulations had several strengths, in that they supported rapid calculation (useful for calculation-intensive aspects such as line-of-sight, etc.), avoided emotional bias, facilitated record-keeping and analysis, and are not always expensive. Validation of an educational simulation could be achieved through repeated use, and in many cases simulations only need to be “good enough” since they can be overridden by the White Cell. Digital simulations also can support humans-in-the loop to address aspects that aren’t easily modeled. Simulations are data-driven, which is both a down-side (data preparation) and a strength (supporting multiple use once the data is in place). Mistakes will be made, even by the computer and its model. These can often be treated as “fog of war,” although one can also roll back the simulation or override the digital simulation.

  • Hide the simulation from the users, if possible.
  • Choose an appropriate simulation.
  • Reduce complexity (and be prepared to use subject matter expertise).
  • Emulate the functionality of C3I

Next, Mark Gould (Dstl) discussed CAEn—Close Action Environment, a digital wargame intended primarily for analytical purpose. CAEn allows detailed terrain and unit/platform simulation for platoon and company actions in a simulated area up to 5x5km. He discussed human-in-the-loop digital wargaming, and the issue of corrective human intervention in game outcomes (versus trying to develop sophisticated AI). Analytically, the CAEn team will follow through the critical elements of the wargame, and replicate this with a focus on critical junctures. This allows them to assess how plausible these are. He concluded by identify key strengths of CAEn:

  • Honest (if low-res) graphics.
  • Unique blend of rigour and creativity.
  • Relatively quick or cheap (for what it does).
IMG_3330.JPG

CAEn in action during the games fair.

He suggested that CAEn should not be used if you are short on time and money, if large scales (terrain and/or forces) are required, or if large amounts of quantitative data is required.

1437498535547.jpeg

The next plenary session featured Stacie Pettyjohn (RAND) speaking on strategic gaming—why it is languished, and how it can be improved. She started by defining strategic gaming, and contrasting this with operational-tactical gaming. Operational games vary in terms of both structure (low/high) and transparency (low/high). However, strategic games tend to be loosely structured, and most are (low structure/low transparency) seminar games. Indeed, she suggested, most seminar games are BOGSATs in which players “admire the problem” and many players arrive with their own “pet rock” talking points, resulting in few innovative ideas. She suggested that manual boardgames are often more useful (when a problem is well understood), since they provide a structure that focuses player attention. However boardgames do risk being dismissed by some audiences as juvenile. There was also a risk that the game rules might distort or misrepresent key strategic dynamics or interests.

As an example of a strategic boardgame Stacie pointed to the Countering-ISIL game she is developing at RAND, based on ideas that emerged from the rapid game design session at last year’s MORS wargaming meeting. Having cofacilitated that session with Brian Train, Robert Leonhard and others I was particularly happy to see how those initial seeds of idea had developed!

IMG_3345.JPG

Stacie Pettyjohn discusses strategic wargaming.

The next panel, chaired by Colin Marston (Dstl) looked at successful real-world wargames. Jeff Appleget (NPGS) offered a presentation, coauthored by Rob Burks, on wargaming at the Naval Postgraduate School. He offered several examples: wargaming hybrid warfare in the Arctic (as part of a course on the topic); countering ISIL’s foreign fighter recruitment; exploring the distributed lethality operating concept; defence support to civil authorities (using AFTERSHOCK as part of game design process); wargaming the US Army Pacific CONPLAN (which required considerable security vetting); undersea communication and other technologies; and mass atrocity response. On occasion they’ve done wargames for foreign sponsors, including Norway (hybrid threats) and the Royal Canadian Navy (non-lethal weapons for boarding parties), and also delivered wargaming courses in the US and abroad.

Roger Mason addressed wargaming in the intelligence community. One challenge, he noted, is that once one designs a game it is rarely really clear what is done with it after you hand off the game. National intelligence agencies design their own games, use commercial games, commission games, and monitor other people’s games. Publicly-funded think-tanks (like RAND or CNA) are wargaming too, as are commercial intelligence providers (such as Wikistrat and Stratfor) and academic institutions. Even the Vatican uses scenario planning.

Ivanka Barzaska (KCL) discussed understanding how missile defence affects nuclear deterrence and stability through gaming. She argued that Cold War era modelling of nuclear exchanges are outdated, since contemporary use would like be more limited and constrained. Her research proposes three strategic gaming events, using NATO and (ideally) Russian participants. The games are not intended to test hypotheses, but rather would form part of a mixed-method exploration of highly uncertain issues, acting as a semi-structured interview of sorts. The games could also serve as an informal Track 2 or Track 1.5 process to help educate official about the impact of BMD.

In the subsequent Q&A it was questioned whether wargames were actually having any effect on policy, or whether they had simply become trendy tick-the-box processes.

After lunch, Stephen Downes Martin (NWC) chaired a session on wargaming innovations. Paul Vebber (Naval Undersea Warfare Center) discussed wargaming for innovation. He identified several types of innovation, and emphasized that wargaming ought to form part of the broader cycle of research. Wargaming is particularly useful for issues that involve substantial human decision-making. He spoke of the value of gaming at the various stages of addressing a problem: problem framing, problem exploration, solution framing, synthesis (and game design), solution exploration (playing the game).

Ellie Bartels (RAND) then explored resolving hidden information in open adjudication. Specifically she argued that open adjudication can be a key way of gaming emerging issues (especially when Control doesn’t know more about the issue than the players). However, many emerging issues (deterrence, hybrid warfare, cyber, terrorism, etc.) involve hidden information. Games may hide motivations, actions, capabilities, or effects. Masking actions, capabilities, and effects is more challenging than masking motives. She went on describe how to capture fog of war effects with limited map visibility, separate maps,  or flipped counters, as well as the use of cards to keep information private. She also addressed alternative models for hidden information such as face validity (whereby players challenge processes when results seem implausible) and zero-knowledge protocols (where results are validated by repeated partial observation). There were a couple of interesting suggestions from the audience on additional approaches during the Q&A.

IMG_3346.JPG

Ellie addresses the challenge of hidden information in open adjudication games.

Laura Hoffman (KCL) offered her perspective, as a former student, on wargaming at KCL. She was very enthusiastic about Phil Sabin’s conflict simulation course (and indeed disagreed with some of the cautionary notes I had sounded during my keynote address), although initially she felt out of her depth. She noted that she playtested and revised the game so many times that her friends grew sick of it, and was forced to play with family members over Skype. Laura found game design changed her perspective, offered an opportunity for a deep dive into her topic (the war in Darfur), and was a different learning approach. Subsequently she served as a teaching assistant in an undergraduate course, which provided a further opportunity to understand the educational value of gaming. She argued that megagaming and other wargaming was a better learning experience than model United Nations, and that there is genuine student interest in conflict simulation. One interesting question from the audience raised the potential difficulty of grading a game design.

The last part of the conference was taken up breakout sessions devoted to the challenge of how might we institutionalize wargaming and build wargaming capacity? Different working groups addressed this in the context of particular groups and audiences:

  • frontline (military, emergency service) personnel
  • defence science and technology
  • military education and training
  • historical analysis/conflict analysis
  • academia
  • industry
  • hobby gamers

I co-facilitated the academic breakout group with Phil Sabin (KCL). Much of the discussion focused on the need to establish professional respect and validation for educational wargaming. I was struck how the challenges differed across disciplines and university settings. In my own field of political science, there is widespread support for games-based learning, and games-based analysis wouldn’t be difficult sell either. Conversely, Phil has often commented on the resistance he sometimes encounters from fellow military historians. In civilian university, especially in North America, instructors have considerable latitude as to what and how they teach. In the military, however, there are often hierarchies and institutional requirements that deter or inhibit pedagogical innovation. We also addressed labelling (“wargaming” vs “conflict simulation”), student interest, whether there ought to be an academic wargaming journal (I’m dubious), and how best to support the sharing of ideas and experiences (in part, to encourage others  who might be interested in gaming but reluctant to try it to “come out of the closet”). There might also be scope for using new technology—for example, YouTube videos—to provide lessons in game design and facilitation.

The hobby game group noted that they were an “expendable and deniable” resource for more serious gaming, able to bring historical knowledge, a different subculture, and extyensive experience in game design and mechanics. Their contributions might include support to Red and Blue (and other teams), playtesting, and facilitation expertise.

The industry group addressed both the industry as a resource and gaming within the business sector. They noted the importance of having access to senior people. The concept of red teaming sometimes provides a hook which can lead on to wargaming. They also noted that interest among individuals doesn’t necessarily translate into continuing  institutional support.

The historical analysis group identified a lack of resources (time, money, people) as the major obstacle to institutionalizing wargaming and creating a safe space for experimentation. There was a general feeling that new officers were not necessarily well-prepared for the uncertainties of future war, and that getting at officers early in their careers and exposing them to quick (possibly recent-historical) wargames could be very useful. Those with experience in recent campaigns could be a valuable resource for this. There was also a need for outreach to, and sharing with, the broader wargame community.

The professional military education group also stressed the need for accessible games, and the value of having games recommended on military reading lists. There needed to be more outreach and publication in appropriate venues, and more collection of evidence as to the effectiveness of wargaming as a teaching and learning technique. Student feedback is essential.

The defence science and technology group raised the need for more cumulated knowledge, and the value of better understanding client needs. They pointed to a degree of disconnect between wargamers and technology. There was interest in, but debate about, an accredited professional society. Publication in peer-reviewed journal was also seen as valuable.

Finally, the “front-line” military and civil group  reported. They noted that wargames were occurring at various points in the planning cycle: risk assessment, planning, training, and exercise/response/capacity development. Wargaming is a planning tool that offers insight into how groups of people will respond to a challenge. Ivor Gardiner enthusiastically emphasized the value of wargaming as a cheap, highly effective training method that saves lives.

Phil Pournelle made some overall comments. He identified one key cross-cutting theme was that of credibility: of the method, with superiors, and among participants. He also emphasized the importance of games being somewhere where it was “safe to fail,” and he also emphasized the importance of “catching them young.” Phil noted the vast reservoir of knowledge in the hobby and industry. He challenged the wargaming community to better understand the analytical needs of the US DoD, UK MoD, and other clients, and thereby be better able to make the case for wargaming. Matt Caffrey offered three more observations: first, the professional wargaming community does want the support of hobby gamers and industry; second, that we needed to document and archive wargaming more carefully, lest it be lost to history; and finally that no matter what wargamers do, young men and women will die—but if we wargame well, perhaps fewer lives will be lost and fewer resources spent.

In summing up the conference, Phil Sabin said he thought it was the best yet. I have to agree. The presentations and discussions were excellent, the atmosphere was enjoyable and productive, and the networking opportunities were outstanding.

Slides and recordings will be have now been posted soon to the Connections UK website. You’ll also find an account of the conference at Bob Cordery’s blog, Wargaming Miscellany.

Kings1.jpg

Exploring US engagement in the Middle East: A crisis simulation

AtlanticCouncilreport.png

Some weeks ago I posted a report on the game methodology that Bilal Saab, John Watts and I developed for a crisis simulation held at the Atlantic Council in Washington DC. The report from that game has now been released.

With the current American election campaign and change in presidential administrations due in January 2017, the debate over appropriate levels of US engagement in an unstable Middle East assumes vital importance. Should a new administration be more proactive in seeking to address threats, resolve conflicts, support allies, and deter foes? Should the new US president be wary about excessive American involvement in complex overseas problems, and focus on other concerns and issues closer to home? What should be done directly by Washington, and what is best addressed by local actors, alliances, and coalitions of the willing? What is the appropriate balance between doing too little and trying to do too much?

Objectives and Design

We focused in our June 23, 2016 crisis simulation on how differing levels of US engagement might affect Washington’s ability to respond to a regional crisis and how differences in US posture and policy might affect the political-military calculations and behavior of key regional and international actors. Approximately fifty former and current officials, diplomats, academics, and journalists from several countries took part as players or observers.

ac2

ac3

The game, I thought, went well. They key findings outlined in the report include the following:

  • The fundamental policy question that needs to be addressed is primarily one of how the United States engages in the Middle East, rather than simply how much.
  • US policy levers can only influence, not control, events in the region.
  • Adversaries may not be fully deterred by a greater American military presence, but rather focus on other arenas where American power is more limited.
  • Gulf partners are reluctant to act without US support—but may do so if they feel they have been abandoned.
  • Gulf partners will seek to use US power as a proxy for their own.
  • Russia and China cannot act as substitutes for the United States in its role as regional crisis manager.
  • Europe and other US coalition members cannot provide an alternative for US leadership.
  • Both US teams felt that their alternative policies gave them more freedom than the current administration’s approach, but in different ways.
  • Regional conflict and sectarian tensions provide fertile ground for crisis escalation.
  • Iranian behavior is deeply problematic and partly driven by a desire to be seen as having a legitimate role in the regional order.
  • While cyberattacks may be an increasing part of the landscape of conflict and hybrid warfare, they pose real challenges in terms of US and allied response.

Although it isn’t addressed much in the report, I thought the game also highlighted the profound policy challenges and dilemmas associated with the Syrian civil war. In the PURPLE game, the US team significantly increased US engagement in Syria, responding to Syrian barrel bomb attacks by shooting down regime helicopters and eventually declaring a safe zone along the Turkish border. However neither of these policies worked out entirely as intended. The Russians continued—and in some cases expanded—air operations, while the shoot-down of Syrian helicopters led Iran to double down on its support for Damascus by directly deploying several thousand combat troops to Syria. In northern Syria the declaration of a safe zone led the Syrian Kurdish YPG to declare sovereignty over Kurdish-controlled areas, an action which seemed likely to bring about a Turkish military intervention—thereby raising the spectre of a coalition member attacking the supposed coalition-protected safe zone.

Connections UK 2016: Civil War in Binni

Connections UK.jpeg

The serious conference component of the Connections UK 2016 professional wargaming conference starts tomorrow, with two days of plenary presentations, working groups, and game demonstrations. Today, however, many of the participants gathered to play the Civil War in Binni megagame, designed by Jim Wallman.

BinniBanner.jpg

The fictional country of Binni featured a dictatorial President, sectarian divisions, multiple rebel groups, terrorism, refugees, humanitarian crisis, conniving neighbours, a concerned and often divided  international community, covert intervention, and UN agencies. New elections were held, but under the regime’s old electoral laws which strongly favoured the incumbent. When the President was reelected in a dubious ballot, Christian militias seized the capital. The President was killed, and the country seemed poised to collapse deeper into chaos.

I served with Stephen Downes-Martin as the UN Control team, and my after-action review slides can be found here (although they will likely make little sense to anyone who wasn’t there). The photos below are courtesy of Tom Mouat. Everyone seemed to enjoy themselves a great deal, and some serious points were also made about wargame design and execution.

IMG_4990-2.JPG

Initial conditions.

vlcsnap-00005.png

Moves underway at the map table.

IMG_4997.JPG

Global News Network at work.

IMG_5015.JPG

News headlines.

IMG_5040.JPG

Influence-peddling.

IMG_5006.JPG

UN Security Council meeting.

IMG_5003.JPG

The UN SRSG checks the map table as various negotiations continue.

IMG_5059.JPG

Christian militias seize the capital.

Gaming foreign policy (at the FCO)

forgncom-eur01
Today I spent an enjoyable day at the UK Foreign and Commonwealth Office, running an abbreviated version of the ISIS Crisis matrix game and discussing how gaming approaches can contribute to policy analysis in a foreign ministry setting. About a dozen people participated, most of them FCO research analysts.

My notes for the session can be found here. I started by highlighting the ways in which serious games could be used to explore issues of crisis, conflict, foreign policy, and related issues. We then launched into ISIS Crisis. This proceeded rather more slowly than usual, partly because foreign ministry staff tend to more spend a little more time verbally framing their actions, but mainly because we devoted considerable time to discussing strengths, weakness, and possible variations in the game methodology as we went along.

ISIS started things off by organizing a successful terrorist attack against a cruise liner in Greece, which boosted their morale and reputation. Alleged civilian casualties from an unsuccessful US drone strike against a senior ISIS leader were also used in jihadist propaganda.

In Iraq, the Kurds tried to take advantage of Baghdad’s focus on Mosul to consolidate their control of the city of Kirkuk. The Iraqi government responded by bolstering its own forces in the Kirkuk area. This resulted in a tense stand-off that was eventually defused through Iranian mediation. The Kurds then sought to use the incident to reopen negotiations on a range of revenue-sharing and constitutional issues, but the central government showed little interest.

The incident also put the long-planned campaign to recapture Mosul behind schedule. When Iraqi forces sought to regain the initiative by pressing forward in a poorly-coordinated fashion they suffered heavy casualties from determined ISIS resistance.

Meanwhile, the failure of Baghdad to address Sunni grievances, coupled with a growing Iranian role and the continued presence of Shiite militias in Sunni areas only increased Sunni alienation. The Sunni opposition player was increasingly disinclined to cooperate with the central government in counter-ISIS actions, and successfully sought funding from Saudi Arabia (which only worsened relations between Baghdad and Riyadh).

We finished up with a discussion of the game and its methodology. I made the point that games need to be designed for their intended purpose, that they could be useful in generating questions and issues for further examination, and that they usually worked best when they formed part of a broader process of policy analysis.

I hope the participants found the day as useful as I did. Particular thanks are due to Owen ElliottHead of the FCO’s Africa Research Group, who arranged the session.

New Chair of Wargaming Department at US NWC

nwc-logo-colorThe Provost of the US Naval War College has just announced the new Chair of the Wargaming Department within the Center for Naval Warfare Studies. This position was announced some months ago for open competition.

The winner of that competition is CAPTAIN Richard A. Labranche, USN. His full biography can be seen on the Naval War College faculty page.

h/t Stephen Downes-Martin 

Clashing in the classroom

the-clash-london-calling-xpost-rmusicwallpapers.jpg

Yeah I’m working in Harrisburg
Working hard in Petersburg (working for the clampdown, working for the clampdown)
Ha! Gitalong! Gitalong!
Beggin’ to be melted down

What do early 1980s  The Clash punk lyrics  have to do with serious games? Nothing at all, other than I’m writing this in the airport in Harrisburg, PA. Ever since we flew over Three Mile Island on the incoming flight I have had the song Working for the Clampdown stuck in my head.

IMG_3208.jpg

The morning sun bathes scenic Three Mile Island in a faintly radioactive light. It would have been nice to linger, but we had to SCRAM

Rather than working for the clampdown, however, I spent Saturday in nearby Carlisle taking part in a US Army War College panel discussion on wargaming in the classroom. The primary focus, not surprisingly, was on professional military education (PME). I also ran two of the several demonstration games featured at the event. Over two dozen people participated.

IMG_3196.jpg

Setting up the event.

The first panelist was Peter Perla (CNA), who made the general case for the value of wargaming. Key to what Peter had to say was the emphasis he placed on process rather than outcome: while the outcome of a game is not irrelevant, it is what goes in the mind of the player that is of key importance. I couldn’t agree more, and it points to why building an engaging game narrative is such an important part of effective wargaming.

I presented next, identifying a number of serious game “worst practices” that was partially inspired by a 2004 US Naval War College and CNA study on (analytic) wargaming pathologies. Specifically, I addressed the dangers of:

  • Gaming for gaming’s sake. Problems soon arise when instructors devote inadequate attention to how and why they are using a game, and how this might support course learning objectives. Gaming enthusiasm is no substitute for effective teaching. One also needs to be clear about the opportunity costs of using scarce contact hours for games that might be used for other activities. It should be noted that learning is not the only reason to game in a classroom: a game can also serve break the ice in a  new group, promote networking, and to help assess student abilities.
  • Assuming that games teach themselves. How will you know that players are learning the appropriate lessons? Properly prebriefing and debriefing games is essential. I also pointed to the danger of gamer mode, whereby players exploit game rules (such as unflankable map edges, zone of control rules, or using disposable units as “speed bumps”) or computer AI to secure victories in ways that would not work in real life or otherwise be inappropriate.
  • Not listening to participants. If instructors want to know what students are taking away from the game, and whether the experience was worthwhile, be sure to ask them. While self-assessment is not always a reliable indicator of actual learning, eliciting feedback will help to identify problems and shortcomings.
  • All game, no gaming. One can take this last point further, and elicit feedback on the game system itself and encourage students to suggest possible game modifications. Indeed, encouraging students to think as game designers, and not just as game players, appears to improve learning outcomes. In the context of professional military eduaction is also serves to enhance critical knowledge of wargaming, thus leaving participants better-equipped to assess the value of future games, derive the greatest value from a game, or even help to design or facilitate one.

James Sterrett ( Deputy Chief, Simulations Division,  Digital Leader Development Center, US Army Command and General Staff College) then offered some thoughts based on his experience of supporting and encouraging classroom wargaming at CGSC. He emphasized the practical considerations that affect what sort of game will be useful and appropriate, and the need to design educational wargames and scenarios so that they are fit for educational purpose. One-size-fits-all solutions, he suggested, rarely work well. He also highlighted that not all instructors are the same, and that classroom wargaming approaches need to take account of a teacher’s strengths and weaknesses too.

Finally, James Lacey (Marine Corps War College) focused most of his comments on the obstacles to the use of wargames to teach strategy in professional military education. He suggested that PME institutions tend to be too rigid and structured, and discourage instructors from experiment. He also mentioned some of the criticism his critique, has received and—in typical fashion–pushed back hard. In subsequent discussion Peter noted that despite the current DoD push for more and better wargaming (inspired by the DEPSECDEF memo of February 2015), this has largely focused on analytical gaming with no clear direction from the top to more fully and effectively integrate gaming into PME.

IMG_3197.jpg

Playing AFTERSHOCK. What is the UN so happy about?

After a Q&A period, the rest of the day was devoted to demonstration games. I ran a session of AFTERSHOCK: A Humanitarian Crisis Game. The foreign military contengents of the joint HADR-Task Force did an exceptional job of quickly repairing the damaged airport and port. A critical moment in the game came when the NGO team unintentionally delayed implementation of a water infrastructure project in District Five, only to see the area stricken soon after with a dangerous outbreak of cholera. The United Nations—possibly having learnt from its real-life experience in Haiti—already had a cholera response programme readied, and was able to both halt epidemic and provide improved water facilities to prevent future outbreaks.

IMG_3199.jpg

The United Nations responds to the cholera emergency.

The players were headed for a well-earned victory when we had to call an end to the game for reasons of time (not to mention my need to eat before the cafe at the US Army Heritage and Education Center closed for the day).

After a quick sandwich, I also ran several turns of the ISIS Crisis matrix game. The Iraqi government sought to build on its successes earlier this year in Fallujah and Ramadi by launching a bold, Patton-esque thrust along the Euphrates Valley towards the border town of al-Qa’im—hoping thereby the sever an important ISIS line of communication and further isolate Mosul. The attack, however, was hastily organized and went disastrously wrong. Local Sunni tribes were angered that the campaign had been supported by Shiite militias rather than coordinated with them, while ISIS benefitted from both a  morale boost and the capture of significant military equipment. ISIS also plotted terrorist attacks in Iraq and abroad—one of which, aimed against NATO facilities in Brussels, was foiled in the nick of time by an alert Belgian police officer.

The failure of the operation also aggravated growing tensions between Baghdad’s Iranian and US allies. Prime Minister Haider al-Abadi used the defeat to justify a large-scale purge and reform of the Iraqi armed forces in an attempt to build a more competent and professional military. In doing so, however, he relied heavily on Iranian advice, advisors, and money—causing the US to temporarily withdraw some of its own advisors in protest. Washington also signalled its dissatisfaction with the Iraqi central government by providing the Kurdish Regional Government with heavier weapons. That, of course, only further annoyed Baghdad, which briefly closed its airspace to US aircraft. Meanwhile, Iraqi Sunni leaders were dismayed both by growing Iranian influence and by the government’s failure to deliver on its promise of a new law that would see more petroleum revenues invested in Sunni areas. Scandals and acrimony dominated the political process, and national unity seemed more distant than ever. Amidst all this, ISIS capitalized on the disarray by rebuilding its network of supporters in government-controlled areas of Anbar province.

All in all, I very heard some thoughtful commentary at the event, made new contacts, played some games, and otherwise very much enjoyed myself. I’m very grateful to MAJ Dennis Davis and his colleagues at the US AWC Center for Strategic Leadership for having me down.

UPDATE: You’ll also find a report on the event by John Carter McKnight (Harrisburg University of Science and Technology) at his blog Aporia.

Oh, and as for that song…

 

 

IMPACT: A Foresight Game

2016-08-07-1470548680-2872122-IMG_08552.JPG

Policy Horizons Canada  (“an organization within the federal public service that conducts strategic foresight on cross-cutting issues that informs public servants today about the possible public policy implications over the next 10-15 years.”) is working with Idea Couture to develop IMPACT: A Foresight Game. According to a piece in the Huffington Post by Robert Bolton of Idea Couture:

The objective of foresight is not to predict the future, but to prepare for many futures. That’s what makes a game such an ideal medium for learning foresight thinking. In more linear formats, such as a films or novels, the narrative is static. But board games are dynamic and can generate a new scenario every time you play.

In Impact, each player takes on the role of a character with a unique job from the future and a set of preferred future conditions that will make their job secure and prosperous. Players compete to achieve their character’s preferred future world by playing Impact cards, which trigger events influencing the various domains of society. The roleplaying aspect of the game also encourages empathy as players embody characters and think from different perspectives about what it means to create a preferable future.

Like the discipline of foresight in general, Impact brings to light the rates and trajectories of change, and the potential second order effects and disruptions that might occur. For organizations, maintaining a competitive intelligence practice that formally tracks rates of social, technological, environmental, economic, and political change is part of ensuring your own resilience.

The content of the game is based on Policy Horizons Canada’s report, MetaScan 3, a foresight study that explores how disruptive technologies may shape the economy and society. So, when you’re playing Impact, you’re engaging with real life technological events that are unfolding today (such as a recent scientific discovery or the formation of a new kind of tech startup) and imagining the possible ways these events could influence society tomorrow.

Players learn about developments in fields like nanotechnology, artificial intelligence, Internet of Things, biotechnology, and robotics; and are prompted to consider their industry, environment, and policy implications.

Robert tells me they hope to launch a Kickstarter for the game in the fall. If so we’ll announce it here—and we hope to be offer a PAXsims playtest review of the game too.

2016-08-07-1470550224-9008080-IMG_4443.JPG

Simulation and gaming miscellany, 24 August 2016

wordle240816.png

PAXsims is pleased to present some recent items on conflict simulation and serious (and not-so-serious) games that may be of interest to our readers.

I’ll be off soon to participate in the day-long public event on wargaming in the classroom at the US Army War College on August 27, followed by running a matrix game for the UK Foreign Office and then participating in the always-excellent Connections UK professional wargaming conference at King’s College London on 6-8 September. Because of that, the next “simulations miscellany” update may not be until mid-September.

PAXsims

olalertbanner.jpg

A forthcoming issue of the British Journal of Educational Technology will contain an article by Matt Hardy and Sally Totman on “Teaching an old game new tricks: Long-term feedback on a re-designed online role play.”

Despite an extensive history of use in teaching Political Science subjects, long-term scholarly studies of online role plays are uncommon. This paper redresses that balance by presenting five years of data on the Middle East Politics Simulation. This online role play has been run since the 1990s and underwent significant technical upgrade in 2013–14. The data presented here covers student feedback to this upgrade process and the factors that can influence their response. Key indications are that students tend to recognise when something is fit (or not) for its purpose and will forgo attractive and well-appointed online environments if the underlying learning exercise is valued. However, there are limits to this minimalism and whilst designers do not need to replicate every Internet trend, attention needs to be paid to broader changes in technology, such as access platform and changing avenues of political communication. The study demonstrates that long-term monitoring of online role play exercises is important to allow informed changes to be implemented and their impacts properly assessed.

PAXsims

Public health epidemiologist (Preparedness) Dr Henning Liljeqvist has used AFTERSHOCK: A Humanitarian Crisis Game to simulate working in humanitarian settings with students enrolled in the University of New South Wales/World Health Organization course on Communicable Diseases in Humanitarian Settings at the UNSW School of Public Health and Community Medicine. The video below shows how he ran the game in the classroom.

A few of the rules were modified to make things run more smoothly (for example, regarding logistics infrastructures upgrades). Of particular note, however, was the skillful way he debriefed the game, linking game processes to real-life humanitarian experiences, highlighting where the game was more or less realistic, and challenging students to think of their own modifications via use of the blank cards provided with AFTERSHOCK.

PAXsims

At War is Boring, Robert Beckhusen argues that “U.S. Army Has Too Many Video Games.”

There’s just a few problems. Some of the Army’s virtual simulators sit collecting dust, and one of them is more expensive and less effective than live training. At one base, soldiers preferred to play mouse-and-keyboard games over a more “realistic” virtual room.

He is absolutely right in noting that many purpose-designed digital games and simulators get inadequate use in the military or prove unfit for their intended purpose (often for practical reasons of time, accessibility, flexibility, and so forth). There are also many good commercial off-the-shelf games that can be usefully integrated into training.

However, one has to be careful about arguing that “It’s unnecessary to strap soldiers into an immobile vehicle and make them scan a wrap-around screen if they can accomplish the same basic tasks with a mouse and keyboard.” A mouse and keyboard can be very unlike a real combat environment, and there is significant risk that soldiers will learn the skills and develop the muscle-memory to win the game—but not develop skills that translate well into actual combat environments. In many commercial games, for example, cover and visibility to not function much like the real thing, while AI opponents may act in unrealistic ways.

PAXsims

keep-calm-and-just-keep-ranting.pngThe use of commercial/hobby wargames in professional military education has much to commend it, as James Lacey (among others) has convincingly argued. On the other hand, professional wargaming addresses a broad array of analytical and educational purposes, not all of which are adequately served by off-the-shelf games. Moreover, while the hobby game experience can inform and contribute to professional game design and execution, it can be a bit of a blinder too if you aren’t careful.

I say all that because this current thread at BoardGameGeek on “revitalizing (manual) wargaming in the military” illustrates the tension well. In particular, read Brant Guillory’s truly epic rant on the topic. He’s right, too!

 

Tom Mouat joins PAXsims as associate editor

4497985550_268449d699_o.jpg

I am very pleased to announce that Tom Mouat will be joining PAXsims as one of our associate editors.

Tom Mouat MBE is the Directing Staff Officer for Simulation and Modelling at the Defence Academy of the UK. He holds an MSc in Defence Modelling and Simulation and a Post-Graduate Certificate in Education. In his 37 years’ service in the British Army he has served worldwide, including operational tours in Bosnia and Iraq, designed and run training events from Battlegroup to Corps level and spent five years in Defence Procurement as a Requirements Manager in the acquisition of Simulation Systems. He holds commendations from the Ministry of Defence Chief Scientific Officer and the Head of Defence Procurement, and has been a contributing author to a number of books on wargaming. He is also currently on the management board of the Connections UK professional wargaming conference.

Tom has also been a frequent contributor to PAXsims, on topics including the recent Connection (US) wargaming conference, the development of migrant cards, matrix games, and his creation of Sandhurst Kriegsspiel. He is also well known for his MapSymbs TrueType military map marking fonts, as well as an extensive online repository of matrix game and other materials—all of which can be found at his website

%d bloggers like this: