PAXsims thoughts on Ducharme on COA analysis gaming

Earlier this week, Devin and ! both listened to a great talk by Naval War College’s Dr. Doug Ducharme for the MORS Wargaming Community of Practice on best practices for wargaming in support of Course of Action (COA) analysis. This is second of three posts: the first summarized Doug’s talk, and the third will have some thoughts from Devin.

I found Doug’s presentation, as well as the discussion that followed his talk, to be very insightful and thought provoking. It was particularly useful that Doug offered concrete guidance for game designers to improve their practice. The suggested best practices mirror well with my own experiences, and serves as a useful set of guidelines for new gamers. However, there were two points that I want to explore more: Doug’s distinction between educational and analytical gaming, and his distinction between free and rigid adjudication.

Doug argued that all games are experiential. What differentiates educational and analytical games is whether the goal of the game is to change the participants, or to change our base of knowledge. This definition is related, but somewhat different from what I’ve used in my own work. In past work, I’ve defined the types of game purposes using the 2×2 below:

As a result, I tend to think of analytical games as seeking to gain a better understanding of a problem, while education games seek to make people better able to solve similar problems in the future. I need to think more about how the distinction Doug points to fits into this model.

Doug’s definition also suggests to me a somewhat troubling fact: the majority of events that are run to improve US strategy today are actually focused on improving decision makers’ future capacity. On one hand, I think gaming can provide excellent educational value and professional development. On the other, I don’t want that to come at the expense of thinking though strategy and plans to make them as robust as possible. I left Doug’s talk hoping that the comment made by another participant that “all games are both educational and analytical” is right!

The second point I want to tease out a bit more is Doug’s definition of adjudication methods. The talk, and the discussion after, clarified for me something that has been bothering me about how gamers talk about adjudication for a long time. A lot of discussion around gaming for analysis argues that the more rigid the system of game is, the more analytical it is. As a qualitative/mixed methods person, this rush to quantification always rubs me the wrong way, and I think this talk gave me a new way to frame why it bothers me.

I think that most of the time when gamers talk about free or rigid methods, we are actually conflating two different ideas. The first concept is a decision made by the game designer about how structured a technique to use to capture and analyze data about adjudication. Here, we can think about a spectrum that ranges from very loose adjudication, where rulings are made with few restrictions (and likely little documentation), to a very rigid system with detailed protocols for documentation and adjudication. The second concept deals with how specified of a model is used to generate the outcomes of player decision. Unless a game designer misses something in their research, this factor is limited by the state of knowledge on the issue being gamed. In some cases, we may have a very concrete and detailed theory of what should happen, but other times our models of cause and effect are less well developed, and we are left to deal with some pretty underspecified models.

While I do think that it is easier to establish structured adjudication rules when we have a well specified theory behind our adjudication, I don’t think the two concepts are necessarily the same. For example, one participant on the call referenced matrix gaming, which can provide a great deal of structure to game adjudication, even when causal models behind adjudication are fairly nebulous.

Treating the two design criteria like they are connected, or even the same, lets us get away with under-designing games when we are dealing with complicated poorly defined issues. For example, often “free” method games relay on expert judgment for adjudication, who make determinations about the effects of player action without providing much more justification then their credentials. However, by having less structure in the adjudication, game designers often give themselves a pass from looking carefully at what mental models experts are using to determine outcomes. As a result, we end up not ever really knowing how specified the model that drove the action of the game actually was, producing enviably nebulous and unsatisfying post-game analysis.

I’d argue that game designers should treat structured approaches to adjudication as critical to good game design. Then, even when the underlying models are underspecified, games can contribute to clarifying the models that do exist, and over time, to increasing model specificity. This is a concept that has been discussed with regard to wargaming emerging issues, but I think it needs to be applied much more broadly.

This is a topic that a lot of my recent work has focused on, and I’m due to speak to the MORS COP on the topic next month. I’m hoping to be able to share some of my thoughts here in advance of that presentation. As a result, even more than usual, I’d love folks’ feedback on these ideas!

Ducharme on COA analysis wargaming

Earlier this week, Devin and Ellie both listened to a great talk by Naval War College’s Dr. Doug Ducharme for the MORS Wargaming Community of Practice on best practices for wargaming in support of Course of Action (COA) analysis. This is the first of three posts: the first summarizes Doug’s talk, and the second and third provide some thoughts from Ellie and Devin.

Wargaming is the recommended technique in military doctrine for analyzing COAs during the joint operations planning process’s 4^th step. In actual practice, restrictions on staff time, skills, and commander involvement can all critically compromise the ability of the military to actually follow through on this. Doug states that he has seen an increase in the attention paid to these games in the last few years. However, he stated that there is not enough work done to document what gaming methods do and do not lead to successful COA analysis.

To set up his discussion of COA analysis gaming best practices, Doug started by defining gaming (using Peter Perla’s often-cited definition), and discussing how games differ from one another. He established that games can be defined along two axes: 1) whether the game has an educational or analytical purpose, and 2) whether the game examines concepts or capabilities. In this model, COA analysis is defined as being educational and conceptual.

Doug noted that with increased interest in COA analysis games, there has also been interest in incorporating other analytical techniques to support COA analysis. In particular, leveraging campaign analysis techniques has become more popular. Doug used his two-by-two to show why this can be an uncomfortable melding. In Doug’s model, campaign planning is an analytical technique, focused on capabilities. This places it in the opposing quadrant to the educational, concept-focused purposes of COA analysis gaming.

He then moved on to lay out five best practices for COA analysis gaming:

1. While doctrine suggests several methods for COA analysis, it does not offer strong guidance about how to select techniques. Given that games, by definition, are focused on decision making, Doug recommends defaulting to the critical events method which focus analysis on decisions and their potential impact.
2. Doug argued that the use of an active red cell is critical to COA wargaming. He specified that the cell’s objective should be to improve the COA, not to “win” the game, and that there should be a facilitator in the cell who can remind participants of this goal if they go off track. He also has found it helpful to keep the red cell to a roughly equal size with blue, and staff it with both intelligence officers and planners. These strategies create an active, but not overly competitive, red that can provide a strong critique of the COA.
3. Doug argued that rather than defaulting to a format of sequential moves with alternating action by red and blue, COA wargaming moves should ideally be made simultaneously to better mirror reality. If turns must be sequenced, game designers should determine who ought to have initiative based on the scenario in play, rather than defaulting to a blue first move.
4. Doug described adjudication options as a plane, with one axis running from move-step to running time, and the other axis from a free to a rigid method of adjudication. He argued that even when using relatively free methods of adjudication, having a structured process to evaluate player decisions is important. He also argued that most COA Analysis games have “open adjudication” with fairly move-step time, and fairly free adjudication methods. He also tied this point back to his earlier discussion of the difference between COA Analysis and campaign analyses, which have much more rigid adjudication rules.
5. Finally, Doug stressed the importance of providing clear criteria for evaluating COAs in advance. Doing so is critical to determining how to assess the COA’s strengths and weaknesses. This then naturally leads into the next step of JOPP, COA comparison, where pros and cons are discussed.

Doug ended his talk by arguing that if we are looking to add rigor to the COA analysis process, it would be better to focus on approaching games with an analytic mindset rather than trying to incorporate campaign planning tools that may not be the right fit. He provided a few examples the use of Analysis of Competing Hypothesis, and Analytic Hierarchy Process as tools to strengthen COA analysis games to show how post game analysis can also strengthen findings.

Boardgames and the indirect surveillance state

Big Brother is watching–for boardgamers buying suspicious game titles, that is.

A current thread on BoardGameGeek describes PayPal investigations triggered when gamers have used the service to buy games with certain words in the title. The PayPal query looks like this (with personal details redacted):

Dear [X],

As part of our security measures, we regularly screen activity in the PayPal system. During a recent screening, we noticed an issue regarding your account.

PayPal’s Compliance Department has reviewed your account and identified activity that may be in violation of United States regulations administered by the Department of the Treasury’s Office of Foreign Assets Control (OFAC).

PayPal is committed to complying with and meeting its global regulatory obligations. One obligation is to ensure that our customers, merchants, and partners are also in compliance with applicable laws and regulations, including those set forth by OFAC, in their use of PayPal.

To ensure that activity and transactions comply with current regulations, PayPal is requesting that you provide the following information via email to compliancetransactions@paypal.com:

1. Purpose of payment [XXXXXXXXXXX] attempted on [DATE] in the amount of $[XX] including a complete and detailed explanation of the goods or services you intended to purchase.
2. Explanation of [WORD] in the above transaction.

Please go to our Resolution Center to provide this information. To find the Resolution Center, log in to your account and click the Resolution Center subtab. Click Resolve under the Action column and follow the instructions.

If we don’t hear from you by [DATE], we will limit what you can do with your account until the issue is resolved.

We thank you for your prompt attention to this matter. We apologize for any
inconvenience.

Thus far, games that have triggered a PayPal investigation include:

• Cuba Libre!
• Drive on Damascus 
• Cuba: The Splendid Little War 
• Santiago de Cuba 
• Shining Path: The Struggle for Peru
• Tupamaro 
• Kandahar
• Target: Iran

You can guess which keywords are popping via their internal transaction-monitoring algorithms!

All of this is an example of the powerful indirect effect of Treasury Department rules—and the consequent fear of financial institutions that failure to adequately monitor transactions might not only violate US law but also leave them open to civil lawsuits.

h/t Rory Aylward and Brian Train

UPDATE (22/01/15):

Jon Compton at One Small Step Games recounts his own experience with this issue:

We recently released the game Shining Path. Every time someone purchases the game via PayPal, the transaction is held pending investigation. Takes about a day, and then the money shows up in the account. On the flip side, the map artist for the game lives in Europe. When we paid her via PayPal, the transaction was held up almost a week, and we had to write a lengthy explanation for why we were paying a foriegn national for something that contained “Shining Path” in the title.

First Battle (1979)

The simulation folks at the US Army Command and General Staff College, together with the Combined Arms Research Library, have been doing some searching in the archives and have come up with some treasures. One of the most recent finds is a set of manual tabletop wargaming rules, First Battle (1979):

The FIRST BATTLE simulation system is designed to exercise division commanders and staffs in the control and coordination of combined arms operations. The system is a flexible training tool that can ultimately be applied to any scenario, level of control, or mode of play FIRST BATTLE has undergone extensive field evaluation and incorporates a variety of supplemental and optional features for user adaptation. The three modes of play are conducive to a progressive training program so that once the Open Mode is mastered along with the Basic Rules, optional rules and supplements may be applied to the more sophisticated modes of play — the Closed Mode and the Command Post Exercise.

As James Sterrett noted when he passed on the link, there’s “not a lot of pax in this sim,” but readers will find it interesting nonetheless.

(h/t James Sterrett)