The following report was contributed by Major Tom Mouat, Directing Staff officer responsible for Modelling and Simulation at the Defence Academy of the UK.
I thought that PaxSims readers might like to hear how running a Matrix Game went down as part of the Cyber Operational Awareness Course at the Defence Academy of the UK.
The course “aims to contextualise Cyber within Defence by raising awareness of wider operational perspectives.” It lasts 3 ½ days and is Rank Ranged OR7-OF4 (Civilian D-C1), (Staff Sergeant to Colonel) which is somewhat unusual, but I think the course benefitted from the wide rank range due to the inclusion of senior NCOs, many of who were specialists in their own right and very interested in the subject. This certainly added to the classroom discussions.
The course is currently only available to UK MOD personnel and, while generally conducted at a low classification, takes place in a restricted area in case questions and discussion stray onto more classified topics.
On previous courses there was a “Planning Estimate” carried out, lasting about a day and featuring a 50 page background briefing about an entirely fictional scenario. This attracted a lot of criticism as it covered many things that had no cyber relevance and the background briefing was coma inducing because it was completely fictitious and was extremely difficult to follow (both for the students and the instructors).
It was decided to replace the Planning Estimate with a Cyber Matrix game. Matrix Games are extremely low overhead, free-form games, built around an evolving narrative through the means of a series of logical “Arguments” proposed by the participants to advance their position in the game. They are useful for rapid assessment of scenarios, as an alternate analytical method and for education. They are particularly suited to subjects where “effects” are more important than weapon system performance.
This represented something of a challenge as previously we had conducted Matrix Games with relatively few players and the course required the game to run with 35 participants. We elected to run the game with teams of about 4 players representing the different actors and use a couple of the more experienced participants to assist in the adjudication of the “Matrix Arguments”.
We conducted a trial with about 10 participants and a completely fictitious scenario which, while it ran adequately, was lacklustre. The scenario suffered from the same difficulty in identifying the different actors in the conflict and being able to present them in a realistically nuanced way. It was quite fun, but failed to bring out the learning points required, so required a significant re-think.
In the end the scenario was completely re-written to involve fictitious states as the primary actors, but within the real geo-political context of the Baltic States. This enabled us to reduce the briefing considerably and promote a better understanding of the scenario through the means of analogies. The teams were allocated as follows:
- The NATO sympathetic State.
- Their “cyber supporters”.
- The Main Protagonist State, massing troops on the border and fomenting rebellion in border cities.
- Their “cyber supporters”.
- The USA.
- An independent third party hacking group based in the Far East.
- The Press (Global Network News – GNN)
- Assessors (a couple of experienced students intended to help with adjudication).
In addition a map was constructed showing the military deployments, as well as things like refugee camps and significant infrastructure installations in order to provide player teams things to focus on.
The game was run in an afternoon in a large classroom with 2 projectors – one with the maps and deployments on it and one with a PowerPoint show with the “Press Headlines” from each turn. This served as a record of the results of the Matrix Arguments as well as a summary of international opinion at the end of each turn (each of which represented “about 2 weeks”).
We also decided to provide the actors at the start of the game with a number of “Capability Cards” representing certain cyber capabilities, to focus their minds but also to bring out specific learning points.
Team briefs included these Capability Cards, the much reduced background briefing (now limited to 4 pages and a map) and a one page briefing specific to the team (including their objectives).
In between turns the players were also asked general “cyber awareness” questions, such as “Who was responsible for the leak of 750,000 confidential US documents to Wikileaks in about 2010?” (Bradley (Chelsea) Manning). The correct answer would provide the team with an opportunity to re-roll the dice if there was chance of failure in an argument, and was intended to represent the “research” element of cyber operations – the better the research / reconnaissance phase, the higher the likelihood of success.
The game went well with good engagement all round. Student quantitative feedback scores were the highest for the Matrix Game session than any of the other sessions in the course and qualitative feedback comments were also good.
Points to note:
- Adding a “Press” team to record results and present “International Opinion” worked very well and was very useful as part of the game debrief (and was fun).
- The “Capability Cards” as specific pointers towards educational learning points worked well – they were used (or not used) appropriately, sold to other Teams, patched and made redundant, and generally added to the outcomes. This supports Canadian findings that player perceptions can be heavily influenced by the information provided and physical presentation. In an educational context this can be very useful.
- There was one criticism in the qualitative feedback that the person running the game “didn’t always listed to the full arguments”. This is an easy trap to fall into when running Matrix Games, in that the person running the game ends up acting as an “Umpire” and occasionally imposing his world view, rather than as “Facilitator” for the event (again supporting the Canadian findings above). The inclusion of the “Assessor” team was intended to mitigate against this, but they weren’t sufficiently briefed.
- While the game was very successful, the post-game briefing was merely adequate. More effort was needed to provide a structure to the debrief and identify the specific learning outcomes. The Capability Cards helped (especially with those teams who chose NOT to use them), but there is room to provide more obvious team objectives in order to bring out additional points.
Overall the inclusion of a Matrix Game in a cyber-focussed course was a success in promoting discussion and understanding of a subject resistant to conventional teaching methods, so we are likely to use it again in the future. I believe that Matrix Games are particularly suited to education about cyber operations as they avoid getting bogged down in unnecessary technical detail, while promoting insight and understanding.