The following article was written by Steven Wagner, Senior Lecturer in International Security, Brunel University London. An earlier version was presented to the International Studies Association annual meeting in March 2022.
In this paper, I share and discuss a simulation I run for my module on Intelligence History: Failure and Success, which is part of our MA in Intelligence and Security studies. My goal here is to share some evidence I have collected about the pedagogic value of the exercise, and the after-action report which students submit (and are assessed on). I will publish some drafts of the briefing papers along with this report.
As a DPhil student, my supervisor, Rob Johnson, ran a similar kriegspiel for staff college and he kindly lent me his briefing materials which helped inform my own simulation design. As a postdoc at McGill, Rex Brynen encouraged me to consider an exercise like this in the style of a matrix game. Rex regularly uses this medium for teaching, assessment, and research, and is a true champion of this community. He and Maj. Tom Mouat were very helpful in providing feedback during my work designing this simulation. Their Matrix Game Construction Kit, which my department kindly financed, was central to my game design as well. Our print service made up some nice maps and tokens for us too.
The scenario I run is drawn from my research. It is based on the 1929 riots in Palestine, also known as the “Buraq Revolt”. This was a major watershed for the Zionist-Palestinian conflict, as well as for British authorities who had ruled the country since the First World War. In August 1929, following nearly a yearlong campaign by both the Muslim and Jewish communities to arouse support for the defence of the Western or “wailing” wall in Jerusalem, communal violence broke out. Palestinians saw the moment as a popular revolt – the first effort since 1921 to resist British colonial policy which they feared would lead to their displacement by Jewish colonists. The Jewish community of Palestine saw this as a merciless attack by mobs against defenceless communities who had lived in Palestine long before the Zionist movement began to prepare for the rise of a Jewish state. Recent research has shown that it in fact strengthened ties between Zionist colonist institutions and those older non-Zionist Jewish communities.[i] The government was caught off-guard: most administrators and the chief of police were away on holiday. Investment in defence had declined for years, and the police were a tiny force of limited capability. The RAF – in charge of regional defence – were called in aid of the civil power to help quell the revolt. Understrength itself, it brought in troops from Egypt and Malta, as well as a battleship, cruiser and aircraft carrier. Within a week, 133 Jews had been killed, mainly by Palestinians, and 116 Palestinians were killed, mainly by British forces.
My simulation is set up with four or five teams: The Supreme Muslim Council, led by the grand mufti Amin al-Husayni, later notorious for his association with Nazi Germany. The Jewish community is represented by the dominant Labour Zionist party which controls nearly all national institutions – including secret militias. In some iterations of the game I have had a team for right wing “revisionist” Zionist party which played an antagonising role before the conflict although I have since removed them. Then there is the Air Officer Commanding, which controls the RAF, Transjordan frontier force, regional intelligence collection units, and if needed, reserves from abroad. Finally, and lastly in the order of play, we have the Palestine Government, a British colonial government with no legislative arm which controls the police, its criminal investigation department, the post office and censor, and other arms of civil administration.
The simulation begins on the day which violence spilled over. The game simulates a crisis of colonial security amid popular uprising and communal violence. What makes my design a bit different is that teams propose both an “action”, as in a usual matrix game, and a “query” for their intelligence sources and informants. The latter looks different for each team, as they each have organised these quite differently and for varying purposes. None the less, queries could consist of questions decisionmakers might ask of their experts based on data they would already plausibly have, or they can task them for collection and analysis. The goal is for them to explore the relationship between intelligence and decision making, policy, and in some instances, tactics.
This scenario presents each team with a set of trade-off dilemmas. If the Jewish community fights back, their secret arms stores are exposed to British and Palestinian scrutiny. If they complain to the British or embarrass them, they risk alienating their main patrons. If the Muslim community overtly supports rioters and revolutionaries, they risk imprisonment, loss of their official jobs, or worse. If they fail to at least secretly support revolution, then they risk popular support amongst Palestinians. If the RAF appears too weak, its decade of “Air Control” could come to an end. If it cracks down too hard, it could damage British policy interests. The list goes on.
Historians tend to avoid counterfactuals. As a researcher, it is exciting to use this exercise to explore the utility of simulation as a means of thought experiment. I am not trying to prove that X could have happened in real life, but rather, to highlight the conditions and variables which were critical to the unfolding of the simulation, as compared to real life. In other words, I am curious: was there a way to prevent revolt or the declaration of an emergency? Could Palestinians have found a way to exploit the moment and bring an end to the Zionist policy? What historical forces stood in the way of each?
In the latest iteration of this simulation, I skipped the emergency phase of the scenario and instead focusing the students on the phase between the restoration of civil authority and the arrival from London of a commission of inquiry – so it was less focused on the map, armed forces, arms stores, and violence, and more focused on subterfuge and diplomacy as each team tried to create favourable conditions and collect supportive witnesses who might testify in front of the commission. Many also tried suppressing evidence which harmed their interests. I also had to reconfigure the teams accordingly. I have attached both briefings.
This has also made for an interesting mode of assessment. Originally, I had intended to assess the gameplay as a means of evaluating student analytic capability. Ie, since matrix games require convincing argument, I could reward students for their success in that process. However, this is hard to document. The rules for assessment in both UK higher ed, and at Brunel, mean that this would be a tricky thing to assess fairly and accurately, unless I record the entire eight-hour simulation.
So, I instead asked students to keep a game diary based on a template I provided. Before the simulation they meet in teams and plan their strategy and opening moves. The diary template asks them each turn to evaluate how their team is making decisions, sticking to policy & strategy, as well as to interpret the scenario as it unfolds. They have only their experience and my master narrative to draw-upon for this. They are told to keep their diaries secret until after the game.
After the game, they sit together and compare notes. As a group they create a brief reflection on their achievements and failures, noting specifically where intelligence impacted decisions, security, etc. Then, they are asked to present individual “After-Action Reports”. They share and compare perspectives here. They must compare and find areas of agreement and contradiction in their diaries. I ask them to identify what details they chose to record and ignore compared to their teammates, and to explain it. For example, how could it be that two to four of you sat through the same events on the same team, but produced such different accounts or interpretations of the game. Can this be explained as an analytic bias? Is it a matter of perspective? Or did you disagree about things like priorities, strategy? Etc. I have also asked them to identify the key conditions for success and failure.
Perhaps it would be most beneficial to illustrate the pedagogical value of the AAR by showing some student responses. In their AAR’s, students show off both their historical literacy and analytic skills as they reflect upon their contributions, successes, failures, and compare the narrative to true events. Some students give a cursory answer which dwells on my questions as though it were a list. However, the best and brightest offer some of the most interesting and sober accounts of self-reflection I have seen. They really reveal during this exercise who I would hire as an analyst. I note here in particular that women and especially women from racialised minority groups have shared some of the most important insights I have come across. For example:
I think some differences in our approaches as events may have sometimes been because of our ethnic and religious backgrounds. I found that I focused a lot on society and faith – highlighting the literacy rate of Palestinians [at that time], Friday prayers for Muslims (which was echoed by a team member who is from the Middle East) and giving suggestions about the capabilities of the Jewish groups on a Saturday given that it was their holy day, whereas others who may not be so religious and come from European backgrounds did not consider these factors initially. I think this was a strength in our group as we were able to give different approaches which the rest of our team may not have thought of, and this broadened our range of approaches.
Reflections like this show the importance of broadening the way we educate analysts. It needs to go beyond structured analytic techniques and other social-scientific skills. Students and future analysts must understand people, and how the world works. They need to be able to collaborate like this and feel confident to share this kind of perspective. This is another example from last year:
I recall one episode in the latter part of the game where [X] had a mathematical argument on the effect of one outcome. My interpretation of the game was closer to that of an art… I believe this difference is an outcome of our own backgrounds. [X] being an expert in cyber-security and myself being a military officer. We had relatively few disagreements on how to interpret, rather, we had to spend more time to find consensus on which actions which align with our strategy.
That example illustrates how students with varied experience and coming from different disciplines learnt to communicate and work well together. Another intelligence professional from the armed forces compared her professional experience with wargaming with the classroom exercise. She led the RAF team and had expected the government to declare an emergency early. Yet the government team did everything they could to prevent this outcome – holding out for days. She felt she had misled her team by planning for an early emergency. She also offered interesting criticism, saying that she expected adjudication to eliminate implausible orders and that, after some were allowed, her plans had been disrupted. I agreed, and this is perhaps one of the core challenges in running a historically-based scenario. The drive for realism is a major constraint on students’ creativity, but it is also necessary for the pedagogical design.
However, my favourite reflections tend to dwell on explanations for failure. This one comes from a team representing the Muslim community’s leadership, often accused in real life of planning the revolt.
Our intelligence queries became progressively more difficult to decide upon because of the state of emergency declared by the government. It meant that we were censored and persistently spied upon by the other players[’ characters] which meant that it was a struggle to [successfully draw results from our queries]. We therefore had to rely heavily on our movements instead… we ended up making poor decisions which put [us] at risk.
Here, the student is describing how, during the last day of the emergency, they ordered direct support for revolutionaries and rioters and exposed the Palestinian leadership to danger. Secrecy was vital to their long-term political survival. The students panicked but also failed to separate their interests as players (who knew the game was ending) from those of their characters (who had to live the rest of their lives). Thereby, they were more prone to risky decisions. Another student commented on the same situation:
I had nothing better to suggest so although I didn’t think it was a good idea, I went along with the move. I think this can be reflective of the craft of intelligence in real life scenarios. We encountered a large failure to due our approach to events under pressure. We were not vocal enough when we felt the move was not suitable and I think this scenario highlights that disagreements should be highlighted when dealing with intelligence matters and we should not just go along with everyone else.
However, even though they made stupid decisions in-game, this level of honesty and self-awareness earned them good grades. Analytic staff who can demonstrate this quality, I am confident, will be less prone to systematic error and bias. I think it is also worth highlighting here the value of exposing students to lessons like this in a scenario, before they are on the job. This requirement for self-reflection also makes for easy grading criteria: it clearly separates A’s from B’s.
Another student remarked: “my most significant personal takeaway is that there is no “winning “ and “losing”, just a trade-off of outcomes based on the available information you have and how you are going to use that to go forward. They added that although they hadn’t worked in intelligence, they were struggling in the module to understand the storied history of intelligence failure. They concluded that the simulation highlights the complexity of intelligence, its tendency to change rapidly, and for decisionmakers not to always be preoccupied with it. Previously, before our course, they struggled to understand how, “if you have all the information, how could you make strategic mistakes”? I doubt any of my students are still confused about that after this module, and especially the simulation.
Moving forward, I will try to address problems of scale with the game. We typically never complete the first phase of the scenario before our eight-hour allotment is complete. Some students have suggested short weekly adjudication sessions online, spread across the year to achieve that. I am also grappling with the glut of detail in the game briefings. Students remark that they are complex. I designed it from my own research so that the details are precise and accurate, and thereby it could offer as many realistic parameters as possible during simulation. However, this means that, for the students, the briefing materials are long and require lots of preparation and background reading compared to other assignments. So, I am looking for ways to simplify things, or even for new scenarios which are simpler.
I welcome your feedback and questions. I hope I have also been able to share something helpful from my teaching practice which has broader implications for intelligence education and pedagogy: That the simulation gives fodder for students to practice skills such as empathy (with their characters, and each other); as well as honest self-reflection and an appreciation for perspective. I’ve tried to teach them how historical and intelligence analysis overlap, including issues of sources, and an appreciation that bias is part of the analytic process. I think the scenario helps teach them how to spot such biases, and to grapple with them in a team setting.