PAXsims

Conflict simulation, peacebuilding, and development

Daily Archives: 15/01/2023

Majnemer and Meibauer: Fictitious country names affect experimental results

Jacklyn Majnemer (MIT) and Gustav Meibauer (Radboud University Nijmegen) have published a very interesting article in International Studies Quarterly, 7, 1 (March 2023) exploring whether fictitious country names in survey vignettes affect experimental results. The answer: yes they do.

Using fictitious country names in hypothetical scenarios is widespread in experimental international relations research. We survey sixty-four peer-reviewed articles to find that it is justified by reference to necessary “neutralization” compared to real-world scenarios. However, this neutralization effect has not been independently tested. Indeed, psychology and toponymy scholarship suggest that names entail implicit cues that can inadvertently bias survey results. We use a survey experiment to test neutralization and naming effects. We find not only limited evidence for neutralization, but also little evidence for systematic naming effects. Instead, we find that respondents were often more willing to support using force against fictitious countries than even adversarial real-world countries. Real-world associations may provide a “deterrent” effect not captured by hypothetical scenarios with fictitious country names. In turn, fictionalization may decrease the stakes as experienced by respondents. Researchers should therefore carefully explain rationales for and expected effects of fictitious country names, and test their fictitious names independently.

In Table 2 below you can see that respondents were more willing to use military force against “Celesta,” “Drakhar,” or “Minalo” than they were either a friendly real country (Canada) or a hostile one (Iran).

The research here focuses on survey responses, not serious game play. However the findings may have some interesting implications for strategic-level wargames using fictional country names, which may be more prone to escalation than similar games using real countries.

Interestingly, the authors also suggest that the more “real” a country sounds, the less fictionalization effects are evident:

Our results suggest that the more clearly fictitious a country name, the easier to condone attacking it—fictionality and its perceived costlessness can therefore embolden respondents to provide more aggressive responses.

These results point to the relevance of perceived realistic-ness: the more “real” a country name sounds to respondents, the weaker the fictionalization effect. In particular, there seems to be a deterrent effect associated with realistic-ness, for example, of being able to imagine more easily the consequences associated with attacking Iran, especially bar any additional information that “fills out” the scenario. 

The explanation they suggest for this is deterrence: respondents are better able to imagine the costs of an attack when the survey question asks about a real country rather than a fictional one. However, there may also be an empathy factor here—it’s easier to imagine killing and maiming actual Iranians or Canadians than it is “Minalans,” “Brakharis,” or “Celestians.”

In professional wargames, it is sometime necessary to use fictionalized countries, usually because of political sensitivities. In experimental games there may also be a desire to exert better control of key variables than is possible using a real-life settings. Both reasons apply, for example, to a recent series of NATO experimental wargames that examined Intermediate Force Capabilities in a fictional conflict between the Illyrian Federal Republic and Hypatia (the latter backed by Organization for Collective Security).

An unclassified NATO STO SAS wargame in 2022. You’ll note the Illyrian Federal Republic operations orders (OP IRKALLAN FREEDOM), in a conflict that seems rather reminiscent of a real one in some ways, but set in the northern Aegean.

If Majnemer and Meibauer’s findings do indeed expand beyond international relations survey research to wargaming, there are several implications. One is the need to provide game participants with a rich and realistic fictional environment and to work hard to promote narrative engagement. Another is the need to caveat experimental findings, especially as they relate to use-of-force decisions but possibly other things as well, such as risk aversion or casualty sensitivity more broadly.

CNN Academy journalism simulation

CNN Academy is a journalism training program run by CNN in collaboration with university programs around the world. In December, more than eighty of those students, together with a number of their instructors, travelled to Abu Dhabi to take part in an five day intensive news-gathering simulation. Although simulation has been used in journalism programmes before, this was an industry first in terms of scope, scale, and complexity.

As with most educational simulations, the intent here was to challenge participants to put to work the knowledge they had acquired in their studies in a “safe to fail” environment. We didn’t make it easy, either.

This wasn’t the first time I had supported journalism training using simulation methods, but those past efforts were an ancillary to a simulation largely designed for other purposes.

Below I’ll discuss the setting and scenario for the simulation, the simulation mechanisms we used, and some of the key lessons learned. There will be a few things I won’t reveal, however—we want to keep them a secret for future iterations! I was the primary simulation designer and game controller. CNN staff also contributed to the design (notably Alireza Hajihosseini, John Sanders, and Mohammed Abdelbary), and most of the roles in the simulation were played by CNN journalists. Jim Wallman (Stone Paper Scissors) codirected the simulation. The simulation was hosted at the Yas Creative Hub of twofour54, and we also made use of their Kizad movie production backlot.

Setting and Scenario

There were several important considerations in establishing the setting and scenario for the simulation. We decided early on that we wanted to use a fictional country. One reason for doing this was to allow us the freedom to craft a narrative that would fully engage a broad range of journalism skills. We also wanted to avoid an Orson Welles “War of the Worlds” -type situation where something in the simulation somehow leaked into the real world and generated confusion or concern.

The problem with a fictional country, however, is providing sufficient detail and depth to be useful and believable. Fortunately, we already had one such country setting available: a fictional conflict-affected country that had been used in my peacebuilding course at McGill for almost two decades. A tremendous amount of historical, political, economic, and cultural information had already been produced for this over the years, both by me and by generations of McGill students. That setting was modified and updated—McGill students will be pleased to know the civil war there is now finally over—for use by CNN Academy.

As for the precise scenario on which participants would be reporting, we needed something that was dramatic enough that it would credibly attract global media attention. We decided on a major environmental disaster. This had multiple elements to it: the immediate disaster, and its associated human and environmental cost; the broader social, political, and economic ramifications; and the complex web of crime, corruption, and politics that had allowed it to happen. This was not a simple plot or easy to unravel, and students had to use a broad range of investigative techniques to fully understand what was going on.

A summary of our rather complex story, rendered suitably murky in case we ever want to reuse some of it.

Everything about the scenario, setting, and simulation structure was written into a 24 page “master scenario guide,” which was updated as necessary as new elements were added.

Simulation Mechanics

Students arrived in Abu Dhabi having taken part in CNN Academy webinars and other instructional content, but with no information on the simulation other than that there might be one. It’s fair to say that none of them anticipated how intense it would be. We immediately grouped them into teams of four or five students and threw them in the deep end: they were told there was breaking story and a forthcoming press conference to cover, given initial details about the situation, and provided with a detailed country brief. They only had a short time to get to know their team, consisting of students from two or three different journalism programmes, as well as read up the country where they had just been “sent” to report. Then they started news-gathering.

Surprise! You’re all about to take part in an intense, stressful simulation this week.

Participants were also given access to a team email address and to a Twitter-like social media platform populated by a constant stream of fictional social media posts about the disaster, mixed in with actual news items about the rest of the world harvested in real time from CNN and other media feeds. About four hundred of the social media posts had been pre-scripted and pre-timed before the simulation, but others were injected live while it was all going on. This assured that there were new potential developments regarding the story almost 24 hours a day. The teams also received both scripted and live emails during the sim, and could “reach back” to their producers for advice and information. Both the email and social media servers were closed so they couldn’t leak into the real world.

A team hard at work.

On the first four days (Monday-Thursday) students participated in five simulated press conferences and many one-on-one interviews. The various spokespersons and interviewees—more than two dozen in total—were played by CNN staff, as well as myself and Jim Wallman. Other online characters might interact via email or social media direct messages.

A press conference underway.

Each role had a role briefing written up, detailing the character’s identity, personality, motivation, and information, along with key talking points. All of our roleplayers had been provided with this in advance. In addition, I also held a series of online orientation session via Zoom for the simulation staff in the weeks running up to the simulation.

In any event, CNN journalists turned out to be terrific improvisational actors! Quite apart from their acting skills, all were well aware of the challenges in covering press conferences or interviewing sources and were able to use their professional experience to keep students on their toes. Teams that did a particularly good job of conducting interviews might be given additional information or contacted later with news tips.

Particularly memorable was a trip to the affected area—represented in this case by twofour54 Kizad movie backlot, much of which is constructed to look like a war-torn city. Here they were paired up with CNN photojournalists and were free to roam about and interview the “local inhabitants.” It was a remarkable experience.

Another “local resident” being interviewed—in this case me, in my stylish green PAXsims “game control” safety vest!

All of this simulation activity over the first four days was interspersed with a series of lectures on various aspects of modern journalism, including newsgathering best practices, mobile storytelling, commercial operations, and the art of the spectacle.

On Thursday students were expected to submit a pitch to their producer for a video report on the disaster. This took the form of a full “paper edit” of their proposed piece, including script and visuals. In addition to whatever video they had shot themselves or had been shot for them on location, we provided additional B-roll to use in these reports. No one got much sleep at this point.

Editing one of the video reports.

The top six submissions were given feedback, access to studio facilities, and an editor the next day to produce their report. The rest of the participants had a chance to relax and see some of the sights of Abu Dhabi. After lunch we all reassembled to screen the semi-finalist videos and announce a winner.

The winning team of student journalists with Becky Anderson (CNN).

Reflections

It all went very well—better than expectations. No major mishaps were encountered. All of the tech (John Sanders) and logistics (Shivon Watson) ran brilliantly. The CNN folks were enthusiastic and engaged, as well as being terrific roleplayers. Maitha Khalifa and her team at the Yas Creative Hub were outstanding hosts and their facilities were top-notch.

A post-event participant survey indicated a very high evaluation of CNN Academy experience, the acquisition of relevant skills, engagement, and willingness to recommend the experience to others.

There were a great many teachable moments during the simulation. Some of the ones that most stood out to me were:

  • The pressure of the simulation caused some students to lose sight of the importance of soft skills. For all the changes in the media brought about by rapidly changing information and communication technologies, “people skills” remain at the center of good journalism. Journalists need to understand those they are reporting on and develop a rapport. They need to treat traumatized populations with sensitivity. They need to develop sources. They need to listen carefully as well as ask questions. They need to be able to follow leads in new directions, especially when an interview reveals new information. They also need to be able to tell a complex story in a way that is interesting and understandable to their audience. Technology changes some of the ways this is done, but most of these skills would have been immediately recognizable to a good reporter a century ago.
  • Teamwork is essential. Every team consisted of a mix of experiences, expertise, language skills—not to mention gender and national origin. The teams that did best worked hard on collaboration, information management, tasking, and generally getting the best out of everyone in a harmonious fashion.
  • The simulation also highlighted the importance of fact-checking and research. Not everything students were exposed to was true. Politicians and others spun the story in ways that made them look good, and all of the interviewees filtered their comments through their own perspectives and beliefs. Locals residents didn’t always know exactly what was going on,. There were lots of rumours online. And whenever you have more than eighty students talking amongst themselves they going to accidentally generate their own rumours through a sort of broken game of “telephone.” The best teams verified what they heard, and didn’t just run with it.
  • Media ethics matter. We sprinkled a few ethical challenges in the simulation (I won’t say what they were in case we reuse them). A few fell for the traps!

Although we did a short debrief at the end of the simulation (including a reveal of the full “plot” and how various elements could be discovered), and although the accompanying journalism professors were constantly providing advice and feedback to their students, it would have been nice to have had more time for this. CNN Academy plans to post a series of debrief “blog posts” for students to the CNN Academy hub in the near future to build on the immediate feedback they received in Abu Dhabi.

During one of the classroom sessions, CNN journalists share their insights with students.

For other coverage of the CNN Academy simulation, see:

%d bloggers like this: