PAXsims

Conflict simulation, peacebuilding, and development

Assessing simulation effectiveness in Brynania

school_is_hell_9_types_of_college_teachers700pix

Back in 20120, Michael King and I asked students in my POLI 450 (Peacebuilding) course about the learning effectiveness of the Brynania civil war simulation. I never did get around to do much with the results, so I have posted them below. I’ve highlighted the largest learning effects in green, and the smallest in red.

Sim2010ResultsAs you can see, students reported learning in every single area we asked about. Learning was highest in those areas related to process and operational constraints (that is, the bureaucratic politics,  “friction” and “fog of war/peace”)—precisely those areas that it is hardest to explore through conventional lectures and readings. Students also overwhelmingly reported that a week of simulation was more valuable to them than a week of readings or lectures. Very importantly, however, they saw the experiential learning of the simulation as being complimentary with the more conventionally-delivered course material, synergistically improving their understanding of prior course readings. As I noted in a previous post, there were no statistically significant gender differences in learning.

Self-reported learning effects were smallest in those areas that related to the development of personal skills like leadership, time management, or empathy (although even here there was agreement that the simulation had delivered some learning). In addition to data above, we had also asked students to self-assess their skills in such areas both pre- and post-simulation. That part of the study revealed no substantial, statistically-significant changes.

In interpreting the data, some caveats are in order:

  • First, this is self-reported learning, not learning that has been assessed by some more objective measure like exam results. There is some evidence that self-reported measures may overstated the effectiveness of simulation-based methods, especially when these are assessed in the immediate aftermath of a simulation that students have enjoyed.
  • Second, these findings hold true for this simulation and these (highly-motivated) students. While they do provide further evidence of the general value of simulation methods, other simulations and audiences could well report significantly different effects.
  • Third, simulation-based learning cannot really be separated from the broader curriculum in which it is embedded. We know, for example, that the simulation debrief plays an essential role in good learning outcomes. In addition, however, learning outcomes are likely enhanced when the simulation and non-simulation material are designed to compliment and synergize each other.

A final thing we did in the questionnaire was to assess whether simulation participation had any measurable effect of student career goals. Here there were suggestions of a very slight decline in interest in legal or military careers, but generally no significant effect.

One response to “Assessing simulation effectiveness in Brynania

  1. J.F. 29/06/2013 at 5:31 pm

    Note from a previous student (2013 edition): The choice between joyful learning for life vs regurgitating for grades led to forfeiting some 10 % of final grades in other courses due to overfocusing on this simulation. This particular case study needs another 5-10 years to see whether the strategy paid off. The point is however that it’s not just about learning more than a week of readings, it’s about putting way more hours into it. Sometimes by rerouting time from paperwriting in other courses, sometimes by sacrificing a free Saturday night for recording muppet voices for a video asking for grants for one’s simulated aid organisation. As stated previously, final outcomes should be seen over the next ten years, if not fifty.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: