PAXsims

Conflict simulation, peacebuilding, and development

World Bank assessment of EVOKE

A couple of pieces have been posted on the World Bank’s EduTech blog assessing its experience earlier this year with the EVOKE social networking and education project. The verdict from Rovert Hawkins is very positive:

By the time the EVOKE adventure ended 19,324 people from over 150 countries registered to play, far exceeding expectations.  Players submitted over 23,500 blog posts (about 335 each day), 4,700 photos and over 1,500 videos. The site received over 178,000 unique visitors and 2,345,000 page views with time per visit averaging over eight minutes.  For the month of March EVOKE generated just under 10% of what the World Bank’s entire external website generated with regard to page views (1.1 million versus 12.1 million).  Phenomenal numbers…  Across the board EVOKE exceeded our expectations.

Over the course of the 10 weeks, players posted ideas, found friends, commented on projects, shared information, rated the quality of the information shared, discussed, argued, created and acted.

Librarians donated time to do research. Someone developed a wiki for the game. Teachers created their own online community within the game. Some players developed an online conference for sharing the best ideas. One player wrote a song about EVOKE. Others planted gardens.

Players went into their communities to learn about challenges on the ground and shared potential solutions to what they saw and heard.  One player collected all of these ideas into a single blog post.

EVOKE I think has created space for dialogue around serious issues that may not be discussed in other social networking forums.

Based on the results of an external survey and evaluation, the World Bank also reports that the project’s goals were met:

Based on the survey, EVOKE had a strong effect on:

  • Developing new ideas about global challenges
  • Developing new ideas about local challenges
  • Learning about potential solutions to….
  • Learning about people in other countries
  • Learning about sustainability

EVOKE had less effect on building a network of colleagues and friends and gaining self confidence.

You’ll also find a South African media report on the positive effect of EVOKE in one Cape Town school here.

The survey results look impressive, although it isn’t clear how sampling was done. If it was done from in-game participants (and especially those who then responded) there would be some fairly obvious problems of potential response bias. There is also the medium vs the message problem that I raised before in my own reflections on EVOKE: while large numbers of people did participate, what were they actually learning? Much of the content of EVOKE, whether in the form of the central storyline or a large percentage of the blogposts, simply didn’t really reflect development best practices. Some of it was flat wrong. Many participants might think they learned about, say, sustainable development–but did they? How do we know?

This is always a problem with evaluating experiential learning techniques, as educational psychologists have long recognized. If you ask students whether they’ve learned something, they tend to say yes—especially if they’ve had fun. If you test this objectively (by, for example, comparing them with a control group that didn’t use such methods) the effects sometimes appear much more modest. If you measure the benefits against the opportunity cost–in other words, spending the same amount of money and time in other forms of educational engagement–you might even get another picture altogether.

Perhaps EVOKE’s final report will address these issues. Certainly I would be pleased to see that it has had positive, objective effects, as well as the subjective ones reported by those who participated.

According to World Bank:

At the end of the game, 74 project ideas (known as ‘evokations’) were submitted and the following rewards were made:

  • 10 projects were given seed funding of US$1,000
  • 22 projects were provided with a post-game mentor
  • 15 projects were invited to the EVOKE summit in September
  • 25 projects are competing in the EVOKE Global Giving Challenge.

The EVOKE challenge on Global Giving is happening now!  The challenge provides players with an opportunity to put their ideas to the test – raising funds on-line and expanding their networks of supporters.  Please visit the EVOKE challenge on Global Giving at http:// www.globalgiving.org/evoke and support one of the EVOKE innovators in their quest to get their ideas off the ground!

Also, there will be an “EVOKE summit” later this month to celebrate and reflect on the project’s accomplishments:

The EVOKE summit will be taking place at the end of September in Washington, DC.  At this summit we will bring together a number of the players, mentors and development team to celebrate the incredible work of the EVOKE community and reflect on how to build on the lessons learned for future iterations.  Please share any ideas you have!

Let’s hope that in doing so they also seek to learn from the constructive criticism offered of the project from the outside.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: