I recently came across a 2007 paper by Warren Fishbein and Gregory Treverton on Rethinking “Alternative Analysis” to Address Transnational Threats, published by the CIA’s The Sherman Kent Center for Intelligence Analysis. The paper itself is a based on a longer series of workshops and associated reports published by RAND (Enhancing Warning for Transnational Threats: Workshop Reports).
In the paper, the authors argue (emphasis added):
One of the greatest challenges of any “alternative” effort is to effectively communicate the message to those who occupy decision-making roles. Decision-makers are buried by both information and tasks. Motivating them to spend time reading sophisticated analysis in general, let alone analysis that queries existing analytic lines, is a considerable challenge. Several presenters at the workshops who had been senior officials in the terrorism area stressed the extent to which information overload had grown in the post-September 11 environment. Moreover, in the transnational domain, many potential key consumers are in middle and lower operational levels, or outside the government, and thus have even fewer contexts for understanding intelligence information. And, of course, in order to carry out “alternative sense-making,” alternative points of view would need to be put before this wide array of harried individuals on a fairly regular basis.
One way to accomplish this is to rethink the concept of the intelligence “product.” Intelligence organizations continue to insist upon written prose and formal briefing as the “gold standard” for disseminating information even though adults rarely retain more than ten percent of what they are “told” either orally or in written form. Instead, more experiential, interactive formats, as discussed at the workshops, might better capture the attention and imagination of intended audiences and strengthen retention of insights.
- Use of web-logs would give consumers—particularly non-senior consumers without formal feedback processes—the opportunity to tap in from time to time on debates within the analytic community and to pose questions themselves.
- RapiSims—rapid simulations enabled by increasingly sophisticated spreadsheet-based programs—would allow consumers to manipulate variables to generate alternative outcomes. Decision-makers could quickly and easily explore a range of possibilities in a way that is more likely to be retained than if presented in a long and dry formal tome.
- Half day “gaming” sessions—intentionally kept brief to allow even the most harried to participate on occasion—could help decision-makers experience, at a minimum, the uncertainties surrounding an issue.
I’m a little dubious about RapiSims, to be frank. So much depends on the assumptions embedded (and therefore hidden) within the underlying computational algorithms that it only seems useful to me in cases where the relationship between quantifiable data is itself fairly widely accepted and uncontroversial.
I can see value in the use of well-structured simulations/gaming in exploring and projecting potential developments, and the response of actors to those developments. However, most simulation structures (unlike most well-written intelligence products) only allow one to easily pursue a single linked path of causality, and not examine a series of possible outcomes and the associated drivers and decision-trees. In terms of examining multiple possible paths and their implications, it might well be that red cell analysis, or even a well-moderated open bear-pit type discussion among analysts and policy folks, is a better investment of time and effort. I also wouldn’t underestimate the amount of give-and-take that goes into a good verbal intel briefing, especially when the policymaker on the receiving end knows how to ask hard questions of the briefer.
As noted in my previous posting below, I think that simulations can be a very useful way of engaging current and potential policymakers under certain circumstances. However, I’m wary of treating it all with a sort of faddism, or of denigrating the very considerable cost-effectiveness of other methods.
The same applies outside of the intelligence community context, in training and education settings too. It is important that we ask, when we devote substantial time and effort to a simulation, whether it is worth the opportunity cost of time that would have otherwise be devoted to lectures, seminar discussions, written assignments, or other more traditional methods.
It often is worth it—indeed, that’s the very reason for this website. It isn’t always, however.