The Wisdom of Crowds
December 16th, 2006 | by Craig |If you remember a couple years back, the Department of Homeland Security wanted to do a predictive market on terrorist attacks.
Further, you’ll recall that the idea was ridiculed so badly in the press that the program folded up.
Too bad the media didn’t take some time to do a little research and find out how well they can work.
This is why I generally look at Tradesports to see how things are going to go in elections or even football games. Sure, sometimes they can be wrong; you can never fully factor in outliers, but overall they’re pretty damned accurate.
6 Responses to “The Wisdom of Crowds”
By Dani on Dec 16, 2006 | Reply
I remember the uproar over this- “speculating on terrorism” or “profiting from terror, ” were some of the misinformed accusations. You wonder if these people had ever tried to figure out what a futures market actually is- or maybe they knew and still wanted to paint it in a bad light for political gain.
The article answered a question I have had for a while- to whome would you open the market? Obviously not terrorists, but “High-quality intelligence.”
I guess that means terrorism experts, whoever they are. Is there a definition for that?
There may be a lot the man on the street knows or intuits, that may be lost in translation to the experts. In the examples given, employees were doing the speculating- not just management. Many companies adjust to changing markets by querying salesmen or other folks dealing directly with the consumer. I would think about opening the market to all law enforcement, at least in some areas (where they can be trusted not to be terrorists themselves).
By Matt Singer on Dec 16, 2006 | Reply
I read somewhere that Tradesports underperformed significantly in political races this year. Markets are OK, but they rely on mass participation and the idea that people trading actually have solid information and aren’t just randomly trying to, well, speculate on terror.
By Craig on Dec 17, 2006 | Reply
The thing about the Policy Analysis Market was that (in my reading of it), they weren’t trying for mass public involvement; it was only going to be for people who had some expertise on the subject matter. Speculation was one of their concerns and something they accounted for.
Which, incidentally, could be why Tradesports underperformed this year — too many uninformed yutzes trying to make a quick buck.
By Dani on Dec 17, 2006 | Reply
Matt, I wonder about that “solid information”. If it’s solid, a market wouldn’t relaly be needed, would it? The expert in question would just notify the Dept. of Homeland Security, wouldn’t they? I think the purpose of the market is to capitalize on hunches, trends and educated guesses- but which are shared by numerous people who ought to know what’s going on. A lot of little brushstrokes that eventually paint a picture that can be acted upon.
I really think it’s a decent attempt to quanitfy and objectify knowledge. Students of epistemology will attest to the fact that people are far from logical, and terrible at estimating risk. That’s why we have all kinds of scoring systems. Medicine is full of them- APACHE illnes scores, Glasgow coma scales, Bishop’s scores, etc. Otherwise, there’s too much information and too many emotional distortions to really get a grip on the situation, and make a plan.
By Shane C. Mason on Dec 18, 2006 | Reply
I think that the thing that bothers me about it was touched on by Dani here. It would be nice if our intel could find out about pending attacks and stop them rather than relying on a predictive model of any type.
By Craig on Dec 18, 2006 | Reply
Sure it would be nice, but the human intelligence side was gutted in the 90’s.
I don’t think that anyone has suggested that predictive markets replace good intel; rather they should be another tool in the toolkit.