Thinking about using a prediction market to rank ideas? Read this first.

There are lots of great ideas in our community (be it a corporation, supplier network, government agency), so let’s use a prediction market to let the crowd prioritize and rank those ideas. Sounds great, right? Wrong.

Imagine I am a participant in an idea market. I come to the market and see two ideas about green innovation that have been submitted by my colleagues. One is practical and implementable — as lightbulbs burn out, replace with energy efficient ones. The other is never going to happen, but sounds nice — put windowboxes in every window of our 100,000 square foot facility to combat greenhouse gases. It’s day one, so both of these ideas have the same “price.” Which do I buy? The one I think my colleagues will find more appealing, regardless of my belief in the idea. Take it one step further. Imagine I submitted one of the ideas. Which do you think I vote for then? Which do I ask my friends to choose?

Prediction markets are powerful tools, designed to aggregate unbiased tacit knowledge in a well-incented environment that doesn’t respect hierarchies. They can reveal truths and patterns in organizations. They can help us plan for the future in a clear and confident manner, and to make mid-course corrections to avoid unpleasant outcomes. This last word is very important. Prediction markets rely on outcomes. Without concrete, objective, quantitative outcomes on which to reconcile the markets, the whole house of cards falls.

Recently, we have read of several high-profile prediction market implementations designed to rank ideas or to forecast trends. These ideas and trends may be 1,000s deep, years from inception, or in the case of some about technology adoption and societal change, decades. Therefore, filtering is impossible and there is no way to reward participants for their prescience about the true outcome. The only thing to measure and reward, therefore, is accuracy about popularity. Generally, a close would only happen on either implementation of the idea (decades hence) or on a date chosen by the market organizer (in which case the market is closed at the final market prediction on that date). Either way, the way to earn money (after all, markets are about profit) in a reasonable time horizon is to know which way the wind is blowing, and bet in that direction.

This problem isn’t new. In 1936, Keynes referred to a similar phenomenon, now known as a Keynesian beauty contest. In it, he describes a contest where players are asked to pick the most beautiful face from a set of choices, with the prize going to one of the players who picks the most popular. The fairly obvious strategy is not to pick the face you find most appealing, but the one you think will appeal to the majority of participants. Inducting one step further, a player might choose what he thinks other players perceptions of popular beauty would be, and so on.

Clients and prospects frequently ask about the use of prediction markets for idea analysis. Unfortunately, standard prediction markets are not the best way to achieve this. However, there are alternatives:

• As a way to collect sentiment about different ideas, surveys work well and are simple to implement.
• Shorter term forecasts where the players forecast which idea will be adopted by an investment committee are also a possibility. Note, here we are not asking which is the best, but which will pass the committee’s criteria. This can be useful for helping to filter many ideas, but again can suffer from information cascades as popular ideas get more votes, rise to the top and are considered. An idea late to the market is unlikely to fare well.
• For ideas that are closer to realization, prediction markets begin to make sense. Imagine having two alternative proposals, A and B. It is certainly reasonable to ask: Were we to pursue A, our revenues/time to market/costs would be ___. Ask the same question for B. One of these sets will have a close as one will be pursued. In the other market, do a push (refund all bets).

In summary, an “ideas market” sounds like an intriguing idea to rank innovation, however, a market mechanic is not the best way of achieving this aim.