Planned date = Jan 15, Crowd Forecast = Feb 10….. DANG!

The realistic crowd forecast is important for managing the project and shaping customer expectations. However, this sensitive information needs to be handled well otherwise it could cause issues around employee morale or with partners or investors.

This issue around transparency of the results has really restricted the use of prediction markets in the enterprise, but Crowdcast has developed and patented a solution.

We have now enabled 3 transparency settings to allow our admins to share as much as they want with the participants.

Three Settings:

You can share everything, share comments, or share only the question.

Share with your team: This is the default setting and shares the Crowdcast Curve, the comments, the beliefs behind those comments. Use this for metrics which are ok to be shared with your team.

Share comments and anonymous individual bets: With this setting, the Crowdcast curve no longer appears. Use this setting for metrics where the commentary and social side is important, but where the metric may not be shared.

Share the question only: This shows only the question. Payoffs are ‘location invariant’ — a user can’t determine the shape of the Curve by moving their bets around. Comments are hidden. Use this for your most important metrics.

For all the settings, the Admins and the users with “Executive” access can still see the crowd forecasts on the Dashboard page.

Some implementations of prediction markets compute probabilities of outcomes. To use Dr. Kambil’s example, one could ask, “Will the DJIA end above 10,000 before the end of the year?” Market participants who are bullish about the Dow buy shares. Those who are bearish sell. As trades come in, the market mechanism adjusts the probability of the event actually taking place.

While this is pretty interesting, getting to a collective forecast that gives you a probability distribution of possible outcomes, rather than just a point forecast, of the DJIA itself is awkward. To do it, you’d have to launch different markets around buckets of outcomes — as in, DJIA in 8,000-9,000, DJIA in 9,000-10,000, DJIA in 10,000-11,000, and so on — and then reason about the probabilities of each.

Crowdcast builds probability distributions automatically. This has two important implications. First, it enables a simple user interface and finer grained expression of beliefs. Rather than asking people to choose predefined buckets, they can select a precise range, as wide or narrow as they wish. And second, it supports some great applications for business. For instance, you can get alerts when the likelihood of hitting your target ship date falls below 50%.

Dr. Kambil also discusses the information dissemination “feature” of prediction markets. Prediction markets, like the public stock market for instance, not only collect information, but they also distribute it. While this is great for some applications — surely you’d want to know the price (and other indicators) of Apple stock before you bought some — it’s a problem for others.

In the enterprise, it is often the case that the more valuable and important a metric, the more secret it is. Revenues or earnings per share are but two examples. We’ve struggled with this reality for some time. Initially, our take was that we should simply concentrate our efforts on problems characterized by “public” metrics. Then we implemented access control lists, which enabled our customers to publish questions about sensitive metrics to a subset of the participants.

While this worked for some applications, the overall approach just didn’t sit well. There is knowledge in the enterprise about revenues and plenty of other sensitive metrics. And we’re all about harnessing knowledge. Won’t let the cat out of the bag yet, but we’ve cracked it! Details very soon.

As I mentioned last time, our initial attempt at running the project revolved around a project plan and regularly scheduled meetings. The project plan was standard — milestones, subordinate tasks, dependencies. Task owners came up with costing estimates. Since we were an experienced team, we added fudge factors liberally. Once the plan was in place, the team leads — engineering, product management,qa — agreed to meet weekly to track progress.

The alpha release milestone was critical, but comfortably far away, so the first few meetings were genial. But a week before ship it became ugly. Turned out that the port from our development environment to test (which simulates production) was in trouble. This was one of those on-going tasks. Deal was that as new APIs came on-line, they’d be ported for testing. Somewhere along the way, this stopped.

How could this have happened? How could it have taken us until the last minute to find out?

After the dust settled, we saw that the task reached 85% completion weeks ago and stalled (the problem was a serious bug in some 3rd party code, which was waiting for a patch). The engineer who owned the task, and indeed others, knew that the alpha milestone was at risk, but lacked a good way to give their knowledge voice. Since everything else was tracking well, intentionally or not, the problem that derailed the project was swept under the rug.

On reflection, it wasn’t a problem of scoping, project sponsor buy-in, staffing, or implementation. Rather, it was a problem of communication. That was when we decided to drink the Kool-Aid.

In the first post in this series, I described our experience with using Crowdcast to gain insight into risks around the next important milestone — the beta release — in the new mech project. Recall that the system rewards people for revealing rare and relevant information early. Moreover, since bets are placed anonymously, participants’ incentives are uniform and aligned. As bets come in, the system aggregates them in real-time and generates alerts when new information reduces the likelihood of success.

The first screenshot is of a piece of the console, which shows crowdcast summaries. The second is a part of the detailed view, which shows the distribution of beliefs as well as how they are tracking relative to target.

My bet on the beta crowdcast caused an alert to fire. It was an early warning for Huned, which gave him time to understand the root cause of the problem and make adjustments to keep the project on track. Next time I’ll put all the pieces together and present a complete overview of the platform that the new mech of ours enables.