Planned date = Jan 15, Crowd Forecast = Feb 10….. DANG!

The realistic crowd forecast is important for managing the project and shaping customer expectations. However, this sensitive information needs to be handled well otherwise it could cause issues around employee morale or with partners or investors.

This issue around transparency of the results has really restricted the use of prediction markets in the enterprise, but Crowdcast has developed and patented a solution.

We have now enabled 3 transparency settings to allow our admins to share as much as they want with the participants.

Three Settings:

You can share everything, share comments, or share only the question.

Share with your team: This is the default setting and shares the Crowdcast Curve, the comments, the beliefs behind those comments. Use this for metrics which are ok to be shared with your team.

Share comments and anonymous individual bets: With this setting, the Crowdcast curve no longer appears. Use this setting for metrics where the commentary and social side is important, but where the metric may not be shared.

Share the question only: This shows only the question. Payoffs are ‘location invariant’ — a user can’t determine the shape of the Curve by moving their bets around. Comments are hidden. Use this for your most important metrics.

For all the settings, the Admins and the users with “Executive” access can still see the crowd forecasts on the Dashboard page.

The US Intelligence Agencies have thousands of highly-trained analysts with access to a vast store of privileged information.  However, they have great difficulty efficiently synthesizing and surfacing this info actionable information.  Often the “a broken watch is right twice a day” phenomenon encourages analysts to stick with the status quo rather than flag new conditions.  These kinds of forecasting behaviors have real costs in dollars and in lives.

Despite its importance in modern life, forecasting remains (ironically) unpredictable. Who is a good forecaster?  How do you make people better forecasters?  Are there processes or technologies that can improve the ability of governments, companies, and other institutions to perceive and act on trends and threats?  Nobody really knows.  The goal of the Good Judgment Project is to answer these questions.

We will systematically compare the effectiveness of different training methods and forecasting tools in accurately forecasting future events.  We also will investigate how different combinations of training and forecasting work together.  Finally, we will explore how to more effectively communicate forecasts in ways that avoid overwhelming audiences with technical detail or oversimplifying difficult decisions.

Over the course of each year, forecasters will have an opportunity to respond to 100 questions, each requiring a separate prediction, such as “How many countries in the Euro zone will default on bonds in 2011?” or “Will Southern Sudan become an independent country in 2011?”  As the project evolves, we’ll iterate the mechanisms and the training, arriving at an idealized and verified system.

The Good Judgment research team is based in the University of Pennsylvania and the University of California Berkeley. The project is led by psychologists Philip Tetlock, author of the award-winning Expert Political Judgment, Barbara Mellers, an expert on judgment and decision-making, and Don Moore, an expert on overconfidence. Other team members are experts in psychology, economics, statistics, interface design, futures, and computer science.

If you’re interested in participating, we’d love to have you.  Head on over to our registration site to sign up.  We’re looking forward to sharing the results with you, and leveraging the valuable insights of The Good Judgment Project with our clients.

But before we get the blog cooking again, I want to take a moment to highlight the great press we’ve enjoyed as a result of our efforts.  The way the world views you is so much more telling than how you view yourself, so here are some great themes that the press pulled from our conversations.

• Robin Wauters at TechCrunch summarized our offering nicely, saying, “Crowdcast’s solutions, composed of a SaaS software platform, consulting and support services – aim to bridge the gap between traditional business intelligence and enterprise social network applications. The idea is for businesses to align their employees with the purpose of the company, bringing all their insights, plans and experience together in one place, all in order to create “insanely accurate” business predictions and outcomes.
• VentureBeat’s Anthony Ha highlighted that “Crowdcast is all about finding the knowledge in your workforce.”
• Jeff Nolan at Enterprise Irregulars zeroed in on one of our key differentiators, saying, “Where Crowdcast appears to have departed from previous efforts to implement crowdsourcing inside the enterprise is that rather than being an ideas-like site where anything can get posted and then promoted/demoted by community members, they are building games that are designed to reveal outcomes. In other words, they form every interaction around a tightly focused question that requires an answer and through virtual currency create competition in the form of betting that narrows the result to what is considered the most probable outcome, which is then measured according to what actually occurs over time.”
• VentureWire’s Ty McMahan really nailed our point-of-view, when he said, “Crowdcast’s vision is for employees to participate in sharing and socializing insight on what’s planned and what the outcome of business activities will be. The company wants to change the sources executives use to make business decisions by mining previously unavailable insight from across the organization.
• Alex Williams at ReadWrite notes that our “$6 million from Menlo Ventures in a deal that bodes well for a new generation of companies with crowdsourcing and business intelligence offerings.“
• Mike Vizard at IT Business Edge was excited by a promising future, where “longer term… an organization can soon start to map out who knows what within their organization.”
• Amanda Coolong over at This Week In had me on for a great interview.  Thanks Amanda!  My favorite quote was from Dave Linthicom, “… the info is there, they know it’s there, they see tidbits of it, but they can’t mine it for it’s value. So, if you’re working on that problem, kudos to you because you’re going to make a billion dollars.”

PEHub, VatorNews, Silicontap, and others helped us spread the message far and wide.  We look forward to talking more as Crowdcast continues to evolve and champion the message that Your People Know.

Thanks, everyone.

As I mentioned last time, our initial attempt at running the project revolved around a project plan and regularly scheduled meetings. The project plan was standard — milestones, subordinate tasks, dependencies. Task owners came up with costing estimates. Since we were an experienced team, we added fudge factors liberally. Once the plan was in place, the team leads — engineering, product management,qa — agreed to meet weekly to track progress.

The alpha release milestone was critical, but comfortably far away, so the first few meetings were genial. But a week before ship it became ugly. Turned out that the port from our development environment to test (which simulates production) was in trouble. This was one of those on-going tasks. Deal was that as new APIs came on-line, they’d be ported for testing. Somewhere along the way, this stopped.

How could this have happened? How could it have taken us until the last minute to find out?

After the dust settled, we saw that the task reached 85% completion weeks ago and stalled (the problem was a serious bug in some 3rd party code, which was waiting for a patch). The engineer who owned the task, and indeed others, knew that the alpha milestone was at risk, but lacked a good way to give their knowledge voice. Since everything else was tracking well, intentionally or not, the problem that derailed the project was swept under the rug.

On reflection, it wasn’t a problem of scoping, project sponsor buy-in, staffing, or implementation. Rather, it was a problem of communication. That was when we decided to drink the Kool-Aid.

In the first post in this series, I described our experience with using Crowdcast to gain insight into risks around the next important milestone — the beta release — in the new mech project. Recall that the system rewards people for revealing rare and relevant information early. Moreover, since bets are placed anonymously, participants’ incentives are uniform and aligned. As bets come in, the system aggregates them in real-time and generates alerts when new information reduces the likelihood of success.

The first screenshot is of a piece of the console, which shows crowdcast summaries. The second is a part of the detailed view, which shows the distribution of beliefs as well as how they are tracking relative to target.

My bet on the beta crowdcast caused an alert to fire. It was an early warning for Huned, which gave him time to understand the root cause of the problem and make adjustments to keep the project on track. Next time I’ll put all the pieces together and present a complete overview of the platform that the new mech of ours enables.

In response to these ideas, Enterprise 2.0 pioneer and MIT Principal Research Analyst, Andrew McAfee, took to his blog to highlight some parts that really stuck with him. Andy’s synthesized version of the article really spoke to us and the ideas that we find valuable at Crowdcast. Andy highlighted a couple of values that we think are essential to smart business: breaking down hierarchies to unclog communication and fostering collaboration through peer effects, which allow people to get recognized by their peers for what they do rather than by their organizational rank.

Companies are beginning to understand the importance of communication within the ranks of their organization, not only to improve workflow, but also to improve access to employee intelligence. From there, managers can start to really monitor the pulse of their company. Ultimately, harvesting wisdom and gleaning well rounded insight is a competitive advantage — a very timely and relevant conversation to have as companies look ahead into 2010.