Dr. Ajit Kambil, Global Research Director of Deloitte, authored an interesting piece on the use of prediction markets by CFOs. He presents a nice summary of how prediction markets work and their benefits — that they are a great way to aggregate dispersed insights and capture information that changes over time.

Some implementations of prediction markets compute probabilities of outcomes. To use Dr. Kambil’s example, one could ask, “Will the DJIA end above 10,000 before the end of the year?” Market participants who are bullish about the Dow buy shares. Those who are bearish sell. As trades come in, the market mechanism adjusts the probability of the event actually taking place.

While this is pretty interesting, getting to a collective forecast that gives you a probability distribution of possible outcomes, rather than just a point forecast, of the DJIA itself is awkward. To do it, you’d have to launch different markets around buckets of outcomes — as in, DJIA in 8,000-9,000, DJIA in 9,000-10,000, DJIA in 10,000-11,000, and so on — and then reason about the probabilities of each.

Crowdcast builds probability distributions automatically. This has two important implications. First, it enables a simple user interface and finer grained expression of beliefs. Rather than asking people to choose predefined buckets, they can select a precise range, as wide or narrow as they wish. And second, it supports some great applications for business. For instance, you can get alerts when the likelihood of hitting your target ship date falls below 50%.

Dr. Kambil also discusses the information dissemination “feature” of prediction markets. Prediction markets, like the public stock market for instance, not only collect information, but they also distribute it. While this is great for some applications — surely you’d want to know the price (and other indicators) of Apple stock before you bought some — it’s a problem for others.

In the enterprise, it is often the case that the more valuable and important a metric, the more secret it is. Revenues or earnings per share are but two examples. We’ve struggled with this reality for some time. Initially, our take was that we should simply concentrate our efforts on problems characterized by “public” metrics. Then we implemented access control lists, which enabled our customers to publish questions about sensitive metrics to a subset of the participants.

While this worked for some applications, the overall approach just didn’t sit well. There is knowledge in the enterprise about revenues and plenty of other sensitive metrics. And we’re all about harnessing knowledge. Won’t let the cat out of the bag yet, but we’ve cracked it! Details very soon.

Watch Andy McAfee, one of the most influential management science thinkers, weigh in on the importance and value of collective intelligence in the enterprise.

I wrote about our own experience with Crowdcast here and here. My main thesis was that, in order to be viable, collective intelligence tools must support decision makers first and foremost. After all, it’s their challenges we’re striving to address. This realization was the impetus behind our Executive Dashboard, which I describe here and also on our website.

This feature was unveiled last week during the keynote at SAP’s GRC 2010 conference in Orlando. The keynote was presented by Narina Sippy, SVP and GM of SAP’s Governance, Risk, and Compliance solutions. The theme of her address was arming yourself for risk in your business. She told a great cautionary tale about Toyota’s recent debacle, interviewed John Hummer of Hummer Winblad, and then introduced Ranga Bodla and our own Mat Fogarty to demo the products. They presented a compelling story about a pharma company managing its risks with GRC, while keeping a finger on the pulse of the organization with Crowdcast. The presentation couldn’t have gone over better.

We took advantage of our bully pulpit at the conference to announce Crowdcast’s formal relationship with SAP. Indeed, this development is hugely validating for our space in general and for Crowdcast in particular. I believe that our charter of supporting decision makers is a big part of the story.

Today, our CEO, Mat Fogarty, took to the stage to present at the Financial Forecasting and Planning Summit in sunny San Diego.

The goal of the Summit is to bring together leaders in the industry to discuss innovation and financial planning methods that are key to future business success. Presentations during the conference focus on new technologies and solutions, and how these developments are revolutionizing the modern business world.

For those of you following from home, Mat’s presentation, titled “The Game of Risk: Leveraging Collective Intelligence for Successful Product Introductions,” spoke to the challenges associated with planning and bringing a new product to market.

According to IDC, market leaders get 43% of their profits from new products. To ensure success, companies must have reliable insight into risks as well as opportunities. Mat discussed ways companies can leverage the wisdom of their employees to produce timely, unbiased, and accurate forecasts that inform key decisions around new product introduction.

If you are interested in more details around the conference (ongoing until tomorrow) – check out all the ways to follow the event here.

A couple weeks back I wrote about our “new mech” project, a wholesale redesign and reimplementation of our knowledge aggregation mechanism. In this post I will talk about why traditional project management techniques failed us and how Crowdcast helped us to recover.

As I mentioned last time, our initial attempt at running the project revolved around a project plan and regularly scheduled meetings. The project plan was standard — milestones, subordinate tasks, dependencies. Task owners came up with costing estimates. Since we were an experienced team, we added fudge factors liberally. Once the plan was in place, the team leads — engineering, product management,qa — agreed to meet weekly to track progress.

The alpha release milestone was critical, but comfortably far away, so the first few meetings were genial. But a week before ship it became ugly. Turned out that the port from our development environment to test (which simulates production) was in trouble. This was one of those on-going tasks. Deal was that as new APIs came on-line, they’d be ported for testing. Somewhere along the way, this stopped.

How could this have happened? How could it have taken us until the last minute to find out?

After the dust settled, we saw that the task reached 85% completion weeks ago and stalled (the problem was a serious bug in some 3rd party code, which was waiting for a patch). The engineer who owned the task, and indeed others, knew that the alpha milestone was at risk, but lacked a good way to give their knowledge voice. Since everything else was tracking well, intentionally or not, the problem that derailed the project was swept under the rug.

On reflection, it wasn’t a problem of scoping, project sponsor buy-in, staffing, or implementation. Rather, it was a problem of communication. That was when we decided to drink the Kool-Aid.

In the first post in this series, I described our experience with using Crowdcast to gain insight into risks around the next important milestone — the beta release — in the new mech project. Recall that the system rewards people for revealing rare and relevant information early. Moreover, since bets are placed anonymously, participants’ incentives are uniform and aligned. As bets come in, the system aggregates them in real-time and generates alerts when new information reduces the likelihood of success.

The first screenshot is of a piece of the console, which shows crowdcast summaries. The second is a part of the detailed view, which shows the distribution of beliefs as well as how they are tracking relative to target.

My bet on the beta crowdcast caused an alert to fire. It was an early warning for Huned, which gave him time to understand the root cause of the problem and make adjustments to keep the project on track. Next time I’ll put all the pieces together and present a complete overview of the platform that the new mech of ours enables.

Yesterday we announced an exciting partnership with Peer Insight, a strategic innovation consultancy that focuses on identifying emerging business opportunities. Our prediction markets platform will allow Peer Insight to tap the knowledge of their clients’ employees and translate it into crowd forecasts, or crowdcasts, of a variety of key metrics. Peer Insight’s recommendations, now backed by the collective wisdom of their clients, will confirm key business model assumptions, drive new solutions, and create new opportunities for increased revenue and growth.

Next month, we will be leading a webinar series with Peer Insight around harnessing collective intelligence in the enterprise. Stay tuned for more info and updates on how to sign up here on the blog.

For more information on our partnership, click here.

At TED, the speakers are as remarkable as their talks. In her blog post titled Transformative Transparency, Ogilvy PR’s Virginia Miracle called out Monitor 360‘s Doug Randall, who gave a presentation in which he described an on-going project Crowdcast is doing with Monitor 360 for the intelligence community around crowd sourcing insights from people on the ground to inform critical military decisions:

The difference between Data and Insight was the topic of a 5 minute TED U talk by Monitor 360′s Doug Randall. He told the story of the US Government having access to reams of data, but relying on on the ground, crowd-sourced insights to combine with that data to make decisions related to military strategy. Algorithms can not replace the value of human instinct. The key to culling the insights is asking the right questions and tapping into the right “crowd.”

So nice to see the concepts that we’ve obsessed over and worked to champion make a showing at an amazing event like TED.

We completely redesigned our knowledge aggregation mechanism — or “new mech” as it has endearingly became known here at Crowdcast — several months back.  The new mech is interesting for a number of reasons.  Here are two: it’s a novel and robust way to collect information (we’ve already filed two patents on it), and integrating it into theCrowdcast platform benefited a lot from managing the project using Crowdcast.  We will describe the new mech and our experience with using Crowdcast for managing complex projects in this and the next couple of posts.

The new mech was designed by our very own Leslie Fine and two UC Berkeley PhDs and implemented using MATLAB.  When it came time to hook it into the Crowdcast platform, we decided to use the MATLAB runtime rather than port the algorithms over to Ruby.  Among other benefits, this path would make incorporating mech changes much more efficient.  On the flip side, project logistics became quite hairy.  In addition to our dev guys here in San Francisco, we’d need to cozy up to the folks at MathWorks (authors of MATLAB) in Massachusetts and get much more tightly coupled to our math guys in Berkeley and a telecommuting systems integration consultant.

We finally got it working, but only after a painful false start.  Initially we managed the project the way we’d done it at other companies — Gantt charts and meetings.  That failed miserably.  Milestones slipped left and right.  People were stressed and unhappy.  Then we started to manage the project using Crowdcast.  The change to overall performance was astounding.  Will have more to say about this later.  First, here’s where Crowdcast came in.

Rev two of the project entailed creating a series of crowdcasts around our milestones.  Crowdcasts are accurate, unbiased predictions that incorporate insights of a diverse group of people.  Crowdcasts aggregate two components: “bets” on the outcome and a comments that explain participants’ rationales.

One of the crowdcasts was around the beta release date of the new mech.  Huned created the crowdcast and invited everyone involved to participate.  He specified a target date, which is not made available to participants, so that he could monitor performance over time.  In particular, the system would alert him if thecrowdcast moved too far outside of his target.

About a month back when the project was in full swing I had a chat with our integration consultant.  He’d just started load testing the system, but it kept falling over due to bugs in the MATLAB runtime (in fairness to MathWorks, we were pushing the envelope on the capabilities of some of MATLAB’s features).  I believed that I had rare and relevant information to share — the beta release was at risk because of MATLAB bugs — so I rushed to my desk to place a bet.

At the time, the crowd — all participants who’d shared their insights on this crowdcast before me — believed that beta would happen on January 15 +/- 7 days.  Given my information, I thought it’d be at least a week later, so I selected a range of January 20-31 and bet $5000 of my virtual money.  I also left a comment explaining what my point of view.  Relative to crowd beliefs, I had about a 25% chance of being correct, so the system offered me a payoff of about $19,200.  If someone were to place the same bet just after me, the odds would now be higher, and so the payoff offered lower.  Neat thing about this is that the system rewards those who not only have accurate information, but are early to reveal it.

Behind the scenes, Crowdcast infers my perception of the probabilities of various outcomes by looking at my bet range, wager amount, and total portfolio value. It then updates the crowd curve using this information. All this heavy lifting enables us to do some really amazing things. For instance, we can “slice and dice” crowdcasts, to analyze sentiment by department or region. In the next post, I’ll talk about how these capabilities helped us identify potential problems and course-correct to navigate around them.

On Tuesday, Brooke and I attended the Catalyst Girls in Tech conference. It was a wonderfully organized and star-packed event aimed at supporting the career development of women in the tech industry. Heather Harde, CEO of TechCrunch, keynoted, backed up by an impressive list of speakers and workshop leaders.

Two first impressions:

1. You know you’re at a women’s conference when the refreshments are smoothies and cucumber water.
2. There are an incredible number of amazing, confident and brilliant women in San Francisco.

To summarize for those who may have missed out, here are a few observations and highlights:

• My favorite talk was “The Art of Negotiation”, by Katherine Barr, a partner at Mohr Davidow Ventures. She focused on key negotiation tools originally created by the Harvard Negotiation Project, now taught in MBA programs all over the world. I was riveted. In my experience, women dont always stand up for what they deserve or self promote in the same way men do (for more thoughts on this, check out Clay Shirky’s recent controversial blog post). Katherine’s enlightened approach was both sensitive to this and forceful. Interestingly, I noticed that in the negotiation exercises, women frequently converged on tit-for-tat strategies, perhaps more so than we would have seen in a male-dominated room. (Of course, this was not the strategy I employed when besting Brooke in 5/5 exercises — sorry Brooke, don’t mess with a game theorist!)
• Most women we spoke to who worked at startups were in the B2C space. One attendee hypothesized, and I agree, that this is because women are often the target audience of B2C products and services, and people founded or joined companies geared towards issues they feel strongly about and are engaged with in their daily lives. This is unfortunate though, as, “ninety per cent of the fastest-growing companies in the country sell to other businesses” (Malcom Gladwell, Jan 18, 2010, New Yorker) I hope B2C doesn’t manifest as the women-led startup ghetto.
• While the business playing field seems to be evening out for men and women, the conference was a thought provoking experience in determining how this is playing out in the tech sector. There seemed to be a pervasive attitude in the room that we should ignore potential differences, and assume that in business men and women are the same. To me, this is a shame. I embrace the fact that men and women have different work, negotiation, social and management styles. There are advantages to each, and we should honor and leverage that to best advantage.

I loved learning from and networking with women tech community, and witnessing their broadening scope as professionals and innovators. A huge “thanks” to the women at Girls In Tech who are working hard to propel careers and raise awareness of women, not only as key contributors to this industry, but as leaders.

In the New York Times’ weekly Corner Office column, the January 16th interviewee was Cristobal Conde of Fortune 500 company, SunGard. He spoke to the collaborative management methodologies that had been instilled at his company and how they altered their day-to-day workflow. Among other tactics, Conde mentions cutting back on micromanaging and using Yammer, a Twitter-like service for enterprise, meant for internal communication. Conde points out that while this is superior to top-down management techniques, it’s really about time – these days everybody has identical access to information, therefore everyone should, essentially, have a say.

In response to these ideas, Enterprise 2.0 pioneer and MIT Principal Research Analyst, Andrew McAfee, took to his blog to highlight some parts that really stuck with him. Andy’s synthesized version of the article really spoke to us and the ideas that we find valuable at Crowdcast. Andy highlighted a couple of values that we think are essential to smart business: breaking down hierarchies to unclog communication and fostering collaboration through peer effects, which allow people to get recognized by their peers for what they do rather than by their organizational rank.

Companies are beginning to understand the importance of communication within the ranks of their organization, not only to improve workflow, but also to improve access to employee intelligence. From there, managers can start to really monitor the pulse of their company. Ultimately, harvesting wisdom and gleaning well rounded insight is a competitive advantage — a very timely and relevant conversation to have as companies look ahead into 2010.