Forecasting Clinical Ops Metrics

Posted on February 3rd, 2012 by mat

Next week we will be attending the SCOPE conference in Miami. Primarily serving clients in clinical operations in pharma and biotech companies, the conference is concerned with improving metrics, forecasts and management of clinical trials.

Building on our success with Genentech, we are looking forward to learning more about this opportunity and how Crowdcast can help bring drugs to market more efficiently.

Did you know:
- The average drug costs over $1B to get to market?
- Every day of delay costs approx $1.4MM (assumes $500MM revenue per year)?
- Only 6% of clinical trials are completed on time, and 72% of trials run over schedule by more than one month?
- Clinical trials last 42% longer than expected in Phase I, 31% longer in Phase II, and 30% beyond planned deadlines in Phase III – all because of recruitment delays?

If pharma companies could accurately assess and manage their clinical trials, especially around enrollment of patients, we assume that they could deliver their drugs to market faster and significantly increase profits.

The problems they face are myriad. They work with many sites (clinics, hospitals etc) with few patients per site. The trials are often international with varying regulations. Much of the work is outsourced to CROs, making a direct line of site with the clinic harder. In addition, the conditions change with alternative trials overlapping and competing for the limited number of potential patients required. Finally, the leader of the project and the CROs may have reason to be optimistic to ensure funding and selection of their particular site.

What they need is a way to aggregate accurate, real time forecasts, from a diverse set of players and reward for accuracy and insight :)

Transparency Management – Control Access to the Crowd Forecast

Posted on December 9th, 2011 by mat

Crowdcast is a powerful tool, producing unbiased, timely forecasts of your most important metrics. Controlling access to these crowd forecasts is key as the metrics may well be different from the “official” forecasts. For example, in a project management setting, the crowdcast may indicate that a key milestone will slip.

Planned date = Jan 15, Crowd Forecast = Feb 10….. DANG!

The realistic crowd forecast is important for managing the project and shaping customer expectations. However, this sensitive information needs to be handled well otherwise it could cause issues around employee morale or with partners or investors.

This issue around transparency of the results has really restricted the use of prediction markets in the enterprise, but Crowdcast has developed and patented a solution.

We have now enabled 3 transparency settings to allow our admins to share as much as they want with the participants.

Three Settings:

You can share everything, share comments, or share only the question.

Share with your team: This is the default setting and shares the Crowdcast Curve, the comments, the beliefs behind those comments. Use this for metrics which are ok to be shared with your team.

Share comments and anonymous individual bets: With this setting, the Crowdcast curve no longer appears. Use this setting for metrics where the commentary and social side is important, but where the metric may not be shared.

Share the question only: This shows only the question. Payoffs are ‘location invariant’ — a user can’t determine the shape of the Curve by moving their bets around. Comments are hidden. Use this for your most important metrics.

For all the settings, the Admins and the users with “Executive” access can still see the crowd forecasts on the Dashboard page.

Project Good Judgement Update

Posted on September 12th, 2011 by leslie

Earlier this summer I posted about our project with The Good Judgment Team on pushing the boundaries of forecasting methodologies.  I’m thrilled to say that a week ago, we went live, with more than 3,000 participants worldwide making forecasts and, in some cases, arguments, about the future of political, economic, and technological trends.  To date, we have well over 20,000 forecasts in the system across more than dozen experimental permutations, and it’s growing every minute.

The project represents one of the first controlled and scientific studies of forecasting training and methodologies.  We’re honored to be a part of it, and look forward to sharing the results with you as we learn.

Crowdcast and the Efficient Forecasting Frontier

Posted on June 27th, 2011 by leslie

Crowdcast is all about bringing the power of collective intelligence to bear on real business metrics.  Most of the time we’re practitioners, working with clients to apply our technology and knowledge to their challenges in project risk management and sales metrics.  Every once in a while, though, we get a great opportunity to step back and challenge the state of the art.  Our current project, joint with an all-star cast of academics and funded by IARPA is such an opportunity. 

The US Intelligence Agencies have thousands of highly-trained analysts with access to a vast store of privileged information.  However, they have great difficulty efficiently synthesizing and surfacing this info actionable information.  Often the “a broken watch is right twice a day” phenomenon encourages analysts to stick with the status quo rather than flag new conditions.  These kinds of forecasting behaviors have real costs in dollars and in lives.

Despite its importance in modern life, forecasting remains (ironically) unpredictable. Who is a good forecaster?  How do you make people better forecasters?  Are there processes or technologies that can improve the ability of governments, companies, and other institutions to perceive and act on trends and threats?  Nobody really knows.  The goal of the Good Judgment Project is to answer these questions.

We will systematically compare the effectiveness of different training methods and forecasting tools in accurately forecasting future events.  We also will investigate how different combinations of training and forecasting work together.  Finally, we will explore how to more effectively communicate forecasts in ways that avoid overwhelming audiences with technical detail or oversimplifying difficult decisions.

Over the course of each year, forecasters will have an opportunity to respond to 100 questions, each requiring a separate prediction, such as “How many countries in the Euro zone will default on bonds in 2011?” or “Will Southern Sudan become an independent country in 2011?”  As the project evolves, we’ll iterate the mechanisms and the training, arriving at an idealized and verified system.

The Good Judgment research team is based in the University of Pennsylvania and the University of California Berkeley. The project is led by psychologists Philip Tetlock, author of the award-winning Expert Political Judgment, Barbara Mellers, an expert on judgment and decision-making, and Don Moore, an expert on overconfidence. Other team members are experts in psychology, economics, statistics, interface design, futures, and computer science.

If you’re interested in participating, we’d love to have you.  Head on over to our registration site to sign up.  We’re looking forward to sharing the results with you, and leveraging the valuable insights of The Good Judgment Project with our clients.

Hacking Work – top 100 Disruptive Heroes

Posted on May 5th, 2011 by mat

Collective intelligence will, in time, radically change the way information flows in a corporation. To the good of the corporation and its stakeholders. However, this change is powerful but we are early in the development of enterprise collective intelligence. The pioneers of today will be the enterprise leaders of tomorrow. We do not intend to disrupt, but collective intelligence will change the enterprise.

Tech Review – Top 50 Most Innovative Tech Companies

Posted on February 24th, 2011 by mat

Congrats to the Crowdcast team.  We were chosen as one of the world’s most innovative companies by Tech Review, MIT’s tech mag.

http://www.technologyreview.com/business/32397/

This recognition reinforces our proposition that corporations have the information they need to make decisions.  However, current methods of gathering and transmitting data up the hierarchy can lead to biases and mistakes.  Collective intelligence can solve this, and Crowdcast is at the leading edge of making this happen.

Congrats again team.

Mat

MIT’s Tech Review = How Bets Among Employees Can Guide a Company’s Future

Posted on December 10th, 2010 by mat

Internal prediction markets enable colleagues to wager on the fate of crucial projects and the success of products in the pipeline.

  • Thursday, December 9, 2010
  • By Chris Taylor

The need to predict the future, as exciting as it sounds, crops up in corporate life in terribly mundane ways. Case in point: large videogame companies need to know where to put their marketing dollars many months before they complete their games. Inevitably, some games will be stinkers, hardly worth the investment of an ad campaign. But how do you know which ones?

Here’s how one very large videogame company used to guess the answer: its marketing people would predict the score their games in progress would garner on the website Metacritic, which aggregates game reviews. But why would the marketing people know more than the game’s developers?

Read more on “MIT’s Tech Review = How Bets Among Employees Can Guide a Company’s Future” »

Jay Margolis Joins Crowdcast Team as VP Engineering

Posted on November 1st, 2010 by mat

Welcome Jay.

Jay Margolis will lead the engineering and operations groups at Crowdcast.  Previously, Jay was Senior Director of Engineering at Jaspersoft, where he ran product development of the Jaspersoft Business Intelligence Suite, bringing powerful yet easy-to-use web-based ad hoc reporting and analytics to everyday users.  Before that, Jay held leadership positions at Primavera Systems and Evolve Software, developing Project Portfolio Management solutions used by large professional services and IT organizations to improve decision making across large numbers of projects and resources.  Jay has over 20 years experience in enterprise software development and holds a BS in Applied Mathematics from Carnegie Mellon University.  (he also plays the guitar real well!)