Drug Baron

Evidence that “pick the winners” is precisely the wrong strategy

| 12 Comments

The asset-centric platform at Index Ventures is built around stringency.  The entire concept arose from the realization that drug discovery and development is a complex process, in a low-validity environment (meaning there is often too little data available to make decisions), where the amount of capital required is steeply graded from little to vast sums.

These parameters together imply a requirement for a filter that is more tolerant of false negatives (a decision to kill a project that would, if continued, succeed) than of false positives (a decision to continue a project that will fail later).   We call that stringency.

But success is not just about ratcheting up stringency – its about when to apply it as well

It turns out that the “gradient” of filter stringency with time is the major difference between the “pick the winners” strategy, that has some vocal supporters in the pharma industry, and the alternative that DrugBaron calls “kill the losers”.

Moreover, stringency is easier to apply in theory than in practice.  As DrugBaron has noted previously, cultural factors bias the kill/continue decision making process in favour of continuing – which automatically lowers the filter stringency.  And these cultural factors can be very strong indeed.

Our entire asset-centric platform has therefore been constructed to build in stringency, and to counteract the cultural bias towards continuing.  This is achieved in at least three ways: by stripping away the complexity due to having more than one asset in a company, everyone involved can see more clearly the risks and benefits of continuing with a single asset.  And at the same time, by providing an environment where entrepreneurs are protected from the consequences of the kill – so long as they have done “the right things” with an asset. And perhaps most importantly, by ensuring each entrepreneur has a single shot at a time to achieve a significant upside, they are incentivized to ask repeatedly if their current project really is the best use of that singular opportunity.  Equally importantly, its built to grade that stringency from low to high as the capital at risk increases.

But, as DrugBaron commented only recently, the benefits of such a model remain substantially theoretical – just like the arguments against “pick the winners” as a strategy.  These strategies are relatively new, so data is difficult to come by.  The asset-centric concept has only been mature from a handful of years, and while the first exits are certainly encouraging, the jury remains out as to whether asset-centric investing really can deliver a step-change in return on capital deployed. The “pick the winners” strategy in pharma is even more recent.

Against this background, it was certainly exciting to see the conclusions of a recent analysis by Boston Consulting Group (BCG) looking at factors that predicted success in pharma R&D.  It provides actual evidence for the principles of asset-centric investing, and against “pick the winners” – the key, as we suspected all along, is not only high stringency filters, but calibrating stringency in line with capital at risk.

According to BCG, the most important factor turns out to be number of project kills before the end of phase 1.  That would appear to be a biomarker for filter stringency – having lots of cheap, early-stage projects and then ruthlessly killing most of them.

Its also consistent with the notion of “kill the losers” in direct contrast to the “pick the winners” strategy

For the avoidance of doubt, these two strategies are truly opposites (though they sound kind of superficially similar).  “Pick the winners” is about selecting projects you are confident will succeed and then investing heavily in them; “kill the losers” instead focuses on the projects that are very obviously not going to be successful and kills them as early and cheaply as possible.  The difference is what happens to the majority – the projects that are neither obviously doomed nor already destined for the stars.  “Pick the winners” leaves them on the table – since all the funding has been dedicated to the supposed stellar projects.  “Kill the losers” though keeps them going – because they know its impossible to pick the actual winners until much later in the process.  But, critically, it keeps them going only to the next kill/continue decision point, and only on the minimum amount of capital necessary to survive.

Those pursuing the “pick the winners” strategy therefore start fewer projects (applying their most stringent filter at the outset).  The ones they do back, they back large – and irreversibly.  So the number of early kills are small.  Those in this camp keep control of costs by reducing the total number of projects in the portfolio.

On the “kill the losers” side of the street, the activation energy barrier to starting a new project is low – as long as it has the potential to be a big success, its time to start collecting data.  But those experiments are carefully designed, low cost experiments to inform the kill/continue decision.  Nothing is done that would not be actionable information in this way.

The result is a lot of projects get started, and the majority have to be killed as the data accumulates.  That shows up as early stage project kills in the BCG data set.  Those in this camp attempt to control costs by keeping the amount spent per project to an absolute minimum but keep the number of projects as high as possible.

The difference is not, therefore, about stringency – but about when you chose to apply the stringency as capital is expended

“Pick the winners” applies maximum stringency at the start, before money is spent, and when the amount of information to feed into the decision is at its minimum.  Stringency is lower once the project has been anointed as a winner, and consumes capital attempting to realize that vision. In marked contrast, “kill the losers” gradually turns up the stringency of the filter in proportion to the capital at risk.

In the pharma world, then, the BCG dataset suggests that reality matches theory: “kill the losers” is conceptually stronger, and – we now know – associated with greater success in the long run.

Venture capital and biotech start-ups are a very different beast to pharma.  But it is tempting to extrapolate from the BCG dataset, and read from their analysis support for the asset-centric investing model too.  There are many similarities between asset-centric investing – at least as we practice it at Index Ventures – and “kill the losers”.  The key commonality is the provision of small tranches of capital to support a sequence of kill/continue decisions (with no ‘spare’ funds to expend on nice-to-have information that isn’t directly contributing to the quality of those go/no-go filters).

The difference between “kill the losers” in pharma and asset-centric VC investing is probably only one of magnitude.  Our idea of keeping costs low is typically at least an order of magnitude lower than the same concept inside a pharma company.  But with the latest BCG analysis in hand, there is at last solid (if not cast-iron) evidence to keep those of us attracted by the theory of asset-centric investing sticking firmly to our hymn-sheet.

 

 

  • Simon Pledger

    interesting article!

  • Simon Pledger

    Interesting article this!

  • http://www.hollyip.com/ Suleman Ali

    Whilst meta-analysis like this is very useful, I’ve yet to see a scientific explanation of what we didn’t know about a drug and so got it wrong. Perhaps every project that fails fails for different scientific reasons, and so no overall conclusion can be reached. Or perhaps such data would be a commercial secret. However it would be interesting to see if there was a pattern in what went wrong with the science

    • nlesh

      You are spot in here, although I like the asset centric model, it does not mitigate the risk of any drug failing and yes there are so many different parameters determining failure. Furthermore the current climate pushes you as an entrepreneur towards single, preferably late stage assets for another reason : namely available funding, suffocating innovation to my mind

  • Ian Skidmore

    David,
    Although I led a project in my old company, many years ago, actually called Picking the Winners the philosophy was essentially what you describe, Kill the Losers and invest in the rest. Fear of failure is often the driving force behind continuing to back a loser. The message here is The only failure is failure to make the right decision at the right time!

    Regards

  • davidgrainger

    Thanks for the comment Ian. Completely agree – I will keep that as a quote that sums up drug development: “The only failure is failure to make the right decision at the right time”

  • JacktheRipper

    After many years in the Biotech world I have reached the conclusion that risk is much like energy – you can’t create or destroy it, you can merely move it around. And so it is with the single asset centric model.

    There is a contradiction at the heart of your model. You (and I believe with good reason) promote the concept of being indifferent to the outcome of any given project – expose them to the wilderness of data and only the fittest will survive. But although you decry the ability of organisations to pick winners your model requires the people that work for these single-asset companies do i.e. the one project/product they are going to hook their future to.
    People tend to dowhat they are incentivized to do and if your wellbeing is tied to continued employment, then continuing your employment is what you will strive to do. This is a ‘corrupting’ influence on the integrity of the outcomes (a.k.a turkeys not
    voting for Xmas). Look how many biotech companies become functionally single
    asset companies as they roll-up the company behind their lead program – and how
    many times has that ended in tears?

    If it makes sense for VCs to diversify their risk – then why doesn’t it make sense for the employees?

    • davidgrainger

      On risk: agree it can only be moved around, and the aim of drug development has to be to move as much risk to the front end as possible – so the largest risks are discharged first, while the capital at risk is lowest.

      Your second point is specifically about the asset-centric investing model as a tool to avoid the problems I describe. You are 100% correct that in isolation, the individual tied to an asset-centric company would be incentivized to continue rather than kill ‘their’ project – which is why we dont operate them in isolation. Rather, that person can abandon their project any time and start a new one (assuming only that they are doing the right things in progressing the asset). So they are constantly asking the question: does the chance of success of this one project remain higher than the chance of success of a random new project beginning from the very start? They are therefore incentivized to call time on the project as soon as its chance of success dips below that threshold.

      Finally you raise a point that I raised myself many times. Why should VCs, companies, investors all seek diversification yet the entrepreneur be forced to hang his hat on a single project? He doesn’t – the entrepreneur has a choice whether he wants to work in the asset-centric model or a conventional “fully built” diversified company. The reason he may chose the former though (we hope), is because the overall chance of success is so much higher precisely because of the stringency the model imposes. Better to be nailed-on to a single asset with a high chance of winning, than diversified across many that will all fail.

  • http://vishrasayan.blogspot.in/ Murali Apparaju

    Thanks a lot for sharing this.

    You have mentioned that while this model has been developed primarily for applying to the 1) early stage venture fund spread across a portfolio of (single asset?) organizations by Index Ventures, it is equally applicable to 2) the drug discovery program with multiple pipeline candidates of a big/ mid pharma company.

    At the end of my first read though, the model appeared a lot more suited to the drug-discovery program of a big-pharma!

    My above surmise is based solely on how you mentioned/ worded the ‘costs’ at each stage – to say it appeared like it was the ‘total cost’ for advancing that stage & not a ‘portion of the total-cost’ that a typical venture firm would’ve risked to fund (i.e. assuming there are other venture partners participating in each funding round at that very decision stage….) – my question hence is;

    - Is the expectation that the model will be run by the lead VC based on ‘total cost’ at each stage of the asset in question? If yes, how easy or tough it is to make the venture partners to get on one-line while making decisions on what to kill & what to keep?

    My other question is if this model will make more sense if applied specific to each therapeutic segment than a potpourri of indications? If yes, how easy or tough it is for a signle VC or a group of Venture partners to agree & disagree on the decisions?

    • http://vishrasayan.blogspot.in/ Murali Apparaju

      right comment on a wrong post – my apologies.

  • http://vishrasayan.blogspot.in/ Murali Apparaju

    Dear David, I pasted my comments on your other article here by mistake – please delete he same. (guest comments starting with “thanks a lot for sharing this”)

  • Pingback: Saving the winners to improve pharma R&D | Pharmagellan