What Are IDEAs Made Of: Assumptions
Mike Rea
IDEA Pharma
I assume that you're over the age of 18. I have reasons to believe that assumption is probably right. I assume that 99% of all readers of this column are over 18. I am 95% confident that 99% of the readers are over 18. I am 100% confident that no readers are younger than 5, and 90% confident that none are older than 80. I assume the average age of the readers is 45. I am pretty uncertain about that one, if you want me to be right to plus or minus one year.
How many assumptions in that opening paragraph? Let's assume...
Every assumption in that paragraph has an underlying assumption: that a basic reading age is required to read this column (an assumption that seems unfounded, given the obviousness that pervades it every fortnight…), that readers have internet access and an awareness that there is such a thing as a pharmaceutical industry, so are probably of working age and reasonably IT-savvy. Those are the robust assumptions. But, what does 'robust' mean, and who decides?
Let’s imagine if some (rather odd) business idea were to be hatched where I would be paid for visits near to publication date where I assume that 90% of all visits to this page would come in the first 5 days, 5% the week after, and 5% for the whole of the rest of the time that it sits here. However, I would be on shaky ground were I not to qualify or quantify those assumptions. Even if I showed that those figures were exactly true for every previous column, it would be an assumption that previous history would predict future performance. If I polled 100 potential readers, and 90 said that they would read it in week one, five said week two and five said 'oh, some time later,' it would be an assumption that they knew how they would behave in the future. In that trivial example, it might matter less that I was accurate than that I was roughly right. My bank wouldn’t care less if it were 85% in week one than if it were 10%. So, I could probably take a view on those sub-assumptions – previous history has a repeating pattern, and why would people not know their pharmaphorum browsing behaviour? There is reason to believe they are reliable indicators.
However, the bank may be real sticklers for accuracy and indicate a penalty for a margin of error of 2%. My level of confidence in the assumption would need to be remarkably high and would need to stay high for each of those sub-assumptions. I would be looking for all kinds of evidence before accepting the deal.
"In early phase, being roughly right about opportunity is much more important, whereas revenue forecasting for on-market brands needs a higher degree of accuracy."
Those situations are entirely analogous to the way that assumptions work in pharma. In early phase, being roughly right about opportunity is much more important, whereas revenue forecasting for on-market brands needs a higher degree of accuracy. Unfortunately, that does not lead to assumptions being treated that way.
The joke that 'assume makes an ass out of you and me' is both corny and somewhat revealing. Assumptions have two dimensions: the assumption itself and the level of confidence that it is right. They are also pack animals - assumptions rarely come in ones. It is critical that the assumptions set is comprehensive / exhaustive. Achieving 100% confidence in a set of sub-assumptions will prove fruitless if a key assumption is not included, and its inclusion changes the overall conclusion, like assuming that both rabbits you bought as pets were indeed female.
There are all kinds of pitfalls in assumptions. A recent example: we were asked to examine the potential erosion of a brand post-patent expiry, and given the 'assumption' that the first year drop would be 10%. If I assume 10% and make planning decisions on that basis, I will be cross (and sacked) if it turns out to be 90%. On the other hand, if it turns out to be 8%, I'm probably not so concerned. Change that original assumption to 30%, however, and it will matter if the erosion turns out to be 10%, as there was a whole lot of revenue that I could have incorporated into R&,D budgets.
So, sticking with the 'assumption' that the first year erosion is 10%, this is hopefully underpinned by a host of sub-assumptions that are increasingly robust further down – an 'assumptions pyramid' where only the 10% is visible at the peak. Unfortunately, sub-assumptions are assumptions themselves. And multiplying levels of confidence together only reduces confidence. While this could seem like a race to the bottom, where granular assumptions continue unravelling until it gets down to the level of 'Dr F Charles, of Boise, Idaho indicates he will switch to the generic formulation, and has always done so in previous cases, whereas Dr K Miller of Boise, Idaho…' it is always more important that the assumption set is exhaustive in breadth than that it is fully elaborated in depth. All numbers are assumptions, but all assumptions aren't numbers.
Using the writers'/ speakers' favourite cliché, a dictionary definition… An informed assumption (interesting choice of word, that 'informed'): supposition, presumption, belief, expectation, conjecture, speculation, surmise, guess, premise, hypothesis, conclusion, deduction, inference, rare illation, notion, impression. Even ignoring that no-one knows what an illation is, even less a rare one, each of those dimensions of 'informed' carries a different character. There are guesses and deductions, there is belief and conclusion. Fortunately, some of those levels of 'informed' are more easily trusted than others. "Trust me, we used Monte Carlo simulation, and we've been doing this for 10 years" when attached to a black box number, is the refuge of the scoundrel.
"'Roughly right' is an underused concept in pharma."
'Roughly right' is an underused concept in pharma. Preference is given to numbers that run to two or three decimal places - they sound more accurate, have what is now called 'truthiness' (sounds true just because of the way it is presented). Several examples have been encountered where NPV has been assessed with 50 separate assumptions, but where one 'assumption' is given as '1.0% share'… Seems reasonable, and probably unimportant, until the scale is investigated, and it runs 0.5, 1.0, 1.5, 2.0, etc… Call that little number wrong, even by one point, and the number you get at the end of all 50 'timeses' is out by either +100% or -33%. Yet that red flag was raised in no part of that calculation. Even stranger, one company found that different therapeutic areas and brands across the company had produced 12 different estimates of the number of people over 65 in the US in the year 2015, buried away in their individual forecasts - there was no standard company assumption for that critical number.
So, assuming you've got this far, it may be reasonable to ask 'so which number do we trust?' The only reasonable answer to that is, 'it depends…' Depends on what you're going to do with it, and how 'right' it needs to be.
About the author:
Mike Rea is a Principal with IDEA Pharma, who enjoys taking a look outside the industry to learn how it can think differently. For direct enquiries he can be contacted on mike.rea@ideapharma.com and for more information on IDEA Pharma please see http://www.ideapharma.com.
Do we make too many assumptions in pharma?
