Comparative effectiveness evidence: Comparing apples and oranges?
Jeanni van Loon
Mapi Values
Development plans typically identify comparators that will meet regulatory needs and go some way to meeting the needs of other stakeholders in key markets. However, there is increasing recognition that it is neither possible nor economically viable to generate head-to-head comparisons with all the relevant comparators. That said, the emergence of cutting edge techniques with increasing acceptability to regulators and payers now means that clinical evidence should comprise randomised controlled trials (RCTs) and indirect treatment comparisons. Such evidence from indirect comparisons can complement and reinforce that from RCTs. Using this approach will increase the efficiency of R&,D expenditures and also go some way to meeting the increasing demands for evidence on effectiveness compared to alternative treatments
"Using this approach will increase the efficiency of R&,D expenditure and fills the gap to meeting the increasing demands for evidence compared to alternative treatments."
Recently, we have seen an increased use of observational data sources by national authorities and Health Technology Assessment (HTA) bodies, but mainly at a later stage - informing reimbursement decisions some years after launch for example. Pre and peri-launch, decision makers are faced with the challenge of making a decision while there is a lack of comparative data with all relevant comparators, so pharmaceutical companies need as much evidence to hand as possible. In the absence of relevant head-to-head RCT comparisons (i.e. direct treatment comparisons), an indirect treatment comparison (ITC) can help to overcome this challenge.
What is an indirect treatment comparison?
An ITC uses techniques whereby existing data from a variety of sources are compared, providing relative efficacy data for different comparators simultaneously, and without actually conducting a direct head to head trial. The example below from Cope et al. 2011 illustrates the concept:
A new COPD treatment demonstrated superior lung function improvement to placebo in RCTs, however no head-to-head RCT data were available for a comparison to ‘fixed dose combinations’ - where an inhaled steroid is added to the bronchodilator. As a high percentage of patients receive this combination as first line therapy it was relevant to evaluate the effect of the new drug relative to combination treatment – in this case via an ITC.
A systematic literature search identified all the relevant evidence and included 15 placebo-controlled trials. Studies were assessed in terms of design and patient characteristics to ensure that these studies were appropriately comparable. The individual study results for the different endpoints with their uncertainty were presented (e.g. change of baseline of the trough forced expiratory volume in 1 sec (FEV1) at 12 weeks was one of the main outcomes of interest). The study results were synthesized in a network meta-analysis (NMA). A NMA can be seen as multiple meta-analyses of different comparators combined in one analysis. Since all studies in the network were comparing treatments to placebo in the NMA, the placebo effect was subtracted from the change from baseline data for each active treatment to assess the relative treatment effect of the intervention. Doing this for all studies, the relative efficacy between each of the competing interventions can be obtained. In this way the new therapy was indirectly compared to fixed dose combinations. In such analyses the differences in patient characteristics across RCTs (e.g. proportion current smokers against ex-smokers) can of course influence results (causing bias), and therefore appropriate methods and adjustments have to be used. The team at Mapi Values comprises experts in the field, who can advise on the potential risks.
"An ITC uses techniques whereby existing data from a variety of sources are compared, providing relative efficacy data for different comparators simultaneously, and without actually conducting a direct head to head trial."
It is of utmost importance that all evidence is considered in such analyses, with the starting point of any ITC being a thorough systematic literature review. The identified studies have to be evaluated from a clinical perspective in terms of comparability - we have to compare like with like.
It is often argued that ITCs are only needed when direct comparisons are not available, but it is important to realize that both direct and indirect evidence from RCTs can be included in the network. All these studies contribute to the so-called ‘total body of evidence’. Even when the results of the direct evidence are compelling and clear, combining them with the results of indirect estimates (a mixed treatment comparison (MTC)) incorporates all available evidence.
Potatos or potatoes
Any comparison of outcomes between interventions is called ‘relative effectiveness’ in Europe, whilst in the USA the term ‘comparative effectiveness’ is in its infancy. These terms do not make any distinction whether the evidence base consists of RCTs designed for regulatory purposes (i.e. efficacy trials), or ‘real world’ pragmatic randomized studies (effectiveness). The terms ITC, MTC, and NMA are currently being used interchangeably.
I would point interested readers to the International Society of Pharmacoeconomics and Outcomes Research’ (ISPOR) guidance on the interpretation of ITC and NMA, which is there to assist policymakers &, health care professionals in using findings for decision making.
"...a growing number of policy makers and health care professionals in Europe, Australia, and North America accept ITC data for market access decision making,"
The value and acceptance of indirect comparisons
It’s a fact that a growing number of policy makers and health care professionals in Europe, Australia, and North America accept ITC data for market access decision making, some even require such analyses.
Whilst this innovative use of evidence is predominantly used by HTA and reimbursement bodies with an interest in assessing the clinical and cost effectiveness of new technologies, regulatory bodies such as EMA are also acknowledging this kind of evidence in regulatory approvals.
“The applicant has provided evidence derived from a meta-analysis…..although the strength of evidence derived from this meta-analysis is considered inferior to results of a clinical trial, the conclusion that tocilizumab and anti-TNF medications appear to have comparable efficacy is acknowledged and supportive….”
Assessment report EMEA, 2010
About the author:
Jeanni van Loon is Senior Vice President of Mapi Values Netherlands and Global Lead for the company’s Clinical &, Economic Evidence discipline.
Mapi Values is a global healthcare consultancy business with offices in the USA, UK, France, Netherlands and Japan. Their passionate people help bring healthcare products and services to the market successfully by investigating and developing scientific evidence to understand the benefit of healthcare products and services to patients, clarifying and communicating the value to decision-makers.
De Molen 84, 3995 AX Houten, The Netherlands | Tel: +31 (0) 30 63 697 63
www.mapivalues.com
What is your opinion of indirect treatment comparisons?
