Three companies submitted economic models to NICE: Astellas, Novartis and Bristol-Myers Squibb.
Novartis’ submission
Novartis, the company that produces EVL, submitted a simulation model of an individual patient’s health experience for the lifetime remaining after renal transplantation in the English NHS. The following treatments were evaluated for a group of simulated people of mean age 45.7 years (SD 12.7 years), mean weight 70 kg (SD 10 kg), 68.5% of whom were male, and mean MDRD eGFR 9.03 ml/minute/1.73 m2 (SD 7.9 ml/minute/1.73 m2):
The model was specified as monthly transitions between six health states:
Moving between these states is associated with changes in direct health-care costs, whereas HRQoL (utility) changes are accounted for transitions between the states of having a functioning graft to a failed graft, and from any of these to the absorbing state of death. In addition, the model accounts for the changes in mortality risks, utilities and monitoring costs (outpatient specialist visit) with renal function. Although the costs associated with AEs emerging following transplantation were measured for six type of events [proteinuria, BK virus (BKV) infection, CMV infection, hyperlipidaemia, wound and hypertension], only for two of these was the loss of utility measured in the analysis (proteinuria and hypertension).
The model assumes that AR may happen up to 3 years after a transplant, and applies the same probabilities of this type of event to first and subsequent transplantations. The probability of chronic rejection (i.e. graft failure) is independent of renal function in the model. Once a patient’s graft fails, dialysis is started and given until the time a new transplant is received, which is determined by a random normal distribution process with mean of 36 months (SD 12 months). This feature of the model is what gives it its discrete event simulation nature.
The model allows different rates of change in renal function (eGFR) between the first year (during which they are specific to the immunosuppressive treatments) and the second, third and subsequent years, when the rate of eGFR change is common to all treatment arms in the model.
The model parameters for the EVL and MPA regimens were populated with efficacy and safety outcomes at 12 months from the study by Tedesco-Silva et al.,359 a multicountry trial that compared EVL 1.5 mg/day with mycophenolate acid 1.44 g/day in people receiving a primary kidney-only transplant in the period October 2005–October 2008. The values for the TAC regimen were obtained from a trial reported by Larson et al.,154 which compared TAC with SRL in people receiving a kidney-only transplant (79% of whom were primary transplants in the TAC arm) in the period April 2001–January 2004 in the USA. The source of the efficacy and safety data for the MMF regimen was the multinational trial report by Vítko et al.,177 which compared EVL with MMF in primary transplant patients who were recruited between August 1998 and August 1999.
The indirect nature of the relative efficacy data used as inputs to the cost-effectiveness model of the three comparisons submitted by Novartis presents some problems for valid estimation. In addition to the different dates when the trials were conducted and the type of transplant (primary only or mixed) for the EVL–TAC comparison, there were differences between the two studies in terms of the use of induction. Tedesco-Silva et al.360 reported that participants in their trial of EVL were administered two BAS 20-mg doses: one within 2 hours before transplantation and the other at 4 days post transplantation ‘or according to local practice’,360 whereas Larson et al.154 reported that all people received thymoglobulin 1.5 mg/kg/day on days 0, 1, 2, 4 and 6 post transplant. The sample of TAC participants was also slightly older but more balanced in terms of sex, and had a higher proportion of living donor transplants. The major issue, however, is the fact that the actual amount of TAC use in the efficacy trial was different from the dose used to cost the same regimen in the model. Larson et al.154 report that the TAC was started at a 3 mg twice daily. The estimated mean daily dosing at 1 year, separately reported for the first 59 people randomised to TAC, was 6.3 mg per day (SD 0.9 mg per day).361 The model, however, applied costs to the TAC arm at a quoted BNF recommended dose of 0.25 mg/kg/day for a group of individuals of 70 kg mean weight, thus resulting in a mean daily dose of 17.5 mg, which is considerably higher than the actual drug use that corresponds to the efficacy outcomes used by the model. The dose behind the TAC drug acquisition costs used in the Novartis submission is also larger than the mean daily doses for immediate-release TAC reported by Tedesco-Silva et al.,349 which Astellas adopted in its submission, and which are consistent with the report of Dean et al.361
In relation to the data sources for the comparison of EVL with the MMF + CSA regimen, the trial samples differ in terms of the period covered by the study and the country mix. The proportion of cadaveric donors transplant recipients was 46.6% in the EVL group compared with > 90% in the MMF + CSA regimen.150 Moreover, the MMF regimen was given without induction therapy, in contrast with the trial that provided the outcome data for the EVL model arm.359 The same issues applied to the comparison of MPA with MMF + CSA, as the data source for MPA was the same trial as that for EVL.107
Costs
Immunosuppressive costs of the MPS + EVL treatment regimens were based on the dosing protocols of the individual trial that was the source of efficacy data, whereas the costs of drug acquisition for the comparators, that is, the TAC and MMF + CSA regimen, were based on BNF-recommended starting dosages. Other health-care costs included the costs of monitoring GP visits, which increased with higher CKD state. The cost of an AR event was taken from that reported by McEwan et al.310 The annual costs of dialysis, £22,877, were obtained from a 2011 NICE costing report362 on organ donation for transplantation. Retransplantation involved an estimated cost of £17,736, a weighted average of NHS reference costs 2012/201364 for transplant procedures for varying ages and donor types.
Utilities
Estimates of utilities were derived from the study by Neri et al.,363 who reported EQ-5D health states measured in a cross-sectional study of people with kidney-only transplants in the UK, valued using UK tariffs, as a function of CKD states. As renal function deteriorated so did the HRQoL (utility) values experienced by the simulated patient in the model. The model accounted for negative impacts on HRQoL (disutilities) of two adverse effects, proteinuria (reduced utility by 0.043) and hypertension (reduced utility by 0.010).
Results
EVL + reduced CSA vs. TAC + MMF
Novartis reports a life expectancy at transplantation in a patient group of mean age 45.7 years (SD 12.7 years) of 25.71 life-years under the EVL immunosuppression compared with 23.39 life-years under TAC, and discounted QALYs of 8.86 and 7.37, respectively (Novartis’ submission, table 5.18, Base-case analysis – deterministic ICERs). Given the discounted costs per patient that result under these options, £135,358 for EVL and £140,972 for TAC, EVL was found to be the preferred option, as it is less costly and more effective than TAC.
Further results accounting for uncertainty in model inputs relating to uncertain parameters (ARRs, chronic rejection rates, rate of change in eGFR after 12 months post transplant, health-state utilities and event costs) confirmed that the probability of EVL being cost-effective was 100% at thresholds ranging from £0 to £200,000 per QALY.
EVL + reduced CSA vs. MMF + standard-dose CSA
The EVL regimen was found to produce 1.76 extra years of life over the MMF with CSA regimen in the base case of a cohort of mean age 45.7 years. This corresponded to 0.99 extra discounted QALYs (Novartis’ submission, table 5.18, Base-case analysis – deterministic ICERs). The EVL containing triple therapy was also associated with £59,354 extra discounted costs over the MMF + CSA regimen, and a practically identical ICER figure, given the 0.99 discounted QALY benefit with EVL.
In probabilistic sensitivity analysis (PSA) accounting for the uncertain parameters (as listed for the results of the EVL vs. TAC comparison), the EVL had a 0% probability of being cost-effective relative to MMF for cost-effectiveness thresholds ranging from £0 to approximately £86,000 willingness to pay per QALY, and was still < 15% at £200,000 per QALY.
The fact that the PSA yielded a willingness to pay per QALY threshold at which EVL had a 50% chance of being cost-effective (> £200,000 per QALY), which was more than three times its deterministic ICER of £59,354, indicates that the model has important non-linearities and that using the deterministic values for decision-making is incorrect. Although this warning would not have made any difference to a decision based on a £30,000 per QALY threshold (i.e. both determinist and probabilistic results led to the same conclusion) for this comparison or the previous one discussed (i.e. EVL vs. TAC), the distinction does matter for interpreting the results of the third comparison presented by Novartis – of EC-MPS vs. MMF, discussed next.
EC-MPS vs. MMF + standard-dose CSA
In the deterministic base-case analysis, the mycophenolate regimen was found to result in 25.48 life-years, and 8.69 discounted QALYs per patient (table 5.18, Base-case analysis – deterministic ICERs). Mycophenolate acid had an extra 1.31 life-years and 0.80 discounted QALYs per treated patient relative to MMF. Given its additional discounted costs of £10,588, EVL had an ICER of £13,209 per QALY relative to MMF with CSA.
In the PSA that accounted for the effect of uncertain parameter estimates (as listed in the results of EVL relative to TAC), mycophenolate acid had a 50% chance of being cost-effective at a threshold value of around £28,000 willingness to pay per QALY.
Although the deterministic ICER for MPA is below the lower cost-effectiveness threshold adopted by NICE (£20,000), the willingness-to-pay threshold corresponding to the 50% probability of mycophenolate acid being cost-effective in the PSA is ≈ £28,000) suggesting that EVL may be borderline cost-effective, in relation to the £30,000 maximum acceptable amount NICE is willing to pay for a QALY. This comparison shows that the deterministic results are potentially misleading for informing decisions or deriving model predictions about treatment outcomes in this model.
Critique
The Novartis model uses a patient simulation model of monthly cycles to calculate the costs and health outcomes of immunosuppressant regimens over the remaining lifetime (i.e. 50 years post transplantation). The main strength of the model is its account of the occurrence of clinical events that determine health status, that is, AR, and graft and patient survival, as well as the effect of renal function on costs and HRQoL.
The study failed to conduct adequate evidence synthesis, as their methods of identification of relevant evidence on efficacy was not systematic, as acknowledged by the authors. The model analyses were based on data from single trials, and their analyses were restricted to undertake pairwise indirect comparisons of the treatments investigated in each of those individual trials. This led to results that were at odds with findings from the systematic review of the clinical evidence undertaken by PenTAG (see Chapter 3, Summary of pairwise comparisons) which found no statistically significant improvement in efficacy outcomes (AR, graft failure, death) of EC-MPS compared with MMF, whereas the Novartis model-based analysis produced an extra 1.31 life-years for EC-MPS. Therefore, the results by Novartis are likely to be biased, and consideration of additional efficacy evidence from direct and indirect comparisons would have allowed the company to provide a more reliable technology assessment.
Some errors were identified in the calculation of unit costs of immunosuppression for the CSA component of the EVL regimen, which was common to two other comparators, but is not part of the current standard clinical practice in England. This had the effect of underestimating costs for the CSA-containing regimens.
The model accounted for some important AEs, but omitted one of the most important determinants of patient and graft survival: PTDM.
A major flaw in the model is the assumption that graft failure occurs independently of the GRF or the occurrence of AR. The probability of graft failure (labelled chronic rejection in the submission) is based on 12-month post-transplantation trial data for each regimen, which, given that this probability is constant over the 50-year time horizon of the model, casts serious doubt about the validity of the findings.
In summary, the main strength of the Novartis analysis is its account for the effect of differences in GRF between treatment arms on current costs and utilities. Its main limitations are:
The use of treatment effectiveness data from single selected RCTs, not systematic reviews or meta-analysis, and based on pairwise indirect comparisons of those trials. The estimated effectiveness of EC-MPS compared with MMF is therefore substantially greater than that estimated from the assessment group’s systematic review and meta-analysis.
The model structure contains the assumption that graft failure occurs independently of GRF or the occurrence of AR. Instead, the probability of graft failure is based on the trial-derived rates at 1 year post transplant, which are then assumed to remain constant throughout the modelled period.
Regimens involving CSA (including the EVL regimen) had incorrect unit costs for CSA; this would underestimate the cost of those regimens.
The estimate of the annual cost of dialysis is from an unusual source, and substantially lower than current costs as in the NHS reference costs.
The AE PTDM is not included in the model (despite others being included).
Bristol-Myers Squibb’s submission
The following regimens, all following BAS induction, were compared in the Bristol-Myers Squibb submission:
Two patient populations were studied, namely standard criteria donor recipients, and the ECDs recipients of de novo renal transplants. In addition, the submission presented subgroup analyses for people of weight of > 90 kg.
In its review of the effectiveness evidence, the company justifies its exclusion of SRL from the analysis arguing that, in practice, its use ‘is generally restricted to treating renal transplant patients whose renal function is steadily declining on TAC or CSA, and in whom other measures (such as dose adjustment) have not been successful’ (Bristol-Myers Squibb’s submission, chapter 3, efficacy section). As for TAC-PR, the company argued that there was insufficient direct or indirect evidence to include it as a comparator. EVL was excluded from the analysis because it lacks UK marketing authorisation. As for MMF and MPS, the company states that they were not included as comparators because they are required to be given with CCSs as part of triple therapy containing BEL, TAC or CSA.
The evidence used to populate the efficacy and safety parameters in the model used in the Bristol-Myers Squibb analysis was derived from the BENEFIT59) and BENEFIT-EXT142 trials, which compared BEL with CSA. The efficacy and safety parameter values for BEL relative to immediate-release TAC were obtained from indirect comparisons in a NMA of 32 studies, 29 of which compared TAC with CSA and three studies (including BENEFIT59 and BENEFIT-EXT142) of BEL compared with CSA.
In making the case for BEL the submission argues that the i.v. mode of administration is likely to result in increased adherence to treatment relative to TAC and CSA, which are administered orally and require routine monitoring to drug exposure and dose adjustment. The company claims that this would be expected to result in improved outcomes with BEL over the CNI comparators. Further, in setting the context of the economic evaluation (Bristol-Myers Squibb’s submission, chapter 6, Cost-effectiveness of BEL) the company states that the drivers of the evaluation were the acquisition cost of BEL, the number of years of functioning graft and the costs and utility (HRQoL) of dialysis following graft failure, which led it to perform subgroup analyses in those whose expected graft survival is short. Therefore, because ‘post-transplant renal function is a well-established predictor of graft survival this analysis focused on people with a post-transplant eGFR < 30ml/minute/1.73m2 as these people represent those for whom improved post-transplant renal function is most likely to have significant health and cost benefits’.
The analysis is based on the 3-year outcomes from the pooled data from BENEFIT59 and BENEFIT-EXT,142 including renal function (eGFR), and the cumulative incidence of NODAT, AR, PTLD, graft failure and death, where eGFR of < 15 ml/minute/1.73m2 was assumed to identify people with graft failure. The Markov model developed by Levy et al.334 was then used to extrapolate these outcomes to the long term. To avoid repeating the description in Chapter 4 (see Identified studies), the main features of this model are summarised here.
The model represents annual transitions among the following health states:
functioning graft (including distinguishing four categories of renal function according to National Kidney Foundation Kidney Disease Outcomes Quality Initiative)
GFR stage 2 (GFR2) = ≥ 60 ml/minute/1.73 m2
GFR3a = 45 ml/minute/1.73 m2 ≤ GFR < 60 ml/minute/1.73 m2
GFR3b = 30 ml/minute/1.73 m2 ≤ GFR < 45 ml/minute/1.73 m2
GFR4 = 15 ml/minute/1.73 m2 ≤ GFR < 30 ml/minute/1.73 m2
graft failure/dialysis defined as:
functioning re-graft/retransplantation
death.
The probabilities of transitions between these states were populated by time to event models estimated by Levy et al.334 using US registry data. The survival models were the following:
Weibull time to event models for graft survival [two models: (1) graft failure 1–4 years after transplant and (2) graft failure > 4 years]
Weibull time to event model for patient survival [two models: (1) death with a functioning graft 1–4 years after transplant and (2) death with functioning graft (DWTG) of > 4 years]
exponential survival model of time from graft failure to retransplant
exponential survival model of time from retransplant to graft failure
exponential patient survival on dialysis (after graft failure)
exponential patient survival after retransplant.
The Weibull survival model adjusted for covariates including patient age, sex, baseline eGFR, weight, NODAT, AR events, PTLD, donor type and other, calendar year, and patient and donor characteristics.334 The conditioning of these models’ predictions on baseline eGFR allowed the derivation of separate survival curves for the different starting (i.e. at 3 years post transplant) renal functioning health states in the model. In order to assign costs and utilities for each starting eGFR group, the total time spent with a functioning graft predicted from the survival models (adjusted for death risks) was allocated to different eGFR categories by assuming that eGFR declined linearly over time from its starting level (the midpoint of the starting eGFR stage) until reaching graft failure, which was associated with an eGFR level of 15 ml/minute/1.73 m2. Thus, for example, the group of people who entered the Markov model in GFR2 (at 3 years post transplant) at the midpoint GFR level of 67.5 ml/minute/1.73 m2; those in these groups who experienced graft failure, say on the fifth annual cycle (i.e. 8 years post transplant), would be assumed to have traversed from eGFR2 to eGFR5 at an annual rate of 10.5 ml/minute/1.73 m2 [ = (67.5 – 15)/5 ml/minute/1.73 m2]. Thus, the members of this illustrative group of modelled people would have made a transition from GFR2 to GFR3a in the first year (at the end of which they would reach a GFR level of 57 ml/minute/1.73 m2), remain in eGFR during the second year (to finish it at a GFR level of 46.5 ml/minute/1.73 m2), then make a transition to, and end the third year in, GFR3b (at a GFR level of 36 ml/minute/1.73 m2), make a transition to GFR4 in the fourth year (to end the year at GFR level of 25.5 ml/minute/1.73 m2) and experience graft failure at the end of the fifth year (GFR level of 15 ml/minute/1.73 m2). In the model, some people die without graft failure, and they were assumed to have remained in the same eGFR stage as that in which they entered the model [on the basis of regression analysis of United States Renal Data System (USRDS) data on which the survival models were estimated].
After calculating expected costs and outcomes in the Markov model for each starting eGFR stage over 37 years (which, added to the initial 3-year period, amounts to the modelled horizon of 40 years adopted in the base case), the expected costs and outcomes for the whole population were calculated by a weighted average of the expected costs and QALYs across starting model stages. The proportions were the frequency distributions of people at 3 years post transplant across functioning graft stages (approximated by a normal distribution using mean and SD of eGFR values), dialysis stage and death. Finally, the expected costs and QALYs over the extrapolated Markov phase were added to costs and QALYs associated with the observed trial outcomes in the trial to calculate total QALYs and costs over 40 years for each trial arm in BENEFIT59 and BENEFIT-EXT.142
Efficacy parameter estimates
The main inputs for the model were those estimated from the NMA at 36 months. These are presented in , which reproduces table in the industry submission (Bristol-Myers Squibb’s submission, section 6.1, Model inputs, table 28). In the model, the effect of NODAT on graft and patient survival curves is accounted for by applying HRs from the literature.365 PTLD and CVD were accounted for in the model by assigning a 50% chance of death to each of them. The sources of these estimates were not given.
Relative effect of TAC and BEL vs. CSA at 36 months
According to the Bristol-Myers Squibb submission, the distribution of the patient cohort at the start of the Markov model for each of the three regimens evaluated – BEL, TAC and CSA – was calculated from the pooled BENEFIT59 and BENEFIT-EXT142 trial data on GFR outcomes at 36 months post transplant. They assumed that GFR level followed a normal distribution to derive the distribution across functioning graft states, and used the observed means of 38.6 and SD of 22.93 for CSA, 54.64 for BEL (from the BENEFIT59 trials) and 44.8 for TAC (from NMA relative to CSA). But the assumption of normally distributed GFR is problematic, as it implies that in the CSA arm, 4.6% of people at the end of the trial phase (and therefore at the start the Markov model phase) have a negative GFR value. However, inspection of the model’s Excel spreadsheets revealed that these values were not used in the model, but rather a mean of 50.80 and SD of 21.80 for CSA, which implies that 0.9% of people have a negative GFR value at 3 years post transplant. The means for TAC and BEL were, in turn, 58.47 and 66.96, and they also applied the SD 21.80 for CSA (these imply negative GFR values for < 0.4% of people).
To validate the survival curves underpinning its Markov model, which were estimated from US data, the company compared the predictions from its Weibull survival models with UK data from the NHSBT 2013 report355 (these have been discussed in relation to the model submitted by Astellas, submission section 6.1). The predicted survival curves from the Bristol-Myers Squibb model by type of donor (DBD and DCD) are compared with the corresponding UK data points at year 1, 2 and 5 post transplant. Owing to the difficulty of visualising the chart presented by Bristol-Myers Squibb (Bristol-Myers Squibb’s submission, figure 22), the 5-year survival curves reported by the NHSBT 2013 report are reproduced in , alongside the corresponding predictions in the survival model informing the Markov model in the company’s submission. It shows that the model predictions for the DBD graft survival (DBD predictions based on USRDS) converge towards actual UK data for the corresponding donor type. The model predictions based on the DCD patient population, however, appear to diverge from the trend observed in UK data for each donor type. This is of concern, as predictions from this model were used to extrapolate 3-year trial outcomes for 37 years.
Validation of first adult kidney-only graft survival predictions of the Bristol-Myers Squibb model (based on US data from the USRDS) with NHS data (NHSBT) by donor type.
Changes in eGFR stages were associated with changes in utilities and costs. Utilities were derived from a cross-sectional study of UK renal transplant patients.338 AEs including AR, NODAT and PTLD were given estimated annual utility losses of 0.50, 0.06 and 0.44, respectively, reported from the literature.
Costs
The submission provides actual data on estimated costs of clinical events following transplantation in standard practice at a single centre in Wales (). The analysis has been published as part of a multinational study report (described in Chamberlain et al.42 Assessment of cost-effectiveness, Results), which shows some common and divergent practice between this site and other European centres. Briefly, costs were estimated in a retrospective analysis of computerised records from the Cardiff Renal Transplant Database, related to all individuals aged ≥ 18 years who received a kidney-only transplant recorded between January 1998 and December 2005. They were followed up to 3 years, and the analysis included those in whom data were recorded for at least 12 months after transplant and whose data included their most recent transplant in the studied period.
Costs and utilities by GFR in the Bristol-Myers Squibb model
The study provided evidence that was previously unavailable for the UK on actual costs of post-transplantation care and events stratified by GFR at 1 year post transplant. The sample for analysis included 370 people in whom a variety of treatment regimens were used. Of the 20 different treatments used in this period, triple therapy with TAC steroids and AZA was the most frequent (19%), followed by triple therapy with TAC, steroids and MMF (18%). The next most frequently used regimens were double therapy with TAC and AZA or TAC with MMF (9% each). By the second year the proportion of people on these TAC triple regimens had declined (to 14% and 12% of the sample), whereas the proportion of people on the double therapy TAC had increased (to 14% and 13%). The same observation was made from 24 months to the 24+ months’ follow-up point.
Another aspect of this data source is the observed number of TAC immunosuppressant doses used over the follow-up period in this sample. Although, the dose of TAC, given as part of triple therapy alongside MMF and steroids, was continually reduced over the first year from the mean of 10.31 mg at month 1 to 6.36 mg at month 12, and was 5.73 mg and 5.71 mg at month 24 and month 24+, respectively, the dose was kept at 11.23 mg throughout the observation period in the triple regimen that included AZA (Bristol-Myers Squibb’s submission, appendix 5, Preliminary report PORTRAIT database study Cardiff).
On the basis of the resource-use estimates from the PORTRAIT study report, the TAC drug regimen and the CSA regimen costs were estimated. Drug use was valued at BNF 67369 prices [for TAC, the average price of immediate-release TAC 1 mg of 50- and 100-capsule packs was used; for CSA, the average prices of Capimune® (Mylan), Capsorin® (Morningside Pharmaceuticals Ltd), Deximune® (Dexcel Pharma Ltd) and Neoral® (Novartis), 30-capsule packs, were used]. Administration costs included one laboratory test per outpatient appointment to determine CNI level, and accounted for the observed number of outpatient appointments in years 1, 2 and 2+. The costs of BEL administration included the costs of i.v. infusion, which were obtained from a previous HTA report on abatacept (from which BEL was derived, and that has the same method and frequency of administration). Thus, the annual drug acquisition and administration costs of the regimens in the first year of the model for a 75-kg patient were £13,472 for BEL, £3937 for TAC (immediate-release TAC) and £1972 for CSA. These costs were smaller in the second and subsequent years by about 30%, 25% and 15% in the BEL, TAC and CSA arms, respectively.
Results of Bristol-Myers Squibb’s analyses
In the base-case results for a cohort of people with a starting average age of 43 years, at 40 years post initial transplant 11% of people would be alive under BEL, whereas that would be 8.8% under TAC and 7.4% under CSA. By that point, in 75.6% of people the graft would have failed under BEL, whereas that would have happened in 73.8% of people under TAC and 76.9% under CSA. Correspondingly, 19.3% of people received retransplantation under BEL, 19.2% under TAC and 20.6% under CSA.
When comparing total discounted costs, BEL resulted in incremental costs of £91,001 over TAC and £92,216 over CSA. In turn, the incremental discounted QALYs were 0.62 relative to TAC and 0.97 relative to CSA. The incremental cost per additional QALY of BEL relative to TAC was £147,334, whereas that for TAC relative to CSA was £3375.
These results were driven by the higher costs of BEL immunosuppression, which, despite its associated savings in dialysis costs relative to the other regimens (£15,469 relative to CSA and £2248 relative to TAC), incurred seven and three times the cost of immunosuppression of the CSA (additional costs £109,402) and TAC (£95,159 difference) regimens, respectively. These results were confirmed by PSAs and deterministic sensitivity analyses, which showed the ICER to be insensitive to variation in uncertain parameters.
The submission presented additional analyses for a special group of people with a shorter expected graft survival than that for the overall patient population. This is referred to as ‘subgroup analysis’ by the company, and implemented by defining the group as those people with GFR of < 30 ml/minute/1.73 m2 at 1 year post transplant. They implement a post-hoc adjustment to the model so that the effect of eGFR improvements within that range may be accounted for in the model, which originally was specified in discrete eGFR categories and thus restricted all people entering the model in the same category to having the same benefits. The company found that, in these people, BEL results in higher benefits (0.46 extra QALYs in both comparisons) and lower costs (–£1478 relative to CSA and –£4166 relative to TAC).
However, this analysis suffers from a logical flaw. It assumes that those people whom the company claims to have identified as able to benefit from their drug regimen may be identified with precision. In fact they may not. The meaningful definition of subgroup analyses in a setting where risk and uncertainty influence the outcomes of treatment such as this, so that the outputs of a decision model are mathematic expectations of cost and benefits, identifies a selected group of people for special management on the basis of observable characteristics defined at the outset. The defining characteristic of the selected group of people in the subgroup analysis by Bristol-Myers Squibb is an outcome of treatment, and thus not known at the time of transplant (which would be required for sound decision-making analysis about choice of maintenance treatment).
A subgroup analysis presented by Bristol-Myers Squibb finds that BEL may be cost-effective in people with body weight of approximately 90 kg and more. At this body weight, BEL use incurs minimal vial wastage, thus maximising effectiveness for the given cost.
Critique
The model captures all the most important clinical outcomes and AEs arising post transplantation, and accounts for the role of renal function as a prognostic factor for long-term graft survival and its contemporaneous effects on HRQoL and costs. It also accounts for the effect of short-term AR on longer-term graft and patient survival.
A major strength of the evidence presented by Bristol-Myers Squibb is the cost study used to populate the costs of immunosuppressant drug use and administration in the model and the costs associated with renal function. This evidence has been reported as part of a wider study42 in a peer-reviewed publication.
The major limitation of this study is the questionable generalisability of the values used to populate the transition probabilities of the model used to extrapolate short-term trial outcomes to 40 years. The survival models that inform the transition probabilities to the key events, that is, graft failure after transplant, time to retransplantation after graft failure, and possibly patient survival with a functioning graft, may reflect the experience of a patient population that does not correspond to that of the UK.
Another issue is the use of efficacy differences between regimens at 3 years post transplant to populate the entire initial 3 years, as if these differences had occurred from day 1 and remained constant until the end of the third year post transplantation, which we know was not the case, and bias the analysis in favour of BEL, the company’s drug. In fact, inspection of the model spreadsheet reveals that discounting was not applied to the first 3-year costs and benefits.
A methodological limitation is the assumed linear, constant decline in eGFR, which was the driver of the Markov model used to extrapolate outcomes beyond 3 years, in order to estimate quality of life over the graft survival period conditional on initial eGFR value. This, in turn, reflected the limited information available on renal function from registry data; studies using multicentre cohorts could potentially address this issue by measuring, rather than imputing, renal function periods of longer than 2–3 years, which are typically found in the experimental literature.
In summary, the Bristol-Myers Squibb model has numerous strengths, but has the following main limitations:
The use of US data to extrapolate the survival data for key transition probabilities to 40 years (graft failure, time-to-retransplantation after failure).
The use of efficacy differences between regimens at 3 years post transplant to invalidly calculate benefit differences throughout the first 3 years in the cost-effectiveness model, which favours the company’s drug, BEL.
Lack of accounting for the costs of concomitant regimens used in the triple-therapy regimens investigated by the RCTs, which served as the source of efficacy values in the model (discussed in
Comparison between the model submissions).
Lack of discounting of costs and QALYs in the first 3 years of the analysis, which invalidly raises the benefits of BEL proportionally more than it increases its incremental costs.
The assumed linear decline in eGFR 3 years post transplant at a rate with no validation or sensitivity analysis of this assumption.
A ‘subgroup analysis’ based on people with poor GRF at 1 year, but who would not be identifiable at the time of starting maintenance immunosuppression (and therefore also outside the scope of this technology assessment)
Another subgroup analysis, of those with a body weight of 90 kg, should be disregarded, as this subgroup is based only on the cost differences that would be affected by the patient’s weight.
Comparison between the model submissions
Besides the treatment comparisons, the company submissions also differ in terms of the models used to evaluate those treatments (). highlights how useful the evidence provided in each of the economic evaluations may be to inform the decision-making. Given the necessity to extrapolate short-term outcomes reported in trials with typical follow-ups of 1–3 years, the main differences between extrapolating models used by the three companies are reflected in the choice of surrogate outcome used to drive the disease course in people with renal transplantation and the duration of any relative effects of treatments.
Summary of the economic analyses in company submissions
Evers checklist: quality of published economic evaluation studies
The submission by Astellas uses a Markov structure to model the disease evolution and the effects of treatment in the relevant cohort of people. In this model the occurrence of BPAR in the first year post transplant (for the first transplant and any second transplant occurring in the first year of the model) affects the probability of graft failure in subsequent years. Renal function plays no role in this model. In contrast, differences in eGFR changes between the triple-therapy regimens in the first year drive the modelled outcomes of subsequent years in the model by Novartis. Although the risk, costs and HRQoL consequences associated with ARs are accounted for in this model, these events do not affect graft survival. Graft failure is thus as likely to occur while individuals are at CKD stages 1 and 2 as when they are at CKD stage 5, and any state in between those two extremes for that matter. The model by Bristol-Myers Squibb, unlike that by Novartis, assumes that eGFR at the end of year 1 determines graft survival. However, unlike Astellas and similarly to Novartis, the Bristol-Myers Squibb model allows for the costs and consequences of changes in eGFR over time in the functioning graft state and for the effect of eGRF on the probability of patient death. An additional advantage of the Bristol-Myers Squibb analysis over that of Novartis is its allowance for the effects of AR in the first year post transplant to affect patient and graft survival thereafter, as the analysis by Astellas does for the graft survival only.
The figures adopted by the Novartis submission seem to underestimate the costs of twice-daily immediate-release TAC doses. Their cost per mg for TAC is £0.82, whereas the weighted average figure for the market share of the different presentations used by Astellas is £1.618. On the other hand, the mean daily dose at 70 kg body weight for TAC in the Novartis submission is 17.5 mg, whereas the average daily dose for the first year used by Astellas is 7.17 mg. This results in an average maintenance monthly cost of TAC that is 24% higher in the model by Novartis than in the model by Astellas (i.e. £438 vs. £353 per month).
Other differences were found in terms of the unit costs of the MMF therapy. Novartis used a £9.65 price per pack of 50 tablets of 500 mg each, obtained from market data [Commercial Medicines Unit (CMU) Electronic Market Information Tool (eMit) 2014370], whereas Astellas used a price almost 10 times higher: £82.26 per pack of 50 capsules of 500 mg, citing BNF 2014.56 The effect of the chosen MMF price is also different across the submitted analyses, as MMF is a concomitant medication across all immunosuppressive regimens analysed in the evaluation by Astellas, whereas in the Novartis analysis MMF is not part of the regimens involving the company’s own therapies (i.e. EVL and EC-MPS). Thus, although across submissions the treatment regimens that include the companies’ drugs may be associated with increased effectiveness, a higher MMF price has different implications across the submissions: it makes it less attractive for the NHS to adopt such a regimen (as people live longer and incur higher drug costs) in the Astellas analysis, whereas the opposite occurs in the Novartis case (as only the cost of comparator regimens increases).
Although the three models submitted to NICE for this assessment varied in terms of the way the health course of an individual evolved and the use of immunosuppression affected such path, accounting of costs was similar in some aspects once the cycle length of models was taken into account. presents the most important costs for those elements that were common across the models.
Major cost elements (£) in the model submissions
Although the acquisition costs of TAC are comparable across the three industry submissions, only the one by Bristol-Myers Squibb reports any estimates of drug administration, which have the merit of being based on observed data as opposed to assumptions about compliance with dosing guidelines or protocols. With respect to immunosuppression costs, it may be noted that Bristol-Myers Squibb did not account for costs of other concomitant drugs that are part of triple-therapy immunosuppression (e.g. MMF + CCSs, which were given in BENEFIT59 and BENEFIT-EXT142).
More importantly for the results is the observation that Bristol-Myers Squibb used an estimate of dialysis costs372 that was twice the size of the estimate adopted by Novartis (NICE costing guideline 2011374) and almost 13% higher than that of Astellas.371 Given the driving influence of dialysis costs for cost-effectiveness and an issue to be discussed next in relation to the time spent on dialysis in the models, the quality of evidence gained by the Bristol-Myers Squibb model in estimating immunosuppression-related costs and event costs may have been partly offset by an overestimation of the cost savings to be obtained from reducing the time for which people experienced dialysis.
In , the key features of the effectiveness elements of the analyses performed by the companies are presented. A salient aspect of the comparison model specifications is the longer expected time to retransplantation at the time dialysis starts for those people whose graft fails in the Bristol-Myers Squibb model. It is noted that this estimate was derived from an exponential survival model from an older patient sample in the USA (Medicare-covered transplant-only people). This model has a hazard (instantaneous probability) of receiving a transplant that is constant over time and that is predicted according to donor and patient characteristics (Levy et al.334). In the Bristol-Myers Squibb model these characteristics are fixed over time and result in the constant annual probability of 4% of receiving a transplant while on dialysis. This means that the expected waiting time for a retransplant in a US sample with the Bristol-Myers Squibb’s model characteristics (which match the BENEFIT59 and BENEFIT-EXT142 sample characteristics), as detailed in the Bristol-Myers Squibb submission, is 16.5 years at the start of dialysis. This waiting time is clearly longer than the waiting time currently expected in the UK, which may be closer to the values adopted by Astellas and Novartis in their models.
Key features of effectiveness analysis in industry models
In any case, the median time to retransplant may also be unrealistic for the USA, even after considering issues about socioeconomic barriers to access and related features of that system. After inspection of the estimated coefficients of the exponential model reported by Levy et al.334 (supplementary material, file 1, and reproduced by the Bristol-Myers Squibb submission as appendix 4, table 1), the age covariate (which remains fixed at 40.3 years throughout the 40 annual cycles of the Markov model, so that those proportions of the cohort who experience graft failure early in the model have the same probability of receiving a retransplant in any given cycle as that people who experience graft failure in the latter part of the modelled time horizon) is positively associated with the probability of retransplant, which means that those who start dialysis at older ages have shorter expected waits for a retransplant and suggests that the model was estimated in a cohort of much older people than the Bristol-Myers Squibb’s modelled age of 40 years (e.g. for graft failure at age 70 years the model yields an expected wait of approximately 10 years to receive a retransplant).
The overestimation of time to retransplant in the Bristol-Myers Squibb model that was just described has the implication of overestimating the time on dialysis with its associated costs and loss in quality of life. This, in turn, means that the model is likely to overestimate the benefits of any advantages in terms of graft survival that BEL has over its comparators, TAC and CSA. Likewise, this probably exaggerates the costs savings and quality-of-life gains of TAC over CSA, which suggest that its ICER (£3375; this was not stated in the Bristol-Myers Squibb submission but implicit in their numbers and calculated from them by PenTAG) is an underestimate. presents a summary of model outputs for the three industry model submissions.
Results of model-based analyses submitted by the companies
