Uncertainty of the model’s transition probabilities: For the induction phase, the estimation of the relative efficacy of compared treatments was based on an NMA. For the maintenance phase, a second NMA was used for the efficacy of biologics other than ustekinumab, and the IM-UNITI trial informed the transition probabilities for ustekinumab and conventional therapy (using individual patient data [IPD]). The CADTH Common Drug Review (CDR) clinical review identified limitations of the available indirect comparisons and could not make any definitive conclusion regarding the comparative efficacy of ustekinumab versus infliximab, adalimumab, and vedolizumab for induction. The CDR clinical review also identified several serious limitations of the methodology (treatment-sequence analysis) and of the evidence base for the indirect comparisons of the maintenance phases of treatment and could not make any conclusion regarding the comparative efficacy of ustekinumab versus infliximab, adalimumab, and vedolizumab for that analysis. In addition, the manufacturer highlighted in its submission that, if the NMA for the maintenance phase is considered inappropriate because of potential carry-over effects from the induction phase, then placebo response rates in the maintenance phase may vary across trials, thus influencing the relative treatment effects used as inputs for the analysis of the maintenance phase. Finally, use of IPD means there is no adjustment with a comparator when estimating treatment effect size. Since ustekinumab had the largest absolute treatment effect sizes, using IPD biases the results in favour of ustekinumab. CDR could not test the effect of this limitation on the uncertainty with the model’s transition probabilities.

Uncertainty of the effectiveness of infliximab: There were limitations on assessing infliximab for two reasons. In the case of the population with an FCTO, there were different definitions of response used in clinical trials comparing ustekinumab with other biologics. In the case of the population with a failure of anti-TNF therapy, there was a lack of data in this population, given that infliximab was the first approved biologic. The manufacturer’s submission instead used data from adalimumab for the population with a failure of anti-TNF therapy, although the patients’ baseline characteristics in these clinical trials differed from those in the ustekinumab trials. CDR could not test this limitation with enough certainty.

Utility values for model’s health states: The manufacturer assumed utility values from a Canadian study that asked a cohort of patients with CD to rate three hypothetical disease states representing mild (0.82), moderate (0.73), and severe disease (0.54) using a standard gamble approach.⁸ The publication also reported utility values for remission (0.88), chronically active therapy–responsive (0.86), and therapy-resistant (0.74). For the manufacturer’s model, the remission, response, and no-response health states were assigned utilities of 0.88, 0.73, and 0.54, respectively. There is inconsistency in how the utility values from the publication were used in the manufacturer’s model, which raises uncertainty with regard to the results of the analysis. CDR conducted two scenario analyses, one applying more appropriately to the model the utilities from the Canadian study used by the manufacturer, and the other using an alternative source of utility values (Table 13).

Overestimation of QALYs for ustekinumab mixed dosage (every eight weeks/every 12 weeks): The submitted model indicated that a weighted average for costs and QALYs for every eight weeks and every 12 weeks dosages was applied to estimate the results for the mixed-dosage regimen. On verification, the model did not apply the weighted average for QALYs but appeared to default to the every eight weeks QALYs, which were more favourable than the every 12 weeks QALY results. This error overestimated the ICUR results for the mixed dose. This was corrected by CDR.

Adjustment of the maintenance-phase transition probabilities using real-world evidence: The manufacturer acknowledged that adjusting these transition probabilities to real data was challenging because of limited real-world evidence. The manufacturer also recognized that a limitation of the approach was its application to the transition at one year, at the end of the trials, at which time point the approach resulted in an “uptick” effect. The manufacturer argued that the uptick meant that the proportions of patients in remission and response in the final cycles of year 1 were artificially low and that the uptick was simply a correction.¹ However, this favoured ustekinumab. The manufacturer’s argument for the relevance of this approach with regard to its uncertainty is not convincing. CDR reanalysis excluded the impact of real-world evidence on the transition probabilities.

Time horizon: The submitted model used a time horizon of 25 years. Although CD is a chronic condition, the limitation with a long time horizon is that a significant proportion of patients are expected to experience a waning efficacy of therapy over 25 years. Therefore, the benefits in long-term survival may bias against the disutility in those who lose response more rapidly. CDR conducted an exploratory analysis testing a time horizon of 10 years (Appendix 3).