The main limitation of the manufacturer’s submission relates to the lack of data presented on UCDs. This has led the manufacturer to use a reasonably simple model structure and to populate this structure with extrapolated relationships between outcomes and clinical opinions. In some cases, this appears to be due to the rare nature of UCDs, but at several points assumptions do not appear to be justified.

The manufacturer’s model contains multiple flaws:

• The primary outcomes reported from the clinical trials of GPB and NaPBA were ammonia levels. The manufacturer cites an estimated relationship found between ammonia levels and HACs, and uses linear interpolation between these two estimated points to further estimate HAC risks for dietary control. The points that the manufacturer uses to estimate a linear relationship (at differences of 10 μmol/L and 25 μmol/L versus GPB) are small compared with the difference (91.7 μmol/L versus GPB) then used to estimate a relative risk for dietary control versus GPB. As an approach, this is flawed, and the manufacturer’s submission on this point is discussed in more detail in Appendix IV.
• The probability of liver transplantation is estimated based on the reanalysis of a single paper reporting the ages at which liver transplantation occurs. The probabilities used appear to be based on a flawed interpretation of the data provided and underestimate the true probabilities, especially among older patients, as a result of both mortality and censoring.
• The parameters drawn within the manufacturer’s PSA are flawed. The model inappropriately uses normal distributions to draw cost parameters, where gamma or lognormal distributions should be used instead, as the distributions should be skewed. When estimating the relative risks of HACs, the model uses lognormal distributions but incorrectly identifies these distributions by setting the mean of the lognormal distribution to the logarithm of the mean costs.
• The PSA does not include all items that might be allowed to vary, given the available data. Where these are allowed to vary, only 1,000 model runs were considered. Given the large amount of uncertainty about key outcomes (such as the relative risk of HACs), a higher number of model runs is likely to be necessary to provide clear results.
• In Subgroup 3, the estimated starting age for treatment is at birth, while the indication is for patients aged two and older. The subgroup age should be at least two years of age.

The manufacturer identifies that adherence to NaPBA is difficult for many patients. However, adherence was not considered an issue within the model. Normally, this would appear to be a conservative assumption favouring alternative treatments and disadvantaging GPB. However, in this case, the impact of this is likely to be increased cost-effectiveness of GPB versus NaPBA, since (1) this does not consider the possibility that GPB will not be tolerated and (2) the alternative treatment (dietary control) appears to be more cost-effective generally than either GPB or NaPBA.

There were some comments from the clinical expert consulted by CADTH Common Drug Review (CDR) that the main clinical study for GPB (versus NaPBA) may have recruited patients who have somewhat less severe UCD than the general patient population. This may have the effect that the benefits of GPB versus NaPBA may be accurate within the group considered but underestimated overall, since there may be greater benefits in more severe phenotypes (or mix of phenotypes). This cannot be fully assessed with the data provided, since this reflects the trials submitted. It is worth noting, however, that the subgroup used in the economic analysis to represent dietary control represent a group of patients with higher ammonia levels who may have a more severe mix of phenotypes than those recruited in the GPB trials. If this is the case, there is a danger that naive indirect treatment comparisons of the type necessary (given the lack of head-to-head data or analysis of patient-level data) may over-estimate the cost-effectiveness of GPB versus dietary control.