Background
Bowel cancer is the third most common cancer in the UK, accounting for 13% of new cancer cases and around 10% of all cancer deaths. The likelihood of surviving 1 year after diagnosis is around 73% and of surviving 5 years is around 55%. Most bowel cancers are initially treated with surgery, but around one in six will spread to the liver. When this happens the cancer in the liver can sometimes be treated by further surgery or, when surgery is not initially possible, chemotherapy may be used with the aim of shrinking the tumour to make surgery possible. Kirsten rat sarcoma viral oncogene (KRAS) mutations make some tumours less responsive to treatment with biological therapies, such as cetuximab. There are a variety of tests available to detect these mutations. These vary in the specific mutations that they detect, the amount of mutation they detect, the amount of tumour cells needed, the time to give a result, the error rate and cost.
Objectives
To compare the performance and cost-effectiveness of KRAS mutation tests (commercial or in-house) in differentiating adults with metastatic colorectal cancer (mCRC) whose metastases are confined to the liver and are unresectable and who may benefit from first-line treatment with cetuximab in combination with standard chemotherapy from those who should receive standard chemotherapy alone.
Methods
Assessment of clinical effectiveness
Thirteen databases, including MEDLINE and EMBASE, research registers and conference proceedings were searched from to January 2013. A web-based survey of UK laboratories gathered data on the technical performance of KRAS mutation tests. The systematic review included studies of tumour KRAS mutation testing in adults with colorectal cancer (CRC) and unresectable, liver-limited metastases. Eligible study designs were randomised controlled trials (RCTs)/controlled clinical trials comparing cetuximab plus standard chemotherapy with standard chemotherapy in participants with known tumour KRAS mutation status, and studies providing data on the accuracy of KRAS mutation testing to predict tumour response to cetuximab plus standard chemotherapy. Search results were screened for relevance independently by two reviewers. Full-text inclusion assessment, data extraction and quality assessment were conducted by one reviewer and checked by a second. RCTs were assessed for quality using the Cochrane risk of bias tool. Diagnostic accuracy studies were assessed using the QUADAS-2 tool. There were insufficient data for meta-analysis. For accuracy studies we calculated sensitivity and specificity together with 95% confidence intervals (CIs). Survival data were summarised as hazard ratios and tumour response data as relative risks with 95% CIs.
Assessment of cost-effectiveness
We considered the long-term costs and quality-adjusted life-years (QALY) associated with different tests followed by treatment with either standard chemotherapy or cetuximab plus standard chemotherapy. The analysis took a ‘no comparator’ approach, which implies that the cost-effectiveness of each strategy will be presented only compared with that of the next most cost-effective strategy. The de novo model consisted of a decision tree and a Markov model. The decision tree was used to model the test result (wild type, mutant or unknown) and the treatment decision. Patients with a KRAS wild-type test result received cetuximab plus standard chemotherapy; patients with a KRAS mutant or unknown test result received standard chemotherapy. The long-term consequences in terms of costs and QALYs were estimated using a Markov model with a cycle time of 1 week and a lifetime time horizon (23 years). Health states in the Markov model were:
progression-free first line – never operated
progressive disease second line – never operated
progressive disease second line – unsuccessful resection
survival after curative resection
progression-free first line – unsuccessful resection
progressive disease third line – never operated
progressive disease third line – unsuccessful resection
dead.
We presented two analyses: ‘linked evidence’, including only tests for which data on test accuracy were available, and ‘assumption of equal prognostic value’, including all tests for which information on technical performance was available. In the linked evidence analysis, test accuracy and resection rates were test specific. Probabilities for (progression-free) survival were assumed to depend on the health state that a patient is in (e.g. survival after successful resection) and did not differ between the tests (test independent). In the assumption of equal prognostic value analysis, tests were assumed to differ solely by technical performance (i.e. proportion of failed tests), retrieved from the online survey of NHS laboratories in England and Wales. All other parameters were assumed to be equal.
Results
Five studies (seven publications) were included in the review.
What are the technical performance characteristics of the different KRAS mutation tests?
No studies assessed the technical performance of KRAS mutation tests. Fifteen UK-based laboratories completed the online questionnaire (response rate 50%). Pyrosequencing, using in-house methods, was the most commonly used test (nine laboratories) followed by the cobas®
KRAS Mutation Test (Roche Molecular Systems, Branchburg, NJ, USA) (three laboratories). Sanger sequencing was used by two laboratories, one laboratory used the Therascreen®
KRAS Pyro Kit (QIAGEN, Hilden, Germany) and one used high-resolution melt analysis (HRM) and direct sequencing. More than half of the responding laboratories reported that KRAS mutation testing was carried out on request (e.g. from a pathologist or oncologist); only one laboratory reported routine testing of all CRC samples. There were no clear differences between tests in terms of batch size, turnaround time, number of failed samples or test cost. With the exception of those using Sanger sequencing, all laboratories reported a limit of detection for percentage mutation of ≤ 10%.
What is the accuracy of KRAS mutation testing for predicting response to treatment with cetuximab plus standard chemotherapy and subsequent resection rates?
Two studies provided data on the accuracy of KRAS mutation testing for predicting response to treatment in patients treated with cetuximab plus standard chemotherapy. The sensitivity and specificity estimates for the Therascreen kit for predicting objective response were 74.6% (95% CI 62.1% to 84.5%) and 35.5% (95% CI 19.2% to 54.6%) respectively. Estimates for pyrosequencing and matrix-assisted laser desorption ionisation time-of-flight (MALDI-TOF) mass spectrometry for predicting potentially curative resection following treatment were 52.0% (95% CI 31.3 to 72.2%) and 45.6% (95% CI 37.0 to 54.3%) respectively.
How do outcomes from treatment with cetuximab plus standard chemotherapy vary according to which test is used to select patients for treatment?
Four RCTs provided data on the clinical effectiveness of cetuximab plus standard chemotherapy compared with that of standard chemotherapy. Two trials used the LightMix® k-ras Gly12 assay (TIB MOLBIOL, Berlin, Germany), one used pyrosequencing together with MALDI-TOF mass spectrometry and one used pyrosequencing alone.
All studies reported improvements in objective response rate for patients with KRAS wild-type tumours who were treated with cetuximab plus standard chemotherapy compared with those treated with standard chemotherapy. There were no clear differences in the treatment effects reported by different studies, regardless of which KRAS mutation test was used to select patients.
What is the cost-effectiveness of the use of different KRAS mutation tests to decide between standard chemotherapy or cetuximab plus standard chemotherapy?
Linked evidence analysis
The linked evidence analysis included two tests, that is, only those tests for which evidence on test accuracy for prediction of either resection rate or objective response was available. We have data from the COIN and CELIM trials only; the COIN trial used pyrosequencing to test for KRAS mutations and the CELIM trial used an earlier version of the Therascreen KRAS RGQ PCR Kit. We assumed that the differences between the outcomes of these trials were exclusively caused by the different tests used. In addition, we assumed that all patients with KRAS wild-type tumours respond perfectly to cetuximab – or will all have a liver resection after cetuximab – and all patients with KRAS mutant tumours do not, and also that test accuracy based on objective response can be compared with accuracy based on resection rates.
Pyrosequencing results in the lowest total cost. The Therascreen KRAS RGQ PCR Kit is the more expensive but also more effective strategy, with an incremental cost-effectiveness ratio (ICER) of £17,019 per QALY gained. The cost-effectiveness acceptability curve (CEAC) indicates that, for lower values of the threshold, pyrosequencing is preferred and that at thresholds of ≥ £17,000 the Therascreen KRAS RGQ PCR Kit is the most cost-effective option. The results of the sensitivity analyses do not differ substantially from the base-case results in the sense that the Therascreen KRAS RGQ PCR Kit is consistently more expensive and more effective than pyrosequencing, with ICERs ranging from £14,860 to £20,528 per QALY gained.
Assumption of equal prognostic value analysis
The analysis based on the assumption of equal prognostic value included all tests for which information on technical performance was available from the online survey of NHS laboratories in England and Wales. This included the tests for which accuracy data, based on either objective response or resection rates, were not available. Therefore, this analysis assessed whether the tests were likely to be cost-effective given an assumption of equal prognostic value based on testing with pyrosequencing (as this was the only test for which full data were available on resection rates following treatment with chemotherapy, with and without cetuximab, for patients with initially inoperable liver metastases and both KRAS mutant and KRAS wild-type tumours) and test-specific information on technical failures within the laboratory only. In the base case and in the first sensitivity analysis, the total technical failure rate (pre-laboratory plus within-laboratory technical failures) is assumed to be equal for all tests. As a result, the strategies in these analyses differ only with respect to costs. In the base case the average QALYs for all comparators are 1.483. The total costs associated with the various testing strategies are very similar. The same applies to the first sensitivity analysis: costs are similar across strategies and average QALYs are equal by assumption at 1.278 (95% CI 1.115 to 1.446).
The second sensitivity analysis assumed that all of the technical failures that occurred were test specific. All other input parameters, such as test costs and test accuracy, were still considered equal. For this sensitivity analysis the cobas KRAS Mutation Test is the least costly and least effective strategy. The HRM analysis and Sanger sequencing have equal costs and effects and their ICER compared with the cobas KRAS Mutation Test is £69,815 per QALY gained. Pyrosequencing and the Therascreen KRAS RGQ PCR Kit are ruled out by extended dominance. From the CEAC it is apparent that the cobas KRAS Mutation Test is the preferred strategy for all threshold values of < £60,000.
Conclusions
Implications for service provision
There was no strong evidence that any one method of KRAS mutation testing had greater accuracy than any other for predicting tumour response or potentially curative resection, following treatment with cetuximab plus standard chemotherapy, in patients with mCRC whose metastases were limited to the liver and were unresectable before chemotherapy. The clinical effectiveness of cetuximab plus standard chemotherapy, in patients whose tumours are KRAS wild type, did not appear to vary according to which method was used to determine tumour KRAS mutation status.
The results of the linked evidence analysis indicated that the Therascreen KRAS RGQ PCR Kit was more costly and more effective than pyrosequencing with an ICER of £17,019 per QALY gained; sensitivity analyses did not show substantial differences compared with the base case. The results of the second sensitivity analysis for the equal prognostic value analysis (including all tests for which information on technical performance was available from the online survey of NHS laboratories in England and Wales) indicated that the cobas KRAS Mutation Test is the least expensive and least effective strategy. It should be noted that substantial assumptions were necessary to arrive at the economic results, in particular the assumption that the differences in resection rates observed between the different studies are solely due to the different tests used. This ignores all other factors that can explain variations in outcomes between the studies. Therefore, these outcomes of the assessment of cost-effectiveness should be interpreted with extreme caution.
Suggested research priorities
Retesting of stored samples from previous studies for which patient outcomes are already known could be used to provide information on the relative effectiveness of cetuximab plus standard chemotherapy and standard chemotherapy alone in patients with KRAS wild-type and KRAS mutant tumours, with mutation status determined using testing methods for which adequate data are currently unavailable. Should quantitative testing become part of routine practice, longitudinal follow-up studies relating the level of mutation and/or the presence of rarer mutations to patient outcomes would become possible. Studies of this type could help to assess which features of KRAS mutation tests are likely to be important in determining their clinical effectiveness. As the uncertainties associated with clinical effectiveness forced the major assumptions in the economic evaluation, this type of research would also facilitate economic analyses of KRAS mutation testing.
Study registration
This study is registered as PROSPERO CRD42013003663.
Funding
Funding for this study was provided by the Health Technology Assessment programme of the National Institute for Health Research.
