The literature searches conducted for this EVA rapid review used a broad approach, with respect to the intervention, and included terms for both CaRi-Heart and FAI. These searches identified a total of 3230 unique references. After initial screening of titles and abstracts, 50 references10,11,21–68 were considered to be potentially relevant and ordered for full-paper screening; of these, two publications,11,52 one full paper11 and one conference abstract,52 which reported results the same study, were included in the review. All potentially relevant publications provided by the company were identified by our searches. shows the flow of studies through the review process. Appendix 2 provides details, with reasons for exclusion, of all publications excluded at the full-paper screening stage.
Flow of studies through the review process.
In addition to the studies included in this report, our searches of trials’ registries and information provided by the company identified one relevant ongoing study,69 the details of which are provided in Appendix 2.
Overview of the included CaRi-Heart Risk prediction model study
Based on the searches and inclusion screening described above, two publications11,52 relating to a single study were included in this rapid review; the Rapid review results section of this report cites the primary publication only.11 This publication is a full report of the development and validation of the CaRi-Heart Risk prediction model and provides information relevant to research question 1, ‘What is the prognostic performance of CaRi-Heart®, in people with stable chest pain, who are undergoing CTCA where: a) the dependent variable is cardiac death? b) the dependent variable is MACE?’ and research question 2 ‘What is the prevalence of “low”, “medium” and “high” CaRi-Heart® Risk in people with no evidence of CAD, people with evidence of non-obstructive CAD and people with evidence of obstructive CAD, based on currently available CTCA imaging?’ Table 2 provides a brief overview of the key features of the CaRi-Heart prediction model study and Table 3 provides a summary of the baseline characteristics of the training/development and validation cohorts included in this study.11
Overview of the included CaRi-Heart Risk prediction model study
Baseline characteristics of patients in the included CaRi-Heart Risk prediction model study
A further publication,10 which reports an assessment of the ability of the perivascular FAI to predict clinical outcomes in patients undergoing CTCA, is also cited in this section. This article did not meet the inclusion criteria for our rapid review because it reports an evaluation of the prognostic performance of FAI and not of CaRi-Heart Risk. The article is cited, where it has provided a source of additional information about the training/development and validation cohorts used in the included study,11 including definitions dependent and independent variables in the CaRi-Heart Risk model.
We did not identify any studies which addressed research question 3, ‘What are the clinical effects of using CaRi-Heart® to assess cardiac risk?’ or research question 4, ‘What are the costs, from a UK NHS and Personal Social Services perspective, using CaRi-Heart®, as an adjunctive investigation for assessment of cardiac risk, in people with stable chest pain, who are undergoing CTCA?’
Study quality
This section describes the results of PROBAST assessment for Oikonomou study.11 PROBAST assesses both the risk of bias and concerns regarding applicability of a study that evaluates (develops, validates or updates) a multivariable diagnostic or prognostic prediction model. It is designed to assess primary studies included in a systematic review.20 PROBAST assessment includes four steps: specification of the systematic review question(s) (Table 4), once per systematic review; classification of the type of prediction model evaluation (Table 5), once for each model in each publication assessed and for each relevant outcome; assessment of risk of bias and applicability (Table 6), once for each development and validation of a distinct prediction model in a publication; overall judgement (Table 7), once for each development and validation of a distinct prediction model in a publication.20
PROBAST step one (specification of the systematic review question for the CaRi-Heart EVA)
PROBAST step two (classification of the type of prediction model evaluation for the included CaRi-Heart Risk prediction model study)
PROBAST step three (assessment of risk of bias and applicability for the included CaRi-Heart Risk prediction model study)
PROBAST step four (overall assessment for the included CaRi-Heart Risk prediction model study)
What is the prognostic performance of CaRi-Heart, in people with stable chest pain, who are undergoing computed tomography coronary angiography?
Where the dependent variable is cardiac death
The Oikonomou11 study included a total of 3912 patients who were undergoing clinically indicated CTCA for the evaluation of stable coronary disease. The training/development (USA) cohort comprised 2040 patients, with a median (range) follow-up duration of 53.8 (4–105) months; a total of 85 deaths were reported during follow-up, of which 48 were cardiac.11 The validation (Germany) cohort comprised 1872 patients, with a median (range) follow-up duration of 72 (51–109) months; there were a total of 114 deaths during follow-up, of which 26 were confirmed cardiac deaths and 16 were deaths of unknown cause.11 Numbers of non-fatal adverse coronary events were not reported.
The unadjusted HR, for 8-year cardiac death, per unit increase in CaRi-Heart Risk was 1.10 [95% confidence interval (CI) 1.07 to 1.12] in the training/development cohort and 1.06 (95% CI 1.04 to 1.08) in the validation cohort.11 The HRs adjusted for ‘traditional risk factors’ (smoking, hypercholesterolaemia, hypertension, diabetes mellitus, Duke index, presence of high-risk plaque features and epicardial adipose tissue volume) were 1.05 (95% CI 1.03 to 1.06) in the training/development cohort and 1.04 (95% CI 1.03 to 1.06) in the validation cohort.11
With respect to the subgroups of interest, specified in the scope for this EVA,1 the predictive value of the CaRi-Heart Risk model was consistent across patients with and without obstructive CAD.11 The unadjusted HRs were similar in patients without obstructive CAD, 1.08 (95% CI 1.05 to 1.10) n = 1754 in both the training/development cohort and 1.07 (95% CI 1.04 to 1.07) n = 1405 in the validation cohort, to those in patients with obstructive CAD, 1.04 (95% CI 1.02 to 1.06) n = 286 in the training development cohort and 1.03 (95% CI 1.01 to 1.05) n = 467 in the validation cohort.11 The subgroup of patients without obstructive CAD included those with no to mild CAD (maximum stenosis < 30%), n = 1033 in the training/development and n = 673 in the validation cohort and those with mild CAD (maximum stenosis 30–50%), n = 721 in the training development cohort and n = 732 in the validation cohort.11 No subgroup analysis was presented for patients with no evidence of CAD.
Unadjusted HRs were reported for other clinically relevant subgroups (age, sex, presence or absence of ‘high-risk plaque features’ and CCS and for different race and ethnicity subgroups [white, black and other (Asian, multiethnic)].11 The unadjusted HRs, for 8-year cardiac death, per unit increase in CaRi-Heart Risk, for the whole study population and for all reported subgroups are provided in Table 8.
Hazard ratio for cardiac death per unit increase in CaRi-Heart Risk
The HRs associated with FAI score component of the CaRi-Heart Risk model are provided in Table 9. HRs, per unit increase in FAI score, are given for each of the three major coronary arteries (RCA, LAD and LCX), where FAI score was used as a continuous variable in multivariable Cox regression analysis (adjusted for smoking, hypercholesterolaemia, hypertension, diabetes mellitus, Duke index, presence of high-risk plaque features and epicardial adipose tissue volume).
Adjusted HR per unit increase in FAI score
When compared to a baseline clinical risk model, which included age, sex, hypertension, hypercholesterolaemia, diabetes mellitus and smoking, the CaRi-Heart Risk model showed improved risk discrimination (Δ c-statistic 0.085, p = 0.01, in the training/development cohort and 0.149, p < 0.001, in the validation cohort).11 This improved discrimination appeared to be retained when the extent of coronary atherosclerosis (indicated by the modified Duke CAD index) was added to the baseline clinical risk model; however, data were only presented for the training/development and validation cohorts combined; the c-statistic for CaRi-Heart Risk was 0.863 [standard error (SE) 0.029], the c-statistic for the clinical risk model plus modified Duke CAD index was 0.733 (SE 0.057) and the Δ c-statistic was 0.130 (p < 0.001).11
Where the dependent variable is major adverse cardiovascular events
The Oikonomou11 study evaluated the predictive performance of CaRi-Heart Risk, with cardiac mortality with 8 years as the dependent variable. The study did not assess the ability of CaRi-Heart Risk to predict other outcomes of clinical interest [e.g. MACE or any of the individual components of MACE, such as stroke, myocardial infarction (MI), heart failure or cardiac hospitalisation].11
We did not identify any other studies that assessed the prognostic performance of CaRi-Heart Risk for any dependent variable.
What is the prevalence of ‘low’, ‘medium’ and ‘high’ CaRi-Heart Risk in people with no evidence of coronary artery disease, people with evidence of non-obstructive coronary artery disease and people with evidence of obstructive coronary artery disease, based on currently available computed tomography coronary angiography imaging?
We did not identify any studies that reported the prevalence of ‘low’, ‘medium’ and ‘high’ CaRi-Heart Risk scores for people in the specified subgroups (no evidence of CAD, people with evidence of non-obstructive CAD and people with evidence of obstructive CAD) based on findings on conventional CTCA imaging. However, the Oikonomou study11 reported information about the numbers of patients in various CaRi-Heart Risk categories versus clinical risk categories. These data allowed calculation of the prevalence of ‘low’, ‘medium’ and ‘high’ CaRi-Heart Risk scores in the overall study population.
The prevalence of ‘low’ (< 5%) CaRi-Heart Risk score was 3060/3912 (78.2%) for the whole study population, 1415/2040 (69.4%) for the training/development cohort and 1645/1872 (87.9%) for the validation cohort.
The prevalence of ‘medium’ (5–10%) CaRi-Heart Risk score was 423/3912 (10.8%) for the whole study population, 302/2040 (14.8%) for the training/development cohort and 121/1872 (6.5%) for the validation cohort.
The prevalence of ‘high’ (> 10%) CaRi-Heart Risk score was 429/3912 (11.0%) for the whole study population, 323/2040 (15.8%) for the training/development cohort and 106/1872 (5.7%) for the validation cohort.
Table 10 shows the rates of reclassification, upwards and downwards, using CaRi-Heart Risk score, compared to a risk score derived from the baseline clinical risk model (age, sex, hypertension, hypercholesterolaemia, diabetes mellitus and smoking). Data are reported separately for the training/development and validation cohorts.
Reclassification of risk using CaRi-Heart Risk score
Overall, 242 of 2040 (11.9%) patients in the training/development cohort and 62 of 1872 (3.3%) patients in the validation cohort were reclassified to a lower-risk category when cardiac risk was assessed using the CaRi-Heart Risk model, compared to the baseline clinical risk model. Conversely, 220 of 2040 (10.8%) patients in the training/development cohort and 155 of 1872 (8.3%) patients in the validation cohort were reclassified to a higher-risk category when cardiac risk was assessed using the CaRi-Heart Risk model, compared to the baseline clinical risk model. The rate of reclassification from ‘low’ (< 5%) to ‘high’ (> 10%) risk was 17/1354 (1.3%) in the training/development cohort and 36/1712 (2.1%) in the validation cohort.
What are the clinical effects of using CaRi-Heart to assess cardiac risk?
What are the clinical effects of any changes to treatment, based on CaRi-Heart Risk, in people with no evidence of coronary artery disease, people with evidence of non-obstructive coronary artery disease and people with evidence of obstructive coronary artery disease, based on currently available computed tomography coronary angiography imaging?
We did not identify any studies that assessed the clinical effects of any changes to treatment, based on CaRi-Heart Risk, either for the whole population or for any of the subgroups of interest (people with no evidence of CAD, people with evidence of non-obstructive CAD and people with evidence of obstructive CAD, based on currently available CTCA imaging).
How does CaRi-Heart Risk affect treatment decisions and patient medication adherence in people with no evidence of coronary artery disease, people with evidence of non-obstructive coronary artery disease and people with evidence of obstructive coronary artery disease, based on currently available computed tomography coronary angiography imaging?
We did not identify any studies that assessed whether and how the availability of a CaRi-Heart Risk score might affect treatment decisions or people’s willingness to take medication, either for the whole population or for any of the subgroups of interest (people with no evidence of CAD, people with evidence of non-obstructive CAD and people with evidence of obstructive CAD, based on currently available CTCA imaging).
What are the costs, from a UK NHS and PSS perspective, of using CaRi-Heart, as an adjunctive investigation for assessment of cardiac risk, in people with stable chest pain, who are undergoing computed tomography coronary angiography?
We did not identify any studies that reported information of the costs, from a UK NHS and PSS perspective or any other perspective, of using CaRi-Heart, as an adjunctive investigation for assessment of cardiac risk, in people with stable chest pain, who are undergoing CTCA.
Caristo Diagnostics Ltd provided the following response, regarding costs, to the NICE request for information:76
The price of CaRi-Heart® technology per CTCA scan of a patient to the NHS is yet to be specified but it will cover the costs of:
No further direct costs are expected from the adoption of the CaRi-Heart® technology to the NHS. However, we will test this expectation in the NHS AI award evaluation.
The downstream costs of the clinical action taken as a result of the CaRi-Heart® technology (e.g. further investigations or initiation of clinical management if the CaRi-Heart® risk of a patient is high) will be included in the economic evaluation of CaRi-Heart® and will be compared with the respective costs of care as usual (e.g. downstream costs without CaRi-Heart® analysis).
Caristo Diagnostics Ltd provided the following additional response, regarding costs, to NICE, following submission of our draft report:
CaRi-Heart® analysis is currently available in the private sector at a price of (confidential information has been removed) per case. It is Caristo’s intention to offer the analysis to the NHS at a discounted price. This will be determined by the ongoing health economic work that is currently being conducted by the Department of Epidemiology and Public Health at the University of Oxford. The price is expected to be between (confidential information has been removed) per case. However, this may change on the health economic results are available.
