NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Evidence review for prediction of complications in pre-eclampsia
Evidence review C
NICE Guideline, No. 133
National Guideline Alliance (UK).
Review question: Which tests or clinical prediction models are accurate in identifying or predicting women at risk of severe complications of pre-eclampsia?
Introduction
Women with pre-eclampsia can have varying clinical courses of disease, with some women being monitored successfully as outpatients, while other women will require urgent admission for their condition to be managed in a critical care setting. The identification of women at increased risk of developing severe complications (either themselves, or complications for their babies) from pre-eclampsia is therefore important in order to manage women in appropriate settings. However, it remains difficult for healthcare providers to differentiate between women at increased risk of severe complications and women at low risk.
The aim of this review is to determine which investigations or risk prediction models are useful in identifying women (and babies) at risk of severe complications from pre-eclampsia, in order to guide stratified surveillance and target interventions for those at higher risk.
Summary of the protocol
Please see Table 1 for a summary of the population, intervention (clinical prediction tools), comparator, outcome, timing and setting (PICOTS) of this review.
Table 1
Summary of the protocol (PICOTS table).
For full details see the review protocol in appendix A
Methods and process
This evidence review was developed using the methods and process described in Developing NICE guidelines: the manual 2014. Methods specific to this review question are described in the review protocol in appendix A.
Declaration of interests were recorded according to NICE’s 2018 conflicts of interest policy (see Register of interests).
Clinical evidence
The aim of this review was to assess which clinical prediction model or prognostic test was most helpful at predicting adverse maternal and/or fetal outcomes in women with suspected or confirmed pre-eclampsia (PE).
For a study to be included, it had to report at least one type of clinical predictive performance measure (or sufficient data for this to be calculated) to predict composite maternal and/or fetal adverse outcomes.
Included studies
Two different types of studies were included, namely externally validated clinical prediction model studies and prognostic test accuracy studies (and systematic reviews of these studies). For a study to be considered as externally validated, the performance of the prediction model should have been assessed in a sample of patients that were not used for the development of the tool, as described by Debray 2017.
Externally validated clinical prediction model studies
Eight publications providing external validation of 4 prediction models (fullPIERS, miniPIERS, PREP-L and PREP-S) were included (Agrawal 2014, Akkermans 2014, Almeida 2017, Payne 2014, Payne 2015, Thangaratinam 2017, Ukah 2017a, and Ukah 2018). In the context of this review, prediction models assessed the individualised risk of developing adverse maternal or fetal outcomes by combining prognostic factors of an individual. For further details regarding the characteristics of the prediction models please see Table 2. Study details for the external validation studies are reported in Table 3.
Five studies included women with other hypertensive disorders of pregnancy, in addition to PE: Akkermans 2014, Payne 2014, Payne 2015, Thangaratinam 2017, and Ukah 2018. In these studies, the proportion of women with PE ranged from 43.5% to 98.5%.
Half of the included studies used data from pre-existing datasets of women, which led to some overlap in the sample of patients included. These were the PETRA cohort (Preeclampsia Eclampsia Trial Amsterdam), which was included in Akkermans 2014, Thangaratinam 2017, and Ukah 2018; PIERS cohort (Pre-eclampsia Integrated Estimate of RiSk), which was included in Laskin 2011, Livingston 2014, Payne 2014 and Thangaratinam 2017; PREP cohort (Prediction model for Risks of complications in Early-onset Pre-eclampsia), included in Ukah 2018; and miniPIERS cohort, which was included in Ukah 2017a.
Prognostic test accuracy studies
Six publications were included (Chan 2005, Laskin 2011, Livingston 2014, Thangaratinam 2011, Ukah 2017b, Waugh 2017). These studies aimed to assess the performance of different tests to predict adverse maternal and fetal outcomes. Studies are summarised in Table 4.
See also literature search strategy in appendix B and clinical evidence study selection in appendix C.
Table 2
Description of the prediction models.
Excluded studies
Studies not included in this review with reasons for their exclusions are provided in appendix K.
Summary of clinical studies included in the evidence review
A summary of the studies that were included in this review are presented in Table 3 and Table 4.
Table 3
Summary of externally validated clinical prediction model studies.
Table 4
Summary of prognostic test accuracy studies.
See appendix D for full evidence tables.
Quality assessment of clinical outcomes included in the evidence review
The included studies were individually assessed with AMSTAR, CASP CPR, and QUADAS-2 (see Methods chapter for more details).
Overall, studies were rated as of moderate or high quality. The reasons for rating down the studies assessed with AMSTAR (systematic reviews) were as follows: not performing study selection in duplicate; not providing a list of excluded studies; or not reporting the included studies in adequate detail.
The reasons for rating down the quality of the studies assessed with CASP CPR (clinical prediction model studies) were as follows: lack of clarity regarding whether the sample of women included an appropriate spectrum of patients; lack of clarity as to whether the predictor variables and outcomes were evaluated in a blinded fashion; statistical methods not clearly described; and studies including population from low and middle income countries, which affects the generalisability of the results.
The reasons for rating down the studies assessed with the QUADAS-2 (prognostic test accuracy studies) were as follows: not pre-specifying the thresholds; and lack of clarity as to whether the results were interpreted without knowledge of the results of the index test.
Data obtained from the prognostic accuracy studies were assessed according to the outcomes reported using GRADE methodology. The rating for imprecision was assessed based on sensitivity, as this was a critical outcome measure for the review. The pre-specified thresholds were ≥90% (high specificity) and ≥75% (moderate specificity).
The GRADE method has not been adapted for use with clinical prediction models, therefore these articles were quality assessed at the level of the individual studies.
See appendix F for the quality assessment of the included studies.
Economic evidence
A systematic review of the economic literature was conducted but no relevant studies were identified which were applicable to this review question.
Excluded studies
Studies not included in this review with reasons for their exclusion are listed in appendix K.
Economic model
An economic analysis was undertaken to estimate the cost-effectiveness of risk prediction models for guiding inpatient and outpatient management in pregnant women with pre-eclampsia (see appendix J for the full report of the economic analysis).
Methods
The analysis was developed in Microsoft Excel® and was conducted from the perspective of the NHS and Personal Social Services (PSS) as outlined in the NICE Reference Case (see Developing NICE guidelines: the manual).
Clinical data and model approach
The economic analysis considered strategies where the decision on whether to manage pre-eclampsia in women as an outpatient or inpatient was based on risk thresholds (e.g. to offer inpatient management with a risk score ≥ 10%). The analysis considered the fullPIERS risk assessment tool, which was selected because it has the best available evidence. Other risk assessment tools such as PREP-S could also be used in clinical practice but it was not possible to include them in the economic model because there is insufficient data on diagnostic accuracy (sensitivity and specificity) at various risk levels.
Management strategies based on risk level were compared against each other and also against strategies where it is assumed that all women are managed as either an inpatient or outpatient.
It is unclear which strategy would best represent current clinical practice as there is known to be variation. However, it is thought that inpatient management is generally more common than outpatient management. Note that this does not affect the current analysis as the intention is to compare all strategies against each other to determine the most cost-effective strategy. This is a separate endeavour to estimating cost impact which aims to estimate the change in cost associated with the adoption of a new strategy compared to current practice.
The economic analysis considered women 34-37 weeks of gestation reflecting the population in which the fullPIERS risk prediction model is applicable. The following management strategies were considered in the analysis:
- All inpatient management
- All outpatient management
- Inpatient management if fullPIERS ≥ 5%
- Inpatient management if fullPIERS ≥ 10%
- Inpatient management if fullPIERS ≥ 20%
- Inpatient management if fullPIERS ≥ 30%
The economic analysis was based on accuracy data (sensitivity and specificity) for the prediction of complications at 2 and 7 days for each of the strategies (see Table 5). In the model, the diagnostic results are linked to subsequent management whereby women with positive results are managed as inpatients and women with negative results are managed as outpatients.
Data on the prevalence of adverse outcomes as well as data on the accuracy of fullPIERS at different thresholds were estimated from an external validation study (Akkermans 2014). Accuracy data for the ‘all inpatient management’ and ‘all outpatient management’ were inferred based on the implications of the strategy e.g. all patients managed as an inpatient implies that all patients with complications would be managed as an inpatient and therefore the sensitivity would be 100%.
Table 5
Diagnostic accuracy for women 34-37 weeks of gestation.
It has been assumed that women managed in an inpatient setting would have a reduction in the number of adverse maternal outcomes. There is no good evidence available on which to base this reduction. Therefore it was speculatively approximated using data from Broekhuijsen 2015 (HYPITAT II study), which compared immediate delivery with expectant management. It has been assumed that the reduction in adverse outcomes associated with being managed in an inpatient setting rather than an outpatient setting would be similar to the reduction seen with immediate delivery compared with expectant management. In comparison to expectant management, immediate delivery was found to reduce reported adverse maternal outcomes with a relative risk (RR) of 0.36 (95% CI 0.12–1.11). Therefore, this value was applied in the analysis as an estimate of the reduction in adverse maternal outcomes with the inpatient approach.
Mortality was not considered in the analysis as there is no evidence to suggest that the use of risk prediction models may confer a survival benefit.
Costs
The costs considered in the model reflect the perspective of the analysis, thus only costs that are relevant to the UK NHS & PSS were included. Where possible, all costs were estimated in 2016/17 prices. The majority of costs were sourced from NHS reference costs 2016/17 by applying tariffs associated with the appropriate Healthcare Resource Groups (HRG) code.
It was assumed that there is no cost associated with using the fullPIERS risk assessment tool itself as it is freely available online. Furthermore, it was assumed that there was no additional cost associated with performing the tests required to inform the risk factors in the tools as these tests are already carried out as part of routine clinical practice.
Inpatient costs were estimated using the average cost of a day as an elective inpatient from NHS reference costs 2016/17 (£384.50). The average length of stay (LOS) was based on pre-eclampsia audit data, which reported an average time between diagnosis of pre-eclampsia and delivery of 6 days for women 34-37 weeks of gestation. Outpatient costs were based on the cost of consultant led face-to-face follow-up in the obstetrics service from NHS reference costs 2016/17 (£120.20). The average duration of outpatient management was assumed to be the same as inpatient management and it was assumed that patients would have re-assessments every 2 days.
Birth costs were estimated using data on the proportions of each mode of delivery from Broekhuijsen 2015 (HYPITAT II study). A combined average of the immediate delivery and expectant management arms of the trial was estimated resulting in proportions of 4%, 86% and 10% for spontaneous labour, induction of labour and caesarean section, respectively. Birth costs for the various modes of delivery were sourced from NHS Reference Costs 2016/17 assuming that women with adverse outcomes would have births with complications and co-morbidities (based on CC scores). Birth costs were estimated by taking a weighted average of births recorded in NHS reference costs as an elective inpatient, non-elective long stay and non-elective short stay.
It was assumed that women with an adverse outcome would be admitted to a high dependency unit (HDU). A HDU cost of £860.61 was estimated from NHS reference costs 2016/17, based on the weighted average cost of “adult critical care, 0 organs supported” and “adult critical care, 1 organs supported”.
Based on a combined average of the immediate delivery and expectant management arms from Broekhuijsen 2015 (HYPITAT II study), it was assumed that a NICU admission would be required in 5.6% of births. NICU admission costs were estimated from NHS reference costs 2016/17, based on the cost of neonatal critical care, intensive care (£1,295)
Health-related quality of life
As recommended in the NICE reference case, the model estimates effectiveness in terms of quality adjusted life years (QALYs). These are estimated by combining life year estimates with quality of life (QoL) values associated with being in a particular health state.
QoL data were sourced from the economic analysis conducted as part of the previous guideline (NICE CG107). Pregnant women with pre-eclampsia were assumed to have the same QoL value as normotensive pregnant women. The QoL value for normotensive pregnant women was sourced from Sonnenberg 2004, a cost effectiveness analysis of contraception methods in women of average health and fertility, which found that short-term utility loss due to pregnancy was 0.0375.
Experiencing severe compications of pre-eclampsia was assumed to have the same QoL as being admitted to ICU for any reason. As part of a cost effectiveness analysis of meropenem in the treatment of severe infections in hospital intensive care, Edwards 2006 estimated that the QoL weight for someone who stayed in intensive care was 0.712. It was assumed that the QoL decrement for women with severe disease would last for 2 weeks.
In order to estimate QALYs these values were converted to daily weights and applied for the modelled time horizon.
Results
The base case results of the analysis are shown in Table 6. A ‘dominance rank’ approach was used to compare all strategies against each other, whereby the strategies are rank ordered in terms of cost and then each intervention is compared against the previous intervention that was found to be cost-effective.
A strategy of outpatient management was the least costly strategy overall. All other strategies were found to be more costly and more effective than outpatient management. Inpatient management if fullPIERS ≥ 30% was found to be cost-effective with an ICER value of £10,797 per QALY which is below the threshold of £20,000 per QALY. All other strategies were not found to be cost-effective with ICERs well above the threshold of £20,000 per QALY. Therefore the strategy of inpatient management if fullPIERS ≥ 30% was found to be the optimal strategy in cost-effectiveness terms.
Table 6
Base case results.
Deterministic sensitivity results
A series of deterministic sensitivity analyses were conducted, whereby an input parameter is changed, the model is re-run and the new cost-effectiveness result is recorded. This is a useful way of estimating uncertainty and determining the key drivers of the model result. The results of the deterministic sensitivity analyses are presented in Table 7. It can be seen that the conclusion of the analysis changes in numerous scenarios with outpatient management found to be cost-effective in certain scenarios. Notably this includes numerous plausible scenarios such as where variations in the RR for adverse outcomes is applied or when the cost of adverse outcomes is changed.
Table 7
Deterministic sensitivity analysis results.
Threshold analysis results
A threshold analysis was conducted to determine the RR for adverse outcomes required for the inpatient management if fullPIERS ≥ 30% strategy to be cost-effective. It was found that a strategy of inpatient management if fullPIERS ≥ 30% was cost-effective with a RR of 0.53.
Probabilistic sensitivity analysis results
Probabilistic sensitivity analysis (PSA) was conducted to assess the combined parameter uncertainty in the model. In this analysis, the mean values that were utilised in the base-case were replaced with values drawn from distributions around the mean values. The results of 10,000 runs of the PSA are shown using cost-effectiveness acceptability curves (CEAC) in Figure 1. The CEAC graph shows the probability of each strategy being considered cost-effective at various cost-effectiveness thresholds on the x axis.
Figure 1
Cost-effectiveness acceptability curves.
It can be seen that outpatient management and a strategy of inpatient management if fullPIERS ≥ 30% have the highest probabilities of being cost-effective at all thresholds. At the threshold of £20,000 per QALY, inpatient management if fullPIERS ≥ 30% has a 53% probability of being cost-effective while outpatient management has a 46% probability of being cost-effective. All other strategies were found to have a 0% probability of being cost-effective at the threshold of £20,000 per QALY.
Conclusion
The base case results of the analysis suggest that using the fullPIERS risk model with a threshold of 30% for inpatient management is cost-effective in women 34-37 weeks of gestation. However, it should be noted that there are gaps in the clinical evidence base and therefore several assumptions have been made to run the analysis. Most notably, a speculative assumption was made around the reduction in the number of adverse maternal outcomes. Furthermore, deterministic sensitivity analysis suggested that differences in assumptions have the potential to change the conclusion of the analysis and probabilistic sensitivity analysis demonstrated some uncertainty around the result.
Evidence statements
Externally validated models
fullPIERS model performance
Prediction of adverse maternal outcomes within 48 hours
- Four validation studies of fullPIERS (n=2470 participants) provided moderate to high quality evidence to show the following:
- LR in the lower predicted risk categories (<1% and 1-2.4%) ranged from uninformative to very informative
- LR in the middle risk categories (2.5-4.9%, 5.9-9.9% and 10-19%) ranged from uninformative to moderately informative
- LR in the higher risk category (20-29%) was uninformative
- LR in the highest risk category (≥30%) ranged from moderately to very informative.
- Calibration, as assessed by the calibration slope, was found to be poor in the 3 studies that reported this (Akkermans 2016, Ukah 2017a and Ukah 2018)
- Discrimination, as assessed by the AUC, ranged from moderate to excellent
- Discrimination, as assessed by sensitivity, ranged from low to high (from 57% to 90.6%)
- Discrimination, as assessed by specificity, ranged from low to high (from 65.1% to 94%)
Prediction of adverse maternal outcomes within 7 days
- Two validation studies of fullPIERS (n=1388 participants) provided high quality evidence to show the following:
- LR in the lower predicted risk categories (<1% and 1-2.4%) were uninformative
- LR in the middle risk categories (2.5-4.9%, 5.9-9.9% and 10-19%) ranged from to uninformative to moderately informative
- LR in the higher risk category (20-29%) was uninformative
- LR in the highest risk category (≥30%) was very informative
- Calibration, as assessed by the calibration slope, was found to be poor in the single study that reported this (Akkermans 2016)
- Discrimination, as assessed by the AUC, was found to be poor to moderate
- Discrimination, as assessed by sensitivity, ranged from low to high (from 59 to 90%)
- Discrimination, as assessed by specificity, was found to be very low to low (<75%)
Prediction of adverse maternal outcomes (timeframe not specified)
- One validation study of fullPIERS (n=322), reporting on adverse maternal outcomes (with predictor variables collected within 24 hours of admission) provided moderate quality evidence to show the following:
- LR in the lower predicted risk categories (<1% and 1-2.4%) ranged from to uninformative to moderately informative
- LR in the middle risk categories (2.5-4.9%, 5.9-9.9% and 10-19%) were uninformative
- LR in the higher risk category (20-29%) was moderately informative
- LR in the highest risk category (≥30%) was moderately informative
- Discrimination, as assessed by sensitivity, was very low (25%)
- Discrimination, as assessed by specificity, was found to be very high (95.4%)
miniPIERS model performance
Prediction of adverse maternal outcomes within 48 hours
- Two validation studies of miniPIERS (n=2152 participants) provided moderate to high quality evidence to show the following:
- LR in the lower and middle risk categories (0-24.9%) were uninformative
- LR in the highest risk category (≥25%) was moderately informative
- Discrimination, as assessed by the AUC, was found to be moderate
- Discrimination, as assessed by sensitivity, was found to be low (32.8%)
- Discrimination, as assessed by specificity, was found to be very high (96.2%)
PREP-L model performance
- One validation study of PREP-L (n=648 participants), reporting on adverse maternal outcomes by discharge, provided moderate to high quality evidence to show the following:
- Calibration, as assessed by the calibration slope, was found to be good
- Discrimination, as assessed by the AUC, was found to be moderate to good
PREP-S model performance
Prediction of adverse maternal outcomes within 48 hours
- One validation study of PREP-S (n=339 participants), reporting on adverse maternal outcomes within 48 hrs of admission, provided moderate quality evidence to show the following:
- Observed: expected ratios in the lower predicted risk category (≤15th centile) showed good calibration
- Observed: expected ratios in the middle risk categories (>15-50, 50-85th centiles) showed a range from not good to excellent calibration
- Observed: expected ratios in the highest risk category (>85th centile) showed not good calibration
- Calibration, as assessed by the calibration slope, was found to be moderate
- Discrimination, as assessed by the AUC, was found to be moderate
Prediction of adverse maternal outcomes within 7 days
- One validation study of PREP-S (n=339 participants), reporting on adverse maternal outcomes within 7 days of admission, provided moderate quality evidence to show the following:
- Observed: expected ratios in the lower predicted risk category (≤15th centile) showed excellent calibration
- Observed: expected ratios in the middle risk categories (>15-50, 50-85th centiles) showed a range from not good to excellent calibration
- Observed: expected ratios in the highest risk category (>85th centile) showed poor calibration
- Calibration, as assessed by the calibration slope, was found to be moderate
- Discrimination, as assessed by the C-statistic, was found to be moderate
Prognostic tests
Prognostic test accuracy of urine spot protein or albumin creatinine ratio
Prediction of adverse maternal outcomes/severe pre-eclampsia
- One cohort study (n=321) provided high quality evidence to show that urine spot protein creatinine ratio (sPCR) > 500 combined with maternal age > 35 years demonstrated:
- low sensitivity and high specificity
- very informative LR+ but uninformative LR− to predict adverse maternal outcomes.
- One cohort study (n=959) provided high quality evidence to show that sPCR at a threshold of 30mg/mmol (local lab, recruitment sample) demonstrated:
- moderate sensitivity and low specificity
- uninformative LR+ and LR− to predict severe pre-eclampsia.
- One cohort study (n=959) provided high quality evidence to show that sACR at a threshold of 2 mg/mmol (central lab, recruitment sample) demonstrated:
- high sensitivity and low specificity
- uninformative LR+ but moderately informative LR− to predict severe pre-eclampsia.
Prediction of adverse perinatal outcomes
- One cohort study (n=959) provided moderate quality evidence to show that sPCR at a threshold of 30mg/mmol (local lab, recruitment sample) demonstrated:
- low sensitivity and low specificity
- uninformative LR− and LR+ to predict adverse perinatal outcomes.
- One cohort study (n=959) provided high quality evidence to show that sACR at a threshold of 2 mg/mmol (central lab, recruitment sample) demonstrated:
- high sensitivity and low specificity
- uninformative LR− and LR+ to predict adverse perinatal outcomes.
Prognostic test accuracy of abnormal coagulation
Prediction of adverse maternal outcomes
- One cohort study (n=1405) provided moderate quality evidence to show that a platelet count ≤ 100 × 109/L demonstrated:
- low sensitivity and high specificity
- uninformative LR− and LR+ to predict adverse maternal outcomes within 48 hours.
- One cohort study (n=1405) provided moderate quality evidence to show that abnormal coagulation (international normalised ratio, INR > 1.06 and serum fibrinogen < 3.54 g/L) demonstrated:
- low sensitivity and high specificity
- uninformative LR− and LR+ to predict adverse maternal outcomes within 48 hours.
Prognostic test accuracy of liver function
Prediction of adverse maternal outcomes
- One systematic review (n=568) provided low quality evidence to show that aspartate transaminase (AST) (cut-off 150 U/l) demonstrated:
- low sensitivity and low specificity
- uninformative LR− and LR+ to predict adverse maternal outcomes.
- One systematic review (n=568) provided moderate quality evidence to show that aspartate transaminase (ALT) (cut-off 100 U/l) demonstrated:
- low sensitivity and low specificity
- uninformative LR− and LR+ to predict adverse maternal outcomes.
- One systematic review (n=568) provided low quality evidence to show that lactate dehydrogenase (LDH) (cut-off 1400U/l) demonstrated:
- low sensitivity and low specificity
- uninformative LR− and LR+ to predict adverse maternal outcomes.
- One systematic review (n=737) provided moderate quality evidence to show that LDH (cut-off 600U/l) demonstrated:
- low sensitivity and low specificity
- uninformative LR− and LR+ to predict adverse maternal outcomes.
- One systematic review (n=737) provided moderate quality evidence to show that ALT (cut-off 40 U/l) and AST (cut-off 55 U/l) demonstrated:
- low sensitivity and moderate specificity
- uninformative LR− and LR+ to predict adverse maternal outcomes.
- One systematic review (n=85) provided very low quality evidence to show that AST (cut-off 30 U/l); ALT (cut-off 32 U/l); bilirubin (cut-off 14 µmol/L); gamma glutamyl transferase (GGT) (cut-off 41 U/l) demonstrated:
- high sensitivity and low specificity
- uninformative LR+ and moderately informative LR− to predict adverse maternal outcomes.
Prediction of adverse fetal outcomes
- One systematic review (n=85) provided very low quality evidence to show that AST (cut-off 30 U/l); ALT (cut-off 32 U/l); bilirubin (cut-off 14 µmol/L); GGT (cut-off 41 U/l) demonstrated:
- moderate sensitivity and low specificity
- uninformative LR− and LR+ to predict adverse fetal outcomes.
Prognostic test accuracy of uric acid
Prediction of adverse maternal outcomes
- One cohort study (n=1487) provided low quality evidence to show that uric acid (cut-off 345µmol/L) demonstrated:
- moderate sensitivity and low specificity to predict adverse maternal outcomes within 48 hours.
- One cohort study (n=1487) provided moderate quality evidence to show that uric acid (cut-off 345µmol/L) demonstrated:
- moderate sensitivity and low specificity to predict adverse maternal outcomes within 7 days.
- One cohort study (n=1487) provided moderate quality evidence to show that uric acid (cut-off 345µmol/L) demonstrated:
- moderate sensitivity and low specificity to predict adverse maternal outcomes at any time.
- One cohort study (n=1487) provided low quality evidence to show that uric acid (cut-off >1 SD above the mean for gestational age) demonstrated:
- moderate sensitivity and low specificity to predict adverse maternal outcomes within 48 hours.
- One cohort study (n=1487) provided low quality evidence to show that uric acid (cut-off >1 SD above the mean for gestational age) demonstrated:
- moderate sensitivity and low specificity to predict adverse maternal outcomes within 7 days.
- One cohort study (n=1487) provided low quality evidence to show that uric acid (cut-off >1 SD above the mean for gestational age) demonstrated:
- moderate sensitivity and low specificity to predict adverse maternal outcomes at any time.
Prediction of adverse perinatal outcomes
- One cohort study (n=1487) provided moderate quality evidence to show that uric acid (cut-off >345µmol/L) demonstrated:
- moderate sensitivity and low specificity to predict adverse perinatal outcomes.
- One cohort study (n=1487) provided moderate quality evidence to show that uric acid (cut-off >1 SD above the mean for gestational age) demonstrated:
- high sensitivity and low specificity to predict adverse perinatal outcomes.
Prognostic test accuracy of soluble fms-like tyrosine kinase-1 and placental growth factor
Prediction of adverse maternal outcomes
- One systematic review (n=501) provided moderate quality evidence to show that serum soluble fms-like tyrosine kinase-1 and placental growth factor (sFlt-1/PlGF) ratio ≥ 871 demonstrated:
- low sensitivity and moderate specificity
- uninformative LR− and LR+ to predict adverse maternal outcomes.
- One systematic review (n=237) provided low quality evidence to show that sFlt-1/PlGF ratio >85 demonstrated:
- low sensitivity and low specificity
- uninformative LR− and LR+ to predict adverse maternal outcomes
Prognostic test accuracy of maternal characteristics
Prediction of adverse perinatal outcomes
- One cohort study (n-321) provided high quality evidence to show that maternal characteristics (gestational age <34 weeks and booking systolic blood pressure <115mmHg, in women subsequently presenting with suspected pre-eclampsia) for predicting adverse fetal outcomes showed the following:
- low sensitivity and low specificity
- uninformative LR− and LR+ to predict adverse perinatal outcomes.
The committee’s discussion of the evidence
Interpreting the evidence
The outcomes that matter most
Pregnant women with pre-eclampsia may develop serious complications and these prediction models and prognostic tests aim to identify which women were at a greater risk of these complications, in order that more intensive monitoring and treatment (such as steroids for fetal lung maturity, magnesium sulfate and planned early birth) can be instigated. Accuracy to identify adverse maternal and perinatal outcomes, as defined by discrimination and calibration in the clinical prediction model studies, and as sensitivity in the prognostic test accuracy studies, were therefore considered of critical importance in this review.
For the clinical prediction model studies, discrimination indicates how well the model separates women at higher risk and lower risk of developing adverse outcomes, and calibration defines how well the expected outcomes (as predicted by the model) and the observed outcomes agree. These outcomes were considered critical because they provide information regarding the usefulness of the test in assisting healthcare professionals to make safe decisions regarding management. Maternal outcomes were predicted at different times by the models – most commonly within 48 hours or within 7 days. The committee agreed that the ‘within 48 hours’ time period was the most useful for assessment of short-term risk, and the prediction model could be repeated if required to obtain an ongoing estimate of risk, but that other prognostic models with a longer time frame were also informative.
For the prognostic test accuracy studies, sensitivity was considered to be critical. It represents the probability that a person at risk of developing adverse outcomes is correctly identified as being at risk. The committee considered that it was important to ensure that women at risk of complications were correctly identified, as the consequences of these complications can be severe.
The quality of the evidence
Eight publications providing external validation of 4 different clinical prediction models were included. For these studies, the quality of the evidence was assessed with the CASP clinical prediction rule. The quality of the evidence ranged from moderate to high. Main sources of bias included not describing the population used to validate the model, which is a limitation because it remains unknown how the demographic characteristics of the population compares to the population that the model will be applied to in clinical practice. Another limitation seen across some of these studies was lack of clarity as to whether the predictor variables were evaluated in a blinded fashion, which is a source of bias because it is not clear whether the prior knowledge of some of the outcomes may have influenced the findings. Finally, not reporting the statistical methods used to construct and validate the tool was a limitation seen in some of the studies.
Two systematic reviews of prognostic test accuracy studies were included. The quality of these systematic reviews ranged from low to moderate. Main limitations were not including enough detail about the included population (such as the definition of pre-eclampsia or total number of women) and not including a list of excluded studies.
Six prognostic test accuracy studies were included. A modified version of GRADE, using the same principles for assessing the quality of the evidence, was used as GRADE is not yet available for prognostic test accuracy studies. The quality of the evidence ranged from very low to high. The domain risk of bias was assessed with the QUADAS-2 checklist and the main limitations seen across studies were lack of clarity about whether the results of the reference standard were interpreted without prior knowledge of the adverse outcomes and vice versa. No serious issues were found regarding inconsistency (heterogeneity) since studies were analysed individually. In evaluating the accuracy of the studies, imprecision was assessed using the 95% confidence interval of sensitivity as the primary measure because of the harmful negative consequence of a false negative (for example, death caused by a woman at high risk of developing serious consequences due to severe pre-eclampsia incorrectly identified as being at low risk). Indirectness was not found in any of the studies, as only women with confirmed or suspected pre-eclampsia were included.
Overall, the committee believed that the quality of the evidence was robust enough to base recommendations on, and the evidence reported was consistent with their clinical experience.
Benefits and harms
Moderate to high quality evidence from 5 prospective and retrospective cohort studies showed that the fullPIERS model has good ability to discriminate women at higher and lower risk of developing adverse outcomes due to pre-eclampsia within 48 hours. The committee noted that the accuracy of the fullPIERS model was best at the extremes of risk – i.e. a predicted risk of ≥30% correlated strongly with a high actual risk of adverse outcome. The studies included different populations of women, with some samples also including women with HELLP and/or severe onset pre-eclampsia, and varied rates of adverse events were seen, but the discrimination as assessed by the AUC ROC was found to be good across studies and the likelihood ratio in the highest risk category (≥30%) ranged from moderately useful to very useful.
The committee considered that the fullPIERS could be used in all women with pre-eclampsia, despite the majority of external validation studies only including participants at very preterm gestations (with a median gestational age of approximately 30 weeks). This is because the original development and validation study (von Dadelszen 2011) participants included with a wider range of gestations, with a median (IQR) of 33.9 weeks (30.0 to 36.6) for women who developed adverse outcomes and 36.6 weeks (33.4 to 38.3) for women who did not develop adverse outcomes.
The currently available version of the fullPIERS tool uses aspartate transaminase (AST) as a measure of liver function. However, the committee noted that many units in the UK only measure alanine transaminase (ALT) in routine care. The committee were aware that the levels of these two parameters are highly correlated, and subsequent discussion with the authors of fullPIERS have confirmed that AST and ALT can be used interchangeably in the model, and since the committee meeting the model has been updated to allow for use of either AST or ALT in the future (Personal communication, Peter von Dadelszen).
It was noted by the committee that the PREP models were developed within a UK population, and therefore management was likely to be relevant and representative. Whilst there were fewer external validation studies of PREP-S (as compared to fullPIERS), all validation studies were conducted in high-income countries, similar to the UK. Therefore the relevance of the PREP model and validation to the UK population was felt to be high. The PREP-S model did provide performance data for 48 hours and showed good calibration in the lower risk category, not good to excellent calibration in the middle risk categories, but poor calibration in the highest risk category (although the model over-predicted risk, and therefore was considered to be safe, rather than unsafe). Furthermore, the committee were aware that the high cost of carrying out further validation studies meant that these were unlikely to be conducted. The committee balanced this representation of the population of interest with the other data available on the models and agreed that a choice of fullPIERS or PREP-S should be recommended.
The committee discussed the other models that had been included in the review – miniPIERS and PREP-L. There was a smaller body of externally validated performance evidence for these models compared to the fullPIERS, with only 2 validation studies for miniPIERS, and 1 for PREP-L. The miniPIERS model had a moderately informative likelihood ratio in the highest risk category (compared to moderately to very useful for the fullPIERS). The committee noted that this model was developed and intended for use in low-income settings, where the results of other parameters included in the fullPIERS model (such as blood tests) were not available. Therefore it was not considered to be of such relevance to the UK setting as the fullPIERS and PREP-S models. For the PREP-L model, data was available for adverse maternal outcomes by discharge, and was limited to calibration and discrimination assessed by the C-statistic, although these were found to be good and moderate to good respectively. However, the committee considered that prediction of risk on a shorter timescale (48 hours) was of more value to guide immediate management, such as admission to hospital, as compared to the longer timeframe of PREP-L
The committee discussed the use of the fullPIERS and PREP prediction models in clinical practice. It is suggested by the authors of the fullPIERS model that a ≥30% risk of adverse maternal outcomes within 48 hours is used as a threshold to ‘rule in’ women who require further surveillance and possibly interventions. The committee agreed that at this level the risk is significantly higher than the background risk of adverse outcome for any pregnant woman, and therefore women with a risk of ≥30% should be offered admission to hospital for surveillance and interventions. The developers of the PREP-S model suggest that a risk of complications of 50% or higher should be an indication for transfer to a tertiary unit, but do not recommend which threshold should be used to guide admission to hospital. The committee were aware that the two models might lead to different risk scores for the same woman, due to differences in the clinical parameters included in the models and the outcomes. This would be confusing for clinicians who may then be faced with a score from one model that suggested admission was necessary, and a score from the other model that suggested it was not. The committee agreed that it was not therefore helpful to set a particular cut-off risk threshold when using either fullPIERS or PREP-S, but just to recommend that these models could be used as guides to aid decision-making. The committee were keen that healthcare professionals should not use the fullPIERS or PREP-S models in isolation and as the only threshold to offer admission to hospital, and that they should always be used in conjunction with a full clinical assessment. The committee agreed that there may be a variety of other circumstances in which admission to hospital should be offered – such as severe hypertension (i.e. systolic BP ≥160mmHg), concerns about the baby, concerns about maternal symptoms of pre-eclampsia or biochemical or haematological results that caused concern, and the committee defined these other criteria, based on the International Society for the Study of Hypertension in Pregnancy (ISSHP) definition of pre-eclampsia (Brown, 2018), but emphasised that this list was not exhaustive and if there were any other concerns about the wellbeing of the woman or baby, then women should be admitted.
The committee were aware that the fullPIERS and PREP models do not predict adverse outcomes for the baby. These are also of serious concern for women with pre-eclampsia and health care professionals, and the committee chose to highlight this in a recommendation, to ensure that those utilising the models for risk prediction consider potential risks to the baby in addition to the woman.
The committee agreed that the fullPIERS and PREP tools were free, easily accessible and easy to use, and would help identify women who were at a high risk of developing complications so they could receive appropriate treatment and monitoring. This may lead to a reduction in complications and adverse events.
The committee discussed the fact that a ‘high risk’ score might lead to anxiety in women, and as this is only a risk score, not all of these women would subsequently go on to develop an adverse outcome. In balancing the risk of causing unnecessary anxiety to women and the benefits of identifying at-risk women, the committee thought it was more important to identify at-risk women and that this outweighed the potential anxiety the test result might cause.
The committee agreed that none of the other prognostic test performance measures were as useful as the fullPIERS or PREP-S tools. The group specifically discussed the prognostic ability of urine sPCR and urine sACR. Urine sPCR and sACR had a moderate to high sensitivity, but very low specificity for predicting adverse outcomes arising due to pre-eclampsia. Although an elevated sPCR or sACR are common findings in women with pre-eclampsia, they do not help to discriminate between those who will and will not develop an adverse maternal or perinatal outcome. For this reason, the group decided not to recommend the use of these tests to identify women at high risk of adverse outcome, although they were recognised to be useful for the identification of significant proteinuria, as part of the diagnosis of pre-eclampsia (see Evidence report G).
Cost effectiveness and resource use
A systematic review of the economic literature was conducted but no relevant studies were identified which were applicable to this review question. An economic analysis was undertaken for this question assessing the cost-effectiveness of risk prediction models for guiding inpatient and outpatient management in pregnant women with pre-eclampsia.
The base case results of the analysis suggest that using the fullPIERS risk model with a threshold of 30% for inpatient management is cost-effective in women 34-37 weeks of gestation. It was found to be more costly than a strategy of outpatient management but also more effective and overall was found to be cost-effective with an ICER below the threshold of £20,000 per QALY. All other strategies were found to be more costly and more effective than using the fullPIERS risk model with a threshold of 30% for inpatient management but none were cost-effective with ICERs well above the threshold of £20,000 per QALY. However, there was uncertainty around this result in sensitivity analysis, which showed outpatient management to be cost-effective in numerous plausible scenarios.
The fullPIERS and PREP models require input of parameters that are routinely collected in clinical practice (i.e. gestational age, presence/absence of chest pain or dyspnoea, oxygen saturation, platelet count, creatinine, and a liver function test) therefore the recommendations are not likely to lead to more monitoring or blood tests in women, but will improve the consistency of parameters used across centres.
Currently there is variation in practice regarding admission to hospital of women with pre-eclampsia: some units admit all women, some units admit certain women, and some admit very few. The committee believed that the recommendations may lead to increases in workload and use of resources due to a potentially larger number of admissions for pre-eclampsia in some units, but this may be balanced out by more selective admission to other units. However, there may also be a cost saving, as some adverse events should be prevented, by the prompt identification and appropriate management of women at high risk. Furthermore, the occurrence of an adverse event in the community (rather than in hospital) is likely to incur additional resource use, and potentially lead to a worse outcome for the woman and her baby.
Other factors the committee took into account
The committee discussed the threshold of risk for offering admission to hospital in detail. There was consensus that the level of risk that was acceptable to an individual woman was likely to vary greatly – with some women prepared to accept a higher risk, in order to avoid admission to hospital. The committee agreed that the fullPIERS and PREP tools could help women and clinicians to share decision making regarding place of care, and short term management.
The committee also noted that the fullPIERS tool could be used to predict adverse outcomes in a 48 hour timeframe, and a 7 day timeframe. However, the accuracy of the tool was greater when used to predict risk in the next 48 hours. However, the committee also discussed that the tool could be used repeatedly in the same individual, so a woman who had been assessed as being at low risk could be reviewed again 48 hours later. Also, if there was a change in her condition, the parameters could be re-assessed, and the tool could be used again to predict risk for the next 48 hours.
References
Agrawal 2016
Agrawal S, Maitra N. Prediction of adverse maternal outcomes in preeclampsia using a risk prediction model. The Journal of Obstetrics and Gynecology of India, 2016. 1; 66(1):104–11 [PMC free article: PMC5016414] [PubMed: 27651587]Akkermans 2014
Akkermans J, Payne B, von Dadelszen P, Groen H, de Vries J, Magee LA, Mol BW, Ganzevoort W. Predicting complications in pre-eclampsia: external validation of the fullPIERS model using the PETRA trial dataset. European Journal of Obstetrics and Gynecology and Reproductive Biology. 2014 Aug 1; 179:58–62. [PubMed: 24965981]Almeida 2017
Almeida ST, Katz L, Coutinho I, Amorim MM. Validation of fullPIERS model for prediction of adverse outcomes among women with severe pre‐eclampsia. International Journal of Gynecology & Obstetrics. 2017 Aug 1; 138(2):142–7. [PubMed: 28475234]AMSTAR checklist
Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. British Medical Journal 2017 Sep 21;358:j4008. [PMC free article: PMC5833365] [PubMed: 28935701]Broekhuijsen 2015
Broekhuijsen K, van Baaren G-J, van Pampus MG, Ganzevoort W, Sikkema JM, Woiski M, Oudijk , Bloemenkamp K, Scheepers H, Bremer H, Rijnders R, Van Loon A, Perquin D, Sporken J, Papatsonis D, van Huizen M, Vredevoogd C, Brons J, Kaplan M, van Kaam A, Groen H, Porath M, van den Berg P, Mol B, Franssen M, Langenveld J, for the HYPITAT-II study group.Immediate delivery versus expectant monitoring for hypertensive disorders of pregnancy between 34 and 37 weeks of gestation (HYPITAT-II): an open-label, randomised controlled trial. Lancet 2015; 385:2492–2501 [PubMed: 25817374]Brown 2018
Brown, M., Magee, L., Kenny, L., Karumanchi, A., McCarthy, F., Saito, S., Hall, D., Warren, C., Adoyi, G., Ishaku, S., on behalf of the International Society for the Study of Hypertension in Pregnancy (ISSHP). Hypertensive disorders of pregnancy. ISSHP Classification, Diagnosis and Management Recommendations for International Prqactice. Hypertension, 72, 24–43. 2018 [PubMed: 29899139]CASP checklist
Critical Appraisal Skills Programme. CASP Clinical Prediction Rule Checklist [online]. Available at https://casp-uk.net/wp-content /uploads/2018 /03/CASP-Clinical-Prediction-Rule-Checklist-Download.pdf Chan 2005
Chan P, Brown M, Simpson JM, Davis G. Proteinuria in pre‐eclampsia: how much matters?. BJOG: An International Journal of Obstetrics & Gynaecology. 2005 Mar; 112(3):280–5. [PubMed: 15713140]Debray 2017
Debray TP, Damen JA, Snell KI, Ensor J, Hooft L, Reitsma JB, Riley RD, Moons KG. A guide to systematic review and meta-analysis of prediction model performance. British Medical Journal 2017 Jan 5; 356:i6460. [PubMed: 28057641]Deeks 2004
Deeks, Jonathan J., and Douglas G. Altman. Diagnostic tests 4: likelihood ratios. British Medical Journal 2004 329.7458: 168–169. [PMC free article: PMC478236] [PubMed: 15258077]Edwards 2006
Edwards SJ, Campbell HE, Plumb JM. Cost-utility analysis comparing meropenem with imipenem plus cilastatin in the treatment of severe infections in intensive care. Eur J Health Econ 7(1):72–8 2006. [PubMed: 16429296]Jaeschke 1994
Jaeschke R, Guyatt G, Sackett DL, Bass E, Brill-Edwards P, Browman G, Cook D, Farkouh M, Gerstein H, Haynes B, Hayward R. Users’ Guides to the Medical Literature: III. How to Use an Article About a Diagnostic Test A. Are the Results of the Study Valid?. JAMA. 1994 Feb 2;271(5):389–91. [PubMed: 8283589]Laskin 2011
Laskin S, Payne B, Hutcheon JA, Qu Z, Douglas MJ, Ford J, Lee T, Magee LA, von Dadelszen P. The role of platelet counts in the assessment of inpatient women with preeclampsia. Journal of Obstetrics and Gynaecology Canada. 2011 Sep 1;33(9):900–8. [PubMed: 21923987]Livingston 2014
Livingston JR, Payne B, Brown M, Roberts JM, Côté AM, Magee LA, von Dadelszen P, PIERS Study Group. Uric Acid as a predictor of adverse maternal and perinatal outcomes in women hospitalized with preeclampsia. Journal of Obstetrics and Gynaecology Canada. 2014 Oct 1; 36(10):870–7. [PubMed: 25375299]NHS reference costs 2016/17
National Schedule of Reference Costs 2016-17. NHS trusts and NHS foundation trusts.Payne 2014
Payne BA, Hutcheon JA, Ansermino JM, Hall DR, Bhutta ZA, Bhutta SZ, Biryabarema C, Grobman WA, Groen H, Haniff F, Li J. A risk prediction model for the assessment and triage of women with hypertensive disorders of pregnancy in low-resourced settings: the miniPIERS (Pre-eclampsia Integrated Estimate of RiSk) multi-country prospective cohort study. PLoS Medicine. 2014 Jan 21; 11(1):e1001589. [PMC free article: PMC3897359] [PubMed: 24465185]Payne 2015
Payne BA, Hutcheon JA, Dunsmuir D, Cloete G, Dumont G, Hall D, Lim J, Magee LA, Sikandar R, Qureshi R, van Papendorp E. Assessing the Incremental Value of Blood Oxygen Saturation (SpO 2) in the miniPIERS (Pre-eclampsia Integrated Estimate of RiSk) Risk Prediction Model. Journal of Obstetrics and Gynaecology Canada. 2015 Jan 31; 37(1):16–24. [PubMed: 25764032]QUADAS-2 checklist
Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeflang MM, Sterne JA, Bossuyt PM. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Annals of Internal Medicine. 2011 Oct 18;155(8):529–36. [PubMed: 22007046]Sonnenberg 2004
Sonnenberg FA, Burkman RT, Hagerty CG, Speroff L, Speroff T. Costs and net health effects of contraceptive methods. Contraception 2004;69:(6)447–59. [PubMed: 15157789]Thangaratinam 2011
Thangaratinam, Shakila, et al. Accuracy of liver function tests for predicting adverse maternal and fetal outcomes in women with preeclampsia: a systematic review. Acta obstetricia et gynecologica Scandinavica 2011; 90.6; 574–585. [PubMed: 21355861]Thangaratinam 2017
Thangaratinam S, Allotey J, Marlin N, Dodds J, Cheong-See F, von Dadelszen P, Ganzevoort W, Akkermans J, Kerry S, Mol BW, Moons KG. Prediction of complications in early-onset pre-eclampsia (PREP): development and external multinational validation of prognostic models. BMC Medicine. 2017 Dec; 15(1):68. [PMC free article: PMC5372261] [PubMed: 28356148]Ukah 2017a
Ukah UV, Payne B, Lee T, Magee LA, von Dadelszen P. External Validation of the fullPIERS Model for Predicting Adverse Maternal Outcomes in Pregnancy Hypertension in Low-and Middle-Income Countries. Hypertension. 2017 Apr;69(4):705–11. [PubMed: 28167685]Ukah 2017b
Ukah UV, Hutcheon JA, Payne B, Haslam MD, Vatish M, Ansermino JM, Brown H, Magee LA, von Dadelszen P. Placental growth factor as a prognostic tool in women with hypertensive disorders of pregnancy: a systematic review. Hypertension. 2017 Dec;70(6):1228–37. [PMC free article: PMC5680987] [PubMed: 29084878]Ukah 2018
Ukah UV, Payne B, Hutcheon JA, Ansermino JM, Ganzevoort W, Thangaratinam S, Magee LA, von Dadelszen P. Assessment of the fullPIERS Risk Prediction Model in Women With Early-Onset Preeclampsia. Hypertension. 2018 Apr; 71(4):659–65. [PMC free article: PMC5865495] [PubMed: 29440330]von Dadelszen 2011
von Dadelszen P, Payne B, Li J, Ansermino JM, Pipkin FB, Côté AM, Douglas MJ, Gruslin A, Hutcheon JA, Joseph KS, Kyle PM. Prediction of adverse maternal outcomes in pre-eclampsia: development and validation of the fullPIERS model. The Lancet. 2011 Jan 15;377(9761):219–27. [PubMed: 21185591]Waugh 2017
Waugh J, Hooper R, Lamb E, Robson S, Shennan A, Milne F, Price C, Thangaratinam S, Berdunov V, Bingham J. Spot protein-creatinine ratio and spot albumin-creatinine ratio in the assessment of pre-eclampsia: a diagnostic accuracy study with decision-analytic model-based economic evaluation and acceptability analysis. Health technology assessment (Winchester, England). 2017 Oct;21(61):1. [PMC free article: PMC5672500] [PubMed: 29064366]
Appendices
Appendix A. Review protocol
Appendix B. Literature search strategies
Databases: Medline; Medline EPub Ahead of Print; and Medline In-Process & Other Non-Indexed Citations
Databases: Embase; and Embase Classic
Databases: Cochrane Central Register of Controlled Trials; Cochrane Database of Systematic Reviews; Database of Abstracts of Reviews of Effects; and Health Technology Assessment
Health economics search strategies
Databases: Medline; Medline EPub Ahead of Print; and Medline In-Process & Other Non-Indexed Citations
Databases: Embase; and Embase Classic
Database: Cochrane Central Register of Controlled Trials
Databases: Health Technology Assessment; and NHS Economic Evaluation Database
Appendix C. Clinical evidence study selection
Appendex D. Clinical evidence tables
Table 9. Clinical evidence tables (PDF, 1.3M)
Appendix E. Forest plots
No forest plots were generated for this review question as it is not applicable to this review question.
Appendix F. GRADE tables
Table 10. fullPIERS model performance for prediction of adverse maternal outcomes within 48 hours
Table 11. fullPIERS model performance for prediction of adverse maternal outcomes within 7 days
Table 13. miniPIERS model performance for prediction of adverse maternal outcomes within 48 hours
Table 17. Quality assessment of prognostic test accuracy studies for abnormal coagulation
Table 18. Quality assessment of prognostic test accuracy studies for liver function
Table 19. Quality assessment of prognostic test accuracy studies for uric acid
Appendix G. Economic evidence study selection
Appendix H. Economic evidence tables
No economic evidence was identified for this review question.
Appendix I. Health economic evidence profiles
No economic evidence was identified for this review question.
Appendix J. Health economic analysis
Aim
The aim of this economic analysis is to estimate the cost-effectiveness of risk prediction models for guiding inpatient and outpatient management in pregnant women with pre-eclampsia.
Methods
Existing economic evidence
A systematic literature review was conducted to identify economic evaluations that may be applicable to the current decision problem. No relevant economic studies were identified that were directly applicable.
De novo economic evaluation
Since the current economic literature did not adequately address the decision problem, a de novo economic evaluation was undertaken to assess cost-effectiveness. The analysis was developed in Microsoft Excel® and was conducted from the perspective of the NHS and Personal Social Services (PSS) as outlined in the NICE reference case (see Developing NICE guidelines: the manual).
The modelled time horizon was 20 days, which reflects the estimated amount of time between women being assessed and giving birth (6 days) plus two additional weeks to capture the duration of QoL effects. This short time horizon was selected because the model is focusing on short term outcomes and currently there is no evidence to inform longer term differences between the strategies. Discounting of costs and benefits was not undertaken because of the short time horizon.
Clinical data and model approach
The economic analysis considered strategies where the decision on whether to manage pre-eclampsia in women as an outpatient or inpatient was based on risk thresholds (e.g. to offer inpatient management with a risk score ≥ 10%). The analysis considered the fullPIERS risk assessment tool, which was selected because it has the best available evidence. Other risk assessment tools such as PREP-S could also be used in clinical practice but it was not possible to include them in the economic model because there is insufficient data on diagnostic accuracy (sensitivity and specificity) at various risk levels.
Management strategies based on risk leve were compared against each other and also against strategies where it is assumed that all women are managed as either an inpatient or outpatient.
It is unclear which strategy would best represent current clinical practice as there is known to be variation. However, it is thought that inpatient management is generally more common than outpatient management. Note that this does not affect the current analysis as the intention is to compare all strategies against each other to determine the most cost-effective strategy. This is a separate endeavour to estimating cost impact which aims to estimate the change in cost associated with the adoption of a new strategy compared to current practice.
The economic analysis considered women 34-37 weeks of gestation reflecting the population in which the fullPIERS risk prediction model is applicable. The following management strategies were considered in the analysis:
- All inpatient management
- All outpatient management
- Inpatient management if fullPIERS ≥ 5%
- Inpatient management if fullPIERS ≥ 10%
- Inpatient management if fullPIERS ≥ 20%
- Inpatient management if fullPIERS ≥ 30%
Prevalence and accuracy data
The economic analysis was based on accuracy data (sensitivity and specificity) for the prediction of complications at 2 and 7 days for each of the strategies (see Table 22). In the model, the diagnostic results are linked to subsequent management whereby women with positive results are managed as inpatients and women with negative results are managed as outpatients.
Data on the prevalence of adverse outcomes as well as data on the accuracy of fullPIERS at different thresholds were estimated from an external validation study (Akkermans 2014). Akkermans showed that 32 of 216 women (14.8%) had an adverse outcome after 48 hours and 62 of 216 women (28.7%) had an adverse outcome after 7 days. Accuracy data for the ‘all inpatient management’ and ‘all outpatient management’ were inferred based on the implications of the strategy e.g. all patients managed as an inpatient implies that all patients with complications would be managed as an inpatient and therefore the sensitivity would be 100%.
In clinical practice risk models are likely to only be used to predict short term outcomes. This reflects the available data which suggests a much better performance when predicting short term outcomes (as can be seen from the accuracy data at 48 hours and 7 days). To reflect the manner in which risk models are employed in clinical practice, it was therefore assumed that women that are managed on an outpatient basis would be re-assessed evey two days. In the model this is estimated by applying the 48 hour diagnostic accuracy data again for women that were being managed as an outpatient following the initial test (i.e. initially found to have a risk score under the threshold).
Effectiveness data
It has been assumed that women managed in an inpatient setting would have a reduction in the number of adverse maternal outcomes. There is no good evidence available on which to base this reduction. Therefore it was speculatively approximated using data from Broekhuijsen 2015 (HYPITAT II study), which compared immediate delivery with expectant management. It has been assumed that the reduction in adverse outcomes associated with being managed in an inpatient setting rather than an outpatient setting would be similar to the reduction seen with immediate delivery compared with expectant management. In comparison to expectant management, immediate delivery was found to reduce reported adverse maternal outcomes with a relative risk (RR) of 0.36 (95% CI 0.12–1.11). Therefore, this value was applied in the analysis as an estimate of the reduction in adverse maternal outcomes with the inpatient approach.
Mortality was not considered in the analysis as there is no evidence to suggest that the use of risk prediction models may confer a survival benefit. Also it is unlikely that there would be mortality differences between outpatient and inpatient management strategies.
Costs
The costs considered in the model reflect the perspective of the analysis, thus only costs that are relevant to the UK NHS and PSS were included. Where possible, all costs were estimated in 2016/17 prices. The majority of costs were sourced from NHS reference costs 2016/17 by applying tariffs associated with the appropriate Healthcare Resource Groups (HRG) code.
Risk assessment tool costs
It was assumed that there is no cost associated with using the fullPIERS risk assessment tool itself as it is freely available online. Furthermore, it was assumed that there was no additional cost associated with performing the tests required to inform the risk factors in the tool as these tests are already carried out as part of routine clinical practice.
Inpatient and outpatient management costs
Inpatient costs were estimated using the average cost of a day as an elective inpatient from NHS reference costs 2016/17 (£384.50). The average length of stay (LOS) was based on pre-eclampsia audit data, which reported an average time between diagnosis of pre-eclampsia and delivery of 6 days for women 34-37 weeks of gestation. To avoid the potential duplication of LOS costs associated with the birth itself, the average LOS associated with births was estimated from NHS reference costs (2.09 days) and deducted from the total days from the survey (resulting in 3.91 days). Outpatient costs were based on the cost of consultant led face-to-face follow-up in the obstetrics service from NHS reference costs 2016/17 (£120.20). The average duration of outpatient management was assumed to be the same as inpatient management and it was assumed that patients would have re-assessments every 2 days.
Birth and complication costs
Birth costs were estimated using data on the proportions of each mode of delivery from Broekhuijsen 2015 (HYPITAT II study) and are shown in Table 23. A combined average of the immediate delivery and expectant management arms of the trial was estimated resulting in proportions of 4%, 86% and 10% for spontaneous labour, induction of labour and caesarean section, respectively. Birth costs for the various modes of delivery were sourced from NHS Reference Costs 2016/17 assuming that women with adverse outcomes would have births with complications and co-morbidities (based on CC scores). Birth costs were estimated by taking a weighted average of births recorded in NHS reference costs as an elective inpatient, non-elective long stay and non-elective short stay.
It was assumed that women with an adverse outcome would be admitted to a high dependency unit (HDU). A HDU cost of £860.61 was estimated from NHS reference costs 2016/17, based on the weighted average cost of “adult critical care, 0 organs supported” and “adult critical care, 1 organs supported” (see Table 24).
Based on a combined average of the immediate delivery and expectant management arms from Broekhuijsen 2015 (HYPITAT II study), it was assumed that a NICU admission would be required in 5.6% of births. NICU admission costs were estimated from NHS reference costs 2016/17, based on the cost of neonatal critical care, intensive care (£1,295)
Health-related quality of life
As recommended in the NICE reference case, the model estimates effectiveness in terms of quality adjusted life years (QALYs). These are estimated by combining life year estimates with quality of life (QoL) values associated with being in a particular health state.
QoL data were sourced from the economic analysis conducted as part of the previous guideline (NICE CG107). Pregnant women with pre-eclampsia were assumed to have the same QoL value as normotensive pregnant women. The QoL value for normotensive pregnant women was sourced from Sonnenberg 2004, a cost effectiveness analysis of contraception methods in women of average health and fertility, which found that short-term utility loss due to pregnancy was 0.0375. Therefore the baseline utility value applied in the model for pregnant women with pre-eclampsia was estimated to be 0.9625 (1-0.0375).
Experiencing severe complications of pre-ecalmpsia was assumed to have the same QoL as being admitted to ICU for any reason. As part of a cost effectiveness analysis of meropenem in the treatment of severe infections in hospital intensive care, Edwards 2006 estimated that the QoL weight for someone who has stayed in intensive care was 0.712. The QoL weight for women with complications was assumed to be the product of the QoL value for being admitted to ICU for any reason (0.712) and the QoL value for pregnant women with pre-eclampsia (0.9625). The QoL value for experiencing adverse outcomes was parameterised in the model as a QoL decrement (estimated by deducting the QoL weight for women with complications from the baseline value for pregnant women with pre-eclampsia) and applied accordingly.
Following the methodology adopted in the economic analysis conducted as part of the previous guideline (NICE CG107), it was assumed that the QoL decrement for women with severe disease would last for 2 weeks, reflecting the estimated period of time that women may stay in ICU.
In order to estimate QALYs these values were converted to daily weights and applied for the modelled time horizon.
Sensitivity analysis
Uncertainty was assessed in the economic model through deterministic and probabilistic sensitivity analysis. A series of deterministic sensitivity analyses were conducted, whereby an input parameter was changed, the model was re-run and the new cost-effectiveness result was recorded. This form of analysis is a useful way of estimating uncertainty and determining the key drivers of the model results.
Probabilistic sensitivity analysis (PSA) was conducted to assess the combined parameter uncertainty in the model. In this analysis, the mean values that were utilised in the base-case were replaced with values drawn from distributions around the mean values. Table 25 gives a full list of the input parameters included in the model along with details of the distributions applied in the PSA.
Table 25. Full list of model inputs with details of PSA distributions
Results
Base-case results
The base case results of the analysis are shown in Table 26 and Table 27.
In Table 26, each strategy is compared against inpatient management (the strategy assumed to be the most likely to be used in clinical practice). It can be seen that all risk management strategies as well as a strategy of outpatient management for all women are much less costly and marginally less effective than inpatient management. This results in very high ICER values which indicate that large cost savings are made for each QALY that is lost (note that the ICER interpretation is non-standard because of negative costs and QALYs). Therefore, the results indicate that all risk management strategies as well as outpatient management are cost-effective in comparison to inpatient management.
In Table 27, a ‘dominance rank’ approach is presented which allows all strategies to be compared against each other. This approach involves rank ordering strategies in terms of cost and then comparing each intervention in turn against the previous intervention that was found to be cost-effective.
A strategy of outpatient management was the least costly strategy overall. All other strategies were found to be more costly and more effective than outpatient management. Inpatient management if fullPIERS ≥ 30% was found to be cost-effective with an ICER value of £10,797 per QALY which is below the threshold of £20,000 per QALY. All other strategies were not found to be cost-effective with ICERs well above the threshold of £20,000 per QALY. Therefore the strategy of inpatient management if fullPIERS ≥ 30% was found to be the optimal strategy in cost-effectiveness terms.
Table 26. Base case results in comparison to inpatient management
Deterministic sensitivity analysis results
The results of the deterministic sensitivity analysis are presented in Table 28. It can be seen that the conclusion of the analysis changes in numerous scenarios with outpatient management found to be cost-effective in certain scenarios. Notably this includes numerous plausible scenarios such as where variations in the RR for adverse outcomes is applied or when the cost of adverse outcomes is changed.
Threshold analysis results
A threshold analysis was conducted to determine the RR for adverse outcomes required for the inpatient management if fullPIERS ≥ 30% strategy to be cost-effective. It was found that a strategy of inpatient management if fullPIERS ≥ 30% was cost-effective with a RR of 0.395 or lower.
Probabilistic sensitivity analysis results
The results of 10,000 runs of the PSA are shown using cost-effectiveness acceptability curves (CEAC) Figure 2. The CEAC graph shows the probability of each strategy being considered cost-effective at various cost-effectiveness thresholds on the x axis.
It can be seen that outpatient management and a strategy of inpatient management if fullPIERS ≥ 30% have the highest probabilities of being cost-effective at all thresholds. Outpatient management is initially the preferred option with the strategy having the highest probability of being cost-effective at a threshold of £0 per QALY. As the threshold increases, the strategy of inpatient management if fullPIERS ≥ 30% becomes the preferred option. At the threshold of £20,000 per QALY used by NICE, inpatient management if fullPIERS ≥ 30% has a 53% probability of being cost-effective while outpatient management has a 46% probability of being cost-effective. All other strategies were found to have a 0% probability of being cost-effective at the threshold of £20,000 per QALY.
Figure 2. Cost-effectiveness acceptability curves (CEACs)
The results indicate that the comparison between outpatient management and a strategy of inpatient management if fullPIERS ≥ 30% is of the most importance from a cost-effectiveness standpoint. Therefore this comparison is further examined using the ICER scatterplot in Figure 3 which shows the incremental costs and QALYs for inpatient management if fullPIERS ≥ 30% compared to outpatient management for each of the 10,000 runs of the PSA along with the mean result.
From the ICER scatterplot, it can be seen that the vast majority of results reside on the East side of the graph, indicating that a strategy of inpatient management if fullPIERS ≥ 30% is more effective in the vast majority of modelled scenarios. Some of the results reside in the South East quadrant indicating that a strategy of inpatient management if fullPIERS ≥ 30% is more effective and less costly than outpatient management. The majority of the results appear to reside in the North East quadrant indicating that a strategy of inpatient management if fullPIERS ≥ 30% is more effective and more costly than outpatient management. Overall it can be seen that a marginal majority of results lie under the cost-effectiveness threshold line, indicating that a strategy of inpatient management if fullPIERS ≥ 30% is cost-effective more often than outpatient management (which is reflected in the CEAC result).
Figure 3. ICER scatterplot for fullPIERS ≥ 30% in comparison to outpatient management
Conclusion
The base case results of the analysis suggest that using the fullPIERS risk model with a threshold of 30% for inpatient management is cost-effective in women at 34-37 weeks of gestation. However, it should be noted that there are gaps in the clinical evidence base and therefore several assumptions have been made to run the analysis. Most notably, a speculative assumption was made around the reduction in the number of adverse maternal outcomes. Furthermore, deterministic sensitivity analysis suggested that differences in assumptions have the potential to change the conclusion of the analysis and probabilistic sensitivity analysis demonstrated some uncertainty around the result.
Appendix K. Excluded studies
Clinical studies
Table 29. Clinical excluded studies with reasons for exclusion
Economic studies
Table 30. Economic excluded studies with reasons for exclusion
Appendix L. Research recommendations
No research recommendations were made for this review question.
FINAL
Evidence reviews
These evidence reviews were developed by The National Guideline Alliance hosted by the Royal College of Obstetricians and Gynaecologists
Disclaimer: The recommendations in this guideline represent the view of NICE, arrived at after careful consideration of the evidence available. When exercising their judgement, professionals are expected to take this guideline fully into account, alongside the individual needs, preferences and values of their patients or service users. The recommendations in this guideline are not mandatory and the guideline does not override the responsibility of healthcare professionals to make decisions appropriate to the circumstances of the individual patient, in consultation with the patient and/or their carer or guardian.
Local commissioners and/or providers have a responsibility to enable the guideline to be applied when individual health professionals and their patients or service users wish to use it. They should do so in the context of local and national priorities for funding and developing services, and in light of their duties to have due regard to the need to eliminate unlawful discrimination, to advance equality of opportunity and to reduce health inequalities. Nothing in this guideline should be interpreted in a way that would be inconsistent with compliance with those duties.
NICE guidelines cover health and care in England. Decisions on how they apply in other UK countries are made by ministers in the Welsh Government, Scottish Government, and Northern Ireland Executive. All NICE guidance is subject to regular review and may be updated or withdrawn.
- A prognostic model to guide decision-making on timing of delivery in late preterm pre-eclampsia: the PEACOCK prospective cohort study.[Health Technol Assess. 2021]A prognostic model to guide decision-making on timing of delivery in late preterm pre-eclampsia: the PEACOCK prospective cohort study.Duhig K, Seed PT, Placzek A, Sparkes J, Gill C, Brockbank A, Shennan A, Thangaratinam S, Chappell LC. Health Technol Assess. 2021 May; 25(30):1-32.
- Calcium supplementation commencing before or early in pregnancy, for preventing hypertensive disorders of pregnancy.[Cochrane Database Syst Rev. 2019]Calcium supplementation commencing before or early in pregnancy, for preventing hypertensive disorders of pregnancy.Hofmeyr GJ, Manyame S, Medley N, Williams MJ. Cochrane Database Syst Rev. 2019 Sep 16; 9(9):CD011192. Epub 2019 Sep 16.
- Mid-trimester uterine artery Doppler screening as a predictor of adverse pregnancy outcome in high-risk women.[Ultrasound Obstet Gynecol. 2000]Mid-trimester uterine artery Doppler screening as a predictor of adverse pregnancy outcome in high-risk women.Coleman MA, McCowan LM, North RA. Ultrasound Obstet Gynecol. 2000 Jan; 15(1):7-12.
- Review Calcium supplementation during pregnancy for preventing hypertensive disorders and related problems.[Cochrane Database Syst Rev. 2014]Review Calcium supplementation during pregnancy for preventing hypertensive disorders and related problems.Hofmeyr GJ, Lawrie TA, Atallah AN, Duley L, Torloni MR. Cochrane Database Syst Rev. 2014 Jun 24; (6):CD001059. Epub 2014 Jun 24.
- Review Interventionist versus expectant care for severe pre-eclampsia between 24 and 34 weeks' gestation.[Cochrane Database Syst Rev. 2013]Review Interventionist versus expectant care for severe pre-eclampsia between 24 and 34 weeks' gestation.Churchill D, Duley L, Thornton JG, Jones L. Cochrane Database Syst Rev. 2013 Jul 26; (7):CD003106. Epub 2013 Jul 26.
- Evidence review for prediction of complications in pre-eclampsiaEvidence review for prediction of complications in pre-eclampsia
Your browsing activity is empty.
Activity recording is turned off.
See more...