Study identification and selection

For this HTA project, a database of published and unpublished literature will be assembled from searches using a comprehensive search strategy, as well as hand searching, contacting commercial weight management organisations and consultation with experts in the area. We will communicate with major centres of obesity research and the first author of each selected study published in the last five years, with enquiry for any published or unpublished relevant studies not included on our list. Language restrictions will not be applied to electronic searches.

The following databases will be searched: MEDLINE, EMBASE, BIOSIS, LILACS, Pascal, Science Citation Index, Cochrane Database of Systematic Reviews (CDSR), Cochrane Central Register of Controlled Trials (CENTRAL), Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment Database (HTA). In addition, information on studies in progress, from commercial providers like Weight Watchers, Slimming world and unpublished research or research reported in the grey literature will be sought by searching a range of relevant databases including the Inside Conferences, Systems for Information in Grey Literature (SIGLE), Dissertation Abstracts and Clinical Trials.gov. Internet searches will also be carried out using specialist search gateways (such as OMNI: http://www.omni.ac.uk/), general search engines (such as Google: http://www.google.co.uk/) and meta-search engines (such as Copernic: http://www.copernic.com/). Citations identified by the search will be selected for inclusion in the review in a two-stage process using predefined and explicit criteria regarding populations, interventions, outcomes and study design. First, a master database of the literature searches will be constructed by amalgamation of all the citations from various database sources. The citation will be scrutinised by two reviewers. Copies of full manuscripts of all citations that are likely to meet the selection criteria will be obtained. Two reviewers will then independently select the studies, which meet the predefined criteria. These criteria will be pilot tested using a sample of papers and agreement between reviewers will be measured. Disagreements will be resolved by consensus and/or arbitration involving a third reviewer.

Study quality assessment and data extraction

The quality of the selected primary randomised controlled trials (RCT's) and observational studies will be assessed based on accepted contemporary standard. Following the GRADE methodology, the quality assessment and reporting of results will be done separately for each outcome, since even within one review the quality of the evidence can vary between outcomes. We define quality of evidence as ‘the extent of confidence that an estimate of effect is correct’. The GRADE system classifies quality of evidence into one of four levels: high, moderate, low and very low (Table 3).

TABLE 3

Quality of evidence and definitions.

To assess the quality, we consider first of all risk of bias (internal validity), i.e. the extent to which design, methods, execution and analysis did not control for bias in assessment of effectiveness (Table 4). Furthermore, we explore the (in-) consistency of results (heterogeneity), (in-) directness of the evidence (to the question under consideration, including surrogate parameters), (im-) precision of the results and publication bias. Deficiencies on those criteria in the body evidence from RCTs will lower the quality of the evidence from high to moderate or low, perhaps even very low. Deficiencies in the body of evidence from non-RCTs will lower the quality of evidence from low to very low.

TABLE 4

Criteria for assessing risk of bias.

Individual studies will be described by study type, intervention, numbers taking part, population denominator (eg pregnant women or fetuses) and study quality. In addition to using study quality as possible explanations for differences in results (heterogeneity), the extent to which primary research met methodological standards is important per se for assessing the strength of any conclusions that are reached. Studies' findings will be extracted in duplicate using pre-designed and piloted data extraction forms, which we have already developed and used in our previously completed reviews. Any disagreements will be resolved by consensus and/or arbitration involving a third reviewer. Missing information will be obtained from investigators if it is crucial to subsequent analysis. To avoid introducing bias, unpublished information will be coded in the same fashion as published information. In addition to using multiple coders to insure the reproducibility of the overview, sensitivity analyses around important or questionable judgements regarding the inclusion or exclusion of studies, the validity assessments and data extraction will be performed.

Data synthesis

We will use RevMan and Stata softwares to conduct analyses. The former will allow uniformity with Cochrane reviews and the latter will allow the data analytic flexibility that we will need to examine issues not included in the RevMan software. Separate analyses will be performed on randomised and non-randomised data. Any heterogeneity of results between studies will be statistically and graphically assessed, including use of funnel plots. We will explore causes of the heterogeneity and proceed to perform meta-analysis if appropriate. To explore causes of heterogeneity subgroup analyses will be planned a priori to see whether variations in clinical factors e.g. populations, interventions, outcomes or study quality affect the estimation of effects. Individual factors explaining heterogeneity will also be analysed using meta-regression to determine their unique contribution to the heterogeneity. Conclusions regarding the typical estimate of an effect size of the intervention will be interpreted cautiously if there is significant heterogeneity.

Study identification and selection for adverse events

We have purposefully kept the scope of the question of adverse effects of any dietary intervention on pregnant women and their children broad. This will enable us to identify a variety of adverse effects that were previously not known or recognised. The adverse outcomes to be evaluated will be in 3 groups and similar to the outcomes in the effectiveness review, they will be ranked according to their importance: critical for decision-making, important for decision making and not important.

1. clinically significant adverse maternal outcomes in pregnancy and later due dietary interventions in (i) overweight or obese women and (ii) women with normal weight
2. clinically significant adverse fetal, neonatal, childhood and adult outcomes in the offspring of pregnant women undergoing dietary interventions
3. Most common adverse effects that lead to pregnant women discontinuing the intervention

We will design a separate search strategy to identify studies on harm by including adverse effects text words and indexing terms to ensure that they are not missed in the databases previously described. We will use datasets providing counts or proportions attributed to specific interventions or weight change in pregnancy leading to maternal and fetal adverse outcomes, from direct counting or from special surveys. We use the term dataset because some sources are research studies but others are direct counts or other forms of routine data collection (such as vital registration; membership of weight reduction club, web table). We will include only those datasets that represent the target population in the final analysis. In cases of partial data duplication with overlapping datasets, we will select the most recent and largest dataset.

Study quality assessment and data extraction for adverse events

Criteria used to assess study quality will follow the same concept as for assessing study quality for effectiveness: assessing risk of bias, inconsistency of results, indirectness of the evidence, imprecision and publication bias. For assessing the risk of bias in estimating adverse event rates associated with dietary intervention in pregnancy, we will take into account existing checklists for evaluation of randomised and non-randomised studies, including study design and other features associated with outcome (e.g. small for gestational age, pre term delivery etc). For the three possible designs (RCTs, observational studies with a control group, and observational studies without controls (case series)) quality assessment and presentation of results will be done separately. Additionally, information on weight change per se on mother and baby will also be extracted as these could be associated with adverse event rates or severity. The methodological quality of all eligible datasets (‘risk of bias’) will be assessed to investigate internal validity (the extent to which the information is probably free of bias) with the following attributes:

1. reporting of adverse maternal and fetal outcome definition to reduce bias in ascertainment of denominator data in the series (any published definition reported Vs no definition)
2. adequacy of data source to ascertain a capture of denominator data that is as complete as possible (use of multiple data sources, special surveys, or clinical studies vs routine registration enrolment in weight loss programmes, in which adequate attribution of cause of harm has been shown to be questionable for maternal and fetal outcomes, leading to substantial underreporting)
3. use of a robust approach to ascertain that the cause of harm is a representation of the underlying condition that is as true as possible (confidential enquiries, use of multiple sources of outcome vs no special efforts to confirm cause)
4. sufficiently high proportion of cases with attributable cause of harm established (< 5% unclassified).

Quality assessment will be done for each outcome. Randomised studies will start as high quality, observational studies with controls will start as low quality, and uncontrolled studies will start as very low quality. The evidence will be downgraded in the presence of methodological weaknesses and uncertainty; it can be upgraded in the presence of large effects, dose–response gradient and remaining plausible confounding which would reduce a demonstrated effect. Based on these criteria, the datasets will be classified into different quality groups.

Data synthesis for adverse events

The number of adverse events reported in pregnant women and children will be obtained for each intervention to compute a percentage of the total number of women and children in whom the occurrence of that particular adverse event or confirmation of its absence was reported. It is inappropriate to calculate adverse events rates from case studies, thus a qualitative summary will be undertaken. Quantitative adverse events rates calculations will be restricted to series of women undergoing dietary interventions and weight change as identified from RCTs and observational studies, with and without controls (case series). We shall quantify the adverse events as relative risks and 95% confidence intervals. The point estimates of proportions and their 95% CIs will be represented in forest plots to explore heterogeneity and the possibility of the differences being due to chance assessed statistically by Cochran Q test. To explore the presence of heterogeneity and its causes, regression models will be adjusted to the proportions attributed to every individual cause of maternal and fetal complications. The proportions will be transformed with the logit transformation. Explanatory variables considered in these models are: type of intervention and dataset methodological quality items.