Synopsis
The aim of this document is to provide practical help on the choice of target difference used in the sample size calculation of a randomised controlled trial (RCT). Advice is provided with a definitive trial, that is, one that seeks to provide a useful answer, in mind and not those of a more exploratory nature. The term ‘target difference’ is taken throughout to refer to the difference that is used in the sample size calculation (the one that the study formally ‘targets’). Please see the Glossary for definitions and clarification with regard to other relevant concepts. To address the specification of the target difference, it is appropriate, and to some degree necessary, to touch on related statistical aspects of conducting a sample size calculation. Generally, the discussion of other aspects and more technical details is kept to a minimum, with more technical aspects covered in the appendices and referencing of relevant sources provided for further reading.
The main body of this report assumes a standard RCT design is used; formally, this can be described as a two-arm parallel-group trial. Most RCTs test for superiority of the interventions, that is whether or not one of the interventions is superior to the other (Box 1 provides a formal definition of superiority and of the two most common alternative approaches). A rationale for the report is provided in Rationale for this report and a summary of the research stages that were used to inform this report is provided in Development of the DELTA2 advice and recommendations. Appendices 1 and 2 provide fuller details, which have also been published elsewhere.9 The conventional approach to sample size calculations is discussed along with other relevant topics in Appendix 3. Additionally, it is assumed in the main body of the text that the conventional (Neyman–Pearson) approach to the sample size calculation of a RCT is being used. Other approaches (Bayesian, precision and value of information) are briefly considered in Appendix 4, with reference to the specification of the target difference. Some of the more common alternative trial designs to a two-arm parallel-group superiority trial are considered in Appendix 5.
Superiority, equivalence and non-inferiority trials
Rationale for this report
A RCT is widely considered to be the optimal study design to assess the comparative clinical efficacy and effectiveness along with the cost implications of health interventions.1 RCTs are routinely used to assess the use of new drugs prior to, and in order to secure, approval for releasing new drugs to market. More generally, they have also been widely used to evaluate a range of interventions and have been successfully used in a variety of health-care settings. An a priori sample size calculation ensures that the study has a reasonable chance of achieving its prespecified objectives.10
A number of statistical approaches exist for calculating the required sample size.1,11,12 However, a recent review of 215 RCTs in leading medical journals identified only the conventional (Neyman–Pearson) approach in use.13 This approach requires establishment of the statistical significance level (type I error rate) and power (1 minus the type II error rate), alongside the target difference (‘effect size’). Setting the statistical significance level and power represents a compromise between the possibility of being misled by chance, when there is no true difference between the interventions, and the risk of not identifying a difference, when one of the interventions is truly superior, whereas the target difference is the magnitude of difference to be detected between sample sets. The required sample size is very sensitive to the target difference. Halving it roughly quadruples the sample size for the standard RCT design.1
A comprehensive review conducted by the original Difference ELicitation in TriAls (DELTA) group14,15 highlighted the available methods for specifying the target difference. Despite there being many different approaches available, few appear to be in regular use.16 Much of the work on identifying important differences has been carried out on patient-reported outcomes, specifically those seeking to measure health-related quality of life.17,18 In practice, the target difference often appears not to be formally based on these concepts and in many cases appears, at least from trial reports, to be determined based on convenience or some other informal basis.19 Recent surveys among researchers involved in clinical trials demonstrated that the practice is more sophisticated than trial reports suggest.16 The original DELTA group developed initial advice, but this was restricted to a standard superiority two-arm parallel-group trial design and limited consideration of related issues.20 It did not provide recommendations for practice. Accordingly, there is a gap in the literature to address this and thereby help improve current practice, which this report seeks to address. The Medical Research Council (MRC)/National Institute for Health Research (NIHR) Methodology Research Panel in the UK commissioned a workshop to produce advice on this choice of ‘effect size’ in RCT sample size calculations. From this resulting DELTA project, this report was produced. The process of its development is described in Development of the DELTA2 advice and recommendations.
Development of the DELTA2 advice and recommendations
The DELTA2 project had five components: systematic literature review of recent methodological developments (stage 1); literature review of existing funder advice (stage 2); a Delphi study (stage 3); a 2-day consensus meeting bringing together researchers, funders and patient representatives (stage 4); and the preparation and dissemination of an advice and recommendations document (stage 5). Full details of the methods and findings are provided in Appendices 1 and 2 and are summarised here.
The project started in April 2016. A search for relevant documentation on the websites of 15 trial-funding advisory bodies was performed (see Appendix 1, Methodology of the literature reviews, Delphi study, consensus meeting, stakeholder engagement and finalisation of advice and recommendations). However, there was little specific advice provided to assist researchers in specifying the target difference. The literature search for methodological developments identified 28 articles of methodological developments relevant to a method for specifying a target difference. A Delphi study involving two stages and 69 participants was conducted. The first round focused mainly on topics of interest and was conducted between 11 August 2016 and 10 October 2016. In the second round, which took place between 1 September 2017 and 12 November 2017, participants were provided with a draft copy of the document and feedback was invited.
The 2-day workshop was held in Oxford on 27 and 28 September 2016, and involved 25 participants, including clinical trials unit (CTU) directors, study investigators, project funder representatives, funding panel members, researchers, experts in sample size methods, senior trial statisticians and patient and public involvement (PPI) representatives. At this workshop, the structure and general content of the document was agreed; it was subsequently drafted by members of the project team and participants from the workshop. Further engagement sessions were held at the Society for Clinical Trials (SCT), Statisticians in the Pharmaceutical Industry (PSI) and Joint Statistical Meetings (JSM) conferences on 16 May 2016, 17 May 2017 and 1 August 2017, respectively. The main text was finalised on 18 April 2018, after revision informed by feedback gathered from the second round of the Delphi study, the aforementioned engagement sessions and from funder representatives. Minor revisions in the light of editorial and referee feedback were made prior to finalising this report.
