Randomization

A large and long-standing empirical body of evidence supports the superiority of RCTs for measuring treatment effect compared with nonrandomized designs.^27,105,160 As a study design element, randomization is powerful because it minimizes selection bias, thus increasing the likelihood that differences among treatment groups are actually the result of the treatment rather than some other prognostic factor.

The randomization domain seen on Table 8 and Grid 2 includes three empirically based elements: an adequate approach to sequence generation and appropriate allocation concealment, both of which result in group comparability at baseline. Studies of these three elements may overlap; some also address the issue of double- or triple-blinding. The process of randomization has two distinct parts. The first is how the random sequence is produced and the second is how patients' treatment group allocation is concealed. Methods of generating the sequence that are not truly random (e.g., using odd and even year of birth) and methods of concealment that can be subverted (e.g., peeking inside assignment envelopes) may allow investigators or clinicians to "rig" the study groups. This may result in study groups that are not similar in terms of their prognostic factors at baseline.

Schulz and colleagues reported that only one-third of RCTs in obstetrics and gynecology reported an adequate method of randomization. ¹⁶¹ They noted that observed differences in the baseline characteristics of study groups further suggested that randomization was improperly done. Studies that failed to report an adequate approach to sequence generation were unlikely to report adequate allocation concealment, and nearly half of the studies did not report an adequate method of allocation concealment. ¹⁶²

Allocation concealment may be more important than the exact procedures for generating the randomization sequence. Chalmers et al. found substantial case fatality differences among studies of treatments for myocardial infarction depending on whether the study was randomized and whether allocation was concealed. ¹⁰⁵ Case fatality rate differences were 8.8% for studies that were randomized and properly concealed, 24.4% for unblinded randomized studies, and 58.1% for nonrandomized studies in cardiology, neurology, and pulmonology. Moher and colleagues found that trials with inadequately reported allocation concealment had significantly exaggerated estimates of treatment effect compared with studies that adequately reported concealment. ⁴¹

Blinding

Allocation concealment inherently implies blinded assessment. Although usage differs, "single-blinding" generally refers to the study subject or patient not being aware of the treatment allocation, whereas "double-blinding" typically means that neither the patient nor the caregivers know the treatment group assignment. However, the principle of double-blinding more generally means that the treatment assigned and received is masked to all key study personnel (e.g., investigators, caregivers, subjects, outcome assessors, data analysts) as well as participants. The study by Colditz et al. found that RCTs that did not employ double-blinding were significantly more likely to show a treatment effect. ²⁷ Not all interventions can be successfully blinded; for health services research, it is difficult to mask participants and caregivers to factors such as their type of health care coverage or the type of clinician caring for them. Just as not all interventions can be randomized, not all interventions can be kept from those who are participating in the study.

Statistical Analysis

As in any study design, bias can be introduced at any point from design to reporting but the analysis strategy for RCTs is key. It is rare for studies to have totally complete follow-up of participants, and subjects leave the study for a variety of reasons. If the reason for a subject's withdrawal is related to the therapy received or the outcome of interest, then bias may be introduced. If the study is analyzed on the basis of which treatment was actually received (an efficacy analysis) rather than by treatment assigned (an intent-to-treat analysis) then randomization is not maintained. Bias is even further increased when less adherent patients have significantly different outcomes and adherence is related to group assignment; underlying prognostic characteristics may be related to adherence and/or treatment effect, as well.

Chene and colleagues examined withdrawal issues, comparing an intent-to-treat analysis with an efficacy analysis in an HIV drug study. The relationship between adherence to the drug and outcomes was significant. The intent-to-treat analysis indicated that drug was not effective, which was not supported by the efficacy analysis. ¹⁶³ Lachin reported similar results in a study of an Alzheimer's drug where substantial numbers of participants withdrew from the RCT because of drug side effects. ¹⁶⁴ Both the efficacy and intent-to-treat analyses supported the new drug, but only the latter supported its effectiveness at higher doses.

These statistical challenges are similar to those noted by Khan and colleagues comparing crossover trials to parallel-group RCTs evaluating infertility interventions. ³⁵ They found that crossover trials overestimated effectiveness by an average of 74% -- subjects who became pregnant were no longer eligible to be "crossed over" to the next treatment in the sequence of treatments being tested.

Funding and Sponsorship

RCTs may be subject to bias related to the author's competing interests. Djulbegovic et al. found that pharmaceutical industry-sponsored studies were more likely to result in favorable evaluations of new treatments. ¹⁶⁵ That studies conducted to support the efficacy of new treatments tend to show more favorable results is consistent with the drug approval process. Because of the expense, large phase III studies to support regulatory approval will only be conducted if the pharmaceutical company is relatively certain that its new treatment is efficacious. However, this may not be the situation for smaller RCTs where not as much financial investment is involved; an example is the comparison between brand-name and generic levothyroxine for treating hypothyroidism.^166-167

Djulbegovic and colleagues also noted that the choice of a comparative therapy known or suspected of being less effective -- that is, in violation of the equipoise principle -- might account for much of the bias found. ¹⁶⁵ A study by Cho and Bero has been used to support the potential for conflict of interest based on funding sources. They found that studies published in pharmaceutical company-sponsored symposia proceedings were significantly more likely to favor the new drug of interest than were studies published in peer-reviewed journals. ¹⁶⁸