Initially a panel of Key Informants gave input on the Key Questions (KQs) to be examined; these KQs were posted on Agency for Healthcare Research and Quality’s Effective Health Care (EHC) website for public comment in September 2017 for 3 weeks. Members of the Beers Criteria Panel and the American Geriatrics Society membership were asked for input. We revised the KQs based on comments. We then drafted a protocol for the systematic review and recruited a panel of technical experts to provide high-level content and methodological expertise throughout the development of the review. The finalized protocol is posted on the EHC website at https://effectivehealthcare.ahrq.gov/topics/depression-harms/research-protocol. The PROSPERO registration is CRD42018088648.

We developed an a priori analytic framework to guide the systematic review process (Figure 1). The details of the analytic framework were determined in consultation with the partner, key informants, technical expert panelists and public comment. We identified relevant literature for KQ1 and KQ2 by searching Ovid MEDLINE, Ovid MEDLINE In-Process & Other Nonindexed Citations, EMBASE via Ovid, Cochrane Central Register of Controlled Trials and PsycINFO via OVID from earliest date through May 15, 2018 using subject headings and natural language terms reflecting major depression, older age and the interventions of interest (Appendix A). We supplemented the bibliographic database searches with backwards citation tracking of relevant publications. We searched the clinicaltrials.gov website and the World Health Organization International Controlled Trials Registry Platform (ICTRP) for ongoing studies and those completed with reported results.

Figure 1

Analytic framework. Abbreviations: CNS=central nervous system; ECG=electrocardiogram; ER=emergency room; KQ=Key Question; MDD= major depressive disorder; SIADH=syndrome of inappropriate antidiuretic hormone; SNRI=selective serotonin norepinephrine inhibitor; (more...)

We managed citations using DistillerSR^®. We screened titles and abstracts using two independent reviewers to determine if the citation met inclusion/exclusion criteria (Error! Reference source not found.). When both reviewers agreed that a citations met inclusion criteria, we reviewed the full text for inclusion into the review. A third reviewer resolved disagreements.

Table 2

Inclusion and exclusion criteria for Key Questions.

We contacted corresponding authors when needed for clarification related to inclusion criteria and to solicit data for outcomes that were reported in the methods of the paper but not reported as a numerical result. All authors were given a minimum of 10 days to acknowledge queries. We matched results posted in clinical trial registries, abstracts and meeting presentations to their corresponding full text publication, which was always used as the primary data source, and reviewed for supplemental data. We considered post-hoc and subgroup analyses of included studies when they provide data on the outcomes of interest.

One investigator extracted data into standardized collection forms and evidence and outcomes tables and a second investigator verified the data. Two independent reviewers assessed risk of bias using the Cochrane Collaboration’s Risk of Bias Tool³³ for randomized controlled trials (RCTs) and Newcastle Ottawa Scale³⁴ for observational studies. We classified overall risk of bias for each study as low, moderate or high, according to the collective risk of bias per evaluated domain and the investigator’s confidence in the study results given the identified limitations.³² Risk of bias was considered unclear if the majority of domains evaluated were unclear.

We assessed clinical and methodologic heterogeneity to determine appropriateness of meta-analysis. We based data synthesis on pharmacologic class (e.g., selective-serotonin reuptake inhibitors (SSRI) or serotonin-norepinephrine reuptake inhibitors (SNRI)) while drugs listed in the “other category” (Error! Reference source not found.) were each analyzed individually. We also considered the treatment phase (acute, continuation, maintenance) when synthesizing data. In older adults, the acute treatment phase is generally considered up to 12 weeks of therapy,⁴ followed by the continuation and maintenance treatment phases for which the durations were less clear in this population. Thus, studies that distinguished between continuation (>12 weeks up to 48 weeks) and maintenance phases (48 weeks or longer) were treated accordingly.

When there were two or more trials of similar pharmacologic comparisons and outcomes, we performed random effects meta-analysis utilizing inverse-variance weighting. Between-study variance was estimated using the Paule-Mandel estimator.³⁵ Relative risks (RR) with corresponding 95 percent confidence intervals (CI) were estimated for binary outcomes and mean differences (MD) with corresponding 95 percent CI were estimated for continuous outcomes. Peto’s Odds ratio (OR) and 95 percent CI were estimated for binary outcomes with rare events (<5 percent) in place of a RR.³⁶ For outcomes with zero events in one study arm continuity correction was used,³⁷ except when a Peto’s OR was calculated which does not utilize continuity correction.³⁸ For trials in which differences between groups were not reported for continuous outcomes, we calculated it from differences at baseline and at the end of follow-up using a correlation coefficient of 0.5. For single trials reporting binary outcomes, we calculated RR and 95 percent CI where applicable. If zero events occurred in an arm of a study, we calculated the risk difference (RD) and 95 percent confidence interval which avoids need for continuity correction. Statistical significance was set at a two sided alpha of 0.05. All analyses were performed using the ‘meta’ package (version 4.9-0) in R (version 3.4.3; the R Project for Statistical Computing).

When quantitative pooling of studies was possible, we assessed presence of statistical heterogeneity using the Cochrane p-value (p<0.10 significant) and the I² statistic which represents the percentage (0–100 percent) of variability in the treatment estimate that is attributable to heterogeneity.³⁹ Tests for funnel plot asymmetry were planned when 10 or more studies reported a given outcome, although this never occurred.

We calculated number needed to treat (NNT) or number needed to harm (NNH) for outcomes that were graded for strength of evidence (SOE), had data reported in order to calculate absolute risk, and were found to have statistically significant difference.

Prior to analysis, we consulted our key informants, technical expert panelists and partner to determine subgroups of interest. This included age group, sex, race, ethnicity, risk of falls or history of fracture, dementia or cognitive impairment, nursing facility setting, ≥2 physical (i.e. nonpsychiatric) comorbidities, history of substance abuse, frailty, early versus late onset major depressive disorder (MDD), polypharmacy (defined as 5 or more concurrent prescription medications),³¹ concurrent use of one other medication with central nervous system activity,²⁸ defined as antipsychotics, benzodiazepines, nonbenzodiazepine hypnotics, and opioids. We performed subgroup analysis when two or more trials per subgroup were available for a given outcome. Included studies that were not amenable to pooling were qualitatively summarized.

The decision of which outcomes to grade was aided by ranking of outcome importance by the Technical Expert Panel (TEP) followed by discussion of the ranking results between the TEP, partner and Evidence-based Practice Center (EPC). Two independent senior investigators graded the SOE for the effect estimates calculated for the following selected outcomes: any adverse event, withdrawal due to adverse event, mortality, hospitalization, serious adverse events, arrhythmias, QTc prolongation, falls, fractures, cognitive impairment and syndrome of inappropriate antidiuretic hormone. The investigators discussed their assessments to arrive at a final SOE grade using established guidance.⁴⁰ We evaluated SOE separately for RCT and observational studies. Five required domains included study risk of bias, consistency, directness, precision and publication bias. RCT data began with a grade of high and could be downgraded based on the assessment of the 5 domains. Observational data began with a grade of low and could be upgraded based on assessment of the 5 domains. We did not further contextualize the calculated effect estimates, rather interpretation was based on statistical significance. The SOE was assessed for the effect estimate generated for each comparison and outcome combination as of the following four grades:

• High: We are very confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has few or no deficiencies. We believe that the findings are stable, i.e., another study would not change the conclusions.
• Moderate: We are moderately confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has some deficiencies. We believe the findings are likely to be stable, but some doubt remains.
• Low: We have limited confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has major or numerous deficiencies (or both). We believe that additional evidence is needed before concluding either that the findings are stable or that the estimate of effect is close to the true effect.
• Insufficient: We have no evidence, we are unable to estimate an effect, or we have no confidence in the estimate of the effect for this outcome. No evidence is available or the body of evidence has unacceptable deficiencies, precluding reaching a conclusion.

We assessed applicability of studies using the population, intervention, comparator, outcomes, timing, setting (PICOTS) framework.⁴¹ Characteristics that may have influenced applicability included but are not limited to depression severity, age of onset, other inclusion/exclusion criteria, treatment period (acute vs. longer term), specific antidepressant, outcome definitions and surveillance techniques.

The contextual question (CQ) is not based on a systematic review as the aim of the CQ is to provide a qualitative overview of the state of the evidence without formal systematic review or analytic plans. The findings of the citations pertinent to the PICOTS are presented in the introduction.

Experts in geriatric medicine and psychiatry fields and individuals representing stakeholder and user communities were invited to provide external peer review of this systematic review; AHRQ and an associate editor also provided comments. The draft report was posted on the AHRQ website for 4 weeks to elicit public comment. We addressed all reviewer comments, revising the text as appropriate, and documented everything in a disposition of comments report that will be made available three months after the Agency posts the final systematic review on the EHC website.