2.1. Review Approach

We developed an analytic framework to guide the process of the systematic review (Figure 1). We followed the established methodologies of systematic reviews as outlined in the Agency for Healthcare Research and Quality (AHRQ) Methods Guide for Effectiveness and Comparative Effectiveness Reviews.²⁹ The reporting complies with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.³⁰

The topic of this report and preliminary Key Questions (KQs) arose through a process involving the public and AHRQ (https://effectivehealthcare.ahrq.gov/about/epc/nomination/). Initially a panel of Key Informants gave input on the KQs to be examined; these KQs were posted on AHRQ’s Effective Health Care website for public comment between April 14, 2021, and May 14, 2021, and revised in response to comments. A panel of Technical Experts provided high-level content and methodological expertise throughout development of the review protocol. The final protocol is posted on the Effective Health Care website at https://effectivehealthcare.ahrq.gov/products/accelerated-partial-breast-irradiation/protocol and registered in the International Prospective Register of Systematic Reviews (PROSPERO #: CRD42021284155).

2.2. Key Questions and Contextual Question

2.3. Analytic Framework

Figure 1

Analytic framework for Key Questions. Abbreviations: CQ = Contextual Question; HER2 = Human Epidermal Growth Factor Receptor 2; KQ = Key Question; PBI = partial breast irradiation; WBI = whole breast irradiation

2.4. Study Selection

2.4.2. Inclusion and Exclusion Criteria

The eligible studies for the KQs had to meet all of the following criteria: 1) adult women (18 years and older) with early-stage breast cancer, defined as a small tumor less than or equal to 3 cm that has minimal or no lymph node involvement (N0/1); 2) received one of the six PBI modalities (multi-catheter interstitial brachytherapy, single-entry catheter brachytherapy, 3-dimensional conformal external beam radiation therapy, intensity-modulated radiation therapy, proton radiation therapy, intraoperative radiation therapy [IORT]); 3) compared with whole breast irradiation (WBI) or another PBI modality; 4) reported outcomes of interest (health outcomes and adverse events [AEs]); 5) RCTs and comparative observational studies; for proton radiation therapy, single-arm observational studies with more than 50 patients (as proton radiation therapy was not adequately evaluated in RCTs and comparative observational studies); 6) published in English as peer reviewed full text publication; and 7) publication after the year 2000 (earlier publications are no longer relevant to the current clinical practice). We excluded studies with children (<18 years old), men, and patients with recurrent breast cancer. In vitro studies, studies without original data (e.g., narrative review, editorial, secondary analyses of published trials), single-arm studies with less than 50 patients, and studies published in foreign languages were also excluded. For the CQ, we included all publications that evaluated financial toxicity related to PBI in early-stage breast cancer, regardless of study design and sample size. The detailed inclusion and exclusion criteria for the KQs and CQ are listed in Table 1.

Table 1

PICOTS (population, interventions, comparisons, outcomes, timing, and setting).

Independent reviewers, working in pairs, screened the titles and abstracts of all citations using prespecified inclusion and exclusion criteria. Studies included by either reviewer were retrieved for full-text screening. Independent reviewers, again working in pairs, screened the full-text version of eligible references. Discrepancies between the reviewers were resolved through discussions and consensus. When consensus could not be reached, a third reviewer resolved the difference.

2.5. Data Extraction

We developed a standardized data extraction form to extract study characteristics (author, year, study design, inclusion and exclusion criteria, patient characteristics, intervention, comparisons, outcomes, and related items for assessing study quality and applicability). The standardized form was tested by all study team members using randomly selected studies. Reviewers worked independently to extract study details. A second reviewer reviewed data extraction and resolved conflicts. When the included studies did not report all necessary information (e.g. methods and results), we contacted authors directly. DistillerSR® was used to create data extraction forms and facilitate data extraction.

2.6. Risk of Bias Assessment

For KQs, we evaluated the risk of bias of the included RCTs using the Cochrane Collaboration’s Risk of Bias 2 tool³¹ to assess bias from the randomization process, deviation from intended interventions, missing outcome data, outcome measurement, selective reporting, and other sources. For comparative and single-arm observational studies, we selected appropriate items from the Newcastle-Ottawa Scale.³² For studies reporting on CQ 1 on financial toxicity, we did not evaluate risk of bias since this contextual information was narratively summarized. One reviewer independently rated risk of bias for all studies. A second reviewer reviewed the ratings and resolved conflicts.

2.7. Data Synthesis and Analyses

We qualitatively summarized key features/characteristics (e.g. study populations, design, intervention, outcomes, and conclusions) of the included studies and present the findings in evidence tables for each KQ.

Table 2 lists the definition of outcomes used in the report.

Table 3 lists the categories of AEs and examples. We differentiated acute and late AEs using the original authors’ definition. In most cases, AEs less than 3 months after radiotherapy were defined as acute AEs; while AEs more than 3 months were defined as late AEs.

Table 2

Definition of health outcomes.

Table 3

Categories of adverse events.

We conducted meta-analysis, whenever appropriate (i.e., 2 or more studies) address the same PICOTS (population, interventions, comparisons, outcomes, timing, and setting) and provide point estimates and dispersion measures) to quantitatively summarize study findings based on the similarities of PICOTS presented by the studies. As a priori, we did not combine IORT with the other PBI modalities, because it is distinctly different from other PBI modalities. Radiotherapy in PBI and WBI is delivered by defining a target volume and calculating the dose it receives, whereas IORT is an exception to this standard approach in which the dose received by a defined target is not evaluated.³⁴ We also did not meta-analyze quality of life and did not combine comparative observational studies with RCTs. Last, studies addressing CQ 1 and single-arm studies were described and summarized narratively.

Analyses were based on the “intention-to-treat” principle for RCTs or number of patients initially receiving the interventions at the start of observational studies. For studies with multiple publications, when there was discrepancy between the publications, we prioritized data from later publications with more complete and longer followup. Relative risk (RR) and corresponding 95 percent confidence intervals (CIs) were extracted or calculated for binary outcomes. We dichotomized cosmesis scales to poor/fair versus good/excellent. Since one patient may suffer multiple adverse events, we calculated the incidence rate ratio for this outcome, which is defined as the ratio of the incidence rate of events within a given time between the intervention and the comparison groups. As suggested by empirical work,³⁵ data from noninferiority trials were combined with those from trials that did not apply a noninferiority margin. Meta-analyses were conducted based on length of followup: for health outcomes: ≥1 year to 5 years, >5 years to 10 years, >10 years; for adverse events: ≤3 months (acute AE), >3 month (late AE). We used the DerSimonian-Laird random effect model with Hartung-Knapp-Sidik-Jonkman variance correction to combine direct comparisons between treatments.³⁶ We evaluated heterogeneity between studies using the I² indicator, which measures percentage of variations in effect sizes reported by the studies due to heterogeneity. To further explore heterogeneity, we conducted prespecified subgroup analyses based on age, lymphovascular invasion, adjuvant therapy, accelerated partial breast irradiation suitability, disease stage (ductal carcinoma in situ vs. invasive disease), estrogen receptor (ER) status, Human Epidermal Growth Factor Receptor 2 (HER2) status, histology , hormone receptor status, tumor size, tumor grade, Ki-67 proliferative index, lymph node status, menopausal status, molecular subtype, molecular subtype, progesterone receptor (PR) status, resection margins, planning parameters (IORT immediately after lumpectomy vs. delayed IORT), dose-fractionation schemes (accelerated vs. nonaccelerated), treatment dose (twice per day, once per day, and once every 2 days) and risk of bias (low, moderate, and high risk of bias). We were unable to conduct other prespecified subgroup analyses (e.g. race/ethnicity, socioeconomic status, breast size) as the studies did not provide specific data for these factors. We evaluated the robustness of the findings (i.e., sensitivity analysis) comparing analyses that produced RR as an outcome measure, to hazard ratio estimates reported by the studies, and combining studies at the longest followup. One study³⁷ reported results from a RCT conducted between 1986 and 1990 with antiquated radiation techniques that are no longer relevant to current practice but, otherwise, met our inclusion criteria. We included this study as a sensitivity analysis. We were unable to evaluate potential publication bias due to the small number of studies included in a meta-analysis (n<10). All statistical analyses were conducted using Stata version 17.0 (StataCorp LLC, College Station, TX, USA).

2.8. Grading the Strength of Evidence for Major Comparisons and Outcomes

We graded the strength of evidence (SOE) for KQs following the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews.²⁹ We graded SOE for the critical effectiveness outcomes: ipsilateral breast recurrence, mastectomy-free survival, cancer-free survival, overall survival, and cosmesis. We graded SOE for overall acute and late AEs comparing WBI to combined PBI modalities. These outcomes were chosen because they are either clinically important from a patient’s perspective or highly relevant for stakeholders’ decision making.

SOE derived from RCTs started with a rating of high and SOE derived from observational studies started as low.²⁹ SOE was rated down due to methodological limitations of the studies (i.e. risk of bias); imprecision (based on the size of the body of evidence, number of events, and confidence intervals); indirectness of the evidence to the KQs (focusing on whether the outcomes were important to patients vs. surrogates); inconsistency of results (based on qualitative and statistical approaches to evaluate for heterogeneity); or increased likelihood of reporting and publication bias.

We lowered SOE rating for the risk of bias when all the studies in a particular comparison had high or unclear risk of bias. If estimates from high and low risk of bias studies were available and similar, we combined them and did not rate down SOE. If estimates were different, we only used the low risk of bias estimate and did not rate down SOE (although this could lead to imprecise estimates).

We considered estimates to be precise if the CI of RR did not overlap benefit and harms and the sample size was 1,000 or more. We also considered an effect to be precise if the CI of RR did not overlap the benefit and harms and the risk difference CI was within 30 per 1,000.^{38, 39} We rated down one level if the effect was statistically significant but the sample size <1,000, the effect was insignificant but sample size >2,000, or the effect was insignificant but the risk difference CI exceeded 30 per 1,000. We rated down by two levels if the effect was insignificant and the sample size was 400-1,999. We rated down by 3 levels if the effect was insignificant and the sample size was <400. We also considered whether outcome designation (mortality vs. survival) affected imprecision judgments.

We rated down for inconsistency when I² exceeded an arbitrary cutoff >60 percent and visual inspection of forest plots suggested substantial variability in point estimates.

Based on this assessment and the initial study design, we assigned SOE rating as high, moderate, low, or ‘insufficient evidence to estimate an effect’ (Table 4).

Table 4

Definition of strength of evidence ratings.

We produced summary of evidence tables that provided for each comparison and for each outcome: data source, effect size, SOE rating; and rationale for judgments made on each domain of evidence rating.

2.9. Assessing Applicability

We followed the procedures outlined in the AHRQ Methods Guide to assess the applicability of the findings within and across studies.²⁹ Applicability for each outcome was summarized and presented qualitatively using the PICOTS framework and not a specific checklist or scale. The following factors that may affect applicability have been identified, including patient factors (e.g. age, menopausal status, race, ethnicity, socioeconomic status), tumor characteristics (e.g. nodal status, tumor size/grade, histology), intervention factors (e.g. dose-fractionation schemes, target volumes, planning parameters, number of treatment fields), comparisons (e.g. type of comparators), outcomes (e.g. use of unvalidated or nonstandardized outcomes), settings, and study design features (e.g. observational studies, RCTs). We used this information to evaluate the applicability of the evidence to real-world clinical practice in typical U.S. settings. We reported any limitations in applicability of individual studies in the evidence tables and limitations of applicability of the whole body of evidence in the summary of evidence tables.

2.10. Peer Review and Public Commentary

Experts in the fields of radiation oncology and breast surgical oncology as well as other stakeholders provided external peer review of this draft report; AHRQ also provided a review of the draft report. The draft report was then posted on the AHRQ Effective Health Care website for public comment from July 1 to July 29, 2022.