Searching for the Evidence

A research librarian experienced in conducting literature searches for CERs searched in Ovid MEDLINE^® (January 1990–October 2014), Cochrane Central Register of Controlled Trials (through September 2014), Cochrane Database of Systematic Reviews (through September 2014), Health Technology Assessment (through Third Quarter 2014), National Health Sciences Economic Evaluation Database (through Third Quarter 2014), and Database of Abstracts of Reviews of Effects (through Third Quarter 2014) to capture both published and gray literature. We searched for unpublished studies in clinical trial registries (ClinicalTrials.gov, Current Controlled Trials, ClinicalStudyResults.org, and the World Health Organization International Clinical Trials Registry Platform) and regulatory documents ( vog.ADF@sgurD and U.S. Food and Drug Administration (FDA) Medical Devices Registration and Listing). Reference lists of relevant studies and previous systematic reviews were hand-searched for additional studies. Scientific information packets were solicited from drug and device manufacturers, and a notice published in the Federal Register invited interested parties to submit relevant published and unpublished studies.

Study Selection

We developed criteria for inclusion and exclusion of studies based on the Key Questions and the defined population, interventions, comparators, outcomes, timing, and settings (PICOTS) and study designs. Inclusion and exclusion criteria are summarized below. Abstracts were reviewed by two investigators, and all citations deemed appropriate for inclusion by at least one of the reviewers were retrieved. Two investigators independently reviewed all full-text articles for inclusion. Discrepancies were resolved by discussion and consensus.

Data Extraction and Data Management

For treatment studies, we extracted the following information into evidence tables: study design; setting; inclusion and exclusion criteria; dose and duration of treatment for experimental and control groups; duration of followup; number of subjects screened, eligible, and enrolled; population characteristics (including age, race/ethnicity, sex, stage of disease, and functional status); results; adverse events; withdrawals due to adverse events; and sources of funding. We calculated relative risks (RRs) and associated 95% confidence intervals (CIs) based on the information provided (sample sizes and incidence of outcomes in each intervention group). We noted discrepancies between calculated and reported results when present.

For diagnostic accuracy studies, we abstracted the following information: setting, screening test or tests, method of data collection, reference standard, inclusion criteria, population characteristics (including age, sex, race/ethnicity, smoking status, signs or symptoms, and prior bladder cancer stage or grade), proportion of individuals with bladder cancer, bladder cancer stage and grade, definition of a positive screening exam, proportion of individuals unexaminable by the screening test, proportion who did not undergo reference standard test, results, and sources of funding. We attempted to create two-by-two tables from information provided (sample size, prevalence, sensitivity, and specificity) and compared calculated measures of diagnostic accuracy based on the two-by-two tables with reported results. We noted discrepancies between calculated and reported results when present. When reported, we also recorded the area under the receiver operating characteristic curve.^14,15

Data extraction for each study was completed by one investigator and independently reviewed for accuracy and completeness by a second investigator.

Assessment of the Risk of Bias of Individual Studies

We assessed the risk of bias for randomized trials and observational studies using criteria adapted from those developed by the U.S. Preventive Services Task Force.¹⁶ Studies of diagnostic accuracy were rated using criteria adapted from QUADAS-2, a revised tool for Quality Assessment of Diagnostic Accuracy Studies.¹⁷ These criteria were applied in conjunction with the approaches recommended for medical interventions in the AHRQ Methods Guide¹² and in the AHRQ “Methods Guide for Medical Test Reviews.”¹³

Two investigators independently assessed the risk of bias of each study. Discrepancies were resolved through discussion and consensus. Each study was rated as low, medium, or high risk of bias.¹²

Studies rated low risk of bias were considered to have no more than very minor methodological shortcomings, and their results are likely to be valid. Studies rated moderate risk of bias have some methodological shortcomings, but no flaw or combination of flaws judged likely to cause major bias. The category of moderate risk of bias is broad, and studies with this rating vary in their strengths and weaknesses; the results of some studies assessed to have moderate risk of bias are likely to be valid, while others may be only possibly valid. Studies rated high risk of bias have significant flaws that may invalidate the results. They have a serious or “fatal” flaw or combination of flaws in design, analysis, or reporting; large amounts of missing information; or serious discrepancies in reporting. We did not exclude studies rated as having high risk of bias a priori, but they were considered the least reliable when synthesizing the evidence, particularly when discrepancies between studies were present.

Assessing Applicability

We recorded factors important for understanding the applicability of studies, such as whether the publication adequately described the study sample, the country in which the study was conducted, the characteristics of the patient sample (e.g., age, sex, race/ethnicity, risk factors for bladder cancer, presenting symptoms, and medical comorbidities), tumor characteristics (e.g., stage and grade, primary or recurrent, unifocal or multifocal lesions), the characteristics of the diagnostic tests (e.g., specific test evaluated and cutoffs used) and interventions (e.g., treatment dose, duration, and interval) used, and the magnitude of effects on clinical outcomes.¹² There is no generally accepted universal rating system for applicability, which depends in part on context. Therefore, a rating of applicability (such as high or low) was not assigned because applicability may differ based on the user of this report.

Data Synthesis

For studies on diagnostic accuracy of urinary biomarkers, we performed meta-analyses to help summarize data and obtain more precise estimates.¹⁸ We used a bivariate logistic mixed-effects model¹⁹ to analyze sensitivity and specificity, incorporating the correlation between sensitivity and specificity. We assumed random effects across studies with a bivariate normal distribution for sensitivity and specificity, and heterogeneity among the studies was measured based on the random-effect variance (τ²). When few studies were available for an analysis, we used the moment estimates of correlation between sensitivity and specificity in the bivariate model. We calculated positive likelihood ratio and negative likelihood ratio using the summarized sensitivity and specificity.^20,21 For head-to-head comparisons, we used the same bivariate logistic mixed-effects model as described above but added an indicator variable for imaging modalities (equivalent to a meta-regression approach).

All quantitative analyses were conducted using SAS^® 10.0 (SAS Institute Inc., Cary, NC).²² We assessed the presence of statistical heterogeneity among the studies using the standard Cochran's chi-square test, and the magnitude of heterogeneity by using the I² statistic.²³ When statistical heterogeneity was present, we performed sensitivity analyses by conducting meta-analysis using the profile likelihood method.²⁴ We also performed sensitivity and subgroup analyses based on ratings for risk of bias, dose of intravesical therapy, inclusion of high-risk patients, and duration of followup. We stratified trials according to the type of instillation regimen, classified as single instillation, induction therapy (treatment for 4 to 8 weeks), maintenance therapy (treatment for longer than 8 weeks), or other. We calculated pooled RRs for the dichotomous outcomes for bladder cancer recurrence, bladder cancer progression, all-cause mortality, bladder cancer mortality, and local and systemic adverse events. Similar analyses were performed for trials of augmented cystoscopy (fluorescent light or narrow band imaging) versus white light cystoscopy.

Grading the Strength of Evidence for Each Key Question

We assessed the strength of evidence (SOE) for each Key Question and outcome using the approach described in the AHRQ Methods Guide,¹² based on the overall quality of each body of evidence; the risk of bias (graded low, moderate, or high); the consistency of results across studies (graded consistent, inconsistent, or unable to determine when only 1 study was available); the directness of the evidence linking the intervention and health outcomes (graded direct or indirect); the precision of the estimate of effect, based on the number and size of studies and CIs for the estimates (graded precise or imprecise); and reporting bias (suspected or undetected)

Assessments of reporting bias were based on whether studies defined and reported primary outcomes, identification of relevant unpublished studies, and when available, by comparing published results with results reported in trial registries.

We graded the SOE for each Key Question using the four categories recommended in the AHRQ Methods Guide.¹² A high grade indicates high confidence that the evidence reflects the true effect and that further research is very unlikely to change our confidence in the estimate of effect. A moderate grade indicates moderate confidence that the evidence reflects the true effect; further research may change our confidence in the estimate of effect and may change the estimate. A low grade indicates low confidence that the evidence reflects the true effect; further research is likely to change the confidence in the estimate of effect and is likely to change the estimate. A grade of insufficient indicates that evidence either is unavailable or is too limited to permit any conclusion because of the availability of only poor-quality studies, extreme inconsistency, or extreme imprecision.

Key Question 1. Diagnostic Accuracy: Comparison of Urinary Biomarkers

For this Key Question, we included 57 studies (in 60 publications) that evaluated the diagnostic accuracy of urinary biomarkers for diagnosis of bladder cancer.

• Quantitative nuclear matrix protein 22 (NMP22): Sensitivity was 0.69 (95% CI, 0.62 to 0.75) and specificity 0.77 (95% CI, 0.70 to 0.83), based on 19 studies, for a positive likelihood ratio of 3.05 (95% CI, 2.28 to 4.10) and negative likelihood ratio of 0.40 (95% CI, 0.32 to 0.50) (SOE: moderate)
  • For evaluation of symptoms: Sensitivity was 0.67 (95% CI, 0.55 to 0.77; 9 studies) and specificity 0.84 (95% CI, 0.75 to 0.90; 7 studies).
  • For surveillance: Sensitivity was 0.61 (95% CI, 0.49 to 0.71; 10 studies) and specificity 0.71 (95% CI, 0.60 to 0.81; 8 studies).
• Qualitative NMP22: Sensitivity was 0.58 (95% CI, 0.39 to 0.75) and specificity 0.88 (95% CI, 0.78 to 0.94), based on four studies, for a positive likelihood ratio of 4.89 (95% CI, 3.23 to 7.40) and negative likelihood ratio of 0.48 (95% CI, 0.33 to 0.71) (SOE: low).
  • For evaluation of symptoms: Sensitivity was 0.47 (95% CI, 0.33 to 0.61) and specificity 0.93 (95% CI, 0.81 to 0.97), based on two studies.
  • For surveillance: Sensitivity was 0.70 (95% CI, 0.40 to 0.89) and specificity 0.83 (95% CI, 0.75 to 0.89), based on two studies.
• Qualitative bladder tumor antigen (BTA): Sensitivity was 0.64 (95% CI, 0.58 to 0.69; 22 studies) and specificity 0.77 (95% CI, 0.73 to 0.81; 21 studies), for a positive likelihood ratio of 2.80 (95% CI, 2.31 to 3.39) and negative likelihood ratio of 0.47 (95% CI, 0.30 to 0.55) (SOE: moderate).
  • For evaluation of symptoms: Sensitivity was 0.76 (95% CI, 0.67 to 0.83; 8 studies), and specificity 0.78 (95% CI, 0.66 to 0.87; 6 studies).
  • For surveillance: Sensitivity was 0.60 (95% CI, 0.55 to 0.65; 11 studies) and specificity 0.76 (95% CI, 0.69 to 0.83; 8 studies).
• Quantitative BTA: Sensitivity was 0.65 (95% CI, 0.54 to 0.75) and specificity 0.74 (95% CI, 0.64 to 0.82), based on four studies, for a positive likelihood ratio of 2.52 (95% CI, 1.86 to 3.41) and negative likelihood ratio of 0.47 (95% CI, 0.37 to 0.61) (SOE: low).
  • For evaluation of symptoms: Sensitivity was 0.76 (95% CI, 0.61 to 0.87) and specificity 0.53 (95% CI, 0.38 to 0.68), based on one study.
  • For surveillance: Sensitivity was 0.58 (95% CI, 0.46 to 0.69) and specificity 0.79 (95% CI, 0.72 to 0.85), based on two studies.
• Fluorescence in situ hybridization (FISH): Sensitivity was 0.63 (95% CI, 0.50 to 0.75) and specificity 0.87 (95% CI, 0.79 to 0.93), based on 11 studies, for a positive likelihood ratio of 5.02 (95% CI, 2.93 to 8.60) and negative likelihood ratio of 0.42 (95% CI, 0.30 to 0.59) (SOE: moderate).
  • For evaluation of symptoms: Sensitivity was 0.73 (95% CI, 0.50 to 0.88) and specificity 0.95 (95% CI, 0.87 to 0.98), based on two studies, for a positive likelihood ratio of 14.2 (95% CI, 5.2 to 39) and negative likelihood ratio of 0.29 (95% CI, 0.14 to 0.60).
  • For surveillance: Sensitivity was 0.55 (95% CI, 0.36 to 0.72; 7 studies) and specificity was 0.80 (95% CI, 0.66 to 0.89; 6 studies).
• ImmunoCyt^™: Sensitivity was 0.78 (95% CI, 0.68 to 0.85) and specificity 0.78 (95% CI, 0.72 to 0.82), based on 14 studies, for a positive likelihood ratio of 3.49 (95% CI, 2.82 to 4.32) and negative likelihood ratio of 0.29 (95% CI, 0.20 to 0.41) (SOE: moderate).
  • For evaluation of symptoms: Sensitivity was 0.85 (95% CI, 0.78 to 0.90; 6 studies) and specificity 0.83 (95% CI, 0.77 to 0.87; 7 studies).
  • For surveillance: Sensitivity was 0.75 (95% CI, 0.64 to 0.83; 7 studies) and specificity 0.76 (95% CI, 0.70 to 0.81; 8 studies).
• CxBladder: Sensitivity was 0.82 (95% CI, 0.70 to 0.90) and specificity 0.85 (95% CI, 0.81 to 0.88) for evaluation of symptoms, based on one study, for a positive likelihood ratio of 5.53 (95% CI, 4.28 to 7.15) and negative likelihood ratio of 0.21 (95% CI, 0.13 to 0.36) (SOE: low).
• Direct (within-study) comparisons:
  • There was no difference between quantitative NMP22 (cutoff >10 U/mL) versus qualitative BTA in sensitivity (0.69 [95% CI, 0.62 to 0.76] vs. 0.66 [95% CI, 0.59 to 0.73], for a difference of 0.03 [95% CI, -0.04 to 0.10]) or specificity (0.73 [95% CI, 0.62 to 0.82] vs. 0.76 [95% CI, 0.66 to 0.84], for a difference of 0.03 [95% CI, -0.08 to 0.01]), based on seven studies (SOE: moderate).
  • ImmunoCyt was associated with higher sensitivity than FISH (0.71 [95% CI, 0.54 to 0.84] vs. 0.61 [95% CI, 0.43 to 0.76], for a difference of 0.11 [95% CI, 0.001 to 0.21]) but lower specificity (0.71 [95% CI, 0.62 to 0.79] vs. 0.79 [95% CI, 0.71 to 0.85], for a difference of -0.08 [95% CI, -0.15 to -0.001]), based on three studies (SOE: low).
  • Evidence for other head-to-head comparisons of urinary biomarkers was based on small numbers of studies with imprecise estimates and methodological shortcomings, precluding reliable conclusions regarding comparative test performance (SOE: insufficient).
  • Sixteen studies found sensitivity of various urinary biomarkers plus cytology to be associated with higher sensitivity than the urinary biomarker alone (0.8 [95% CI, 0.75 to 0.86] vs. 0.69 [95% CI, 0.61 to 0.76], for a difference of 0.13 [95% CI, 0.08 to 0.17]), with no difference in specificity (SOE: moderate).

Key Question 1a. Diagnostic Accuracy: Patient Characteristics or Presenting Signs or Symptoms

For this Key Question, we included 42 studies that evaluated diagnostic accuracy according to patient characteristics or the nature of the presenting signs or symptoms.

• Effects of tumor stage: Across urinary biomarkers, sensitivity increased with higher tumor stage. Evidence was most robust for quantitative NMP22 (11 studies), qualitative BTA (18 studies), and FISH (8 studies); the association between higher tumor stage and increased sensitivity was least pronounced for ImmunoCyt (10 studies). Sensitivity for carcinoma in situ (CIS) tumors was generally similar to or slightly lower than for T1 tumors (SOE: high).
• Effects of tumor grade: Across urinary biomarkers, sensitivity increased with higher tumor grade. Evidence was most robust for quantitative NMP22 (12 studies), ImmunoCyt (10 studies), qualitative BTA (18 studies), and FISH (9 studies) (SOE: high).
• Effects of tumor size: Two studies found that sensitivity was higher for larger (>1 cm or >2 cm) versus smaller tumors (SOE: low).
• Evidence on the effects of patient characteristics, such as age, sex, smoking status, and presence of other clinical conditions, on diagnostic accuracy of urinary biomarkers was limited and did not clearly or consistently indicate effects on sensitivity or specificity (SOE: low).

Key Question 2. Use of Formal Risk-Adapted Assessment Approach

This Key Question addresses the issue of whether use of a formal risk-adapted assessment approach to treatment decisions decreases mortality or improves other outcomes compared with treatment not guided by a formal risk-adapted assessment approach.

• No study compared clinical outcomes associated with use of a formal risk-adapted approach to guide treatment of NMIBC versus treatment not guided by a risk-adapted approach (SOE: insufficient).

Key Question 3. Effect of TURBT Plus Intravesical Therapy Versus TURBT Alone

This Key Question addresses the issue of whether the use of various intravesical chemotherapeutic or immunotherapeutic agents in addition to TURBT decreases mortality or improves other outcomes compared with TURBT alone. We included 37 studies (in 46 publications) that evaluated intravesical therapy versus no intravesical therapy.

• BCG was associated with decreased risk of bladder cancer recurrence (3 trials; RR, 0.56; 95% CI, 0.43 to 0.71; I² = 0%) and progression (4 trials; RR, 0.39; 95% CI, 0.24 to 0.64; I² = 40%) versus no intravesical therapy. No trial evaluated effects of BCG versus no intravesical therapy on risk of all-cause mortality. One trial found BCG to be associated with decreased risk of bladder cancer mortality, but the difference was not statistically significant (RR, 0.62; 95% CI, 0.32 to 1.19) (SOE: insufficient for all-cause and bladder cancer mortality; low for recurrence and progression).
• MMC was associated with decreased risk of bladder cancer recurrence versus no intravesical therapy (8 trials; RR, 0.71; 95% CI, 0.57 to 0.89; I² = 72%), but there was no difference in risk of all cause-mortality (1 trial; hazard ratio [HR], 1.17; 95% CI, 0.89 to 1.53), and effects on bladder cancer mortality (1 trial; HR, 0.71; 95% CI, 0.34 to 1.46) and bladder cancer progression (5 trials; RR, 0.68; 95% CI, 0.39 to 1.20, I² = 0%) were not statistically significant (SOE: moderate for recurrence; low for progression, all-cause mortality, and bladder cancer–specific mortality).
• Doxorubicin was associated with decreased risk of bladder cancer recurrence versus no intravesical therapy (10 trials; RR, 0.80; 95% CI, 0.72 to 0.88; I² = 46%), no difference in risk of bladder cancer progression (5 trials; RR, 1.03; 95% CI, 0.72 to 1.46; I² = 0%), and no clear effects on all-cause mortality (2 trials) or bladder cancer–specific mortality (1 trial) (SOE: moderate for recurrence; low for progression, all-cause mortality, and bladder cancer–specific mortality).
• Epirubicin was associated with decreased risk of bladder cancer recurrence (9 trials; RR, 0.63; 95% CI, 0.53 to 0.75; I² = 64%) (SOE: moderate), but the effect on bladder cancer progression was not statistically significant (8 trials; RR, 0.79; 95% CI, 0.84 to 1.30; I² = 27%) (SOE: low).
• Gemcitabine was examined in one trial that found no difference between single-instillation gemcitabine versus no intravesical therapy in risk of bladder cancer recurrence (RR, 0.98; 95% CI, 0.70 to 1.36); estimates for progression (RR, 3.00; 95% CI, 0.32 to 28.4), all-cause mortality (RR, 0.50; 95% CI, 0.13 to 2.00), and bladder cancer–specific mortality (RR, 1.00; 95% CI, 0.06 to 15.81) were very imprecise (SOE: low for bladder cancer recurrence; insufficient for all-cause and bladder cancer–specific mortality and progression).
• Interferon alpha was associated with decreased risk of bladder cancer recurrence versus no intravesical therapy that was not statistically significant (3 trials; RR, 0.75; 95% CI, 0.53 to 1.06; I² = 50%), decreased risk of bladder cancer progression (2 trials; RR, 0.33; 95% CI, 0.14 to 0.76; I² = 0%), and no difference in risk of bladder cancer–specific mortality (1 trial; RR, 1.00; 95% CI, 0.15 to 6.75) (SOE: low).
• Interferon gamma was associated with decreased risk of bladder cancer recurrence versus no intravesical therapy (1 trial; RR, 0.72; 95% CI, 0.51 to 1.01), with no difference in risk of bladder cancer progression (1 trial; RR, 1.08; 95% CI, 0.07 to 16.4) (SOE: low).
• Thiotepa was associated with decreased risk of bladder cancer recurrence versus no intravesical therapy that was not statistically significant (5 trials; RR, 0.78; 95% CI, 0.58 to 1.06; I² = 69%), with insufficient evidence to determine effects on progression or mortality (SOE: low for recurrence, insufficient for all-cause and bladder cancer mortality and progression).

Key Question 3a. Comparative Effectiveness: Chemotherapeutic or Immunotherapeutic Agents as Monotherapy or in Combination

For this Key Question, we included 54 studies in 66 publications.

Key Question 3b. Comparative Effectiveness: Tumor Characteristics

For this Key Question, we included 29 studies.

• There were no clear differences in estimates of effectiveness of intravesical therapies in subgroups defined by tumor stage, grade, size, multiplicity, recurrence status, or DNA ploidy (SOE: low).

Key Question 3c. Comparative Effectiveness: Patient Characteristics

• No trial evaluated how estimates of effectiveness of intravesical therapy vary in subgroups defined by patient characteristics, such as age, sex, race/ethnicity, performance status, and comorbidities (SOE: insufficient).
• In patients with recurrence or progression following prior BCG therapy, one trial found maintenance therapy with gemcitabine to be associated with decreased risk of recurrence versus repeat treatment with BCG, and one trial found MMC maintenance therapy to be associated with lower likelihood of disease-free survival than gemcitabine; estimates for progression were imprecise (SOE: low).

Key Question 3d. Comparative Effectiveness: Dosing Frequency, Treatment Duration, Timing

For this Key Question, we included 53 studies (in 57 publications) that compared different doses or instillation regimens of the same drug or different BCG strains.

Key Question 4. For TURBT Patients, Effectiveness of Radiation Therapy Versus Intravesical Therapy or Cystectomy

This Key Question addressed the effectiveness of external beam radiation therapy for decreasing mortality or improving other outcomes compared with intravesical chemotherapy/immunotherapy alone or cystectomy in patients treated with TURBT. One randomized trial (rated moderate risk of bias) compared external beam radiation therapy with no radiation therapy in patients with NMIBC.

• One randomized trial of patients with T1 Grade (G) 3G3 bladder cancer found no effects of radiation therapy versus no radiotherapy (for unifocal disease and no CIS) or radiation therapy versus intravesical therapy (for multifocal disease or CIS) in recurrence-free survival (HR, 0.94; 95% CI, 0.67 to 1.30), progression-free interval (HR, 1.07; 95% CI, 0.65 to 1.74), progression-free survival (HR, 1.35; 95% CI, 0.92 to 1.98), or overall survival (HR, 1.32; 95% CI, 0.86 to 2.04) after 5 years (SOE: low).

Key Question 5. Effectiveness of Urinary Biomarkers Versus Other Urinary Biomarkers or Standard Diagnostic Methods for Surveillance

• No study evaluated the effectiveness of urinary biomarkers to decrease mortality or improve other outcomes compared with standard diagnostic methods or other urinary biomarkers in surveillance of patients treated for NMIBC.

Key Question 5a. Comparative Effectiveness: Tumor Characteristics

• No evidence was found (SOE: insufficient).

Key Question 5b. Comparative Effectiveness: Treatment Used

This Key Question addressed the issue of whether comparative effectiveness differs according to the treatment used (i.e., specific chemotherapeutic or immunotherapeutic agents and/or TURBT).

• No evidence was found (SOE: insufficient).

Key Question 5c. Comparative Effectiveness: Surveillance Intervals

• No evidence was found (SOE: insufficient).

Key Question 5d. Comparative Effectiveness: Patient Characteristics

• No evidence was found (SOE: insufficient).

Key Question 6. Effectiveness of Augmented Versus Standard Cystoscopy

This Key Question addresses the effectiveness of blue light or other methods of augmented cystoscopy compared with standard cystoscopy for recurrence rates, progression of bladder cancer, mortality, or other clinical outcomes in initial diagnosis or surveillance of patients treated for NMIBC. We included 14 trials (in 19 publications) that evaluated clinical outcomes of augmented (fluorescent or narrow band imaging) cystoscopy versus standard white light cystoscopy.

• There was no difference between fluorescent versus white light cystoscopy in risk of mortality (3 trials; RR, 1.28; 95% CI, 0.55 to 2.95; I² = 41%) (SOE: low).
• Fluorescent cystoscopy with 5-aminolevulinic acid (5-ALA) or hexaminolevulinate (HAL) was associated with decreased risk of bladder cancer recurrence versus white light cystoscopy at short-term (<3 months; 9 trials; RR, 0.58; 95% CI, 0.36 to 0.94, I² = 75%), intermediate-term (3 months to <1 year; 5 trials; RR, 0.67; 95% CI, 0.51 to 0.88; I² = 35%), and long-term followup (≥1 year; 11 trials; RR, 0.81; 95% CI, 0.68 to 0.98; I² = 64%), but findings were inconsistent and potentially susceptible to performance bias (because of failure to blind the initial cystoscopy) and publication bias (SOE: low).
• There was no difference between fluorescent and white light cystoscopy in risk of progression to muscle-invasive bladder cancer (9 trials; RR, 0.78; 95% CI, 0.55 to 1.12; I² = 0%) (SOE: moderate).
• Narrow band imaging was associated with lower risk of bladder cancer recurrence at 3 months (3.9% vs. 17%; odds ratio [OR], 0.62; 95% CI, 0.41 to 0.92) and at 12 months (OR, 0.24; 95% CI, 0.07 to 0.81) compared with white light cystoscopy in one trial (SOE: low).

Key Question 7. Adverse Effects: Tests

We included seven studies that evaluated adverse effects of various tests for diagnosis and post-treatment surveillance of bladder cancer.

• Urinary biomarkers miss 23 to 42 percent of patients with bladder cancer and are incorrectly positive in 11 to 28 percent of patients without bladder cancer, but no study directly measured effects of inaccurate diagnosis on clinical outcomes (SOE: insufficient).
• There were no clear differences between fluorescent cystoscopy and white light cystoscopy in the risk of false-positives in two trials (SOE: low).
• There were no clear differences between fluorescent cystoscopy and white light cystoscopy in the risk of renal and genitourinary adverse events in two trials (SOE: low).

Key Question 8. Adverse Effects: Treatments

This Key Question addressed adverse effects of various treatments, including intravesical chemotherapeutic or immunotherapeutic agents and TURBT. We included 22 studies of intravesical therapies that reported harms.

Key Question 8a. Adverse Effects of Treatments: Patient Characteristics

• No study evaluated how harms of treatment vary in subgroups defined by patient characteristic, such as age, sex, race/ethnicity, performance status, or medical comorbidities (SOE: insufficient).

Key Findings and Strength of Evidence

The key findings of this review are described in the summary-of-evidence table (Table A).

Table A

Summary of the strength of evidence.

Urinary biomarkers were associated with sensitivity for bladder cancer that ranged from 0.58 to 0.77 and specificity that ranged from 0.72 to 0.89, for positive likelihood ratios that ranged from 2.18 to 6.10 and negative likelihood ratios that ranged from 0.21 to 0.48. Findings were robust in sensitivity and stratified analyses, although evidence was strongest for quantitative NMP22 and qualitative BTA (SOE: moderate) and relatively sparse for other biomarkers (SOE: low). Across urinary biomarkers, sensitivity was greater for higher stage and higher grade tumors (SOE: high). For qualitative BTA, sensitivity was somewhat higher for evaluation of patients with signs or symptoms of bladder cancer than for surveillance of patients previously treated for bladder cancer, but for quantitative NMP22 there was no clear difference in diagnostic accuracy based on reason for testing. Studies that directly compared the accuracy of quantitative NMP22 and qualitative BTA found no differences in diagnostic accuracy (SOE: moderate). There were too few head-to-head comparisons of other urinary biomarkers to reach firm conclusions regarding comparative accuracy. Sensitivity was increased when urinary biomarkers were used in conjunction with urine cytology (SOE: moderate). No study evaluated clinical outcomes associated with use of urinary biomarkers for diagnosis or surveillance of bladder cancer (SOE: insufficient). Urinary biomarkers miss 23 to 42 percent of patients with bladder cancer and are incorrectly positive in 11 to 28 percent of patients without bladder cancer, which could result in delayed diagnosis or unnecessary cystoscopies and other diagnostic procedures, but no study directly measured effects of inaccurate diagnosis on clinical outcomes (SOE: insufficient).

Most trials found that fluorescent cystoscopy was associated with decreased risk of subsequent bladder recurrence versus white light cystoscopy, but there was no difference in risk of progression or mortality, although data for these outcomes were relatively sparse (SOE: low). In addition, evidence on effects on risk of recurrence was inconsistent, and the only trial²⁵ designed to minimize performance bias (by blinding the cystoscopist to instillation of photosensitizer vs. placebo) found no difference in risk of bladder cancer recurrence.

Intravesical therapy was effective for reducing risk of bladder cancer recurrence versus no intravesical therapy. Compared with no intravesical therapy, BCG was associated with decreased risk of bladder cancer recurrence (RR, 0.63; 95% CI, 0.50 to 0.79) as well as progression (RR, 0.50; 95% CI, 0.32 to 0.77) (SOE: moderate). MMC, doxorubicin, and epirubicin were also associated with decreased risk of bladder cancer recurrence versus no intravesical therapy (RR, 0.66 to 0.80), but effects on bladder cancer progression were not statistically significant (MMC and epirubicin) or showed no effect (doxorubicin). Although trials varied with respect to doses, instillation regimens, and patient populations evaluated, findings were generally robust in sensitivity and subgroup analyses. No intravesical agent, including BCG, was associated with decreased risk of all-cause or bladder cancer–specific mortality versus no intravesical therapy. Evidence on gemcitabine, interferon alpha, and thiotepa was sparse, and we found no randomized trials of valrubicin, paclitaxel, or apaziquone.

Head-to-head trials of intravesical therapy using different drugs showed few clear differences. For BCG versus MMC, the most well-studied comparison, there was no difference on any outcome, including bladder cancer recurrence, progression, or mortality (SOE: moderate). However, BCG was associated with decreased risk of bladder cancer recurrence in the subgroup of trials that evaluated maintenance regimens (SOE: low). Other head-to-head comparisons were evaluated in fewer trials, and in general showed few differences. A possible exception was for BCG versus epirubicin, for which there was some evidence that BCG might be associated with decreased risk of bladder cancer recurrence and progression versus epirubicin (SOE: low). Although doxorubicin was associated with increased risk of bladder cancer recurrence versus epirubicin (RR, 1.56; 95% CI, 1.08 to 2.22), this finding was based on only three trials (SOE: low).^26-28 Evidence to determine the effects of tumor characteristics on estimates of effectiveness of intravesical therapies was limited but indicated no differences in risk estimates based on factors such as tumor stage, grade, multiplicity, recurrence status, and size (SOE: low). However, even if relative estimates of effectiveness are similar, absolute effects will vary depending on the underlying incidence of recurrence, progression, mortality, or other outcomes. Therefore, patients with higher stage, higher grade, multiple, recurrent, or larger tumors would be expected to experience greater absolute benefits. Evidence to determine the effects of patient characteristics, such as age, sex, race/ethnicity, performance status, or comorbidities, on estimates of effectiveness of intravesical therapies was not available.

Results from trials that compared effects of intravesical therapy using different doses or instillation regimens for the same agent were difficult to interpret because of variability in the patient populations, doses, instillation regimens, and other factors. For BCG, there were no clear differences between standard and lower doses in risk of bladder cancer recurrence, progression, or mortality, including in patients with higher risk NMIBC, but there was some inconsistency between trials (SOE: low). Limited evidence suggested that BCG maintenance regimens (>6 weeks) are more effective than induction regimens (≤6 weeks) at reducing risk of bladder cancer recurrence in patients with higher risk tumors (SOE: low). Trials on the effects of dose and duration of other intravesical agents on outcomes reported inconsistent results and were clinically heterogeneous, making it difficult to draw strong conclusions (SOE: insufficient to low). However, there is no evidence that prolonging therapy for more than 1 year is more effective than shorter regimens.

Evidence on harms associated with intravesical therapies was more limited than evidence on benefits. Trials of BCG versus no intravesical therapy found that local and systemic adverse events were relatively common (chemical cystitis or irritative symptoms in 27% to 84% of patients, macroscopic hematuria in 21% to 72%, and fever in 27% to 44%) (SOE: low). BCG was also associated with an increased risk of local adverse events and fever versus MMC (SOE: low to moderate). Standard-dose BCG was associated with increased risk of local and systemic adverse events versus lower dose BCG. Few trials reported harms of intravesical agents other than BCG versus no intravesical therapy or versus another intravesical agent.

The only randomized trial of radiation therapy found no effects on recurrence, progression, or survival in patients with T1 Grade (G) 3 cancers when compared with no radiotherapy (for unifocal cancers and no CIS) or against intravesical therapy (for multifocal disease or CIS) (SOE: low).²⁹

Findings in Relationship to What Is Already Known

Our findings on diagnostic accuracy were generally consistent with prior systematic reviews that found urinary biomarkers insufficiently accurate to replace cystoscopy.^30-32 Estimates for sensitivity and specificity were generally similar in our review and prior reviews, even though we excluded case-control studies and included more recently published studies. In addition, prior reviews did not evaluate potential differences in diagnostic accuracy for testing performed for evaluation of signs and symptoms of bladder cancer versus for surveillance.

Prior systematic reviews^33,34 found fluorescent cystoscopy to be associated with decreased risk of recurrent bladder cancer versus white light cystoscopy, but they were published prior to a recent trial that was the only one to blind the cystoscopist to instillation of the photosensitizer and found no effect.²⁵ Like our report, prior reviews found no effect of fluorescent cystoscopy on risk of progression or mortality. Although prior reviews also found that fluorescent cystoscopy detected more bladder cancers on initial cystoscopy, this was not an assessed outcome for our review.

Our findings regarding the comparative effectiveness and harms of intravesical therapies are generally consistent with prior reviews that found intravesical therapy to be associated with decreased risk of bladder cancer recurrence versus no intravesical therapy^35,36 and found BCG to be associated with decreased risk of bladder cancer progression. Prior systematic reviews that focused on immediate single-instillation therapy also found intravesical therapy to be more effective than no intravesical therapy in reducing risk of bladder cancer recurrence, a conclusion consistent with our finding of no clear difference in risk estimates based on the type of instillation regimen.^37-39 Like our review, a prior systematic review found that maintenance therapy with BCG was associated with decreased risk of bladder cancer versus MMC, despite some differences in the trials that were included, definitions of maintenance therapy, and use of individual patient data in the prior review.⁴⁰ Our findings are also consistent with prior systematic reviews that found BCG to be associated with decreased risk of bladder cancer versus epirubicin,⁴¹ that the evidence on intravesical gemcitabine is limited,⁴² and that the optimal dose and duration of intravesical therapy cannot be determined based on the available evidence.⁴³

Implications for Clinical and Policy Decisionmaking

Our review has implications for clinical and policy decisionmaking. As there are no studies evaluating effects of using urinary biomarkers for diagnosis or surveillance of bladder cancer on clinical outcomes, decisions regarding their use must necessarily be made on the basis of diagnostic test performance. Table B shows estimated probabilities for bladder cancer following use of urinary biomarkers, based on likelihood ratios calculated from pooled sensitivities and specificities. In populations with a pretest probability of 5 percent, the post-test probability increased to 16 to 24 percent following a positive result and decreased to 1.8 to 2.5 percent following a negative result. In settings with a pretest probability of 20 percent, the post-test probability increased to 37 to 60 percent following positive results and decreased to 8.0 to 11 percent following a negative result. Whether urinary biomarkers are sufficiently accurate to rule out bladder cancer and thereby reduce the need for cystoscopy depends on the ability of clinicians to estimate the pretest probability of disease and the acceptable threshold for a missed or delayed diagnosis. Use of urinary biomarkers in combination with urinary cytology increases the sensitivity for bladder cancer, but still misses about 10 percent of cases. Regarding fluorescent cystoscopy, studies have not shown an effect on progression or mortality, and trials that found reduced risk of recurrence may have been affected by performance bias. These findings might inform decisions regarding widespread adoption of fluorescent cystoscopy.

Table B

Post-test probability of bladder cancer using different biomarkers.

Our findings also have implications for use of intravesical therapy. Although intravesical therapy was associated with decreased risk of bladder cancer recurrence, there were no clear effects on bladder cancer–specific or all-cause mortality, and intravesical therapies were associated with local and systemic adverse events. Our findings are consistent with guidelines that recommend BCG as first-line therapy.^10,44 As no intravesical agent was more effective than BCG at reducing risk of bladder cancer recurrence, BCG is the only intravesical agent associated with decreased risk of bladder cancer progression versus no intravesical therapy, and some evidence indicates that BCG is associated with decreased risk of bladder cancer recurrence versus other intravesical agents. However, BCG is also associated with a high risk of adverse events. Some evidence indicates that using lower than standard doses of BCG maintains effectiveness while reducing harms. Other evidence suggests that longer courses of therapy may be necessary for optimal effects, particularly in higher risk patients. Therefore, decisions to use intravesical therapy and regarding the intravesical agent, doses, and regimen selected should take into account the tradeoffs between potential benefits and harms. Benefits are likely to be higher in patients at higher risk for disease progression and harms.

Limitations of the Review Process

Substantial statistical heterogeneity was present in most pooled analyses of diagnostic accuracy; this situation is common in meta-analyses of diagnostic accuracy.^45-47 As noted in the “Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy,” “heterogeneity is to be expected in meta-analyses of diagnostic test accuracy.”⁴⁷ To address the anticipated heterogeneity, we used random-effects models to pool studies and stratified studies according to the reason that imaging was performed and the unit of analysis used. We also performed additional stratified and sensitivity analyses based on the reference standard used, study characteristics (such as country in which the study was conducted, factors related to risk of bias), patient characteristics, and technical factors related to the imaging tests under investigation. Results were generally robust in sensitivity analyses, despite the heterogeneity. We also focused on evaluations of comparative test performance based on within-study comparisons of imaging modalities, which tended to be associated with less heterogeneity than pooled across-study estimates. A limitation of our analysis of within-group comparisons is that we had to treat the two compared groups as independent because we had aggregated data only. Individual patient-level data would be required to take into account the paired nature of the comparisons. Such correlations are generally positive and would be expected to result in more narrow CIs. Although it is possible that this could have caused us not to detect statistically significant differences, the point estimates indicated very little difference between tests.

We did not construct summary receiver operating characteristic curves. Almost all studies of a specific urinary biomarker used the same definition for a positive test, including tests based on a quantitative threshold. Estimates of sensitivity and specificity at different thresholds are needed to construct informative receiver operating characteristic curves.⁴⁸

Statistical heterogeneity was also present in some analyses of intravesical therapies and fluorescent cystoscopy. To address this, we used the Dersimonian-Laird random-effects model to pool studies. The Dersimonian-Laird random-effects model may result in CIs that are too narrow when heterogeneity is present, particularly when the number of studies is small.²⁴ Therefore, we repeated analyses using the profile likelihood method, which resulted in similar findings. Regardless of the method used, meta-analyses based on small numbers of trials can underestimate statistical heterogeneity and must be interpreted with caution.²⁴ We also stratified trials according to factors such as risk-of-bias rating, dose, number of instillations, duration of followup, enrollment of patients with high-risk NMIBC, and other factors. Although statistical heterogeneity remained present in some analyses, with some unexplained outlier trials, results were generally robust.

We excluded non–English-language articles and did not search for studies published only as abstracts. Because of small numbers of trials for meta-analyses involving intravesical therapies, we did not formally assess for publication bias using statistical or graphical methods for assessing sample size effects, as research indicates that such methods can be seriously misleading in such situations.^49,50 For fluorescent cystoscopy, we found one relatively large trial that showed no effect on risk of recurrence versus white light cystoscopy, suggesting that publication bias could have impacted results.⁵¹

Limitations of the Evidence Base

Several limitations of the evidence base limited our ability to reach strong conclusions with regard to several aspects of diagnosis and treatment of NMIBC. Other than quantitative NMP22 and qualitative BTA, urinary biomarkers were assessed in small numbers of studies (6 or fewer), resulting in less precise estimates. In addition, most of the evidence on comparative accuracy was indirect, as few studies directly compared the accuracy of two or more biomarkers against cystoscopy and histopathology.

For fluorescent cystoscopy, a limitation of the evidence base is that few trials reported effects on progression or mortality, and instead mostly focused on evaluating effects on recurrence. In addition, only one trial of fluorescent cystoscopy blinded the cystoscopist to whether the photosensitizer had been instilled, which may have an impact on assessments of recurrence because of performance bias related to knowledge of the type of initial cystoscopy performed.

A limitation of the evidence for all Key Questions addressed in our review is that very few trials were assessed as low risk of bias. Methodological shortcomings included failure to adequately describe randomization and allocation concealment methods, and unblended design. Findings would be stronger if more high-quality trials were available.

Other limitations include the lack of evidence on how use of urinary biomarkers impacts clinical outcomes (including harms), the evidence from only a single randomized trial on effects of radiation therapy for NMIBC, no trials on effects of using a risk-adapted approach, and no studies on how using different surveillance intervals impacts outcomes. Few studies evaluated effects of patient characteristics, such as age, sex, race/ethnicity, performance status, or comorbidities, on diagnostic test performance or effectiveness of intravesical therapy.

Research Gaps

We identified a number of important research gaps. Given the increased sensitivity of urinary biomarkers with cytology, studies on how this combination impacts use of cystoscopy and subsequent clinical outcomes might be helpful for determining its role in diagnosis or surveillance. Randomized trials that adequately safeguard against performance bias associated with use of photosensitizers for fluorescent cystoscopy are needed to determine effects on recurrence, progression, and mortality. Additional head-to-head trials of intravesical therapies that use more standardized instillation regimens and doses, report outcomes in subgroups stratified by patient and tumor characteristics, and include long-term outcomes related to progression and mortality would help clarify optimal treatment strategies. Research is also needed to determine the effectiveness of risk-adapted approaches to guide selection of therapy, including use of nontraditional prognostic markers, effects of different surveillance intervals and protocols, and newer techniques such as electromotive administration of intravesical therapy.