Prevalence

Because the unequivocal diagnosis of acute bacterial rhinosinusitis requires a bacteriologic evaluation (routinely obtained by maxillary sinus puncture), the true prevalence of acute bacterial rhinosinusitis is difficult to obtain reliably. There are no epidemiologic studies that used sinus puncture to estimate the prevalence of this condition in a given population. In this evidence report, the prevalence of sinusitis is estimated from diagnostic test studies and from treatment studies that used sinus puncture as the reference standard. In addition, we review data from several observational studies.

Diagnostic Tests

Studies identified in the literature search described in the Search Strategy and Study Selection sections were included if they presented data prospectively comparing the performance of two or more tests in the diagnosis of acute bacterial rhinosinusitis. These diagnostic tests included clinical criteria (studies had to evaluate a composite measure such as overall clinical impression or a decision aid such as a risk score), radiographs, ultrasonography, or sinus puncture/aspiration. Although studies evaluating computed tomography or magnetic resonance imaging would be eligible, no comparative studies of these tests meeting inclusion criteria were identified (there were several studies of chronic sinusitis). Studies were excluded if diagnostic tests were evaluated on individuals not presenting with symptoms of acute bacterial rhinosinusitis. However, the definition of acute bacterial rhinosinusitis varied among studies; for example, two studies evaluated some subjects with prolonged symptoms (Berg and Carenfelt, 1988; Williams, Simel, Roberts, et al., 1992), whereas others did not provide a definition of sinusitis. To avoid the problem of verification bias (Irwig, Tostetson, Gatsonis, et al., 1994), studies were excluded if some subjects did not undergo all tests being compared.

A comparison matrix (Evidence Table 1) using studies that met the inclusion criteria was constructed to help visualize the number of studies available for analysis. Meta-analyses were performed on comparisons where there were at least three studies or subgroups. These meta-analyses provided results to answer question 2.

Treatment Trials

Evaluation of antibiotics and ancillary treatment trials were considered separately. A matrix of antibiotic comparisons (Evidence Table 9) was constructed using randomized controlled trials meeting the minimum inclusion criterion of including patients with suspected acute bacterial rhinosinusitis of 4 weeks or less in duration. The intersection of a row and a column in this matrix denotes a comparison between two different antibiotics or a dosage or duration comparison of the same antibiotic. The number within a cell denotes the number of comparisons. An empty cell has no comparison available. For example, four comparisons are listed in the cell intersecting the row heading of amoxicillin and column heading of cefixime. The references for the comparisons are provided in Evidence Table 10. A total of 74 unique comparisons from 72 articles were identified.

The comparison matrix is used to help assess whether a relevant clinical question is addressable by determining whether a sufficient number of potentially useful studies is available to conduct a meta-analysis. In consultation with the technical experts, three meta-analyses of antibiotic trials were identified. The meta-analyses are based on an article published by several EPC members in the British Medical Journal (deFerranti, Ioannidis, Lau, et al., 1998). This meta-analysis was updated and provided the basis for the supplemental analyses of antibiotic treatment in this evidence report. Parts of this published article are used in this evidence report with permission from the journal.

Key questions 3 and 4 formulated by the technical expert panel concern the efficacy of antibiotics compared with that of placebo and the comparative efficacy of different antibiotics. Examination of the matrix (Evidence Table 9) identified 10 comparisons of antibiotics with placebo to address question 3. Some of these studies did not meet the inclusion criteria and were excluded from the meta-analysis. Similarly, two additional meta-analyses were identified for question 4.

Trials were eligible for inclusion in a meta-analysis if three criteria were met: (1) the trial compared amoxicillin or a folate inhibitor agent (e.g., trimethoprim/sulfamethoxazole) to another agent, generally one with a broad spectrum of activity including cephalosporins, penicillins with β-lactamase inhibitors, tetracyclines, quinolones, and macrolides; (2) patients were assigned randomly; and (3) the trial evaluated acute sinusitis or an acute exacerbation of chronic sinusitis ("acute-on-chronic"). Both adult and pediatric studies qualified. Trials of subacute or chronic sinusitis (greater than 4 weeks mean symptom duration) were excluded. Although dosage or duration comparison studies would provide useful information, a quick scan of the matrix revealed few such studies, and a meta-analysis was not possible. Placebo-controlled studies were examined to assess the effect of antibiotics on the natural history of acute bacterial rhinosinusitis.

Because the focus of this evidence report is on uncomplicated, community-acquired, acute bacterial rhinosinusitis, we excluded studies with immunocompromised patients such as patients with malignancy receiving chemotherapy, human immunodeficiency virus (HIV) infection, cystic fibrosis, asthma, Kartagener's syndrome, IgA and IgG deficiencies, and trauma or surgery-related sinus infections. Even though HIV infection and asthma are part of the exclusion criteria, we screened the MEDLINE search results for randomized controlled trials specific for these populations with the intention of performing separate subgroup analyses. We found none.

Since we found only 10 randomized controlled trials of ancillary treatments, a comparison matrix is not useful. Because of the heterogeneous mix of treatments, diagnostic definitions, and protocols, a meaningful meta-analysis was not possible for ancillary treatments. The data from these studies were abstracted and summarized in the evidence tables.

Diagnostic Test Studies

For each included study, we extracted test data cross-classifying individuals as having or not having bacterial rhinosinusitis. To calculate sensitivity and specificity in a comparison of two diagnostic tests, it is necessary first to decide which test is the "test of interest" and which is the "reference test." For included studies, comparisons of diagnostic tests were derived in a manner consistent with a "hierarchy" of accuracy: most accurate was sinus puncture, followed by radiography, ultrasonography, and then clinical criteria. Therefore, for example, estimates of sensitivity and specificity of ultrasound were derived with respect to radiography (and not vice versa). Many studies reported data only for "sinuses" and not for "patients"; these sensitivity and specificity data were used in the analyses.

When studies presented test performance data for more than one threshold or cut-point for tests of interest, we extracted data for each cut-point separately. For instance, for the clinical examination compared with radiography, data for overall clinical impressions of intermediate probability and high probability were included separately.

The following data were also extracted from each study: country where the study was performed and publication year of the study; age of study participants and duration of their symptoms; location of the study (hospital, office practice, or emergency department); and type of physicians who evaluate patients (primary care physicians or otolaryngologists). We also noted whether each diagnostic test was evaluated in a manner blinded to the results of the other evaluated tests.

Treatment Trials

Outcomes of interest were clinical "cure," "improvement," and "failure" as assessed within 48 hours of the end of treatment. Although we would have been interested in analyzing other outcome measures such as rates of improvement in the treatment and control arms and relapses, most studies do not report them. Cures and failures were recorded as defined by the individual study; "cure" generally meant resolution of all signs and symptoms, and "failure" generally signified no change or worsening of signs and symptoms. Data on radiographic "cure," "improvement," or "failure" and bacteriologic "cure" or "failure" were also extracted as defined by each study. The main analyses used clinical outcomes as the endpoint most relevant to clinicians because primary care practitioners do not routinely obtain sinus films for uncomplicated acute bacterial rhinosinusitis and almost never perform cultures of sinus aspirates. Furthermore, there is only limited evidence suggesting a correlation between radiographic or bacteriologic failure and clinical outcomes. Separate analyses assessed bacteriologic failures, radiographic failures, and patient withdrawals due to adverse drug effects.

In addition to clinical outcomes, data were also extracted on study design characteristics such as blinding, disease definition, publication year, and age group. These factors provided the basis for sensitivity analyses.

Meta-Analysis of Diagnostic Tests

Evidence Tables 3 through 8 list the items frequently considered in various quality assessments of diagnostic test studies. These items include specification and diagnostic criteria of the reference standard and the test and blinding of the interpreter of a test to the clinical information and the results of the other test. There were too few studies in each of the categories of the diagnostic test comparisons to allow a meaningful sensitivity analysis. Therefore, these items are included for descriptive purposes only. To avoid the problem of verification bias (Irwig, Tostetson, Gatsonis, et al., 1994), studies were excluded if some subjects did not undergo all tests being compared.

Meta-Analysis of Treatment Trials

In each trial, the following characteristics pertaining to the quality of study design, conduct, and reporting were assessed by two investigators with subsequent consensus: blinded vs. unblinded design, specification of criteria for the diagnosis of sinusitis, detailed reporting as to the use of decongestants, and robustness of the assessment of clinical outcomes and completeness of the information on outcomes (losses to followup). The diagnosis of sinusitis was categorized as "firm" if a trial performed sinus aspirations and culture or radiographic evaluations (assessing the presence of air-fluid levels, mucosal thickening greater than 6 mm, or opacification of sinuses). Any other diagnostic criteria, including clinical judgment or nasal swabs, were categorized as "subjective." Outcome criteria were judged to be well-specified when a study scaled symptoms or signs as assessed by patients and/or physicians in a way that could be replicated. Trials, which specified criteria to some extent, noted the signs or symptoms used to evaluate cure, improvement, or failure but were not specific about how these data were evaluated. Trials with unclear criteria made no mention of how clinical outcomes were determined.

In addition to this subject-specific assessment of quality components, we also used a previously developed, validated scale of assessing the methodologic quality of the trials on a scale of 0 to 5 with a value of 5 being the perfect score of a specific study (Jadad, Moore, Carroll, et al., 1996). This scale focuses on randomization, double-blinding, and description of withdrawals and has been widely implemented.

We further explored quality issues by conducting subgroup analyses by dichotomizing studies into two groups using factors thought to be associated with higher quality. Cumulative meta-analyses ordered by methodologic quality of the studies were used to explore possible treatment effect trends as studies with lower quality scores were added to studies with higher quality scores.

Meta-Analysis of Diagnostic Test Studies

We followed the general principles of conducting a meta-analysis of diagnostic test studies described several years ago in the Annals of Internal Medicine (Irwig, Tosteston, Gatsonis, et al., 1994). For each combination of the test of interest and the reference test, a summary receiver operator characteristic (SROC) curve was constructed based on the method described by Moses, Shapiro, and Littenberg (1993). Multiple data points from studies that provided data at different cut-points were used to derive these curves; because these observations were not independent, confidence intervals around SROC curves could not readily be calculated. SROC curves were derived for data points weighted by the inverse of the variance. When studies provided estimates of specificity over a wide range (approximating the total possible range from 0 to 1), the area under the SROC curve was calculated by extrapolation. In addition to the SROC method, random-effects weighted average was used to calculate the average sensitivity and specificity for each comparison. Although pooling these values separately as we did tends to underestimate the true test sensitivity and specificity, they are nonetheless useful estimates of the average test performance. We assessed the appropriateness of this method by noting the distance of the estimates from the SROC curve. Statistical analyses using the SROC curve method was performed using "Meta-Test" version 0.6, a computer program developed by the EPC director (Dr. Lau).

Alternative Method for Summarizing Diagnostic Test Data

In addition to the SROC curve method, we explored a method described by de Bock, Dekker, Stolk, et al. (1997) for combining studies to estimate the sensitivity and specificity of diagnostic tests either when there is no reference standard or when a reference standard is not available for all comparisons. We applied this method to the diagnostic studies for acute rhinosinusitis.

Estimates are obtained by maximizing the likelihood through the Expectation Maximization (EM) algorithm (Dempster, Laird, and Rubin, 1977). Through an iterative procedure, the method estimates the true sensitivity and specificity of each test without assuming that any of the tests is a reference standard. First, starting values are assigned for the sensitivity and specificity of each test and for prevalence of the disease associated with each study. Second, based on these parameter values, the number of diseased and not diseased are estimated for each study. This is called the E-step, for estimation. Third, the maximum likelihood estimates are computed using the estimates from the E-step. This is called the M-step, for maximization. The estimates from the M-step are then used in the E-step, and the process iterates until convergence. The process should be repeated with different starting values to make sure the initial choice of parameters does not affect the final estimates.

The validity of the method relies on two key assumptions. One assumption is that an individual's results on two tests are independent, conditioned on whether the person has the disease. The other assumption is that the sensitivity and specificity of each test do not vary from study to study. Both of these assumptions are violated by the data available in the literature. Most studies did not blind the interpreter of the reference test to the results of the test of interest, so conditional independence is unlikely. Another way conditional independence can be violated is if the disease is not dichotomous, and different tests pick up the disease at different severity levels, as pointed out by the authors of this method. In our meta-analysis of diagnostic tests, the SROC curve analysis is evidence that the sensitivity and specificity for a given modality vary. Indeed, there are several criteria for defining a positive sinus radiograph with sensitivity estimates varying from 0.41 to 0.90.

We abandoned this method because of the violations of its assumptions and based our conclusions on the SROC curve method.

Meta-Analysis of Antibiotics Randomized Controlled Trials

First, treatment outcomes from studies that had placebo arms were pooled to determine the effect of treatment with any antimicrobial on the natural history of acute bacterial rhinosinusitis. Second, two main comparisons between antibiotic groups were made: newer and/or expensive antibiotics including cephalosporins, macrolides, quinolones, tetracyclines, and penicillins with a β-lactamase inhibitor vs. amoxicillin, and newer and/or expensive antibiotics (as above) vs. folate inhibitors.

The general method of quantitative synthesis was followed (Laird and Mosteller, 1990; Lau, Ioannidis, and Schmid, 1997). Pooling of risk ratios, risk differences, and control group event rates were performed using both the Mantel-Haenszel fixed effects model (Mantel and Haenszel, 1959) and the DerSimonian and Laird random effects model (Fleiss, 1993; Ioannidis, Cappelleri, Lau, et al., 1995). The random effects model takes into account the variability of the true treatment effect between studies and provides a wider confidence interval (compared with the fixed effect model) when heterogeneity is present. Heterogeneity between studies was assessed with a chi-square statistic. This test is not very sensitive; therefore, heterogeneity was considered statistically significant if p<0.10 (Lau, Ioannidis, and Schmid, 1997). Weighted rates are also reported; rates were weighted by the inverse of their variance with random effects.

For the main outcome of clinical failure, we performed sensitivity analyses excluding: (1) pediatric studies; (2) studies in which amoxicillin or folate inhibitors were compared against tetracyclines which have been available almost as long, but continue to be more expensive; (3) studies in which patients with resistant organisms had been excluded; (4) studies in which diagnosis of sinusitis was not made on firm criteria; (5) studies with unclear assessment of outcomes; (6) studies that were not double-blind; (7) studies published before 1993; and (8) studies with a "Jadad" quality score of lower than 3. In each case, heterogeneity between excluded and remaining studies was assessed by the chi-square test and deemed significant for p<0.1. Statistical analyses of pooling treatment effects from randomized controlled trials were performed using "Meta-Analyst" version 0.991, another computer program developed by the EPC director (Dr. Lau).