Differences in Disease Distribution by Prespecified Subgroups

We attempted to assess whether disease distributions differed by prespecified subgroups for KQ1. These included comparison between U.S.-based and non-U.S. based studies; patient age group (children younger than 18 years of age versus adults aged 18 years or older; and, within the adult population, adults younger than 65 years of age versus adults aged 65 years or older); ED type (general versus specialty ED [e.g., psychiatric, eye and ear]); and ED disposition (ED discharges versus admissions versus transfers).

Differences by Patient Age Group

We identified one study conducted among pediatric populations,⁹⁰ none among adult populations, four among multiple age groups or populations not restricted by age,^{17, 59, 80, 96} and four among populations where the patient age was unclear or not reported.^{16, 31, 71, 95} While meta-analytic comparisons were hampered by study differences in design and reporting, there were clear differences in disease distribution by age group. These were most clearly illustrated in the largest U.S.-based malpractice claims study, as shown above in Figure 2 and below in Table 5.

Table 5

Variation in diagnostic error malpractice claims (any severity) by patient age decile.

There are fewer ED diagnostic error-related malpractice claims among children (<18 years old, 13%) than among adults (18 years or older, 87%). This is mainly because there are fewer pediatric ED patients (about 30 million, 20-25%¹⁰⁷) than adult ED patients (about 100 million, 75-80%). However, this incompletely accounts for the difference. Table 5 shows that there are proportionately fewer claims per age decile for those ages 0-20 (8%) versus for those ages 21 and older (11%). When considering adults ages 21-60 (for whom age-related mortality has not appreciably reduced the general population),¹⁰⁸ the difference is even larger (17%, ratio about 2:1 for claims in adults versus children). A similar difference in the epidemiology of malpractice claims per population has been reported previously (for all claims, not restricted to diagnostic errors or ED care) using the National Practitioner Data Bank, which showed 5.6 claims per 100,000 population for children versus 10 claims per 100,000 for adults (ratio about 2:1 for claims in adults versus children),¹⁰⁹ so is unlikely to represent a bias in CRICO data. Although all malpractice claims are less frequent in pediatric populations, the plurality (48%) of claims are still diagnosis related (as in adults), and 58 percent occur in the ED setting.¹¹⁰ Although this absolute frequency difference between children and adults could be accounted for by a lower likelihood of a lawsuit being brought when the patient is a child, this seems highly improbable; if anything, one would suspect just the opposite, since legal actions are disproportionately sought when the severity of adverse outcomes is greater¹¹¹ (as would be the case for a child who might otherwise have a “full life to live” were it not for a devastating medical misdiagnosis). The greater likelihood of a lawsuit being brought when the claimant is a child is supported by data from the National Practitioner Data Bank showing higher payouts in pediatric than adult cases, with the highest payouts occurring among the youngest children and the lowest payouts among the oldest adults.¹⁰⁹ Some specific data on the relative frequency of claims, such as those related to lung cancer misdiagnosis in the ED, appear to confirm the general suspicion of a higher likelihood that cases will be brought when patients are younger (see Representativeness of Malpractice Claims Data for Disease Distribution, below).

This leaves two possible explanations—either (a) diagnostic errors are less frequent among children (e.g., because they have less medical comorbidity, so are less “complex”) or (b) harms are less frequent among children (e.g., because they are less often impacted by life-threatening diseases or are more medically resilient when such diseases are present). The rate of diagnostic errors in pediatric acute care settings (5.0%)¹¹² is close to that estimated for the aggregate ED setting (5.7%, see KQ2), suggesting explanation “a” is less likely. Explanation “b” makes sense and corresponds best to the data shown in Figure 2, which show that diagnostic error claim frequency roughly mirrors the relative prevalence of dangerous disease groups in children versus adults (higher prevalence of infections and lower prevalence of vascular events and cancer). Thus, to summarize, there appear to be fewer total (absolute) misdiagnosis-related harms among children, most likely because they are fewer in number (total population), visit the ED less frequently, and less often have a dangerous underlying cause; there is less evidence to support the contention that the rate of diagnostic errors is lower or that harms occur less frequently (or are less severe) when a misdiagnosis occurs and an underlying dangerous cause is present.

Overall, among children, vascular events are less prevalent than in adults while missed fractures and infections (including missed appendicitis) tend to predominate.^{17, 90} As shown in Table 5 with results by age decile, the largest malpractice claims-based study (Newman-Toker, 2019) found that infections accounted for 52 percent in the 0-10 age group and 31 percent in the 11-20 age group. Authors grouped fractures with “other” diseases, but a review of source data from the authors found that, among children under age 18, fractures accounted for just 9 percent (n=25 of 269) of diagnostic error malpractice cases of any harm severity and 7 percent (n=4 of 54) of those resulting in high-severity harms. In a smaller study of pediatric diagnostic error malpractice claims, 24 percent (n=12 of 50) were fractures.⁸⁴ Studies in pediatric populations using other methods showed some degree of concordance (i.e., a relative preponderance of infections), but were not directly comparable because of differences in design and disease categories. One cohort study of patients admitted from the ED, in particular, from Children’s Hospital (Boston, Massachusetts) looking at 10 predefined conditions (notably not including fracture) found the most common of the 10 diseases (regardless of error) were appendicitis (53%), pancreatitis (14%), and sepsis (10%) (n=2,151).⁸² However, the most frequent diagnostic errors (total n=67) occurred with Kawasaki disease (25 percent diagnostic errors [n=17 of 67]; 9% of cases [n=194 of 2,151]), followed by pancreatitis (24% diagnostic errors; 14% of cases) and septic arthritis (18% of diagnostic errors; 8% of cases). The list of diagnostic error frequency after that was appendicitis (10%), sepsis (9%), stroke (including cerebral venous sinus thrombosis) (6%), ovarian torsion (4.5%), and hemolytic uremic syndrome (3%). The diseases with the highest ratio of diagnostic error proportion to overall prevalence were hemolytic uremic syndrome, stroke, and Kawasaki disease. Since this study was conducted at a quaternary care referral center and the age ranges of patients included was not reported, it is unclear the extent to which results are representative of all ED diagnostic errors among children.

Representativeness of Malpractice Claims Data for Disease Distribution

It is known that malpractice claims data represent a biased sample of cases, so it is then reasonable to consider whether bias(es) might influence the distribution of diseases represented in this report. In particular, claims are known to be biased towards higher-severity harms¹⁷; this is self-evident from Tables 3 and 4, since high-severity harms are relatively rare, yet among the malpractice cases there are more high-severity harm cases than low- and medium-severity harm cases combined. This is further reinforced by the much higher fraction of high-severity harms in the malpractice claims than in the large incident report study described above (58%¹⁷ versus 15%¹⁶). It is uncertain what additional biases may be at work, but results from the systematic review do suggest that some specific biases in the malpractice claims data may be present.

It has previously been suggested that diseases with tangible clinical artifacts from the encounter (e.g., radiographs showing missed incidental findings, such as a lung nodule on chest X-ray) make it easier to bring a legal action, leading to overrepresentation of cancer cases in claims, which does appear to be the case in primary care settings.¹⁷ It is possible that this may partially account for the relatively high number of lung cancer cases among ED claims, particularly given the high frequency of obtaining chest imaging in the ED (relative to other types of imaging likely to disclose cancers of the breast, prostate, colon, or other malignancies).

It is unknown the extent to which the same bias might lead to overrepresentation of fractures among ED claims. As mentioned above, there are about 2 million ED cases of fractures per year in the United States, as of 2020, according to the NEISS.¹⁰² With a maximum plausible error rate of 5 percent and a more probable estimate of about 1 percent (see KQ2, Fractures), there are likely no more than 100,000 missed ED fractures per year and probably closer to 20,000 per year in the United States. By contrast, there are an estimated 800,000 strokes and likely 400,000 transient ischemic attacks (TIA) each year in the United States; with a meta-analytically summarized error rate of 17% (see KQ2), this suggests there are roughly 200,000 missed cerebrovascular events annually. In Table 2, fractures outnumber strokes 1.3-fold in U.S.-based malpractice claims and 4.2-fold in U.K.-based incident reports. It is hard to imagine how this discrepancy can be explained other than to suggest missed fractures are overrepresented relative to missed strokes in these data sets. One possible cause, alluded to in the prior paragraph, is the presence of verifiable evidence of the diagnostic error through re-examination of radiographs.

Given that only 1.5 percent of myocardial infarctions are missed, it is possible that missed heart attacks may also be overrepresented in malpractice claims relative to their population prevalence. In terms of population annual incidence, heart attacks and strokes are very similar in the United States,¹¹³ and the rate of missed stroke (17%) is roughly an order of magnitude higher than that for heart attacks, yet there are only 1.5-fold more strokes in claims than there are heart attacks. We speculate here that the rationale could be that “standard of care” expectations are now so high for heart attacks that any missed case probably crosses the legal threshold for care to be considered “sub-standard.” Alternatively, missed strokes could be underrepresented for the opposite reason—because successful legal claims may be infrequent when overall misdiagnosis rates are high (e.g., stroke manifesting with clinical dizziness or vertigo, where error rates are estimated to be roughly 40%,^{21, 103, 114–116} yet claims cases are fewer than expected¹¹⁷). In such cases, if the “standard of care” is effectively to miss (rather than detect) a stroke, a course of legal action may be pursued less often or only infrequently lead to a paid claim. If malpractice data are used to track diagnostic error rates or disease distributions, it will be important to conduct further research into the types, direction, magnitude, and frequency of such biases.

Age-related biases are also a possibility, at least for some diseases. Figure 2 and Table 5 show that the peak age of incidence of missed cancer in malpractice claims is 51-60 years of age, and most of this reflects lung cancer (46% of 122 cases) with the next most common being brain/spinal tumors (19%), hematologic malignancies (8%), and colorectal cancer (7%). However, the peak incidence of cancer cases is 65-74 years, with 71 percent of cases occurring over age 65.¹¹⁸ If the principal mechanism by which lung cancer is missed in the ED is via missed incidental lung nodules on chest X-ray,^{106, 119} then there is no specific reason why this should occur with greater frequency in younger patients than older ones—if anything, they should have less lung pathology that interferes with radiographic interpretation. This suggests a likely age bias to file a legal claim when the patient is younger, rather than older. It is unknown whether this sort of bias may explain some of the skewed distribution in Figure 2 and Table 5 towards more claims among younger and middle-aged patients, who have a lower incidence of dangerous diseases relative to their older counterparts; the alternative explanation is that misdiagnosis is more frequent because younger patients are not thought likely to have dangerous diseases (e.g., stroke).⁶⁴ Child abuse (non-accidental trauma) is a special case in which misdiagnoses are unlikely to result in malpractice claims, even if the underlying problem does result in serious harm to the child, since the abuser (often a parent) is unlikely to draw attention to the underlying cause via a legal claim. Also see KQ1, Differences by Patient Age Group, for additional consideration of potential biases related to pediatric claims.

Other biases could be at work that are not readily apparent from the available literature. For example, disadvantaged or vulnerable populations (e.g., those who are differently abled, racial or ethnic minorities, lower health literacy, lower socioeconomic status, prisoners, immigrants) might be more likely to be misdiagnosed and less likely to file a legal claim. However, we could find no specific evidence to suggest that this would likely impact the distribution of diseases for KQ1. In particular, it is important to note that there was close alignment between the list of diseases from malpractice claims and those reported in diagnostic safety incidents (Table 2), which argues fairly powerfully against a major disease maldistribution based on claims data.

Per-Visit Overall ED Diagnostic Error Rates

We use the term “overall” ED diagnostic error rates to refer to rates measured across presenting symptoms and clinical problems (as opposed to those that are symptom-, disease-, or discipline-specific). Although many studies reported on “diagnostic error” rates, they were mostly misdiagnosis-related harm rates, since they used an adverse event trigger to focus their search for errors. Only two studies addressed diagnostic error (as opposed to adverse events) systematically – one among ED patients who were discharged and the other among ED patients who were admitted to the hospital. These two studies are described below. In aggregate, the weighted average estimated ED diagnostic error rate is 5.7 percent (Moderate strength of evidence [SOE]).

We found just one study that systematically measured overall per-visit diagnostic error rates among patients discharged from the ED.¹³⁷ This study (Nuñez, 2006) was based in a large, university hospital in Spain and began by using an adverse event trigger (72-hour unscheduled returns for the same chief complaint) to identify cases and assess diagnostic errors (which external, masked reviewers defined as a discrepancy between initial and final diagnoses).¹³⁷ Study investigators then purposively sampled from the remaining visits (patients who did not return) to create a comparable population on factors likely to impact diagnostic error rates. Exclusion criteria were “age <14 years, obstetric/ gynecological emergencies, erroneous referral, voluntary withdrawal, and incomplete or unavailable data in the medical records at the hospital or health center.” Of 32,523 eligible patients during a four-month period in 2004, there were 250 unscheduled 72-hour returns; among these study investigators found a diagnostic error rate of 20 percent (Nuñez, 2006, Table 2, including footnote). The control group “consisted of 250 patients who did not return; these comprised the next consecutive patient after each case in an attempt to balance cases and controls with respect to the influence of the attendance team, patient census, day of the week, work shift, and other external factors.” Among the control group, the study investigators found a diagnostic error rate of 4 percent. Thus, diagnostic errors were 5-fold enriched among patients with 72-hour returns, but because the unscheduled return rate was very low at 0.8 percent of all visits (n=250/32,523 visits), the estimated total diagnostic error rate for the discharged ED population was very close to 4 percent. Authors did not report the admission fraction; however, given the small number of unscheduled returns (n=250), an admission fraction anywhere between 1 and 50 percent would produce a weighted average diagnostic error rate of 4.1 to 4.2 percent (with 4.1% being the value for a typical admission fraction of 10-15%). This likely represents a “floor” (minimum) rate estimate because diagnostic errors were based solely on chart review and not systematic patient follow-up. Methodologically, the control group schema was strong with respect to the risk of diagnostic errors in those with unscheduled returns versus those without, but the absolute error rate is of uncertain representativeness even for this individual ED. For example, if every diagnostic error was attributable to a single clinician who was intermittently on call, then the matched population would track that individual’s diagnostic error rate, rather than the average diagnostic error rate for the entire ED.

One high-quality prospective study was identified that examined overall diagnostic error rates and misdiagnosis-related mortality among patients admitted from the ED.⁷ The study was a prospective observational study of 755 consecutive ED patients at a university-affiliated tertiary care facility in Switzerland. They used the primary hospital discharge diagnosis as the reference standard for the final correct diagnosis. They used a rigorous and moderately reliable (kappa 0.54) process of classifying diagnostic differences that only counted clinically meaningful discrepancies for the main analysis of the primary outcomes (hospital length of stay and mortality). They found diagnostic differences in 42 percent of cases (n=319 of 755) and considered these meaningful discrepancies in 12 percent of cases (n=93 of 755). Although the authors demurred labelling these as errors (focusing on “error” as a process failure), these events meet the National Academy of Medicine (NAM) definition of a diagnostic error used in this report, regardless of whether an explicit, preventable failure occurred during the diagnostic process. Diagnostic errors were associated with longer hospital stay (mean 10.3 versus 6.9 days; Cohen’s d 0.47; 95% confidence interval 0.26 to 0.70; P = 0.002) and increased patient mortality (8.6% [n=8] versus 3.8% [n=25]); OR 2.40; 95% confidence interval 1.05 to 5.5 P = 0.038). Note that no post-hospital follow-up was performed, so the authors concluded that their estimates were likely minimum estimates (i.e., some additional diagnostic errors were presumably not captured by the inpatient team and therefore unaccounted for in the study results). The authors defend this approach well, but it is apparently more common than one might expect for the inpatient team to convert a correct ED diagnosis into an incorrect one, as found in one study that focused on the subset of patients with non-specific symptoms at higher risk for diagnostic error.⁶² Whether this is a “floor,” “ceiling,” or intermediate estimate therefore remains unknown.

Per-Visit Overall ED Misdiagnosis-Related Harm Rates

There were seven studies that assessed overall per-visit misdiagnosis-related harm rates, referred to in most of the studies as diagnosis-related adverse events, or similar terminology (Table 8).^{7, 24, 72, 131, 137, 141, 143} Only three of these studies were high-quality, prospective studies (Nuñez, 2006¹³⁷; Calder, 2010¹³¹; Hautz, 2019⁷) and just one included both those discharged and admitted from the ED, in addition to systematic patient follow-up (Calder, 2010).¹³¹ The prospective studies found adverse events and deaths at rates one to two orders of magnitude higher than those found in the various retrospective cohorts identified by revisit triggers. The retrospective studies are likely to represent substantial underestimates, given under-ascertainment as a consequence of design.

Table 8

Overall per-visit ED misdiagnosis-related harm rates.

As noted above, we identified only one high-quality study that assessed overall diagnostic adverse event rates for both admitted and discharged patients with a prospective design using systematic follow-up (Calder, 2010).¹³¹ They enrolled adult patients (≥18 years of age) from high-acuity areas of the ED (Emergency Severity Index triage level 1-3) during random shifts at two university-affiliated hospitals in Canada in 2004. They excluded patients deemed incapable of informed consent (cognitive impairment or major psychiatric illness; critically ill or in distress) or unable to complete 2-week phone follow-up (non-English/French speaker, no telephone, or expected to be unavailable). Of 518 enrollees (369 treat-and-release ED visits and 134 hospital admissions), an impressive 97 percent had a follow-up assessment, with 2 patients withdrawing and 13 lost to follow-up (at equal rates among those discharged versus admitted). They looked for prespecified “flagged outcomes” including deaths, hospital complications, returns, healthcare visits, and new, worsening, or persistent symptoms. They found 22 percent of both discharged and admitted groups had flagged outcome events, which were then assessed via chart review. They found 43 of 107 flagged outcomes represented preventable adverse events and classified 10 of these as diagnostic in nature. Thus, the authors found 2.0 percent (n=10 of 503, 95% CI 1.0 to 3.6) of ED patients enrolled suffered preventable diagnostic adverse events. However, treatment errors pursuant to inaccurate diagnoses were considered management adverse events, rather than being counted as diagnostic adverse events; furthermore, events had to be deemed causally related and preventable with a certainty of at least 5 on a 6-point Likert scale by at least 2 of 3 reviewers. Also, this study was conducted at an academic hospital, and teaching hospitals are known to have lower diagnostic error rates for some conditions (see KQ3). Therefore, this represents a “floor” estimate. One of the 10 patients died of a delayed diagnosis of aortic dissection (rate 0.20%, 95% CI 0.005 to 1.1). Although the severity of the morbidity was not fully quantified, one additional patient was noted to have suffered “permanent disability” from a missed myocardial infarction (rate 0.20%, 95% CI 0.005 to 1.1).

There were four retrospective studies that reported overall per-visit harm rates. All but one (Heitmann, which used the longest revisit window) found much lower rates than the prospective study (Table 8).^{24, 72, 141, 143} These studies all used triggered chart reviews at single institutions, with the trigger being an ED revisit or short-term hospitalization (<72 hours to <30 days), and none used regional health information exchange or insurance claims-based follow-up to ascertain health events. This means that diagnostic errors were not generally counted towards the totals if they (a) were not discovered until after the time window; (b) did not prompt further care within the time window; (c) were discovered at an outpatient clinic visit, rather than via an ED revisit or hospitalization; (d) prompted care at another hospital or health system; or (e) resulted in an out-of-hospital death. Furthermore, all studies used chart review procedures that required reviewers to gauge whether care was “appropriate” or diagnostic errors “preventable,” further reducing the estimates. Such chart reviews are limited by the data recorded, which tend to be systematically incomplete and biased away from relevant details in cases where diagnostic errors have occurred.^{104, 145, 146} This group of studies systematically underestimates harms, and likely does so by a wide margin, given much higher rates in studies not limited by under-ascertainment.

The four trigger-based studies examining returns after ED discharge were each conducted at single institutions (total of 5 EDs, with ED annual visit volumes ranging from 15,000 to 100,000) (Table 8).^{24, 72, 141, 143} Trigger event time windows varied from 72 hours to 30 days, reducing direct comparability across studies. The all-cause return rates ranged from 2.0-2.9 percent at 72 hours, 4.4-6.8 percent at 7 days, and 11 percent at 30 days, suggesting a fairly comparable rate of overall ED returns across studies. However, these returns were at lower absolute rates than those reported using U.S. state-level data (7.5 percent at 72 hours and 22.4 percent at 30 days),¹⁴⁷ perhaps suggesting an academic/teaching hospital bias in the reported studies.¹⁴⁸ The proportion of ED returns attributed to diagnostic error varied from 0.6 to 14.2 percent, with a weighted mean of 1.0 percent (n=94 of 9,277). The overall rate of diagnostic adverse events (returns attributed to diagnostic error) per original ED visit varied over 100-fold across studies (i.e., across hospitals) from 0.01 percent at a large tertiary care ED in the United States to 1.6 percent at a small regional ED in Denmark, with a weighted mean of 0.022 percent (n=94 of 436,861). It was unclear the extent to which these reflected real differences between institutions as opposed to methodological differences in time windows, inclusions, or outcome definitions. Regardless, the rate of diagnostic adverse events in the one high-quality, prospective study (2.0%) is 92-fold higher than the weighted mean from the five retrospective studies (0.022%).

Misdiagnosis-related deaths per ED visit were reported in three of four retrospective studies,^{24, 72, 141} ranging from 0 to 0.007 percent, with a weighted mean of 0.0009 percent (n=4 of 436,173). On an institutional basis in these three studies (representing 4 EDs), each ED would see between 1 and 5 misdiagnosis-related deaths annually (based on their reported ED volumes). Since these studies conducted no systematic searches for out-of-hospital or out-of-hospital-network deaths and the single largest study (Aaronson, 2018, representing 94% of the patients synthesized) used 72-hour returns, rather than 7-day returns, this is, again, likely a substantial underestimate. The rate of misdiagnosis-related deaths in the one high-quality, prospective study (0.2 percent, n=1 of 503) is 217-fold higher than the weighted mean from the three retrospective studies (0.0009 percent). Although the rate of 0.2 percent is based on just a single death (so is imprecise, with a wide 95% CI 0.005 to 1.1), the value is the best estimate from this study and matches data from other sources. However, the confidence interval from the Calder study alone is implausibly wide. Based on data from other sources, we have assigned a +/− 2-fold plausible range to the 0.2 percent estimate (0.1% to 0.4%). This range bound comports well with other available data relevant to estimates of serious misdiagnosis-related mortality (details below in “Plausibility of Mortality Estimates from Higher Quality Studies”).

Per-Symptom ED Diagnostic Error and Harm Rates

Appendix Table B-2 shows included studies reporting on symptom-specific rates of diagnostic error. Six studies reported on rates of diagnostic error among polytrauma patients.^{121, 125, 127, 128, 130} Wilner et al. focused on pediatric patients and reported an 8 percent rate of delayed diagnosis of injury as well as a 0.3 percent rate of clinically significant delayed diagnoses.¹³⁰ The remaining five studies focused on adult populations, had varying definitions of diagnostic delay, and reported delayed diagnosis rates ranging from 0.2 to 40.3 percent.

Kornblith et al. reported that 16.9 percent of patients ‘found down’ had a late-identified injury/medical diagnosis.¹²³ Sun et al. reported a 4 percent rate of diagnostic delay among patients presenting with syncope/near-syncope.¹³⁵

Royl et al. reported a 44 percent rate of diagnostic error for patients seen in the ED with dizziness and for whom a neurology consult was sought. The rate of harm ranged 5 to 6 percent for patients that had a primary diagnosis changed from a benign to a serious condition, and for patients that had one serious primary diagnosis replaced with another serious condition.¹²⁹ Moeller et al reported a 17 percent discordance between the emergency clinicians’ diagnosis and the final diagnosis and a 19 percent discordance between ED trainees’ diagnosis and the final diagnosis among patients that received a neurology consult for any neurological complaint.¹³⁴

Two studies reported on misdiagnosis rates among patients presenting with headache.^{122, 140} Miller et al. included adult and pediatric patients and reported a 1.7 percent rate of missed intracranial diagnoses.¹⁴⁰ Dubosh et al. focused on adult populations and reported a 0.5 percent rate of serious misdiagnosis-related harms. Dubosh et al. also reported a 0.2 percent rate of serious misdiagnosis-related harms for adults presenting with atraumatic back pain.¹²²

Four studies reported on misdiagnosis rates among patients presenting with abdominal pain.^{58, 126, 138, 139} Gallager and Osterwalder focused on adult populations. Gallager et al. reported a 14.1 percent rate of misdiagnosis among abdominal pain patients receiving morphine and 14.6 percent among patients not receiving morphine.⁵⁸ Osterwalder et al. reported a 5.6 percent rate of misdiagnosis and 1.7% rate of patients requiring surgery.¹³⁹ Saaristo et al. reported on adult and pediatric populations, and found the misdiagnosis rate to be 3.3 percent rate; 0.7 percent of the patients with abdominal pain required hospitalization, and 0.06 percent needed immediate surgery.¹³⁸ Crosby et al. focused on pediatric patients and reported a misdiagnosis rate of 1 percent among surgeons, and of 0.3 percent among emergency medicine clinicians. Crosby also reported a 1.6 percent and 0 percent rate of misdiagnosis for testicular pain among surgeons and emergency clinicians, respectively, and equal rates of misdiagnosis for minor head trauma at 0.3 percent across the providers types.¹²⁶ Freedman et al reported a 0.28 percent rate of misdiagnosis among pediatric patients with constipation.¹⁴²

Two studies reported on misdiagnosis rates of adults presenting with dyspnea.^{54, 136} Ray et al. focused on older adults (65 years and older) and reported a misdiagnosis rate of 20 percent.¹³⁶ Pirozzi reported on all adults and found the rate of misdiagnosis to be 5 percent when using point-of-care ultrasound, and 50 percent when not using point-of-care ultrasound (their definition of a misdiagnosis was a discordance between the initial and final ED diagnosis).⁵⁴

One study reported on rates of misdiagnosis among adults presenting with ‘low-risk’ chest pain; they found a 0.5 percent rate of missed or delayed acute coronary syndrome among control patients, and 0% among intervention patients randomized for patient and clinician to receive print-out information on their acute coronary syndrome risk assessment.⁵⁶

One study reported on rates of infection misdiagnosis among older adults (age 65 years and older); they found an 18.4 percent false discovery rate in the ED.¹²⁴

Two studies reported on rates of misdiagnosis among patients receiving radiological imaging.^{132, 133} Chung et al. reported a 2 percent misdiagnosis rate for patients receiving torso imaging that were read by radiology residents during off-hours; 0.3 percent of the cases resulted in a change in management or call back to the ED, and no cases resulted in serious harm.¹³² Filippi et al. reported a 7.2 precent misdiagnosis rate of neurological magnetic resonance imagine (MRI) being read by radiology residents off-hours; 4.2 percent of the cases resulted in harm.¹³³

Stroke

We identified 50 studies (28 of these U.S.-based) that reported on the rate of diagnostic errors and/or misdiagnosis-related harms among over 1.9 million patients with cerebrovascular events.^{55, 64, 66, 69, 85, 87, 88, 120, 122, 144, 159–198} Studies varied significantly in the methodological approach, definitions to assess diagnostic errors, target populations, and inclusion/exclusion criteria. Most of the studies had a low risk of bias. However, 22 studies had an unclear or high risk of bias in terms of patient selection,^{177, 179, 182, 194} the reference standard,^{165, 167–169, 177–179, 181, 182, 186, 188, 195, 198} or the patient flow.^{160, 161, 163, 176, 182, 189, 195, 196}

Stroke: False Negatives

Twenty-three studies reported on the false negative rate (1-sensitivity) for stroke.^{64, 66, 87, 88, 120, 122, 159, 160, 169, 171, 173–175, 177–181, 183, 186, 194–196} Fourteen of these were sufficiently comparable to conduct a meta-analysis (Figure 4).^{66, 85, 87, 88, 159, 160, 171, 178, 179, 181, 183, 186, 194, 196} After contacting the authors, two of these were largely overlapping (Morgenstern, 2004 and Kerber, 2006 [dizziness subgroup]), so we excluded Kerber, 2006 from this meta-analysis. The pooled false negative rate was 15 percent (95% CI 9 to 23; I-squared 99%), with no clinically meaningful or statistically significant heterogeneity based on whether the study included only ischemic stroke, focused on subarachnoid hemorrhage, or had a mixed population that included ischemic strokes and intracranial hemorrhages (high SOE for false negative rate). The highest estimate (false negative rate 40%, 95% CI 38 to 42) was from a large U.S.-based study of patients (n=2303) admitted from the ED with non-stroke diagnoses who were discharged from the hospital with strokes of mixed subtypes, including transient ischemic attack (Chompoopong, 2017).⁸⁵ Authors acknowledged the limitation that some cases may have involved strokes occurring during hospitalization (i.e., not present at the time of admission). The lowest estimate (false negative rate 2%, 95% CI 1 to 3) was from a large Swiss study of patients (n=2200) of only ischemic strokes derived only from patients admitted to the stroke unit or intensive care units (Richoz, 2015).¹⁸⁶ Focusing on strokes admitted to stroke or intensive care tends to inflate diagnostic accuracy and reduce estimates of diagnostic error. Authors acknowledged the limitation that their estimate may have been low because some strokes may never have been detected; their methods note that MRI was not routinely performed—“Systematic diffusion-weighted MRI is not performed in all patients with new neurologic disease in our ED.” In a severity-adjusted analysis, they found worse outcomes and greater mortality.

We further analyzed false negatives excluding any studies with strong case selection filters likely to bias estimates away from the true overall cerebrovascular false negative rate (Figure 5). For this analysis, we excluded 2 studies selecting on case features that confer higher illness severity, which tends to bias towards lower error rates (Richoz, 2015 [stroke unit/intensive care unit admissions]¹⁸⁶; Pihlasviita, 2018 [stroke code activations for possible thrombolysis]¹⁸³). We also excluded 5 studies selecting on case features known to increase false negative risk—3 studies selecting only for younger stroke patients ages 16-50 (Kuruvilla, 2011¹⁷¹; Mohamed, 2013¹⁷⁸; Bhattacharya, 2013¹⁶⁰) and 2 studies selecting on case features linked to posterior circulation stroke (Kerber, 2006 [dizziness]¹⁶⁸; Calic, 2016 [cerebellar stroke location]⁸⁷). The resulting false negative rate point estimate (17%, 95% CI 9 to 27) was slightly higher than the overall point estimate prior to removing these potentially biased studies (15%, 95% CI 9 to 23), but each point estimate fell well within the other’s 95 percent confidence interval. The 17 percent estimate shown in Figure 5 (low selection bias) is more likely to be representative of the real-world ED rate. Because most studies did not capture missed strokes among ED treat-and-release patients or account for missed TIAs (which have higher error rates), this estimate is likely conservative.

Most of the studies did not compare diagnostic accuracy for TIA to that for acute ischemic stroke, but Whiteley, 2011 provided data that permitted such a calculation. Their results suggest that TIAs are more often missed than ischemic strokes (false negative rate 37.8% [n=14/37] for TIA versus approximately 20.8% [n=41/197, assuming equal proportion of hemorrhages among missed cases as overall], p=0.025). However, Morgenstern found that TIA did not predict greater odds of a false negative (odds ratio [OR] 1.02, 95% CI 0.71 to 1.46).

Figure 4

False negative rate for stroke in the emergency department by stroke subtype. CI = confidence interval; ES = effect summary (false negative rate); FN = number of false negatives; SAH = subarachnoid hemorrhage; TP = number of true positives; U.S. = United (more...)

Figure 5

False negative rate for stroke among studies with low selection bias, by stroke subtype. CI = confidence interval; ES = effect summary (false negative rate); FN = number of false negatives; SAH = subarachnoid hemorrhage; TP = number of true positives; (more...)

Stroke False Negatives: Younger Patients

Three studies included younger adult populations of stroke cases (16 to 50 years) and reported on missed cerebrovascular accidents. The pooled false negative rate of cerebrovascular accidents was 14 percent (95% CI 10 to 19, I-squared 0%).^{160, 171, 178} All three studies found higher rates of misdiagnosis among younger patients (either <35 or <40 years of age) within their already “young stroke” cohorts. Another study investigated delayed diagnosis of stroke specifically among children <18 years of age and reported that 65 percent of cases were diagnosed ≥6 hours after hospital arrival and 23 percent were diagnosed after 24 hours.¹⁷⁷ So, although the measured rate of 14 percent for patients aged 16 to 50 is nominally lower than the overall false negative rate for stroke of 17 percent obtained from other studies, this may be artifactual and related to methods or other inter-study differences.

A study using SPADE methods (which assesses misdiagnosis-related harms, rather than diagnostic error rates, per se, since detection is based on diagnostic adverse/unexpected events), found that patients 18 to 44 years of age were 6.7-fold more likely to suffer a missed opportunity antecedent to a stroke hospitalization than their older counterparts ages 75 and above (3.98% vs. 0.59% with P < 0.001 for differences across age groups).⁶⁴ The same study reported (in its supplemental “Appendix 2”) limited details on those under age 18, but, compared with those 18 and over, the odds of a misdiagnosis appeared to be greater. Specifically, the observed to expected ratio for antecedent ED treat-and-release visits for headache prior to a stroke hospitalization were 1.9-fold enriched for adults and 11.0-fold enriched for children.

Included studies did not permit a meta-analytic assessment of the overall rate of stroke misdiagnosis in pediatric populations, but available studies do seem to suggest that younger age is a strong risk factor for diagnostic error and associated adverse events, with the youngest patients (who have the lowest overall risk of stroke) having the highest risk of being missed.

Stroke False Negative: Special Stroke Subtypes (Subarachnoid Hemorrhage)

Three studies reported on false negatives in patients with subarachnoid hemorrhage. A prospective cohort study (n=401) from Western Europe reported 26 percent missed subarachnoid hemorrhage diagnosis, although the cohort included cases misdiagnosed outside of the ED, and the false negative rate in the ED was lower (12%) than the aggregate rate.¹⁸¹ When adjusted for initial case severity (i.e., restricting to patients with mild initial clinical presentations [Hunt and Hess grade 1 or 2], who comprised 59% of all cases), misdiagnosed patients had a 3.89-fold increased odds (95% confidence interval 1.9 to 8.0) of a poor clinical outcome. Two studies used look-back SPADE-style methods to assess diagnostic adverse events (i.e., the subset of false negative cases requiring hospitalization after an initial misdiagnosis). One retrospective cohort study reported 3.5 percent missed cases (observed minus expected ED visits within the last 45 days for patients ultimately hospitalized with subarachnoid hemorrhage) using Medicare data.¹²⁰ Another study reported a 5.4 percent miss rate for subarachnoid hemorrhage based on retrospective data from ED visits in the 14 days prior to hospital admission but, importantly, demonstrated wide variability across institutions (false negative rates ranged from 0-100% across 147 EDs); they found a paradoxical “misdiagnosis is protective” association between missed diagnosis and better health outcomes using crude (unadjusted) 30-day mortality, but were not able to adjust for initial case severity due to the lack of clinical details, leaving unanswered the question of whether earlier diagnosis may have actually improved outcomes in this cohort.¹⁹⁵

Stroke False Negative: Special Stroke Subtypes (Dissection, Cerebral Venous Thrombosis)

One study reported that 3.1 percent of patients with cervicocephalic dissection were treated-and-released from the ED in the prior 14 days with related symptoms.¹⁷⁵ Two studies included cases with cerebral venous thrombosis: one reported 3.6 percent misdiagnosis. They found longer length of hospital stay among misdiagnosed cases, but no unfavorable outcome (again, unadjusted for initial case severity). They also did a chart review on a smaller group of patients with cerebral venous thrombosis and found 6 percent missed diagnosis.¹⁷⁴ The other study reported 20.8 percent diagnostic error rate among cerebral venous thrombosis cases, using the “Safer Dx” Instrument.¹⁷³ Without adjusting for initial case severity, they found worse health outcomes among cases without a diagnostic error compared to those with a diagnostic error (28.6 versus 0%, P = 0.05),¹⁷³ again reflecting the apparent “misdiagnosis is protective paradox.” One study of stroke in polytrauma patients found that 11 of 192 patients (5.7%) had acute ischemic strokes, all of which were initially missed (and no neurologic consultations were obtained initially, despite neurologic findings being noted in four cases); the underlying cause for acute ischemic stroke was discovered by neurovascular imaging to be craniocervical dissection in six cases (two carotid artery, four vertebral artery); median time to diagnosis was 2 days (range 0 to 5).¹⁹⁸

Stroke False Negatives: Symptom—Specific Populations (Dizziness and Headaches)

Stroke patients presenting with dizziness or headache symptoms are prone to be missed. Dizziness increases the odds of misdiagnosis 14-fold over motor symptoms, and those with dizziness and vertigo are missed initially in an estimated 40 percent of cases.²¹ A large, population-based stroke surveillance program in Texas used ED chart review by neurologists (including hospitalization and imaging results) to validate stroke diagnoses, demonstrating that 46 out of 1629 (2.8%) of those with a presenting complaint of dizziness were strokes and 16 (35%) were misdiagnosed in the ED.¹⁶⁸ The same study found that only 5 of 15 cases with isolated dizziness admitted as stroke TIA from the ED were validated as stroke (i.e., false discovery rate of 67%). A second study from the same Texas cohort followed ED patients with dizziness who initially received a non-stroke diagnosis for a median period of 347 days, reporting a stroke incidence rate of 13.2 per 1000 person years (1.32%); this study found that most of that risk occurred in the first 48 hours after ED treat-and-release.¹⁶⁹ A separate U.S.-based study found that stroke hospitalizations were enriched in the 30 days following an ED dizziness discharge, with a false omission rate of 0.3 percent for these diagnostic adverse events; the 180-day cumulative incidence of a major vascular event or death was 0.93 percent.¹⁷⁰

Isolated ED headaches appear to be a risk factor for misdiagnosis of both ischemic stroke and intracranial hemorrhage (both intracerebral and subarachnoid).⁶⁴ A U.S.-based study found that among cases with an ED visit because of headache, 0.3 percent had ischemic stroke and 0.4 percent had any cerebrovascular disease within 1 year.¹⁷⁶ Similarly, a single-site study in the United States with regional follow-up showed that 0.6 percent of patients discharged with a benign headache diagnosis from ED had subsequent cerebrovascular disease hospitalization within 1 year.¹⁴⁴ Another U.S.-based study using state-level data found that stroke cases were enriched in the 30 days following an ED headache visit, with a false omission rate of 0.2 percent (n=4,253 of 2,101,081 headache discharges) (high SOE for false omission rate).⁸¹

Stroke False Negatives: Impact on Care and Outcomes

Multiple studies demonstrated cases who missed acute stroke interventions because of an initial missed or delayed diagnosis, including younger patients.^{88, 159, 171, 183, 194} In one large study of 2,027 confirmed acute ischemic strokes, 1.1 percent of misdiagnosed cases did not receive tissue plasminogen activator, despite being eligible; as expected, the number of cases eligible for tissue plasminogen activator was smaller in the misdiagnosed group than the correctly diagnosed group.⁸⁸ However, another study reported that in 22 percent of misdiagnosed ischemic stroke cases, the error resulted in missed or delayed tissue plasminogen activator administration.¹⁵⁹ This rate was slightly higher in community hospitals compared to academic centers, although the difference was not statistically significant. With regards to possible harms, misdiagnosed cases were readmitted within the next 60 days almost twice as often as correctly diagnosed cases.¹⁵⁹

Using Medicare data and a SPADE-style look-back approach (reflecting diagnostic adverse events), the false negative rate antecedent to stroke hospitalization (defined as observed minus expected prior ED visits) was reported to be 4.1 percent (95% CI 4 to 4.2) within the last 45 days and 3.7 percent (95% CI 3.7 to 3.8) within the last 30 days.¹²⁰ These cases reflect potential missed opportunities to have prevented major stroke after minor stroke or TIA.

Four studies, one from Australia and three from Western Europe, reported on missed ischemic and hemorrhagic strokes among 5,130 patients and analyzed stroke functional outcomes or mortality. The pooled rate of missed ischemic and hemorrhagic strokes was 7 percent (95% CI 3 to 14, I-squared 98%).^{181, 183, 186, 194} Using chart review (and an analysis unadjusted for initial severity), an Australian study found no association between missed diagnosis and worse outcome, but a subgroup of misdiagnosed cases who were admitted under non-neuro service showed worse functional outcomes (Modified Rankin Scale score ≥3, 80% versus 41%, P < 0.0001) and greater in-hospital mortality (15% versus 4%, P = 0.002) when compared to those admitted under the neurology service that was robust to adjustment for initial stroke severity.¹⁹⁴ Using chart review (and an analysis unadjusted for initial severity), a study from Finland found that 0.8% of misdiagnosed cases could have possible or likely worsened outcome, but no deaths were attributed to misdiagnosis.¹⁸³ A Swiss registry-based study with prospective data collection (Richoz, 2015) found in a multivariate, adjusted analysis (which included initial stroke severity) that favorable outcomes were 4.8-fold less likely and mortality 4.3-fold more likely among those with misdiagnosed acute ischemic strokes.¹⁸⁶ As noted in the subarachnoid hemorrhage section above, when adjusted for initial case severity (limiting to Hunt and Hess Grade 1 or 2, n=236), misdiagnosed subarachnoid hemorrhage patients with an initially mild clinical presentation had a 3.89-fold increased odds (95% CI 1.9 to 8.0) of a poor clinical outcome.¹⁸¹

Stroke: False Positives

Nineteen studies reported on the false discovery rate (1-positive predictive value) for strokes.^{69, 161, 164–168, 179, 180, 182, 184, 185, 187, 190–193, 196, 197} After contacting the authors, two of these were largely overlapping (Morgenstern, 2004 and Kerber, 2006 [dizziness subgroup]), so we excluded Kerber, 2006 from this meta-analysis. We have analyzed these by stroke type (Figure 6). The pooled false discovery rate was 21 percent (95% CI 14 to 29), but there was clinically meaningful and statistically significant heterogeneity based on stroke subtype. The most obvious difference was that TIAs were falsely positive at a much higher rate (49%, 95% CI 33 to 64) than ischemic strokes (10%, 95% CI 6 to 16) or brain hemorrhages (10%, 95% CI 7 to 12) (high SOE for false discovery rate). These differences are not surprising, since it is much more challenging to correctly diagnose TIA than completed stroke. The small differences between ischemic stroke, subarachnoid hemorrhage, and intracranial hemorrhage (Holland, 2015 in Figure 6) may be explained by frequent use of computed tomography (CT) scans (often obtained in the ED for neurological symptoms), which are substantially more sensitive for detection of hemorrhages than ischemic strokes.¹⁹⁹ Patients treated with thrombolysis using tissue plasminogen activator were false positives less frequently (erroneous treatment 3.9%, n=13/331).¹⁸³

A single U.S.-based study reported on stroke false discovery rate in the pediatric population by investigating stroke alerts (activated when patient presents with symptoms or signs suggestive of stroke or TIA, prompting rapid neurology stroke evaluation). They found that in 74.2 percent of pediatric stroke alerts, the correct diagnosis was not stroke or TIA. Given the design of this study, the high rates are unsurprising, since stroke alert calls are similar to requesting a neurology consultation for suspected stroke, rather than assigning a diagnosis, per se.¹⁷² Nevertheless, this high rate could also potentially reflect (a) a lower threshold for ordering a stroke consultation among children with neurological symptoms, (b) generally low stroke prevalence among children, or (c) increased probability of a false positive misdiagnosis.

As with false negatives, false positives appear to be disproportionately common among patients presenting to the ED with dizziness and vertigo—one included study from Western Europe found 31 percent of benign ear causes were initially misdiagnosed as stroke.¹²⁹

Figure 6

False discovery rate (referred/admitted) for stroke in the emergency department by stroke subtype. CI = confidence interval; ES = effect summary (false discovery rate); FP = number of false positives; ICH = intracranial hemorrhage; SAH = subarachnoid (more...)

Myocardial Infarction

We identified 15 studies that reported on the rate of diagnostic errors among 869,711 patients presenting with myocardial infarction to the ED.^{25, 63, 65, 77, 120, 200–209} The risk of bias of included studies was generally low.

Myocardial Infarction: False Negatives

Six studies^{25, 63, 77, 120, 202, 206} assessed diagnostic false negative rates for myocardial infarction in routine ED practice, five of which used variations of the SPADE method, look-back approach, based on recent ED treat-and-release visits antecedent to a hospitalization for confirmed myocardial infarction.¹⁴⁵ All were based on either regional or insurance-based capture of both hospitalizations and antecedent ED visits. A meta-analysis was conducted to synthesize those five similarly designed studies^{25, 63, 77, 120, 202} and estimated a false negative rate of 1.5 percent (95% CI 1.0 to 2.2; I-squared 99.7%; Figure 7; high SOE for false negative rate). These studies indicate that very few patients who are ultimately hospitalized with myocardial infarction are discharged from the ED in the 7 to 30 days prior. They do not address patients whose myocardial infarctions may have been mild or silent, never requiring hospitalization. Thus, these studies more closely reflect misdiagnosis-related harm rates than diagnostic error rates, per se.

We can assess the relationship between the measured harm rates and false negative diagnostic error rates from a large (n=10,689), U.S.-based prospective, randomized trial with 99 percent follow-up of patients, which did not meet our entry criteria because it was conducted in 1993.²² That study (Pope, 2000), which used serial measurement of creatinine kinase myocardial band (CK-MB) as the biomarker, found the missed myocardial infarction rate was 2.1 percent (95% CI 1.1 to 3.1). Even in the oldest study included in our meta-analysis (Schull, 2006, patients 2002-2003), troponin tests were available around-the-clock at 55 percent of 153 EDs responding to a study survey (survey response rate 89.5%, n=153 of 171).²⁵ Given the advances in diagnostic testing for myocardial infarction between the time of the Pope et al. study and the studies included in our meta-analysis, it would be expected that missed myocardial infarction rates in the ED would have fallen (i.e., would be below the 2.1% rate identified in the 1993 randomized trial). Accordingly, a measured misdiagnosis-related harm rate of 1.5 percent in our meta-analysis is probably quite close to the false negative diagnostic error rate for myocardial infarction, at least in absolute terms. If the true myocardial infarction false negative diagnostic error rate is 2 percent, then even though the error rate is 25 percent higher in relative terms, the absolute difference is just 0.4 percent. Thus, diagnostic error and harm rates for myocardial infarction appear to be low enough that the gap between the two values is at the level of rounding error.

Figure 7

False negative rate (initially discharged) for myocardial infarction in the emergency department. CI = confidence interval; ES = effect summary (false negative rate); FN = number of false negatives; TP = number of true positives; U.S. = United States (more...)

The sixth and final study (Graff, 2006) also began with a cohort of patients hospitalized for myocardial infarction, but addressed patients admitted, rather than discharged, from the ED. They found that 25.6 percent of myocardial infarction patients were admitted with a non-acute coronary syndrome diagnosis.²⁰⁶ These were mostly other cardiac diagnoses (17.6%) with respiratory diagnoses next (4.7%) and a small proportion of all other diagnoses (3.3%). Patients admitted with non-specific chest pain or coronary artery disease diagnoses were classified with more specific acute coronary syndrome admitting diagnoses (acute myocardial infarction and unstable angina) in the three-fourths of patients who were “correctly” diagnosed on admission (i.e., ED admitting diagnoses were underspecified). The authors found that the non-acute coronary syndrome admitting diagnoses (“diagnostic delay”) were associated with substantially lower quality care (substantially fewer evidence-based therapies applied, including 17% as opposed to 39% undergoing cardiac catheterization) than their counterparts with myocardial infarction who had no delay in diagnosis. Taken together with the meta-analytic results shown in Figure 7, this suggests that EDs are only rarely “missing” heart attacks outright, but diagnostic delays among admitted patients are perhaps substantially more frequent.

One SPADE look-forward study reported that among 325,088 patients who were discharged from the ED with a diagnosis of chest pain or dyspnea, 508 (0.2%) returned to the hospital and were diagnosed with a myocardial infarction (high SOE for false omission rate).⁷⁷ In one additional prospective study of 1114 patients admitted to three academic ED chest pain units, 991 were discharged after a negative chest pain work up and 0.4 percent developed acute coronary syndrome within 45 days.²¹⁰ Finally, a Canadian population-based study looked at 498,291 patients aged 40 years old or older who presented to an ED with chest pain and were discharged after assessment. Overall, 0.7 percent of patients were hospitalized within 30 days for myocardial infarction or unstable angina and 0.2 percent died. This study also demonstrated that higher ED volume was associated with significantly lower adjusted OR for mortality or acute coronary syndrome at 30 days.²¹¹

Myocardial Infarction: False Positives

Three studies assessed the diagnostic false discovery rate for myocardial infarction in routine ED practice.^{201, 207, 209} All three focused on false positive ST-elevation myocardial infarction, using patients referred for immediate cardiac catheterization who were determined not to be having ST-elevation myocardial infarction (STEMI). A meta-analysis produced a false discovery rate of 14 percent (95% CI 7 to 22; I-squared 95%; Figure 8; low SOE for false discovery rate) based on those three studies. No evidence on heterogeneity due to country, recruiting period, or clinician training could be detected due to the small number of included studies.

Figure 8

False discovery rate (cardiac catheterization) for acute STEMI in the emergency department. CI = confidence interval; ES = effect summary (false discovery rate); FP = number of false positives; STEMI = ST-elevation myocardial infarction; TP = number of (more...)

Aortic Aneurysm and Dissection

Twelve studies with an unclear- or low-risk of bias reported on the rate of misdiagnosis among at least 37,638 patients with aortic aneurysm or dissection.^{68, 73, 89, 120, 212–219} Two studies likely had overlapping study populations as they both included patients with ruptured abdominal aortic aneurysm from the same region of Sweden during similar time periods^{216, 219}; we included the most recent study in the analysis.²¹⁹

Aortic Aneurysm and Dissection: False Negatives

We pooled eight studies that reported on missed or delayed diagnoses among patients with ruptured abdominal aortic aneurysm^{89, 213, 219} or acute aortic dissection^{68, 73, 89, 214, 218, 219} (n=1,799). The estimated false negative rate was 36 percent (95% CI 21 to 52; I-squared 98%; Figure 9; moderate SOE for false negative rate). Studies differed in their definitions of missed or delayed diagnoses. Five studies compared patients who were correctly diagnosed in the ED or at initial presentation with those who were misdiagnosed.^{89, 213, 214, 218, 219} Two of these studies provided strict criteria for a correct diagnosis. Ohle et al. classified patients as missed diagnosed if they were not diagnosed within the ED, if they received treatment for an alternate diagnosis in the ED, or if they re-presented at an ED within 14 days of initial visit.²¹⁸ Smidfelt et al. considered patients as correctly diagnosed if aortic aneurysm was mentioned in the medical chart by the ED, if the patient was referred by the ED for an acute CT scan for aortic aneurysm, or if the patient received a laparotomy for suspected ruptured abdominal aortic aneurysm.²¹⁹ Two other studies used time to diagnosis to determine patients who had a short versus long diagnostic time.^{68, 73}

Figure 9

False negative rate (diagnostic delay) for aortic aneurysm or dissection in the emergency department. AAD = acute aortic dissection; CI = confidence interval; ES = effect summary (false negative rate); FN = number of false negatives; RAAA = ruptured abdominal (more...)

We were unable to include three studies in the meta-analysis due to differences in study design and differences in defining missed diagnoses. One was a retrospective cohort that reported a false negative rate of 0 percent among those who received a focused cardiac ultrasound by an emergency physician and 43 percent among those who did not.²¹² Another study reported a misdiagnosis rate of 24 percent among patients transferred to a referral center, including 15 percent of patients misclassified type of acute aortic syndrome (aneurysm called dissection, dissection called aneurysm, or error in type of dissection).²¹⁵ Using Medicare data and SPADE-style methods, the false negative harm rate for diagnosis (defined as observed minus expected prior ED visits in advance of a related hospitalization) was reported as 3.4 percent (95% CI 2.9 to 4.0) for ruptured abdominal aortic aneurysm and 4.5 precent (95% CI 3.9 to 5.1) for aortic dissection.¹²⁰

Three studies reported on the association between misdiagnosis and 30-day mortality.^{213, 216, 217} Pooling these three studies in a meta-analysis suggests a greater risk of 30-day mortality among those who were misdiagnosed than among those who were correctly diagnosed with aortic aneurysm and dissection (risk ratio [RR] 1.21; 95% CI 1.06 to 1.37; I-squared 0%; Figure 10). In addition to these unadjusted results, one study (Smidfelt, 2021) reported an even greater increased risk for mortality among those who were misdiagnosed when adjusted for age, sex, serum creatinine, and first-recorded systolic blood pressure of 90 mmHg or less (adjusted OR 1.83; 95% CI 1.13 to 2.96). The last of these is a proxy for initial case severity (those with low initial blood pressure were misdiagnosed in 28% vs. 44%, P = 0.001), and, as expected, when adjusted for initial severity (which often confounds the relationship between diagnostic error and misdiagnosis-related harms), the impact of diagnostic delay on mortality increases.

Figure 10

Association between initial emergency department delay in diagnosis of aortic aneurysm or dissection and 30-day mortality. CI = confidence interval; RR = risk ratio

Venous Thromboembolism

Five studies with a low-risk of bias reported on the rate of misdiagnosing venous thromboembolism (N=13,459 patients).^{54, 221–224} All of the studies, except one,²²⁴ were conducted outside of the United States.

Venous Thromboembolism: False Negatives

Three studies included patients (n=2,757) with a final diagnosis of pulmonary embolism and reported on the number of patients with a delayed diagnosis, which was defined as a diagnosis 7 days after the onset of symptoms^{221, 223} or a diagnosis between 24 hours and 30 days after an ED presentation.²²² Pooling these three studies in a meta-analysis yielded a false negative rate of 20 percent (95% CI 17 to 24; Figure 11; moderate SOE for false negative rate).^221–223 Heterogeneity was not significant (I-squared 43%). Limiting the meta-analysis to only studies that were conducted after 2010 yielded a pooled false negative rate of 22 percent (95% CI 18 to 25),^{222, 223} indicating no change over time.

We did not include one study in this analysis because of the heterogeneity in study design. This study recruited patients with undifferentiated dyspnea and randomized them to receive immediate or delayed point of care ultrasound.⁵⁴ The sensitivity and specificity of detecting acute pulmonary embolism in the ED were 89 percent and 100 percent, respectively, with immediate point-of-care ultrasound and 83 percent and 100 percent, respectively, with delayed ultrasound.

A second study was not included in this analysis because of heterogeneity in study design. This study was a retrospective interrupted time series evaluating age-adjusted dimer in patients over the age of 50 suspected of having pulmonary embolism (D-dimer ordered, chest related complaints, and no ultrasound order). The primary outcome was use of advanced diagnostic imaging and secondary outcome was diagnosis of pulmonary embolism within 30 days with age-adjusted D-dimer demonstrating a sensitivity of 95.2 percent and specificity of 68.6 percent.²²⁴

Two studies reported the mortality associated with a delayed diagnosis of pulmonary embolism.^{221, 222} One study reported no difference in all-cause mortality at 3 months between those with a delayed (>7 days from symptom onset) versus timely diagnosis (unadjusted OR 0.9; 95% CI 0.4 to 2.0).²²¹ This study showing no difference failed to adjust for baseline initial case severity, and patients diagnosed in timely fashion were clearly sicker at baseline (e.g., oxygen saturation <60 mmHg at presentation, 57 versus 42%, P = 0.03). The other study reported a significantly higher inpatient mortality rate among those with a delayed diagnosis (between 24 hours to 30 days after ED presentation) compared to those with an early diagnosis (unadjusted OR, 45.3; 95% CI 13.2 to 153.4).²²²

Figure 11

False negative rate (diagnostic delay) for pulmonary embolism in the emergency department. CI = confidence interval; ES = effect summary (false negative rate); FN = number of false negatives; TP = number of true positives; W. = Western

Meningitis and Encephalitis

Sepsis

We identified four studies that reported on the rate of diagnostic errors among 3,479 patients presenting to the ED and later diagnosed with sepsis.^{93, 156, 157, 225} All the studies were retrospective cohort studies (three of the four using SPADE-style look-back analyses in large electronic data sets) to identify missed diagnoses at ED or discrepancy in diagnosis between ED and inpatient. Only one study (Morr, 2017) was performed among adults (over 18 years of age). This study focused on review of consecutive hospital admissions from the ED to an internal medicine service; the case records were systematically assessed for evidence of infection, sepsis, and severe sepsis, and the authors reported on lack of recognition of sepsis or severe sepsis.¹⁵⁷

Sepsis: False Negatives

The pooled false negative rate among sepsis patients was 18 percent (95% CI 8 to 32; I-squared 99%; Figure 12; moderate SOE for false negative rate). Subgroup analysis by age showed a significant difference in rate of misdiagnoses among patients under 18 years of age (10%; 95% CI 3 to 21) versus those over 18 years (59%; 95% CI 45 to 72), with rates significantly higher among adults than children. However, the lone adult study (Morr, 2017) used very different methods than the studies in children, focusing on incorrect severity assessment among ED patients admitted with infections, rather than missed opportunities to diagnose infection among ED treat-and-release visits that were followed by sepsis hospitalizations. The Morr, 2017 paper refers to “ED discharge letters” but in the methods section they note that “All medical patients receive a detailed discharge letter upon transfer from the ED to the wards.”; this seems to clarify that the patients are all admitted via the ED, rather than admitted after having been treated and released previously from the ED. Thus, it is likely that the apparent difference in false negatives by age group is methods-related, rather than age-related. It is unsurprising that the rate of ED treat-and-release followed by sepsis hospitalization would be lower than the rate of correctly diagnosed infection requiring admission in which severity (i.e., sepsis) was under-recognized in the ED. Thus, the more generalizable false negative rate is likely 10 percent, rather than 18 percent. Two studies assessed impact of missed diagnosis on health outcomes (30-day mortality), and no significant difference was observed.^{93, 156} Both studies failed to adequately adjust for initial case severity in performing their analyses of adverse health outcomes.

Figure 12

False negative rate for sepsis in the emergency department. CI = confidence interval; ED = emergency department; ES = effect summary (false negative rate); FN = number of false negatives; TP = number of true positives; U.S. = United States; W. = Western (more...)

Arterial Thromboembolism

Spinal and Intracranial Abscess

One study included as part of the review (Dubosh, 2020) addressed missed spinal abscess among 1,381,614 ED discharges for back pain, enabling assessment of the false omission rate.⁸¹ Two others addressed missed cases (false negative rate) in all-comers with spinal abscess but were excluded during the full-text review stage; the nature of these exclusions (described below) is such that they are unlikely to invalidate the study findings, so results are presented here.

Pneumonia

We identified two studies that reported the rate of diagnostic error among 293 patients who were diagnosed with pneumonia.^{136, 229} Neither study addressed all ED patients with pneumonia. One study reported on community-acquired pneumonia among patients 65 years or older with acute respiratory failure¹³⁶; the other study reported on round pneumonia among patients under 19 years of age.²²⁹

Appendicitis

We identified eight studies that reported the rate of diagnostic error of appendicitis among 7,351 patients.^{225, 230–236} Two studies assessed diagnostic error as part of a prospective assessment of different diagnostic imaging in the ED.^{230, 231} Three studies conducted a retrospective analysis.^{225, 232, 233} Two studies examined diagnostic outcome changes before and during the coronavirus (SARS-CoV-2) disease 2019 (COVID-19) outbreak.^{234, 235} One study examined missed diagnostic opportunities at the ED using a look-back method.²³⁶ All approaches may have high risk of biases due to inclusion criteria and sampling.

Appendicitis: False Positives

The pooled false discovery rate for appendicitis diagnoses in the ED was 7 percent (95% CI 4 to 9; I-squared, 0%; Figure 13)^230–233 (moderate SOE for false discovery rate). The studies included a combination of prospective and retrospective cohorts. However, case selection due to inclusion criteria for certain studies limited their generalizability. False positive diagnoses of appendicitis may result in harm by subjecting patients to unnecessary surgical procedures.

Figure 13

False discovery rate for appendicitis in the emergency department. CI = confidence interval; ES = effect summary (false discovery rate); FP = number of false positives; TP = number of true positives; U.S. = United States; W. = Western

During the COVID-19 outbreak, Somers et al. showed that the false discovery rate decreased from 26.1 percent to 2.5 percent.²³⁴ This observation might have been due to changes in patient illness seeking behavior during the pandemic—since volumes were lower, perhaps only those who experienced more severe symptoms or more advanced illness might have ended up seeking care in the ED, making the mix of diagnoses more “obvious.” However, Willms et al. showed no difference in the false discovery rate before and during the COVID-19 outbreak.²³⁵

Fractures

We identified 17 retrospective or prospective studies (4 in the United States^240–243) that reported on the rate of diagnostic errors and/or misdiagnosis-related harms among 138,551 patients with fractures.^{74, 83, 121, 240–253} Studies varied significantly in the methodological approach, definitions to assess diagnostic errors, target populations, and inclusion/exclusion criteria. Most of the studies had a low risk of bias. The retrospective design in most of the studies makes it difficult to know the true rate of diagnostic error. When Enderson and colleagues changed the study design from retrospective to prospective the incidence of missed traumatic fractures increased from 2 to 9 percent.¹⁵² Studies tended to emphasize delay in diagnosis as an endpoint, while clinically meaningful delays affecting outcomes were far fewer in number.

Testicular Torsion

We identified two studies (one in the United States and one in Canada) that reported on the rate of diagnostic errors and/or misdiagnosis-related harms among 262 patients with testicular torsion.^{254, 255} One study was a retrospective review evaluating doppler ultrasound as a means of detecting testicular torsion.²⁵⁴ The other study was a retrospective chart review of ED patients who underwent detorsion and orchiopexy or orchiectomy (2005-2015).²⁵⁵

Other Conditions

We did not find any studies meeting our inclusion criteria that reported on the ED diagnostic error rate for endocarditis, necrotizing enterocolitis, sudden cardiac death, arrythmias, congenital heart disease, ectopic pregnancy, or pre-eclampsia/eclampsia.

ED Treat-and-Release Discharges Versus Hospital Admissions

There is direct evidence that diagnostic errors are more frequent among patients discharged than admitted. Heitmann et al., 2016 found that 1.6 percent of ED discharges and 0.3 percent of patients admitted to a hospital ward via the ED returned within 30 days due to a diagnostic error, and almost all of these (in both subgroups) returned within 7 days.¹⁴³ This likely indicates that hospital admission serves as at least a partial clinical safety net when there is diagnostic uncertainty or error, and comports with U.S. Medicare data showing that EDs with very high discharge fractions (proportion of patients sent home on any given day) are more susceptible to diagnostic errors associated with short-term, unexpected patient deaths.¹⁴⁸

For both stroke and myocardial infarction there was evidence that patients admitted with the wrong ED diagnoses were more frequent than patients misdiagnosed and discharged. Chompoopong, 2017 began with a cohort of patients hospitalized for stroke and found that 40 percent were admitted initially from the ED with non-stroke diagnoses.⁸⁵ Graff, 2006 began with a cohort of patients hospitalized for myocardial infarction and found that 25.6 percent were admitted initially from the ED with non-acute coronary syndrome diagnoses.²⁰⁶ These rates are much higher than the overall false negative diagnosis rates among patients who are discharged (17% for stroke, 1.5% for myocardial infarction). This may suggest that ED clinicians are (appropriately) focused more on correct disposition than correct diagnosis, per se.

There also appears to be evidence that false negatives for dangerous diseases, particularly among those discharged from the ED, are generally less common than false positives (Table 9). Some false negatives are associated with significant adverse outcomes (including death), but we presume that false positives (i.e., those who undergo diagnostic testing for the dangerous disease in question via hospital admission but are found instead to have some more benign underlying cause) are generally less dangerous for patients. This would seem to suggest that ED clinicians are weighting their diagnostic decision-making tradeoffs appropriately based on asymmetry of outcomes (i.e., dangerous diseases are worse to “undercall” than to “overcall”).

Temporal Profile of Diagnostic Adverse Events/Harms

It has been shown previously that the short-term risk of adverse events following a false negative (missed) dangerous disease in the ED follows a characteristic temporal profile. The initial risk is at its peak, then exponentially declines towards a linear base rate over days to months, depending on the specific disease. We identified studies in our review showing this pattern for stroke,^{64, 81, 120, 144, 170} myocardial infarction,^{63, 77, 120} aortic aneurysm/dissection,¹²⁰ multiple vascular events combined,¹²⁰ sepsis,^{78, 93, 94, 256} meningitis,^{81, 93} and spinal abscess.⁸¹ Unsurprisingly, the temporal profile of returns after a missed case seems to mirror the underlying disease biology and natural history, as shown in Figure 14 for stroke.

Figure 14

Cumulative incidence of stroke hospitalizations post ambulatory (ED or other) treat-and-release as “benign dizziness” (a) and cumulative incidence curve for natural history of major stroke following TIA or minor stroke (b). ED = emergency (more...)

This appears to be true, more generally, of diagnostic adverse events in the ED. Specifically Heitmann et al. also showed that most returns linked back to chart review-detected diagnostic errors occur in the first week (Figure 15). This comports with data from the large administrative and electronic health record data studies alluded to above that use symptom-disease pairs (“SPADE” methods),¹⁴⁵ which have found that short-term rehospitalizations occurring at rates statistically above baseline for missed dangerous vascular events and infections occur dominantly in the first month and disproportionately in the first week after ED discharge. This indicates that 72-hour or 7-day revisits are expected to be an enriched source to detect diagnostic error, but that absolute error rates will be substantially underestimated using very short revisit windows for analysis (e.g., 72-hour returns, which are commonly utilized).

Figure 15

Nature of short-term ED revisits. ED = emergency department; SPADE = Symptom-disease Pair Analysis of Diagnostic Error Note: The graph clearly demonstrates that return visits related to diagnostic errors (dots) tend to occur predominantly in the first (more...)

It should be noted that this temporal profile has also been demonstrated for all ED revisits (without regard to underlying cause for the revisit). Using data from the Agency for Healthcare Research and Quality’s Healthcare Cost and Utilization Project (HCUP) family of databases,²⁵⁹ Rising et al. found that 31 percent of ED visits were followed by a revisit within 1 year (3-day revisit rate 7.5%; 30-day revisit rate 22.4%).¹⁴⁷ The modeled cumulative hazard for revisits showed exponential growth over roughly the first 14 days, followed by a linear rise thereafter (best approximated by a double-exponential model, with excellent fit, R² = 0.9997). The authors concluded that the optimal balance between capturing “excess” acute revisits and “expected” revisits would be achieved by using a 9-day return window for quality measurement, rather than the more typically used 72-hour window. However, for diagnostic error detection, relevant windows likely vary in disease-specific fashion.

Representativeness of Malpractice Claims Data for Root Causes

It is known that malpractice claims data represent a biased sample of cases, so it is then reasonable to consider whether bias(es) might influence the root causes of diagnostic error identified. As described above, it was clear from ED incident report studies (e.g., Hussain 2019,¹⁶ Okafor 2016³¹) that the spectrum of root causes identified is quite similar to that found in ED malpractice claims studies—mostly cognitive errors related to bedside diagnostic decision-making (especially clinical examination, test ordering, or integration of test results into diagnostic reasoning). What is not known is whether both malpractice claims and voluntary incident reports might be biased towards cases with cognitive errors by physicians. This question cannot be easily addressed by retrospective studies relying on chart review, since most potential root causes must be inferred (i.e., they are not actually captured or recorded). Nor can it be addressed by diagnostically oriented, experimental vignette-based studies (which only assess for cognitive errors). To address this question rigorously, one would need a cohort study or clinical trial that prospectively captured all potential root causes and then assessed diagnostic errors and root causes. We found no such studies, so this remains an unanswered scientific question.

Patient Characteristics

We identified 108 studies that reported the effect of one or more patient characteristics on diagnostic error. We report the impact of patient characteristics separately by condition. Across conditions, the impact of age, sex, race, and ethnicity were reported far more often than the impact of language, socioeconomic status/income, health literacy, or health insurance.

The most common patient factors studied in relation to the misdiagnosis of stroke were age, sex, and race. Older age was associated with a lower risk of misdiagnosis^{64, 88, 171, 183} and patients with missed stroke were younger than the correctly diagnosed cases.^{160, 164, 174, 175, 186, 187, 192, 195} However, several studies found no age-difference in the time to evaluation for stroke.^{161, 185, 188, 261} Women were more likely to be misdiagnosed^{64, 164, 175, 183, 192, 262} and have a longer time of evaluation.^{185, 188, 261} Black^{64, 174, 188, 262} and Hispanic^{64, 262} patients were also at increased risk of misdiagnosis. Some studies reported no difference by race or ethnicity.^{160, 179}

Twenty studies reported on patients’ characteristics and missed or delayed diagnosis of myocardial infarction. Studies reported mixed results on the effect of age on myocardial infarction misdiagnosis. Age was significantly associated with decreased risk,^{25, 63, 201} increased risk,^{77, 204, 206, 263, 264} or no effect on myocardial infarction misdiagnosis.^{120, 202, 205, 209, 265, 266} Three studies reported higher risk of misdiagnosis of myocardial infarction among female patients,^{120, 264, 267} One study found even among patients who presented with cardiac chest pain and cardiac troponin > 99^th percentile, women were less likely to be diagnosed with MI, to undergo cardiac catheterization, or to be using evidence-based medications within 90 days of discharge.²⁶⁷ The rest of the studies reported no effect by sex on misdiagnosis of myocardial infarction.^{25, 63, 202, 204, 205, 208, 209, 265, 268–270} There were mixed results on the effect of race on misdiagnosis of myocardial infarction. Some studies reported an increased risk among African American patients,^{63, 77, 202} while others reported no significant effect of race on myocardial infarction misdiagnosis.^{120, 206, 208, 209, 266, 271} Several studies found no effect by ethnicity,^{63, 77, 120, 202} or socioeconomic status.^{25, 63, 77, 202} Due to concern for delayed STEMI treatment among women and older patients, one study aimed to assess the performance of a physician-blinded prehospital activation system for STEMI in comparison with standard systems with physician involvement. In the standard system, female sex and age > 75 were independent predictors of treatment delay in hospitals with and without a prehospital notification system. By contrast, with implementation of a physician-blinded prehospital notification system, there was no difference in treatment delay by age and there was a smaller gap in treatment delay among women.²⁶⁴

Eight studies reported on the effect of age, sex, race, and drug abuse on accuracy diagnosing aortic aneurysm and dissection. Studies showed conflicting results about the effect of age on diagnostic delay or missed diagnosis of aortic aneurysm and/or dissection. Some studies reported significant decreased or increased risk of diagnostic delay among older age patients,^{73, 120, 214} while others reported no significant effect of age on delayed or missed diagnoses.^{68, 216–218} Two studies reported increased risk of diagnostic delay among female patients,^{120, 219} others reported no significant difference among male or female patients on missed or delayed diagnosis of aortic aneurysm and/or dissection.^{68, 73, 216–218} Other studies reported no effect of race or drug abuse on delayed or missed diagnosis of aortic aneurysm and/or dissection.^{68, 120, 217} None of the studies on risk of delays in aortic dissection diagnosis found a statistically significant difference between those with a history of Marfan’s syndrome and those without,^{68, 73, 217} although the presence of a known history was, if anything, protective (median time from presentation to diagnosis 2.2 hours for those with a known history versus 4.5 hours for those without, P = 0.066⁶⁸).

We identified one study that reported on the effect of patient characteristics on diagnostic errors among 521 patients under 5 years of age presenting to the ED and later diagnosed with sepsis or meningitis.⁹³ Compared to 30-90 day-old children, older age children (age 91 days-2 years and >2 to 5 years) experienced higher odds of missed diagnosis of sepsis or meningitis in the ED (OR 2.56; 95% CI 1.49 to 4.41 and OR 2.26; 95% CI 1.17 to 4.35, respectively).⁹³

Two studies assessed clinical and patient characteristics associated with delayed diagnosis of appendicitis.^{236, 272} There was no effect of age (among children) or sex on delayed diagnosis of appendicitis in either study. Race, ethnicity and insurance status could not be studied in relation to diagnostic delay due to significant differences in these characteristics between the case and control group and was not assessed in the other study.^{236, 272} Michelson et al., 2021 found 63 percent of children had a delayed diagnosis of appendicitis, of which 76.8 percent were deemed possible or probable missed opportunities to improve diagnosis.²⁷² In comparison with children who received a timely diagnosis, patients with delayed diagnosis of appendicitis had longer hospital length of stay, higher rates of perforation, and a higher likelihood of undergoing two or more abdominal surgeries (OR 8.0; 95% CI, 2.0 to 70.4).²⁷² Lastunen et al. 2021 found that among patients with uncomplicated appendicitis on initial CT, age greater than 60 years was independently associated with progression to complicated appendicitis at the time of operation.²⁷³ Although we did not find any included studies that directly addressed older age (i.e., adult presentations) as a risk factor for misdiagnosis in appendicitis, cross-study differences suggested that it might be a risk factor. In one study, the false negative rate among patients 18 years of age and under was 2.9 percent.²²⁵ In an unrelated study, the false negative rate among patients 17 years of age and older was 30.8 percent.²³³ Although this latter study used a very different method and focused only on missed cases using point-of-care ultrasound,²³³ an older study (from prior to the study period), which directly compared younger and older patients with appendicitis, appears to corroborate older age as a risk factor. In that older study, diagnostic delays among older adults were more common than among children, with contributions from both “patient interval” (from symptoms to presentation) and “clinician interval” (from presentation to diagnosis) delay components—the result appears to be a higher rate of complications and greater mortality.²³⁹

Nine studies reported on the distribution of age and sex among patients with diagnostic errors related to fractures. However, the effect of age or sex on misdiagnosis of fractures was not quantified in any of the studies.

Three studies reported about patient characteristics and delayed diagnosis of testicular torsion.^{254, 255, 274} These studies focused on patient-related delays prior to seeking care (known as the “patient interval” in studies of cancer) as it related to delay in definitive therapies and clinical outcomes. Chan et al., 2019 found that patients with testicular torsion who underwent orchiectomy had significantly longer prehospital pain duration compared with those who underwent testicular salvage (18.75 versus 3.56 hours; P = 0.003).²⁵⁴ For patients who underwent orchiectomy, in-hospital time intervals were not significantly different than those who underwent testicular salvage. Bayne et al., 2017 found that patients in the delayed presentation group were 4 times more likely to have a developmental, cognitive, or social disorder than patients in the acute presentation group (10.6 versus 2.6%; P = 0.02). Half of the patients in the delayed group reported having autism spectrum disorder. Patients reporting a history of recent genital trauma were twice as likely to present in the delayed vs acute setting (14.9 versus 7%; P = 0.07). Misdiagnosed patients were younger and weighed less than those correctly diagnosed in the acute setting (9.9 years versus 12.9 years; P = 0.006; 42.6 kilograms versus 59.2 kilograms; P = 0.01). All boys who were misdiagnosed eventually underwent orchiectomy compared with 24.6 percent of those correctly diagnosed in the acute period (P < 0.0001).²⁵⁵ One study reported on potentially avoidable testis loss in the setting of delayed diagnosis and treatment of cryptorchidism.²⁷⁴ Despite international guidelines which recommend surgical exploration and orchidopexy prior to 18 months of age, the authors found 60% of the patients were above this age when they presented with preventable cases of testicular torsion. Further, there was significant delay (over 6 hours) from symptom onset to presentation to the ED in 72 percent of patients, which was associated with higher rates of orchidectomy (56% versus 23% in those who arrived within 6 hours; P = 0.04). There was not a significant effect of age on the duration of delay in ED presentation, and the effect of sex was not quantified.²⁷⁴

One study looked at rates of concordance and discordance between ED diagnosis and discharge diagnosis across all International Classification of Diseases (ICD)-10 codes and found no difference by age or sex.²⁷⁵ Another study focused on delays (including diagnostic delays) due to difficulty obtaining intravenous access (DIVA).²⁷⁶ Patients with DIVA (or 3.1% of the population) were more likely to be female, black and were more often triaged to a higher acuity track. Throughout the ED, DIVA was associated with delays in median time to completion of lab testing, intravenous fluid and contrast administration, pain medication administration and delayed admission and discharge orders.

Illness Characteristics

We identified 120 studies that reported on the effect of one or more illness-related factors on diagnostic error in the ED. Most of the studies had a low risk of bias and/or low concerns for applicability. However, 12 studies had concerns with patient selection^{58, 160, 177, 179, 194, 200, 244, 266, 271, 272, 277, 278} and 19 studies had concerns with the reference standard.^{176–179, 186, 188, 195, 208, 213, 244, 267, 272, 275, 279–284} We report the impact of illness characteristics separately by condition. The illness characteristics most studied were symptom type and illness severity, followed by mode of arrival and diagnostic tests ordered. Atypical presentations were the strongest and most consistent predictors of increased risk for a missed diagnosis. Comorbidities were often studied as outcome predictors (e.g., mortality), but generally not in relation to diagnostic error, apart from polytrauma, where comorbidities tended to increase the risk of missing a second disease.

Thirty-one studies reported on stroke, including five prospective cohorts, 18 retrospective cohorts, eight registries, two case-control studies, and one cross-sectional study that reported on the illness-related causes of diagnostic error among patients presenting to the ED. We meta-analyzed two studies, including 522 patients, indicating an increased risk of misdiagnosis in posterior circulation stroke (RR 2.51; 95% C, 1.46 to 4.33; I-squared 59%; Figure 18).^{159, 171} Three other studies also confirmed the increased risk of misdiagnosis in posterior circulation stroke but were not included in the meta-analysis because of study design, and lack of sufficient information.^{87, 186, 194} Atypical presentation^{66, 87, 88} and non-specific symptoms,¹⁸⁴ dizziness,^{69, 159, 194} altered mental status,^{88, 194} and loss of consciousness,^{88, 194} syncope,¹⁹⁴ headache,⁶⁹ involuntary movement,⁶⁹ and having a negative Face-Arm-Speech-Time test were associated with increased risk of misdiagnosis.^{161, 194} Compared to the correctly diagnosed stroke patients, misdiagnosed cases had a tendency to present without focal neurological deficits^{69, 88, 183, 186, 194} and with lower clinical severity as judged by the following: (a) ED triage resuscitation/emergency category,^{194, 195} (b) the National Institutes for Health Stroke Scale (NIHSS) score for ischemic stroke/TIA,^{164, 179, 183, 192, 197, 262, 285} (c) the ABCD2 score for TIA,^{69, 184, 185} and (d) the Hunt and Hess and Fisher scale scores for subarachnoid hemorrhage.¹⁸¹ By contrast, the presence of unilateral weakness or numbness were protective against misdiagnosis.⁸⁸ A retrospective review of an acute stroke registry found a bimodal NIHSS distribution among missed stroke cases, indicating that very severe cases may also be at risk of misdiagnosis (e.g., due to presentation in stupor/coma from basilar artery occlusion).¹⁸⁶ Mode of arrival by emergency medical services/ambulance also decreased delayed/missed diagnosis of stroke.^{160, 185, 188, 261, 262} MRI was performed equally often among misdiagnosed and correctly diagnosed cases,^{184, 194} but with longer delays among the misdiagnosed.¹⁹⁴ Unsurprisingly, stroke-specific sequences such as diffusion-weighted MRI and neurovascular imaging were used less frequently among misdiagnosed cases.¹⁸⁴ Whether or not ED patients with neurologic complaints had symptoms highly suggestive of TIA/stroke (e.g., aphasia or weakness), the use of head CT portended increased risk of subsequent stroke.¹⁸⁹ Similarly, having a head CT scan at the index visit for headache was associated with an increased risk of subsequent cerebrovascular event^{91, 176}; however, one study showed that a reduction in the use of head CT scan in ED visits for headache had no effect on the rate of death or subsequent cerebrovascular events.¹⁴⁰ Two studies assessed delayed diagnosis of ischemic stroke/TIA among pediatric patients (n=181).^{177, 286} They found no effect for the type of first contact with the medical sector, pediatric NIHSS, level of consciousness, symptoms, or the location of the brain lesion. One study looked systematically for ischemic stroke among polytrauma cases brought to a trauma center and found 11 acute ischemic strokes among 192 patients (5.7%)—none were detected initially and none had neurologic consultation obtained at initial trauma triage (100% missed); the median time to diagnosis was 2 days (range 0-5).¹⁹⁸ These studies all point to greater miss rates in case presentations with a higher degree of diagnostic difficulty (transient, milder, non-specific, or atypical symptoms²¹).

Figure 18

Risk ratio of misdiagnosis among people who have posterior circulation stroke. CI = confidence interval; RR = risk ratio

We identified 11 studies on myocardial infarction, including two registries, two prospective cohorts and seven retrospective cohorts that reported on the illness-related causes of diagnostic error among patients presenting to the ED. Meta-analysis was not possible because of differences in the definitions of the risk factors and diagnostic error. The rate of misdiagnosis was lower among patients with chest pain^{65, 209, 266} and more severe triage levels.^{25, 287} However, one study found no difference in the median door-to-balloon time between patients with and without angina pectoris²⁶⁵ and another study found no difference in the percentage of triage delay between triage levels.²⁶⁶

We identified 12 studies on aortic aneurysm/aortic dissection, including two registries, one randomized controlled trial, and nine retrospective cohorts that reported on the illness-related causes of diagnostic error among patients presenting to the ED. Meta-analysis was not possible because of missing data and differences in the definitions of the potential risk factors among these studies. Misdiagnosed cases were more likely to present with atypical symptoms,^{68, 73} dyspnea,^{73, 217} systolic blood pressure of above 105 mmHg,^{68, 73, 216} or clinical features resembling myocardial infarction, including angina pectoris,²¹⁷ positive troponin,^{73, 288} and acute coronary syndrome-like findings on ECG.⁷³ Cases who underwent CT scan had less diagnostic delay than those who did not.^{68, 73}

We identified five studies with a low or unclear risk of bias on pulmonary embolism, including one prospective cohort and four retrospective cohorts that reported on the illness-related causes of diagnostic error among patients presenting to the ED with pulmonary embolism. We included three studies (655 patients), reporting on hemoptysis, cough, and pleuritic chest pain,^{278, 289, 290} and four studies (881 patients) reporting on dyspnea^{222, 278, 289, 290} in a meta-analysis (Figure 19). The risk of misdiagnosis increased with the presence of hemoptysis (RR 2.08; 95% CI 1.06 to 4.07; I-squared 0%) and cough (RR 1.75; 95% CI 0.98 to 3.13; I-squared 66.4%), decreased slightly with pleuritic chest pain (RR 0.86; 95% CI 0.54 to 1.37; I-squared 45.6%), and was not related to dyspnea at the index visit (RR 1.00; 95% CI 0.82 to 1.22; I-squared 64.2%). One study reporting an increased risk of delayed diagnosis in the absence of dyspnea was not included in the meta-analysis because of the unclear number of misdiagnosed patients.²²³ In addition, two studies reported on syncope in the clinical presentation of PE.^{278, 289} Kline et al., reported syncope at a higher rate among those diagnosed within 48 hours after leaving the ED (delayed) than patients diagnosed while in the ED at the initial presentation.²⁷⁸ However, Torres et al., found syncope at a similar rate among ED diagnosed patients and those sent home with a wrong diagnosis, but less frequently among patients diagnosed with PE during hospitalization.²⁸⁹ Compared to the correctly diagnosed patients with pulmonary embolism, misdiagnosed cases were less likely to have D-dimer tested in the initial work-up^{222, 289} and more likely to present with pulmonary infiltrates on chest X-ray.^{289, 290}

Figure 19

Risk ratio of pulmonary embolism misdiagnosis in patients presenting with cough, dyspnea, hemoptysis, or pleuritic chest pain. CI = confidence interval; RR = risk ratio

We identified five or fewer studies on other conditions in the ED, where meta-analysis was not possible because of different definitions or statistical measures, or the limited number of reports on each risk factor.

Across these studies, higher triage severity increased misdiagnosed injuries in pediatric or adult trauma patients.^{74, 127, 130, 243, 291} However, clinical and triage severity decreased sepsis misdiagnosis among children⁹³ and adults.¹⁵⁷

Typical symptoms such as isolated scrotal pain for testicular torsion²⁵⁵ and right lower quadrant abdominal tenderness for appendicitis^{272, 292} decreased misdiagnosis. However, atypical presentation, as in isolated abdominal pain for testicular torsion²⁵⁵ and lack of abdominal pain or abdominal pain accompanied by constipation for appendicitis²⁹² increased misdiagnosis. Imaging on first presentation by a single sonography or CT scan decreased diagnostic delay of testicular torsion.^{293, 294} The studies on preoperative imaging for appendicitis were inconclusive. In two reports, CT scan or sonography was performed less frequently among adults with missed appendicitis at index visit than those with a same-day diagnosis.^{272, 292} One study showed no change in the rate of negative appendectomy (i.e., false positive diagnosis of appendicitis) but a delay to surgery with preoperative imaging,²⁹⁵ while another study indicated that preoperative imaging reduced the false discovery rate from 10 percent to 3 percent.²⁹⁶

Clinician Characteristics

We identified 30 studies, including one randomized controlled trial, three prospective cohorts and 17 retrospective cohorts, two case-control studies, five cross-sectional studies, one registry, and one case series that reported on clinician characteristics associated with diagnostic error among patients presenting to the ED. Most of the studies had a low risk of bias and/or low concerns for applicability. However, five studies had concerns with patient selection^{132, 194, 200, 266, 297} and five studies had concerns with the reference standard.^{186, 202, 281, 298, 299} The sources of heterogeneity between studies were differences in the definitions of the clinician factors, study design, and patient selection. Provider type and clinical experience (including training level) were the most frequently reported factors. Most studies were limited by a retrospective design that made it difficult to evaluate some potential risk factors such as clinician fatigue.

Numerous studies addressed accuracy of diagnosing patients based on provider type. For strokes, Arch found that neurological consultation was strongly associated with fewer diagnostic errors (35% [n=20/55] of missed cases were seen by a neurologist, while 95% [n=213/225] of correctly diagnosed cases were seen by a neurologist, P < 0.001).¹⁵⁹ These numbers correspond to a false negative rate of 9 percent (n=20/233) for neurologists which is roughly half of the estimated overall error rate in the ED shown in KQ2 (which includes multiple studies that gave “credit” to correct ED diagnoses employing neurological consultation); however, authors did not report results on a per-symptom or case-mix adjusted basis, so these results could potentially be confounded by indication (i.e., neurologists might have been consulted disproportionately in obvious cases and they might not have fared so well in subtle cases). Richoz found that “ED physicians were a little less than twice as likely as neurologists or neurologists in training to miss the right diagnosis.”¹⁸⁶ Importantly, this was in cases where the ED initially missed the diagnosis (among 43 initially missed strokes by ED physicians, 33 underwent neurological consultation without suspicion for stroke by the ED, and 14 of these were correctly diagnosed as stroke by the neurologist); however, the neurologist also caused the misdiagnosis in four cases suspected to represent strokes by the ED clinician.¹⁸⁶ In two multivariable models, Morgenstern found point estimates that neurology consultation (obtained in just 8.6% of stroke cases) cut diagnostic error by 34 to 51 percent, but results were imprecise and confidence intervals overlapped with no difference.¹⁷⁹ Venkat found that neurology service admission (versus non-neurology service) was associated with lower rates of stroke misdiagnosis (non-neurology admissions were 11% of correctly diagnosed vs. 35% of misdiagnosed cases, p < 0.001).¹⁹⁴ Liberman found that, in cerebral venous sinus thrombosis, fewer misdiagnosed patients had neurology consultations, but the result was not statistically significant (81.8% among misdiagnosed vs. 95.2% among correctly diagnosed, P = 0.19).¹⁷³ Yi found that access to telestroke video consultations did not reduce false positive transfers for mechanical thrombectomy for large vessel occlusions causing stroke.²⁸⁵ In summary, studies that assessed neurologist accuracy found that neurologists generally missed fewer strokes than ED clinicians, but neurologists also missed strokes (even sometimes when ED clinicians correctly suspected them). There was also clear evidence of opportunities for improvement by ED clinicians in stroke diagnosis. Vaghani found a large number of patients who presented with red flags and multiple stroke risk factors did not undergo appropriate ED diagnostic evaluation, and processes failures related to the patient–provider encounter (history and physical examination) were the most frequent cause of diagnostic errors.³⁰⁰ However, one study of ED patients with suspected acute stroke found that formal use of bedside diagnostic stroke scales improved ED clinician sensitivity for detecting stroke over ED clinical impression alone (76% clinical impression vs. 83% using Recognition of Stroke In the Emergency Room (ROSIER) [P = 0.005]; use of the Face Arm Speech Test [FAST] scale by ED clinicians was not statistically different than use of ROSIER by ED clinicians 81% FAST vs. 83% ROSIER [P = 0.39]).¹⁹⁶ Results were said to be similar whether the scales were performed by a physician or nurse. This suggests that relatively simple interventions might be helpful.

For potential surgical conditions, surgeons were less likely to miss ruptured aortic aneurysm than internists.²¹⁹ However, compared to emergency physicians, surgeons were more likely to misdiagnose common surgical complaints in the pediatric ED including head trauma, testicular pain, and abdominal pain.¹²⁶ Data suggest that early recognition of diseases such as testicular torsion needing emergent treatment are sometimes delayed and/or missed; in these cases absence of early surgical consultation was deemed to be the main cause.^{274, 301}

For radiographic diagnoses, specialists in radiology generally provided more accurate diagnoses, and subspecialists were the most accurate when interpreting images in their own subspecialty. ED clinicians (non-radiologists) had significantly higher error rates compared to radiologists and radiology residents when interpreting ED imaging.²⁹⁹ In acute stroke patients, neuroradiologists missed fewer large vessel occlusions on CT angiography than non-neuroradiologists.¹⁶² However, subspecialty radiologists who interpreted ED imaging outside their area of expertise had diagnostic error rates similar to radiology residents.²⁹⁹ Radiologists were less likely to miss acute mesenteric ischemia on CT imaging of patients with acute abdominal pain if clinicians had suspected the diagnosis prior to CT referral.²²⁷ In patients with fractures who had imaging read only by an orthopedic surgeon (without attending radiology backup during the visit), the incidence of false-negative fractures was 2.2 percent,⁸³ which is slightly higher than that generally reported for radiologists (see KQ2 fractures).

Multiple studies addressed accuracy of diagnosing patients at the bedside based on clinical experience, including training level. Less clinical experience of ED clinicians showed a trend towards increased stroke misdiagnosis (≤6 years of experience OR 1.20; 95% CI 0.80 to 1.75)⁶⁹ but was not identified as a predictor of missed myocardial infarction.²⁰² A stroke study “found no significant difference in diagnostic accuracy between neurologists and trained neurology residents.”¹⁸³ Radiology residents were more prone to diagnostic error than attendings in the diagnosis of stroke,¹⁶² subtle pelvic fractures¹³⁰ and interpreting CT scan,¹³² CT angiography,⁵⁵ and MRI.¹³³ Nevertheless, one large study comparing “off hours” initial radiology resident reads to those of attending radiologists (n=81,201) found diagnostic errors in just 0.2 percent.²⁴¹ Earlier year of residency in radiology was associated with greater risk of MRI misinterpretation.¹³³ Although radiology residents had overall suboptimal sensitivity (87%) for detecting intracranial aneurysms on head CT angiography in subarachnoid hemorrhage patients, there was no clear benefit to overall diagnostic accuracy based on year of residency training.⁵⁵

We identified one study assessing training background which found that hospitals with a greater proportion of emergency medicine board certification among their ED clinicians was associated with fewer missed diagnoses of myocardial infarction (median 0.3% [interquartile range 0 to 1.15] for hospitals in the top quartile versus median 2.0% [interquartile range 0 to 33.33] for hospitals in the bottom quartile of emergency medicine board certification).²⁰²

We did not find studies that addressed a clinician’s history of disciplinary action as a predictor. We also did not find studies that addressed clinician fatigue as a predictor.

Fixed Systems Factors

Fixed systems factors were those that would generally not change on a given day or at a given visit for a specific patient. We identified 21 studies that reported on fixed facility or health systems factors that were associated with diagnostic errors/harms.^{25, 63, 64, 78, 93, 156, 160, 178, 188, 189, 195, 202, 208, 244, 253, 281, 302–305} We identified six studies that reported on both fixed and dynamic factors.^{25, 63, 64, 93, 188, 195} Nine studies took place in the United States,^{63, 64, 78, 156, 160, 178, 188, 189, 202} seven studies took place in Canada,^{25, 93, 195, 211, 244, 303, 304} three studies took place in the United Kingdom or Western Europe,^{253, 281, 302} and two studies were based in Australia.^{208, 305}

Twelve studies reported on the association between a facility’s teaching status and rates of misdiagnosis.^{25, 63, 64, 78, 93, 178, 188, 189, 193, 195, 202, 302} Schull et al, 2006, Cifra et al., 2020, and Rosenman et al., 2020 found no association between teaching status and myocardial infarction, sepsis, and stroke misdiagnosis respectively.^{25, 78, 189} All other studies found significantly lower odds of misdiagnosis at academic centers.

Five studies reported on variation by U.S. geographic region in rates of misdiagnosis.^{63, 64, 78, 202, 303} Moy reported significantly higher odds of myocardial infarction misdiagnosis in the Midwest relative to the Northeast.⁶³ Wilson reported significantly lower rates of myocardial infarction misdiagnosis in the Mid-Atlantic, West, South, Central, and Mountain regions of the country.²⁰² Newman-Toker reported non-significantly lower rates of stroke misdiagnosis in the Northeast relative to the Midwest, South, and West.⁶⁴ Cifra reported significantly higher odds of sepsis misdiagnosis in California, Florida, and Massachusetts relative to New York.⁷⁸ Cheong reported on geographic variation in Canada, and found the West had significantly higher odds of appendicitis misdiagnosis that relative to the Maritime region.³⁰³

Five studies reported on facility ownership/business models.^{63, 64, 78, 160, 202} Moy, Newman-Toker, and Cifra found no association between facility ownership and rates of myocardial infarction and stroke misdiagnoses respectively.^{63, 64, 78} Wilson found that public hospitals had higher odds of myocardial infarction misdiagnosis relative to private hospitals.²⁰² Bhattacharya found non-significantly higher rates of correctly diagnosed strokes among young adults presenting to primary stroke centers (PSC) relative to non-PSCs.¹⁶⁰

Four studies reported on the association between population density and rates of misdiagnosis.^{63, 64, 202, 208} Moy and Wilson found a significant association between lower population density and higher rates of myocardial infarction misdiagnosis.^{63, 202} Williams also reported an increased rate of myocardial infarction misdiagnosis in rural regions of Australia, but did not report on significance.²⁰⁸ Newman-Toker found that small metropolitan regions had lower odds of stroke misdiagnosis relative to large metropolitan areas; the effect size was small but statistically significant.⁶⁴

Four studies reported on the association between average ED volume/annual number of visits and diagnostic accuracy.^{64, 78, 93, 211} Ko, in a study following approximately 500,000 Canadian adults with treat-and-release ED visits for chest pain, found that EDs with higher annual volumes of chest pain complaints had significantly lower rates of myocardial infarction/unstable angina hospitalizations and death in the 30 days following those treat-and-release ED encounters.²¹¹ This trend continued until the ED volume reached 1400 annual visits—once volumes exceeded 1400 annual chest pain visits, there was no longer a significant reduction in the rates of acute coronary syndrome hospitalizations or death relative to the lower-volume EDs. Newman-Toker found that lower-volume EDs had significantly higher odds of stroke misdiagnosis, and that moderate-volume EDs had non-significantly higher odds of stroke misdiagnosis relative to high-volume EDs.⁶⁴ Cifra found that rates of pediatric sepsis misdiagnosis were significantly higher in lower-volume EDs.⁷⁸ Vaillancourt did not find a significant association between ED volume and the rate of pediatric sepsis misdiagnosis.⁹³

Two studies reported on access to electronic health records.^{78, 304} Cifra found that hospitals’ accuracy decreased non-significantly when diagnosing pediatric sepsis if the hospital had fully implemented electronic health records.⁷⁸ Gouin found that emergency physicians’ diagnostic accuracy increased non-significantly with use of digital versus conventional radiography viewing using a Picture Archiving and Communications System (PACS).³⁰⁴

Two studies reported on access to testing.^{63, 202} Both studies found that access to cardiac catheterization facilities reduced risk of myocardial infarction misdiagnosis, but the findings in the Wilson study were not statistically significant.²⁰² The Wilson study also found a significant benefit to diagnostic accuracy from being at what was classified as a “high-tech” hospital.²⁰²

Two studies reported on average ED discharge fraction.^{63, 64} Newman-Toker found that higher discharge fractions were associated with increased risk of stroke misdiagnosis.⁶⁴ Likewise, Moy found that higher discharge fractions were associated with significantly higher odds of myocardial infarction misdiagnosis.⁶³ Both studies were compatible with findings from a large Medicare-based study (outside the systematic review) which found that unexpected deaths (associated with apparent diagnostic errors) within 7 days of an ED treat-and-release visit were increased at EDs with higher discharge fractions. In that study, hospitals in the lowest quintile of admission fraction from the ED had the highest rates of early death—3.4 times higher (0.27% versus 0.08%) than hospitals in the highest quintile of admission fraction—despite serving healthier populations, as measured by overall 7-day mortality among all comers to the ED.¹⁴⁸

Two studies reported on average inpatient occupancy rates influencing misdiagnosis rates.^{63, 64} Newman-Toker found that occupancy rates did not affect rates of stroke misdiagnosis.⁶⁴ Moy found significantly lower rates of myocardial infarction misdiagnosis among hospitals with higher (classified as “medium” or “high” relative to “low”) occupancy rates.⁶³ The implications of this finding are uncertain.

One study reported on access to consultants.⁹³ Vaillancourt found that hospitals with access to pediatric consultations improved accuracy among children with meningitis or sepsis.⁹³

We did not identify any studies that evaluated the association between delivery or payment models and rates of misdiagnosis.

Dynamic Systems Factors

Dynamic, context-specific systems factors were those that might change on a given day or at a given visit for a specific patient. We identified 17 studies that reported on dynamic, context-specific systems factors.^{25, 63, 64, 88, 92, 93, 122, 127, 162, 183, 188, 195, 208, 253, 265, 285, 286} Seven studies took place in the United States,^{63, 64, 88, 122, 188, 265, 285} five studies took place in the United Kingdom or Western Europe,^{92, 127, 162, 183, 253} three studies took place in Canada,^{25, 93, 195} and two studies took place in Australia.^{208, 286}

Sixteen studies reported on the rates of misdiagnosis during off-hours.^{25, 63, 64, 88, 92, 93, 127, 162, 183, 188, 195, 208, 253, 265, 285, 286} Newman-Toker reported significantly increased odds of stroke misdiagnosis during off-hours.⁶⁴ Muhm also reported increased cases of misdiagnosis in polytrauma cases during off-hours though did not report on significance.¹²⁷ Rose reported suspected stroke patients had more rapid access to immediate CT scans during off hours.¹⁸⁸ Parikh, Schull, Daverio, and York reported mixed results.^{25, 253, 265, 286} Moy, Vermeulen, Fasen, Williams, Pihlasviita, Madsen, Yi, Mirete, and Vaillancourt reported no effect.^{88, 92, 93, 162, 183, 195, 208, 285}

We identified two studies that reported on the relationship between same-day ED crowding and rates of misdiagnosis and found mixed results.^{63, 64} Unexpectedly, Moy⁶³ found that high levels of same-day ED crowding were associated with lower risk of misdiagnosis (OR 0.78, P = 0.009), but this appears to have been a univariable analysis that might have been confounded by other factors (e.g., high same-day ED admission fraction, which was not measured, but in a similar analysis for stroke⁶⁴ was strongly protective against error). Although Newman-Toker⁶⁴ did not find an association between same-day ED crowding and odds of stroke misdiagnosis, there was an increased risk of stroke misdiagnosis among incomplete ED visits (e.g., patient left against medical advice), suggesting that incomplete diagnostic assessments may be more important than crowding, per se (though it is expected that overcrowding is likely to increase incomplete ED visits). We identified one study that reported on the relationship between same-day ED discharge fraction and rates of misdiagnosis which found a strong association, with higher discharge fraction on the day of the visit (top quintile versus bottom quintile) increasing the odds of a misdiagnosis 6.3-fold (P < 0.001).⁶⁴ We found no studies that assessed the association between handoffs, same-day ED staffing, or same-day ED illness severity and misdiagnosis rates, but the Okafor 2016 incident report study did note inadequate or failed handoffs (5% of all causes) and high workload (11% of all causes) as contributing factors.³¹