Background

Acute myocardial infarction (AMI) is the leading cause of death in the United States. Research into the causes, progression, and treatment of AMI continues to be a national research priority; this research continues to produce substantial progress in the areas of prevention, diagnosis, and treatment of AMI, as well as advances in understanding its molecular and cellular aspects. In clinical medicine, much research has been focused on the early diagnosis and treatment of ACI, which includes both unstable angina pectoris (UAP), which can lead to AMI, as well as AMI itself. Research has shown that early diagnosis and treatment of UAP is beneficial and may prevent AMI.

Thus, as mandated by Congress, in 1991 the National Heart, Lung, and Blood Institute (NHLBI) of the National Institutes of Health (NIH) initiated NHAAP to look for ways to reduce the morbidity and mortality from AMI in this country. Specifically, the Program focused on issues related to the rapid recognition and response to patients with symptoms and signs of ACI in emergency settings, the point at which most of these patients enter the health care system. This ongoing effort brings together scientists, clinicians, and NHLBI staff with a Coordinating Committee that includes representatives of 40 professional organizations.

With a plan to increase public awareness of the need for rapid evaluation and treatment of symptoms that might represent ACI, the NHAAP also studied the needs of emergency department (ED) physicians to be able to handle the potentially larger number of patients with chest pain and related symptoms and, in particular, their need to accurately diagnose ACI from among the large number of such ED patients. Accordingly, in 1994 the NHAAP Working Group on Evaluation of Technologies for Identifying Acute Cardiac Ischemia in the Emergency Department was formed to assess the technologies for diagnosing ACI and AMI in the ED. Members of the Working Group had expertise in the areas of cardiology, emergency medicine, general internal medicine, family practice, and nursing, as well as in the specific disciplines of meta-analysis and health services research. The Working Group reviewed all technologies for diagnosing ACI in the ED. The assessments of these technologies in actual use in EDs, and the nature, extent, and quality of the evidence on which the assessments were based, are presented in the Working Group's 1997 final report, An Evaluation of Technologies for Identifying Acute Cardiac Ischemia in the Emergency Department (Selker, Zalenski, Antman, et al., 1997).

Working Group Process and Methods

The Working Group conducted a formal review and evaluation of the literature on the technologies for diagnosing ACI in the ED. MEDLINE and related electronic database searches were supplemented with the members' knowledge of the literature and ongoing research. All relevant studies published in English on each technology were reviewed, summarized, analyzed, and reported independently by three members in a process analogous to an NIH Study Section. The quality of evidence for each technology was rated as A (high), B (moderate), C (limited), or NK (not known). Similarly, diagnostic accuracy and clinical impact were each rated as +++, ++, +, NK (not known), or not effective (NE).

These ratings were presented by reviewer members of the Working Group in both oral and written format at an initial meeting, revised and updated by the reviewers, and then compiled into a single document by the Working Group co-chairs. This document was reviewed in a second Working Group meeting, at which consensus was reached on the detailed conclusions and recommendations for each technology. The text, recommendations, and conclusions were then updated, reviewed, and approved once more before the final report was completed. The final report also underwent external review by a broad range of experts who were not members of the Working Group.

Summary of Technologies Assessed and Key Findings

Recommendations on the use of a technology were based on the strength of the evidence of its diagnostic accuracy in the ED and on its clinical impact when implemented in decisionmaking. The conclusions of the Working Group are presented below in the same order as reported in the executive summary of the report.

Although the standard electocardiograph (ECG) is a safe, readily available, and inexpensive technology with a relatively high sensitivity for AMI, it is not highly sensitive or specific for ACI. However, the ECG remains an integral part of the evaluation of patients with chest pain, and the Working Group recommended that it remain the standard of care for evaluating patients with chest pain in the ED.

The original ACI predictive instrument, developed and tested in the early 1980s, was found to have excellent diagnostic performance and substantial clinical impact in a high-quality prospective, multicenter trial for both AMI and UAP. The requirement for a programmed calculator was a drawback, but this drawback was expected to be remedied in the newer ACI Time-Insensitive Predictive Instrument (ACI-TIPI). The instrument was recommended for general use.

The ACI-TIPI, a newer version of the original ACI instrument, performed as well as the original instrument in both diagnostic performance and clinical impact. The algorithm has been incorporated into standard ECG machines and its predictions are printed on the ECG itself, making it much more useful than its predecessor. More definitive recommendations were withheld, pending the publication of a large multicenter trial of this new version's effectiveness.

Theprehospital 12-lead ECG was found to havegood diagnostic performance but limited clinical impact. Although it has promise, the Working Group believed that its best use would be in areas with long emergency medical service (EMS) transport times and perhaps in conjunction with prehospital thrombolytic therapy. Its routine use was not recommended.

The Goldman chest pain protocol was found to have excellent diagnostic accuracy for AMI, the purpose for which it was designed, but it does not test for UAP. Its greatest potential benefit was believed to be that of improving the specificity for AMI, which could reduce unnecessary cardiac care unit (CCU) admissions. However, the only trial to study this benefit showed that it had no impact on care or on resource use. Thus, its routine use was not recommended.

The use of a single creatine kinase (CK) test (or its subunit, CK-MB) was not recommended for use in ED triage, but multiple tests over several hours accurately diagnosed AMI. The test does not detect UAP, and its clinical impact has not been studied.

The overall performance of sestamibi imaging studies was encouraging, but the data available for evaluation at the time of the report were insufficient for making recommendations.

The ECG exercise stress test, an extension of the standard ECG, has only modest diagnostic accuracy for coronary disease, and few studies have examined its clinical impact in the ED. Its routine use was not recommended.

Echocardiography performed only modestly well in diagnosing ACI, and its clinical impact in the ED had not been studied. For these reasons, it was not recommended for routine ED use.

Other computer-based decision aids represent a variety of methods to predict AMI. Information about their generalizability and transportability is limited. Their clinical impact was also not known, and they were not recommended for routine use.

The data for troponin T and troponin I biomarkers were encouraging, but these biomarkers had not been adequately studied.

The importance of myoglobin as a marker of AMI was not yet clear, and its use was not recommended.

The use of nonstandard ECG leads for detecting ACI had undergone only limited testing, and their clinical impact had not been studied.

Thallium scanning, body surface mapping, and continuous 12-lead ECG had not been evaluated for ED use and so could not be recommended.

Limitations of the 1997 Report

The 1997 report had three important limitations. First, it contained no details about the individual studies reviewed. Second, it contained no quantitative estimates of diagnostic performance or clinical impact. Finally, although literature based, the report's recommendations were based on consensus, without the aid of a quantitative framework, such as decision or cost-effectiveness analyses, to help integrate the findings. All three of these limitations have been addressed in the updated report. In addition, the 1997 report evaluated studies published as of September 1994. Many studies have been published on this topic since, and they have been reviewed in the updated report.

Issues in the Definition of Acute Cardiac Ischemia

Acute cardiac ischemia is a continuum of clinical states that range from reduced myocardial perfusion at rest to infarction of myocardial tissues. The charge to study the diagnosis of ACI reflects the fact that identifying only patients with AMI misses a large number of ED patients with UAP, who are also at substantial and immediate risk of cardiac events.

About 15 percent of patients with unrecognized UAP admitted to the ED experience AMI within 2 months of admission (Roberts, Califf, Harrell, et al., 1983; Gottlieb, 1987; Gottlieb, Weisfeldt, Ouyang, et al., 1987; Mulcahy, Conroy, Katz, et al., 1990). The overall mortality of untreated AMI is between 12 and 15 percent. Early recognition and treatment of AMI can reduce the amount of myocardial tissue damage, improve cardiac function and survival rates, and reduce long-term complications such as congestive heart failure; and the early recognition and treatment of UAP may prevent the morbidity and mortality associated with untreated cardiac ischemia. Indeed, the mortality rate is nearly doubled among patients with AMI or UAP inadvertently sent home (Pope, Aufderheide, Ruthazer, et al., 2000).

In addition, patients with unrecognized ACI who are inappropriately discharged from the ED often require additional and costly ED visits and diagnostic evaluations. Such discharges may also result in negative consequences for physicians (such as malpractice suits or disciplinary action). Thus, the timely recognition of both AMI and UAP can reduce the morbidity and mortality associated with ACI as well as reduce the health care costs and negative consequences related to missed diagnoses.

Issues in the Populations Studied

Clinical studies of diagnostic tests for ACI vary widely in the patient populations they study. For example, some studies included all patients with chest pain and others included only patients with chest pain in whom clear evidence of AMI was lacking. Still others included all patients presenting to the ED with various symptoms suggestive of ACI. Different inclusion criteria may result in heterogeneous study populations in whom prevalence rates of AMI and UAP differ, which could lead to different study results.

Another difficult issue is the distinction between ED and CCU settings. Studies that evaluated only patients admitted to the CCU will have selected a population at much higher risk for true ACI by having already excluded low-risk patients from the evaluation process, even though all such patients were initially seen in the ED. Differences in study populations are also apparent in the mean times between symptom onset and ED presentation.

Issues in the Use of Diagnostic Technologies

Several technologies have been developed to detect some or all of the possible manifestations and types of ACI. However, some tests currently used in the ED, such as serum CK levels, are intended to diagnose only AMI and not UAP (Table 1). Other technologies, such as echocardiography, may detect either AMI or UAP, whereas still others, such as the Goldman chest pain protocol, may not be sensitive to the diversity of manifestations associated with ACI. In addition, the patient populations for which, and circumstances under which, ED physicians order individual tests vary widely.

Table

Table 1. Technologies for identifying acute cardiac ischemia in the emergency department and their diagnostic use.

It is important to have diagnostic technologies with high sensitivity and specificity for ACI to minimize the chances of missing patients with ACI and also to avoid unnecessary hospitalizations for those not having ACI. Good test performance in isolation, however, does not automatically translate to improved patient care when the technology is used. A test result provides only one piece of information in the complex decisionmaking process of clinicians in the ED. Therefore, in addition to high sensitivity and specificity, a diagnostic technology must also demonstrate desired clinical impact during routine use in the ED. The earlier Working Group report found very few studies that reported the clinical impact of the technologies. Most studies evaluated only test performance and did not assess impact on ED care. Studies that reported clinical impact also differed substantially in how they measured impact. Outcome measurements included 30-day survival rates, discharge rates, re-admission rates, procedure rates, ejection fractions, and so on. Other outcomes included time from symptom onset to treatment administration.

The sensitivity and specificity of a test depend not only on the intrinsic nature of the test, but also on how the users of the test define these characteristics (Figure 1). In most diagnostic tests, the test results of sick and healthy populations overlap to some extent. The less the overlap, the more accurate the test. The interpretation of these overlapping scores is governed by a threshold value placed in the overlapping range by the user. Scores on one side of the threshold are interpreted as positive; those on the other, as negative.

Figure

Figure 1. Effect of threshold on diagnostic test performance. The performance of a diagnostic test is determined by both the innate accuracy of the test, as indicated by the degree of overlap in the test results of patients with and without (more...)

There is a tradeoff, however, between sensitivity (the proportion of true positive results) and specificity (the proportion of true negative results). If the threshold is set to reduce the rate of false-positive results (overdiagnosis), the rate of false-negative results (underdiagnosis) will rise (see Figure 1). The tradeoff between sensitivity and specificity is illustrated in receiver operating characteristics (ROC) curves, but such curves are often not included in studies of diagnostic tests.

The timing of tests also differs greatly among studies. The technologies studied ranged from a single test at presentation to the ED to repeated tests administered up to 14 hours after the patient's initial presentation or that depended on symptom duration (not time from presentation). For example, a single serum CK or CK-MB measurement at presentation has been used diagnostically, but additional serial measurements taken over several hours have additional incremental diagnostic value. Thus, when studies of biomarkers are compared, the timing of the test has to be taken into account. If the data are not presented separately, tests used in combinations may also be difficult to combine with the same tests used alone.

The reference test or criteria to which the index test is compared can also differ among studies. Most studies for the diagnosis of AMI use the World Health Organization (WHO) criteria to define AMI (Gillum, Fortman, Prineas, et al., 1984). However, the definition of the "cardiac enzymes" differed among the studies. Some studies further complicate the matter by using a procedural outcome, such as angioplasty, from which the diagnosis must be inferred. The diagnostic criteria for UAP are often less strict. Although some studies used Braunwald's classification for UAP, other studies defined non-AMI ACI as "evidence of atherosclerotic heart disease," or "non-AMI ischemia." Without a standard definition, we accepted the author's diagnosis of UAP, without necessarily knowing which signs and symptoms were used to make the diagnosis.

As described above, given the clinical and economic consequences, tests for diagnosing ACI need to have both high sensitivity and high specificity. A highly sensitive test is required if the adverse outcomes of missing (and therefore not treating) a diagnosis of ACI are to be avoided. In the worst case, patients with undiagnosed ACI can be discharged home without treatment and then die from a cardiac event. Thus, an underdiagnosis rate of, say, only 2 percent may not be acceptable. At the same time, a highly specific test is required to avoid unnecessary admission of patients to cardiac care units, a practice that increases costs and resource use. About half of the people admitted to coronary care units with the symptoms of ACI do not have coronary artery disease. The economic implication of this overdiagnosis and unnecessary hospitalization is in the billions of dollars each year (Fineberg, Scadden, Goldman, et al., 1984; Pozen, D'Agostino, Selker, et al., 1984).

Inadequate followup of patients discharged from the ED could lead to verification bias. Verification bias may occur when the proportion of patients with confirmed negative test results is small. Thus, studies that do not adequately follow up all the patients discharged from the ED may miss a large number of diagnoses, which can falsely increase test sensitivity and decrease test specificity.

Issues in Implementing the Technologies

A major issue in determining clinical impact and in implementing these technologies is the perspective from which they are studied. Emergency department physicians may be more concerned with avoiding the medical and legal consequences of underdiagnosing (and therefore not treating) ACI, whereas third-party payers may be more interested in reducing overdiagnosis to avoid unnecessary CCU admissions throughout their health care network. Patients are most concerned with the accurate and timely diagnosis of ACI.

New Features in the Updated Report

For this updated report, we re-reviewed and systematically abstracted data from all the studies included in the 1997 Working Group report, as well as from all studies published from October 1994 through December 1998. The abstracted data allowed us to summarize the evidence quantitatively, which was not done in the original report.

Also new in this updated report is the inclusion of meta-analyses. When the data were sufficient and appropriate, we conducted meta-analyses to quantitatively assess test performance with summary receiver operating characteristics curves (SROC). Likewise, when possible, we conducted meta-analyses of trials studying clinical impact.

The updated report also includes decision and cost-effectiveness analyses. Because the diagnosis of ACI in the ED is complex and dependent on many different factors, such analyses are fraught with difficulties and limitations. The number of possible clinical scenarios that can be analyzed is great. Long-term clinical outcomes are difficult to model because the management of patients with ACI can vary. The long-term disposition of patients with noncardiac chest pain is also difficult to model. Thus, no single model, or set of models, can adequately reflect the range of circumstances that are encountered in the ED. As a result, these analyses were conducted not for the purpose of making specific clinical recommendations but for understanding the interactions among the variables studied.

Technologies Assessed in the Updated Report

As directed by the AHRQ, the technologies reviewed in the updated report are:

1. Prehospital 12-lead electrocardiography
2. Emergency department electrocardiography
   Continuous 12-lead ECGs
   Nonstandard ECG leads
   Rest, exercise, or stress ECG assessments
3. The Acute Cardiac Ischemia Time-Insensitive Predictive Instrument
4. The Goldman chest pain protocol
5. Biochemical markers
   Creatine kinase (CK), single and serial measurements
   Creatine kinase subunit (CK-MB), single and serial measurements
   Troponin T
   Troponin I
   P-selectin
   Fatty acid binding protein
   Myoglobin
6. Myocardial perfusion imaging technologies
   Echocardiography (including rest, exercise, and stress assessments)
   Sestamibi imaging (including rest, exercise, and stress assessments)
7. Other computer-based decision aids