A. Class I Error Rates

I. Time Trend

Coefficient for Year = -0.03041 → -0.3041 to use decade as unit of analysis instead of single year

Odds ratio = exp (–0.3041)= 0.7378

<Subtract above from 1.0 to obtain relative decreases>

→ 0.2622

Class I error rates showed relative decrease of 26.2% per decade (p=0.1044)

95% CI = exp (-0.3041 ± 2*0.1825)

= exp (-0.3041 ± 0.3650)

= exp (0.0609, -0.6691)

= (1.0628,0.5122)

<Subtract above from 1.0 to obtain relative decreases>

→ (-0.0628,0.4878)

→ Class I error rates showed relative decrease of 28.0% (95% CI: 48.8% decrease to 6.3% increase) per decade

II. Relationship to Autopsy Rate

Coefficient Autopsy Rate = -0.8122

(-0.8122/10 = -0.08122 to calculate relationship as per 10% change in Autopsy Rate

Odds ratio = exp (-0.08122) = 0.9220

<Subtract above from 1.0 to obtain relative decrease>

(0.0780)

For every 10% increase in autopsy rate, Class I error rate decreased by 7.8% (p=0.1848)

95% CI = exp (-0.08122 (2*0.6007/10)

= exp (-0.08122 (0.12014)

= exp (-0.20136, 0.03892)

= 0.8176, 1.04

<Subtract above from 1.0 to obtain relative decreases>

(-0.0400, 0.1824)

(Class I error rate exhibited 7.8% relative decrease (95% CI: 18.2% decrease to 4.0% increase) for each 10% increase in autopsies.

III. Calculation of “mean” Class I error rate

Because error rates varied with time and autopsy rate (as well as case mix, though to a lesser extent), a true “mean” error rate does not exist. However, we can estimate a “base error rate,” if the predictor variables are all set to their base values (i.e., time=1980, autopsy rate = mean rate of 44.3%, country=U.S., and case mix=general autopsies).

A point estimate for the base error rate can then be obtained from the regression equation,

Prob (Class I error) = 1 / (1 + e^-λ) where λ = β₀ + β₁ X ₁ + β₂ X ₂ + … + u

Intercept= -2.2622 and, for the base probability, the terms for time, autopsy rate and case mix all equal zero, because the equation is centered on these values.

Therefore,

λ= -2.2622 + (1980-1980)(–0.03041) + (-0.8122)(0.443–0.443) + (-0.09338)(-1), where the –1 in the country term reflects the value assigned to U.S. (non-U.S.=+1).

Thus, λ= -2.16882, so that base prob (Class I error) = 1 / (1 + e^{‐(-2.16882)})

→ Base prob (Class I error) = 0.10258→ 10.2%

Base error and 95% CI obtained from software were: 0.1023 and (0.06701, 0.1532)

Therefore, base rate of Class I errors was: 10.2% (95% CI: 6.7–15.3%)

Because the trends over time and the relationship to autopsy rate were not statistically significant for Class I errors, this base error rate provides a reasonable overall mean rate for Class I errors. It is clear, though (as shown in Table below), that the effects of study period (time) and autopsy rate are noticeable enough (even if not statistically significant for Class I errors). Therefore, a true “mean error rate” is not meaningful in the absence of stipulated values for time and autopsy rate. The table below shows how the Class I error rate varies with time and autopsy rate using the regression model above, with country equal to U.S. and case mix equal to general autopsies.

B. Class I Errors: U.S. only

Coefficient for Time is –0.05501

→ -0.5501 to use decade as unit of analysis instead of single year

odds ratio = exp (–0.5501)= 0.57689

<Subtract above from 1.0 to obtain relative decrease>

→ 0.42311

→ Error rate showed relative decrease of 42.3% per decade, but this relationship was not statistically significant (p=0.07)

Coefficient Autopsy Rate = -1.6629

→ -1.6629/10 = -0.16629 to calculate relationship as per 10% change in Autopsy Rate

odds ratio = exp (-0.166291) = 0.8468

<Subtract above from 1.0 to obtain relative decrease>

→ For every 10% increase in autopsy rate, error rate decreases by approximately 15.3%, but this relationship was not statistically significant (p=0.2).

Prob (Class I error) = 1 / (1 + e^-λ) where λ = β₀ + β₁ X ₁ + β₂ X ₂ + … + u

Intercept= -2.0678

For base probability, time, autopsy and case mix terms all equal zero and there is no country terms in the analysis restricted to U.S. only, so

λ = -2.0678

Therefore,

→ Base prob (Class I error) = 1 / (1 + e^‐(-2.0678))

→ Base prob (Class I error) = 0.1117→ 11.2%

The value calculated using the statistical software corroborated this estimate and provided the corresponding confidence interval.

Thus, the mean Class I error rate using data from U.S. only is 11.2% (95% CI: 6.9–17.5%).

C. Major Errors

I. Time Trend

Coefficient for Time is –0.03288 → -0.3288 to use decade as unit of analysis instead of single year

Odds ratio = exp (–0.3288 ) = 0.719787

<Subtract above from 1.0 to obtain relative decrease>

→ 0.2802

→ Major error rates showed decrease of 28.0% per decade (p=0.0058) relative to the base time period of 1980.

Coefficient Autopsy Rate = -1.2846

10% change → -1.2846/10= -0.12846

Odds ratio = exp (-0.12846) = 0.879449

<Subtract above from 1.0 to obtain relative decrease>

→ 0.120551

For every 10% increase in autopsy rate, major error rate decreases by 12.0% (p=0.0003)

The models dropping case mix, country and autopsy rate (one at a time) produced results with statistically significant differences from the above. Consequently, we used Model 1 (including autopsy rate, country, case mix as predictors of major errors) to compute the mean error rate and the range of error rates shown in the table below and the mean major error rate of 25.6% (95% CI: 20.8–31.2%). The point estimate can be obtained manually using the calculation shown below and was compared with the value generated by the statistical software. (The software was required to calculate the confidence interval.)

Prob (major error) = 1 / (1 + e^-λ) where λ = β₀ + β₁ X ₁ + β₂ X ₂ + … + u

Substituting in Intercept (β₀)= -0.9773

and base values of time (1980), autopsy rate (overall mean rate of 44.3%), country (U.S.=-1) and case mix (general autopsies, so that case mix terms CM2,…,CM7 all equal zero), then

λ = -0.9773 + (–0.02223)(1980-1980) + (-0.9603)(0.443–0.443) + (0.08833)(-1)

λ = -1.06563

→ Base prob (major error) = 1 / (1 + e^{‐(-0.88867)})

→ Base prob (major error) = 0.256235 → 25.6%

Calculation performed with statistical software confirmed this estimate and provided the corresponding confidence interval of (0.2077, 0.3116).

Thus, base probability of major error was: 25.6% (95% CI: 20.8–31.2%)

D. Major Errors: U.S. Studies Only

Coefficient for Time is –0.0246 → -0.246 to use decade as unit of analysis instead of single year

Odds ratio = exp (–0.246)= 0.78192

→ Error rate exhibited relative decrease of 21.8% per decade, but this relationship was not statistically significant (p=0.2).

Coefficient Autopsy Rate = -1.0411

10% change → -1.0411/10 = -0.10411

Odds ratio = exp (-0.10411) = 0.90113

For every 10% increase in autopsy rate, the major error rate decreased by 9.9% (p=0.06)

E. ICD Disease Category Discrepancies

Nb: for the analysis below, time was centered at 1975, rather than 1980, as the study periods ranged from 1970-1980. Also, all of the studies using this error classification involved general inpatients or adult inpatients (Case Mix = 1), so case mix was not included in the models below.

Intercept= -2.0252; Standard error = 0.1024

95% CI= -2.0252 ± 2*0.1024 → -2.230, -1.8204

probability of error = 0.1166 (95% CI: 0.0971 -0.1394)

Coefficient for Time is 0.2468

Odds ratio = exp (0.2468)= 1.2800

→ Error rate increased by roughly 28% per year (p<0.0001).

Coefficient Autopsy Rate = -0.2642

5% change → -0.2644/20 = -0.01321

Odds ratio = exp(-0.01321) = 0.9869

For every 5% increase in autopsy rate, error rate decreases by approximately 1.4%, but this relationship was not statistically significant (p=0.1)