NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Tipton K, Leas BF, Flores E, et al. Impact of Healthcare Algorithms on Racial and Ethnic Disparities in Health and Healthcare [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2023 Dec. (Comparative Effectiveness Review, No. 268.)

Appendix CCharacteristics of Key Question 1 and 2 Studies

Table C-1Characteristics of studies addressing Key Question 1

Author/YearStudy ObjectiveAlgorithmControlComponent VariablesStudy Design SettingSource of Study DataPatient Characteristics

Ashana et al. 20211

*Also addressed KQ2

Assess the performance of the Sequential Organ Failure Assessment (SOFA) score and LAPS2 among Black and White patients admitted through the emergency department (ED) with sepsis or acute respiratory failure (ARF).

SOFA score

Laboratory-based Acute Physiology Score version 2 (LAPS2)

Mitigation strategy: modified versions of SOFA and LAPS2

Compared original and modified versions of algorithms

SOFA score: Composed of organ function scores from 6 organ systems (cardiovascular, respiratory, hepatic, renal, coagulation, neurological). In this study, the renal subscore was calculated using creatinine alone, and the highest value of each subscore during the patient’s ED stay was used to calculate total score (continuous variable 0 to 24 points, with higher values representing greater illness severity).

SOFA score modifications:

1) divided score into 4 categories (<6, 6 to 8, 9 to 11, ≥ 12)

2) subtracted one-half point from renal subscore for Black patients whose raw renal subscore >0

3) eliminated renal subscore

LAPS2: 2-stage algorithm in which patients are first stratified into low- and high-mortality-risk groups, and then vital signs and laboratory values are added to the algorithm. Total score is based on risk stratum and most deranged laboratory value and ranges from 0 to 414 (continuous variable, scores > 200 uncommon).

LAPS2 modifications:

1) continuous LAPS2 divided into 8 categories

2) continuous LAPS2 divided into 4 equal categories

Modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated

EDs of academic medical centers

Patients admitted for sepsis or ARF at 27 hospitals (Kaiser Permanente Northern California and Penn Medicine) between 2013 and 2018. Patients were ≥ 18 years with sepsis at all hospitals and ARF at Penn Medicine hospitals and were admitted from the ED to an inpatient location.

Study does not report how race/ethnicity was defined (e.g., self-reported)

Patients (n): 113,158

Race/Ethnicity: 75.6% White; 24.4% Black

Mean Age (SD): 67.7 (15.2) White; 61.7 (16.6) Black

p<0.001

Sex: White 54.1% male and 45.9% female; Black 48.2% male and 51.8% female

p<0.001 for % female between groups

Boley et al. 20222Examine impact of a rapid triage fast-track (FT) model on outcomes in Black non-Hispanic and White non-Hispanic patients presenting to the ED.

Rapid triage fast-track (FT) model

Providers assign an emergency severity index (ESI) score, then determine whether patients meet additional criteria for FT or main ED status.

No comparator. Compared triage process outcomes by racial groups.

Triage process: Nurse checks in patient, determines chief complaint, and obtains set of vital signs. Nurse applies ESI protocol to give patient a score of 1 (most acute) to 5 (least acute). Nurse then determines whether patient is appropriate for FT based on the following requirements:

  1. Patient is able to sit in a recliner.
  2. Patient is ambulatory and able to speak.
  3. Patient’s ESI score is 3, 4, or 5 (lowest acuity).
  4. Patient is determined to be not critical based on the triage determination

Patients identified for FT are seen in a separate 5-bed area of ED. Patients not eligible for FT wait until an ED bed becomes available.

FT status: patients placed in a separate ED section and can receive intravenous fluids, medications, and laboratory and radiology tests. A physician assistant, nurse practitioner, nurse, or ED technician provides care, but an ED physician can be involved if needed (e.g., case’s complexity).

Main ED status: patients wait until an ED bed is available. Care typically provided by ED physician.

Retrospec-tive matched cohort (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Tertiary care hospital (556-bed)

EHR ED encounters during a 1-year period (2/1/2019 to 1/31/2020) after full implementation of rapid triage system.

Black and White patients were exact-matched on potential confounders including presence of abnormal vital signs.

Race and ethnicity collected as separate measures in the EHR based on patient self-reporting and patients can report multiple races.

Race and ethnicity not included as an input variable.

Patients (n): 9704 with 12,330 unique encounters (5151 Black Non-Hispanic encounters; 7170 White Non-Hispanic encounters)

Race/Ethnicity: 58.2% White Non-Hispanic encounters; 41.7% Black Non-Hispanic encounters

Mean Age (SD): 37.4 (36.9) White Non-Hispanic; 36.9 (13.2) Black Non-Hispanic

Sex: 70.4% female; 29.6% male

Carbunaru et al. 20193Compare the frequency of avoided biopsies and missed clinically significant prostate cancer (csPCa) resulting from use of 2 risk prediction algorithms across racial groups in an urban, multi-racial cohort.

Prostate Cancer Prevention Trial Risk Calculator 2.0 (PCPT RC)

Prostate Biopsy Collaborative Group (PBCG) RC

Compared algorithmsInput variables include prostate-specific antigen (PSA) level, digital rectal exam (DRE) result, first-degree family history of PCa (father, brother or son ever diagnosed with PCa) and history of a prior negative prostate biopsy. Both algorithms “take race into consideration”.

Retrospec-tive (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Urology clinics

The sample consisted of consecutive ambulatory patients from urology clinics at 2 privately funded and 3 publicly funded institutions who were undergoing their first prostate biopsy for an abnormal PSA level or digital rectal exam (DRE). Data obtained from a prospectively maintained dataset.

Race/ethnicity was self-reported.

Patients (n): 954

Race/Ethnicity: Black (463, 48.5%), white (355, 37.2%), Other race (136, 14.2%). The Other group included Hispanic (n = 103, 75.7%), Asian (n = 28) and Middle Eastern men (n = 5).

Median Age (IQR): Black, 61 (57, 67); white, 62 (58, 67); other, 62 (57, 67)

Sex: 100% male

Han et al. 20204Characterize individuals who would be selected for lung cancer screening based on risk factors but would not be recommended for screening based on the current USPSTF guidelines.PLCOm2012 Model, which predicts 6-year risk of lung cancer based on demographic, environmental, and clinical risk factors.Current USPSTF guidelines (annual low-dose computed tomogra-phy screening of individuals aged 55–80 years with at least 30 pack-years of smoking and within 15 years since cessation)Age, race/ethnicity, education, body mass index (BMI), chronic obstructive pulmonary disease (COPD), personal history of cancer, family history of lung cancer, and smoking status

Simulation study (modeling using synthetic data – source data for calculation of algorithmic scores is synthetic data and outcomes that would have resulted from using the algorithm are simulated)

1950 U.S. birth cohort aged 50–90 years

Analyses were performed on a simulated dataset of 100,000 individuals in the 1950 U.S. birth cohort, containing: (a) smoking history data generated by the CISNET Smoking History Generator based on data from the NHIS, Cancer Prevention studies I and II, and the Human Mortality Database, and (b) risk factor data generated by the LC Risk Factor Generator based on data from the NHIS, PLCO trial, U.S. Census Bureau, and NHANES.

Sensitivity analyses were performed on a similar dataset representing the 1960 U.S. birth cohort.

Race/ethnicity data generated from U.S. Census Bureau data.

1950 birth cohort:

Patients (n): 100,000

Race/Ethnicity: White (76%), Black (10%), Hispanic (8%), Asian (5%)

Age: Not reported

Sex: Not reported

Metzger et al. 20225Assess the association of race and language with ED triage scores.ESI scoreNo comparator. Compared triage scores by racial groups.

A 5-level triage algorithm

ESI 1 (Immediate medical attention). Study excluded visits with an ESI score of 1.

ESI 2 (Emergency)

ESI 3 (Urgent)

ESI 4 (Nonurgent)

ESI 5 (Minor)

Retrospec-tive cohort (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Pediatric ED

EHR data from July 2015 to June 2016 for patients aged 0 to 17 years.

Study does not indicate whether race and ethnicity was self-reported. Patients were categorized as Non-Hispanic White if White/Caucasian and non-Hispanic ethnicity were reported. Patients were categorized as non-White if any other race was reported (Black/African American, Asian, American Indian/Alaska Native, Native Hawaiian/Pacific Islander, other) or if an ethnicity of Hispanic was reported.

Race and ethnicity not included as an input variable.

Patients (n): 8928 (3086 Non-Hispanic White; 5842 Non-White) with 10,815 visits (3538 Non-Hispanic White; 7277 Non-White)

Race/Ethnicity: 34.6% Non-Hispanic White; 65.4% Non-White (1.2% American Indian/Alaska Native, 14.6% Asian, 23.8% Black, 2.5% Native Hawaiian/Pacific Islander, 38.7% other, 11% more than 1 race)

Median Age (Months at Visit; IQR): 39.2 (14.1 to 88.5) Non-Hispanic White; 33.4 (13.7 to 74.2) Non-White

Sex: 46.8% female; 53.2% male

Miller et al. 20216Investigate whether using the SOFA is associated with deprioritiza-tion of Black patients in currently adopted crisis standards of care (CSC).SOFA -- continuous variable used to predict in-hospital mortality risk, scored from 0 (lowest risk) to 24 (highest risk). Scores are collapsed into tiers for the purpose of prioritizing resources to patients most likely to survive with appropriate care when resources are overwhelmed. This study examined 3 tiering systems, termed A (4 tiers, with scores <6 forming the highest-priority tier and scores ≥12 forming the lowest), B (3 tiers, scores <8 highest priority, ≥12 lowest), and C (4 tiers, scores <9 highest priority, ≥15 lowest).The authors quantified how much the SOFA threshold required for inclusion in a priority tier would have to be increased for Black patients so that mortality would be equivalent for Black and White patients eligible for resource allocation.Blood pressure, hypoxemia, creatinine, bilirubin, platelet count, and the Glasgow Coma Scale.

Modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated

ICUs

The SOFA score was developed using a consensus-based process. In this study, the data source is the eICU Collaborative Research Database, a cohort of patients admitted to ICUs in 208 U.S. hospitals from 2014 to 2015. Eligibility criteria: age ≥18 years; Black or white race; at least 1 SOFA variable recorded within 24 hours of ICU admission; in-hospital mortality documented. Records representing the first ICU stay during a hospitalization were included.

Study does not report how race/ethnicity was defined (e.g., self-reported).

Patients (n): 111,885 patient encounters for 95,549 unique patients

Race/Ethnicity: 16,688 encounters with Black patients (14.9%) and 95,197 encounters with White patients (85.1%)

Mean (SD) age: 63.3 (16.9) years

Sex: 51,464 encounters with women (46.0%)

Obermeyer et al. 20197

*Also addressedKQ2

Quantify racial disparities in health care resource allocation produced by a widely used commercial risk prediction algorithm.The algorithm is used to predict complex health needs in primary care patients; the goal is to direct additional resources to such patients, based on the assumption that they will benefit the most from them. The original algorithm predicts costs over the following year. In the health system studied here, patients scoring above the 97th percentile are automatically identified for enrollment into the system’s care management program. For those above the 55th percentile, their primary care physician is asked whether they would benefit from the program.The authors developed and internally validated 3 new algorithms, predicting different outcomes or “labels” (total costs, avoidable costs, and health).

Features of raw insurance claims data from the previous year, including age, sex, insurance type, diagnosis and procedure codes, medications, and detailed costs.

Race is not an input variable in the original algorithm or in the 3 new algorithms.

Retrospective cohort study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Hospital

Development dataset for original algorithm is not specified. In this study, the data source is all primary care patients enrolled in risk-based contracts at a large academic hospital from 2013 to 2015 and self-identifying as either Black or as white without another race or ethnicity.

Patients (n): 49,618

Race/Ethnicity: 87.7% White, 12.3% Black

Mean Age: 51.3 (White), 48.6 (Black)

Sex: 62% female (White), 69% female (Black)

Pasquinelli et al. 20218Compare 2 different lung cancer screening criteria, USPSTF 2013 and PLCOm2012.PLCOm2012, a validated logistic regression lung cancer risk prediction modelUSPSTF 2013: criteria based on findings of the National Lung Screening Trial (NLST). The Task Force recom-mends low-dose computed tomogra-phy for individuals who meet NLST-like eligibility.

USPSTF 2013: NLST-like eligibility criteria include age 55 to 80 years, ≥ 30 pack-year cigarette smoking history, and having quit smoking within the past 15 years.

PLCOm2012: age, highest level of education obtained, BMI, COPD, personal history of cancer, family history of lung cancer, race and ethnicity, smoking status (former or current), average number of cigarettes smoked per day, duration smoked, years of quitting smoking

Retrospective study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Urban academic medical center (13 federally qualified health centers)

Lung cancer cohort at the University of Illinois Hospital and Health Sciences System between 2010 and 2019. Data collected up until March 15, 2020.

Study does not report how race/ethnicity was defined (e.g., self-reported)

Patients (n): 883

Race/Ethnicity: 56.3% African American; 29.2% White; 7.8% Hispanic; 2.7% Asian; 4.0% Other or missing

Mean Age (SD): 64.8 (9.4)

Sex: 55.8% male; 44.2% female

Presti et al. 20219Externally validate a newly developed prostate cancer risk prediction algorithm, and compare with 2 other calculators.

Prostate cancer risk prediction

Kaiser Permanente Prostate Cancer Risk Calculator (KPPC RC) (range: 0% to 100%)

Compared 2 versions of the algorithm

Version A: age, race (patient-reported), BMI, family history of prostate cancer, number of prior biopsies, PSA level, DRE result

Version B: Version A variables plus prostate volume

(The study also examined a version that did not include DRE result or prostate volume, but did not report results by race or ethnicity for that version.)

Retrospective

Large integrated health care system (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Validation (this study): Kaiser Permanente Northern California (KPNC)

All men with no prior diagnosis of prostate cancer who underwent prostate biopsy at any of 21 KPNC urology departments between 12/2017 and 8/2019 for either an abnormal DRE and/or an elevated PSA and had complete data on all analysis variables. Prospective data collection methods used to capture biopsies during this time.

Study does not report how race/ethnicity was identified.

Patients (n): 4178 (5353 men underwent biopsy; 4178 had complete data)

Race/Ethnicity: 56.7% Caucasian, 11.3% African American, 16.2% Asian/Pacific Islander, 13.5% Hispanic, 2.3% other

Median Age (IQR): 63 (57–67)

Sex: 100% male

Riviello et al. 202210Analyze the association of Crisis Standards of Care (CSC) scoring system with resource prioritization and estimated excess mortality by race, ethnicity, and residence in a socially vulnerable area.CSC scoring systemCompared outcomes by racial and ethnic groups and compared CSC and random allocation lottery in a simulated model to estimate number of excess deaths.

CSC scoring system: aggregate score outlined by Commonwealth of Massachusetts for application in individual hospitals. Score based on points derived from SOFA score and a chronic severity of illness score based on comorbidities or a life expectancy score based on physician assessment. SOFA converted into a 4-point scale: 1 for SOFA < 6, 2 for SOFA 6 to 9, 3 for SOFA 10 to 12, and 4 for SOFA>12. Comorbidities based on a 3-level system: 0 points no significant comorbidities, 2 points major comorbid conditions with substantial impact on long-term survival, 4 points severely life-limiting conditions prior to acute illness. Life expectancy based on a 3-level score: 0 points death not likely in 5 years, 2 points death likely within 5 years, and 4 points death likely within 1 year. Points totaled to create raw ordinal priority score from 1 to 8. Highest scores 1 to 2, intermediate 3 to 5, and lowest 6 to 8. Highest-scores first to receive scarce critical care resources, then intermediate scores, then lowest scores.

Simulation: Simulation of mortality outcomes using CSOC score vs random lottery in a subset of patients receiving ventilation. Created a scenario of scarcity requiring allocation of ventilators using 2 state-recommended cutoff scores of ≤ 2 (highest-priority category patients receive ventilator) and ≤ 5 (both highest and intermediate-priority category patients receive ventilator). Authors ran 10,000 trials randomly assigning individuals to receive a ventilator.

Retrospective cohort (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

ICUs of 6 Boston-area tertiary and community hospitals in Beth Israel and Lahey Hospital systems

EHR data: April 13, 2020, to May 22, 2020. April 28, 2020, hospitals used an estimate of life expectancy instead of comorbidities in response to revised guidelines which required collection of additional data from EHR (discharge dates, vital status at discharge, discharged destination). Attending physicians assessed likelihood that a patient would survive past 1 or 5 years based on baseline health status at time of ICU admission. Race and ethnicity self-reported or reported by patient surrogate and recorded in medical record. Race categorized as other for any self-reported race that was not White, Black, or Asian. Race and ethnicity listed unknown if self-report not recorded. Race and ethnicity not included as an input variable.

Patients (n): 498 (79 Black, 298 White, 11 Asian, 46 Other, 64 Unknown)

Race/Ethnicity (%): 15.8% Black, 59.8% White, 2.2% Asian, 9.2% Other, 12.8% Unknown

Median Age (IQR), years: 67 (56 to 75)

Black 68 (59 to 75), White 69 (57 to 76), Asian 62 (59 to 72), Other 63 (52 to 73), Unknown 59 (50 to 69)

Sex: 38,4% female; 61.6% male

Black (32.9% female, 67,1% male), White (39.3% female, 60.7% male), Asian (36.4% female, 63.6% male), Other (32.6% female, 67.4% male), Unknown (45.3% female, 54.7% male)

Subgroup (n): 244 (16.8% Black, 49.2% White, 2.9% Asian, 10.7% other, 20.5% unknown)

Race/Ethnicity (%): 16.8% Black, 49.1% White, 2.8% Asian, 10.6% other, 20.4% unknown)

Sarkar et al. 202111Examine the performance of 3 severity scoring models.

APACHE IVa generates a risk score for hospital, ICU mortality, and length of stay.

OASIS predicts hospital mortality and ICU mortality of critically ill patients

SOFA characterizes severity state in sepsis but has been used to predict patient outcomes

Compared models

APACHE IVa: 142 patient variables including 116 admission categories and 17 acute physiologic parameters (65.9% of score and includes age, chronic health condition, underlying diagnosis, ventilation status).

OASIS: 10 variables collected in first 24 hours of ICU stay (heart rate, mean arterial pressure, temperature, respiratory rate, urine output, pre-ICU admission length of stay, GCS, age, being placed on a mechanical ventilator at any point during day 1 and admission following elective surgery).

SOFA: composed of organ function scores from 6 organ systems (cardiovascular, respiratory, hepatic, renal, coagulation, neurological). SOFA categories based on proposed categories for COVID-19 ventilator allocation.

Modeling study using real-world data to determine effect of illness severity scores on CSC allocation (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

ICU admissions

eICU-Collaborative Research Database (eICU-CRD) includes > 200,000 discharge episodes across 335 ICUs at 208 hospitals between 2014 and 2015. Data available includes age, sex, ethnicity, vital signs, diagnoses, laboratory measurements, clinical history, problem lists, APACHE IVa scores, and treatments.

Medical Information Mart for Intensive Care-III database consists of >60,000 ICU admissions to Beth Israel Deaconess Medical Center between 2001 and 2012 and includes OASIS as a mortality prediction model.

eICU-CRD

Patients (n): 122,919

Race/Ethnicity: 81.9% White; 12.4% African American; 4.1% Hispanic; 1.5% Asian

Median Age (IQR): 64 (52 to 75)

Sex: 54% male; 46% female

MIMIC-III

Patients (n): 43,823

Race/Ethnicity: 82.14% White; 11.07% African American; 4.07% Hispanic; 2.71% Asian

Median Age (IQR): 64.5 (52 to 76)

Sex: 57% male; 43% female

Note: race/ethnicity is based on definitions within each database. Information is typically entered by an administrator. Patients are either asked which group they identify with or the group is entered based on available records.

Snavely et al. 202112Compare the safety and effectiveness of the HEART pathway among women vs men, and white vs non-white patients, presenting to the ED with acute chest pain.HEART Pathway provides test ordering and disposition decision support to clinicians and risk-based care planning.Compared pre- and post-implementation of the HEART Pathway.

HEART Pathway risk assessment is based on the HEAR score (History, ECG, Age, and Risk factor) and 0- and 3-hour troponin measures.

HEAR score ≤ 3 without elevated troponin is classified as low-risk and recommended for discharge without objective cardiac testing. HEAR score ≤ 3 with elevated troponin leads to a cardiology consult and admission and/or further observation or testing.

HEAR score ≥ 4 with elevated troponin, known coronary artery disease, or ischemic ECG is classified as non-low risk and designated for further testing. HEAR score ≥ 4 without elevated troponin leads to observation/admission and/or cardiology consult or testing.

*Authors focused on low-risk (≤ 3) and non-low risk (≥ 4) groups for analysis.

Preplanned subgroup analysis of a prospective pre-post study (pre-post study).

3 EDs in North Carolina (large urban academic medical center, rural medical center, small community hospital).

EHR (Clarity-EPIC systems) index encounter and claims data.

Race/ethnicity was self-reported.

Patients (n): 3713 pre-implementation; 4,761 post-implementation

Race/Ethnicity Pre-implementation: 66.9% (2,484) White; 28.3% (1,052) Black or African American; 0.6% (21) Asian, 0.2% (9) American Indian, 0.03% (1) Hawaiian or Pacific Islander, 3.9% Other (145), 0.03% (1) Refused to provide information or Unknown

Race/Ethnicity Post-implementation: 65.2% (3,106) White, 28.8% (1,371) Black, 0.6% (27) Asian, 0.3% (16) American Indian, 0.04% (2) Hawaiian or Pacific Islander, 4.9% (234) Other, 0.1% (5) Refused to provide information or Unknown.

Median Age: 54 years

Sex: 46.4% male; 53.6% female

Thompson et al. 202113

*Also addressed KQ2

Assess fairness and bias of a previously validated machine-learning opioid misuse classifier.Natural language opioid misuse classifier using a convolutional neural network. Input is electronic health record (EHR) data from a hospitalization. The algorithm’s goal is “to provide point-of-care education, treatment options, and care pathways to patients who misuse opioids.” Thus, false negatives (Type II errors) represent failures of the model to recommend appropriate resources.2 post-hoc analyses were performed to mitigate the classifier’s bias: (a) the threshold value dividing negative predictions from positive predictions in the subgroup with biased false-negative rate was varied to improve sensitivity without losing specificity, and (b) the classifier was recalibrated by subgroup.Clinical notes

Retrospective cohort study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Hospital and tertiary care academic center

Development dataset: adult hospital encounters from EHR between 2007 and 2017 at a U.S. hospital and tertiary academic center. Opioid-related hospitalizations were oversampled. “The final dataset …consisted of 367 manually labeled cases, age- and sex-matched with controls that had no indications of opioid misuse.” External validation dataset: EHR at a different tertiary care academic center. Dataset included “all unplanned adult inpatient encounters …who were screened between October 23, 2017, and December 31, 2019 (n = 53,974).”

Appears race/ethnicity was self-reported.

The analyses reported in this article were carried out on the external validation dataset.

Patients (n): 53,794

Race/Ethnicity: White (n = 23,345), Black (n = 17,541), Hispanic/Latinx (n = 9252), Other (n = 3836)

Age: not reported

Sex: not reported

Wille et al. 201314Compare ethnic disparities in lung transplantation rates and time to death on the wait list, before vs after introduction of the Lung Allocation Score (LAS).

In 2005, LAS became the main method for determining allocation of deceased donor lungs for transplantation in the United States.

The LAS is a numerical score based on survival models that estimate likelihood of survival both while on the wait list and post-transplant; thus, it reflects the net benefit of transplantation.

In the pre-LAS period, time on the wait list was the sole basis for allocation.Diagnosis (4 categories), age, height, weight, cardiac index at rest, bilirubin, functional status, PA systolic pressure, O2 required at rest, six-minute walk distance, continuous mechanical ventilation, PCO2, increase in PCO2, creatinine (from https://optn​.transplant​.hrsa.gov/data/allocation-calculators/las-calculator/)

Retrospective pre/post-implementation study (pre-post study)

U.S. health care system

The study population consisted of all White and Black non-Hispanic adults who were listed for lung transplantation during 2 time periods: pre-LAS (January 1, 2000–May 3, 2005) and LAS (May 4, 2005–September 4, 2010).

Race/ethnicity was self-reported.

Patients (n): 8765 (pre-LAS), 8806 (LAS).

Race/Ethnicity: White (89.9%), Black (10.1%)

Mean (SD) Age: Pre-LAS: White, 49.3 (12.6); Black, 47.2 (9.6)

LAS: White, 54.0 (13.0); Black, 50.4 (10.5)

Sex: Pre-LAS: 51.3% female

LAS: 45.5% female

Williams 202215Compare number eligible for lung cancer screening between USPSTF criteria in 2013 with revised criteria in 2021, and with more detailed criteria from the PLCOm2012 modelUSPSTF-2021USPSTF-2013 and PCLOm2012Eligibility criteria for USPSTF 2021: Age (50–80), 20+ pack year smoking history, for people who currently smoke or who had quit within the past 15 yearsRetrospective application of USPSTF 2013, USPSTF 2021, and PLCOm2012 criteria on 2019 cohort dataThe Center for Disease Control and Prevention Behavioral Risk Factor Surveillance System (BRFSS), which is a health-related telephone survey that collects data from more than 400,000 adults annually in 50 states, the District of Columbia, and 3 U.S. territories.

Patients (n): 41,544

Race/Ethnicity:

White-Non-Hispanic (n=36,787); Black Non-Hispanic (n=66);

Hispanic (n=786); Other Non-Hispanic (n=1905)

Yoo et al. 202316Evaluate how predictive performance of a clinical calculator affects downstream health outcomes.CHA2DS2-VASc

No comparator. Compared outcomes by racial and ethnic groups

*Calculator outputs are used to guide clinical guideline-based care

CHA2DS2-VASc (start at 0): age 65 to 74 (+1) or > 75 (+2), female (+1), CHF history (+1), HTN history (+1), stroke / TIA / thromboembolism history (+2), vascular disease history (+1), diabetes history (+1).

The algorithm informs the American College of Cardiology (ACC)/American Heart Association (AHA) atrial fibrillation treatment guideline.

2014 ACC/AHA guideline recommendation: do not recommend antithrombotic therapy for male patients with a score of 0 or female patients with a score of 1.

2020 ACC/AHA guideline recommendation: recommend antithrombotic therapy for male patients with a score ≥ 2 and female patients with a score ≥ 3. Consider antithrombotic therapy for male patients with a score of 1 and female patients with a score of 2.

Retrospective cohort (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Stanford Health Care and Lucile Packard Children’s Hospital

Stanford Medicine Research Data Repository (STARR). STARR was linked with the Social Security Administration’s Death Master File to determine out-of-hospital deaths.

Race and ethnicity self-reported. Study used the 5 U.S. Census Bureau categories and used Hispanic as a dedicated ethnicity.

Race and ethnicity are not included as input variables in MELD, CHA2DS2-VASc, or sPESI.

CHA2DS2-VASc

Patients (n): 233,129

Race/Ethnicity: Asian 15% (n=33,927), Black 3% (n=7323), White 76% (n=176,278), Hispanic 6% (n=13,578), Other 1% (n=2,023)

Median Age (25th to 75th percentiles): 77 years (71 to 83)

Sex: 56% male (n=129,621), 44% female (n=103,508)

Zhang et al. 201817Assess impact of the 2014 Kidney Allocation System (KAS) policy change on waitlisting overall and evaluate whether racial/ethnic disparities in waitlisting in the United States changed following implementation.

Kidney allocation

KAS was developed to improve equity related to dialysis time and to patients with high panel reactive antibody. Specific changes to the system include a change in the calculation of waiting time and prioritization of the most sensitized patients. Waiting time starts at dialysis start instead of at the time of waitlist.

Pre-post comparison

KAS uses Kidney Donor Profile Index (KDPI) and Expected Post Transplant Survival (EPTS) score for longevity matching between donors and recipients.

KDPI (donor variables): age, height, weight, ethnicity, history of hypertension, history of diabetes, cause of death, serum creatinine, hepatitis C Virus status, donation after Circulatory Death status (range: 0% to 100%). Options for ethnicity variable: American Indian or Alaska Native, Asian, Black or African American, Hispanic/Latino, Native Hawaiian or Other Pacific Islander, White, or Multi Racial. EPTS score: age, time on dialysis, current diabetes status, and if candidate had a previous solid organ transplant (range: 0% to 100%).

KAS also incorporates a points system to increase priority for patients with high panel reactive antibody (i.e., patients less likely to find a compatible donor), includes pre-registration dialysis time as part of a candidate’s waiting time, provides increased access for candidates with blood type B, uses KDPI scores to inform pediatric priority, and eliminates the payback system (i.e., if an organ was received from another organization, the receiving organization had to pay back an organ to the national pool). *Information from the Organ Procurement & Transplantation Network New KAS FAQs.

Retrospective cohort study (pre-post study)

U.S. medical centers

New patients on dialysis and existing patients on dialysis with end-stage renal disease (ESRD) from the United States Renal Data System (USRDS).

Pre-KAS group: beginning dialysis between 1/1/2005 and 12/03/2014

Post-KAS: beginning dialysis between 12/4/2014 and 12/31/2015

The United Network for Organ Sharing system used to collect information about active and inactive status of newly waitlisted patients from 2005 to 2015.

Study does not report how race/ethnicity was defined (e.g., self-reported)

Pre-KAS (incident patients)

Patients (n): 1,120,655

Race/Ethnicity: 52.1% White; 26.5% Black; 13.7% Hispanic; 4.2% Asian

Age group, N (%):

18 to 39: 7.5%

40 to 49: 10.7%

50 to 59: 19.9%

60 to 69: 24.9%

≥ 70: 47%

Sex: 56.8% male; 43.2% female

Post-KAS (incident patients)

Patients (n): 132,445

Race/Ethnicity: 51% White; 25.1% Black; 13.7% Hispanic; 4.8% Asian

Age group, N (%):

18 to 39: 7.3%

40 to 49: 10.4%

50 to 59: 19.7%

60 to 69: 27.2%

≥ 70: (35.4%)

Sex: 58.2% male; 41.8% female

Prevalent Dialysis Cohort

*Patients eligible for first time waitlisting anytime during period (1/1/2005 to 12/31/2015). Baseline characteristics NR for race/ethnicity, age, and sex.

Patients (n): 1,556,954

Abbreviations: APACHE IVa = Acute physiology and chronic health evaluation; ARF = acute respiratory failure; BMI = body mass index; CAD=coronary artery disease; CG CrCl = Cockcroft-Gault Creatinine Clearance; CISNET = Cancer intervention and surveillance modeling network; CKD-EPI = Chronic Kidney Disease Epidemiology Collaboration; CLRD = chronic lower respiratory disease; COPD = chronic obstructive pulmonary disease; CSC = crisis standards of care; csPCa = clinically significant prostate cancer; DRE = digital rectal exam; ECG = electrocardiogram; ED = emergency department; eGFR = estimated glomerular filtration rate; EHR = electronic health record; eICU-CRD = eICU-Collaborative Research Database; ESRD = end-stage renal disease; FAQ = frequently asked question; ICU = intensive care unit; IQR = interquartile range; KAS = kidney allocation system; KDPI = Kidney Donor Profile Index; KPPC RC = Kaiser Permanente prostate cancer risk calculator; LAPS2 = Laboratory-based Acute Physiology Score version 2; LAS = Lung Allocation Score; LYFS-CT = life-years from screening-computed tomography; MIMIC-III = Medical Information Mart for Intensive Care III; NHANES = National health and nutrition examination survey; NHIS = National health interview survey; NSLT = National lung screening trial; OASIS = Oxford Acute Severity of Illness Score; PBCG = Prostate Biopsy Collaborative Group; PCPT RC = Prostate Cancer Prevention Trial risk calculator; PSA = prostate-specific antigen; SD = standard deviation; SOFA = Sequential Organ Failure Assessment; USPSTF = United States preventive services taskforce; USRDS = United States Renal Data System

Table C-2Characteristics of studies addressing Key Question 2

Author/YearStudy ObjectiveAlgorithmMitigation StrategyStudy DesignSource of Study DataPatient Characteristics
Ahmed et al. 202118Examine the impact of the race coefficient in the CKD-EPI eGFR equation on CKD classification and care delivery.

eGFR

(CKD-EPI)

Removed raceModeling study using cross-sectional data (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Partners Health Care Chronic Kidney Disease Registry data obtained June 2019

Patients (n): 56,485

Race/Ethnicity: 87% White; 3.9% African American; 2.3% Asian; 1.0% Hispanic; 0.1% Native American; 6.1% Other

Median Age: White 77; African American 73; Asian 77; 74 Hispanic; Native American 73; Other 76

Sex: 56.5% female; 43.5% male

Ashana et al. 20211

*Also addressed KQ1

Assess the performance of the SOFA score and LAPS2 among Black and White patients admitted through the emergency department with sepsis or acute respiratory failure (ARF).

Illness severity prediction models

SOFA score

Laboratory-based Acute Physiology Score version 2 (LAPS2)

Simulation analysis adjusted category thresholdsModeling study to evaluate potential effect on severity of illness scores using real-world data (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Patients admitted for sepsis or ARF at 27 hospitals (Kaiser Permanente Northern California and Penn Medicine) between 2013 and 2018 Patients were ≥ 18 years with sepsis at all hospitals and ARF at Penn Medicine hospitals.

Study does not report how race/ethnicity was defined (e.g., self-reported)

Patients (n): 113,158

Race/Ethnicity: 75.6% White; 24.4% Black

Mean Age (SD): 67.7 (15.2) White; 61.7 (16.6) Black

p<0.001

Sex: White 54.1% male and 45.9% female; Black 48.2% male and 51.8% female

p<0.001 for % female between groups

Baugh et al. 202219Evaluate how removal of race from algorithms affects measurement of lung function in patients with COPDPercent predicted forced expiratory volumeRemoved raceModeled potential effect of lung function formulas with and without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Sub-Populations and Intermediate Outcome Measures In COPD Study (SPIROMICS)

Patients (n): 2652

Race/Ethnicity: 20% Black

Mean age (SD): 65 (8.4) White; 58 (8.9) Black

Sex: 46% female

Bundy et al. 202220Evaluate how removal of race coefficient from eGFR affects the Kidney Failure Risk Equation and prediction of 2-year risk of end-stage kidney disease

eGFR

(CKD-EPI)

Removed raceProspective cohortChronic Renal Insufficiency Cohort study (CRIC)

Patients (n): 3873

Race/Ethnicity: 42.1% Black; 57.9%

Mean age (SD): 57.8 (10.9)

Sex: Black 51.2% female; Non-Black 40.9% female

Casal et al. 202121Evaluate how removal of race coefficient from eGFR affects use of anticancer drugs with kidney function cutoffs.

eGFR

(CKD-EPI)

Removed raceModeled potential effect of CKD-EPI formula with and without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Data drawn from National Cancer Institute database of patients enrolled in clinical trials between 1995–2010

Patients (n): 340

Race/Ethnicity: 100% Black

Median age: 57

Sex: 49% female

Coresh et al. 201922Examine an alternative approach to estimating GFR without using race or measuring creatinine through use of a metabolic panel.eGFR

GFR estimated from metabolic panel without adjustment for race or use of creatinine, as follows:

eGFR=exp(3.04584 − 0.450817*ln(Acetyl-L-Threonine) − 0.214876*ln(Beta-pseudouridine) − 0.253004*ln(Myo-inositol) + 0.2265693*ln(Tryptophan))

Modeling of a cross-sectional comparison of eGFR based on metabolic panel without race to eGFR based on creatinine and race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Derived from African American Study of Kidney (AASK) participants; validated in Multi-Ethnic Study of Atherosclerosis (MESA) participants

Patients (n): 265

Race/Ethnicity: 46% Black

Mean age (SD): 71 years (9)

Sex: 47% female

Diao et al. 202323Evaluate how removal of race coefficient from eGFR affects diagnosis and staging of kidney disease, eligibility for kidney donation and transplantation, medication dosing, and eligibility for medical services.

eGFR

(CKD-EPI 2021 and 2009; MDRD 2006)

Removed raceModeled potential effect of 2021 CKD-EPI formula without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Data drawn from NHANES, 2001–2018

The NHANES data were extrapolated to the US population at large.

Patients (n): 44,360 after removing patients with age <18 years or censored age; pregnant patients; and patients without serum creatinine reported.

Race/Ethnicity: 21.5% non-Hispanic Black; 41.9% non-Hispanic White; 26.6% Mexican American or Other Hispanic; 9.98% Other Race – Including Multi-Racial

Mean age (IQR): 45 (26)

Sex: 50.7% female

Doshi et al 202224Evaluate impact of removing Black donor race indicator from the original KDRI formula (without refitting) on perceived GF risk (as implied by KDPI categorization), GF risk discrimination and predictive accuracy, and organ discard probability.Race-free KDRI from the donor-only version of KDRI formula in DonorNetRemoved Black raceRetrospective cohort study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Scientific Registry for Transplant Recipients

Patients (n): 66,987

9,945 from a Black donor

Drawz et al. 201225Evaluate if a modified Framingham Risk Score improves prediction of cardiovascular risk in patients with hypertension.Framingham Risk Score for cardiovascular riskAdded race and chronic kidney disease to Framingham Risk ScoreRetrospective cohort study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT)

Patients (n): 6,604

Race/Ethnicity: 40% Black

Mean age: 64

Sex: 50% female

Duggal et al. 202126Evaluate how removal of race coefficient from eGFR affects medication dosing and risk of kidney failure.

eGFR

(CKD-EPI)

Removed raceModeled potential effect of CKD-EPI formula with and without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Data drawn from NHANES 2015–16, and Veterans’ Affairs Corporate Data Warehouse

NHANES cohort

Patients (n): 227,613,357

Race/Ethnicity: 11% Black

Mean age (SD): 47.4 (17.5)

Sex: 52% female

VA cohort

Patients (n): 4,477,675

Race/Ethnicity: 17% Black

Mean age (SD): 62.9 (15.8)

Sex: 8% female

Elmaleh-Sachs et al. 202127Examine whether race and ethnicity–based spirometry reference equations improve the prediction of incident chronic lower respiratory disease (CLRD) events and mortality compared with race and ethnicity–neutral equations.

Spirometry reference equations

Global Lung Function Initiative (GLI) reference equation

Removed raceRetrospective study of a prospective cohort modeling the effect of race and ethnicity-based equations compared with race and ethnicity-neutral equations (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)MESA Lung Study cohort (2004–2006)

Patients (n): 3344

Race/Ethnicity: 36% White; 25% Black; 23% Hispanic; 17% Asian

Mean age (SD): 65.3 (9.6)

Sex: 50% female

Fairman et al. 202028Evaluate effect of updated pooled cohort equations compared to original equations for cardiovascular risk prediction.ASCVD pooled cohort equations (PCE)Updated PCE were derived from more diverse population and improved statistical techniquesModeled potential effect of updated equations (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)National Ambulatory Medical Care Survey, 2011–14

Patients (n): 12,556

Race/Ethnicity: 10% Black

Sex: 56% female

Foryciarz et al. 202229Evaluate 2 algorithmic fairness approaches to adjust the risk estimators (group recalibration and equalized odds) for their compatibility with the assumptions underpinning the ACC/AHA primary prevention of ASCVD guidelines’ decision rules.10-year ASCVD risk prediction using PCE (pooled cohort equations) for statin initiation
1)

Group recalibrated algorithm model

2)

Equalized odds algorithm model

Modeled 1) group recalibrated and 2) equalized odds models using original PCE cohorts compared with revised PCE model previously published and original PCE model (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)ARIC (Atherosclerosis Risk in Communities Study), CARDIA (Coronary Artery Risk Development in Young Adults Study), CHS (Cardiovascular Health Study), FHS OS (Framingham Heart Study Offspring Cohort), MESA (Multi-Ethnic Study of Atherosclerosis) and JHS (Jackson Heart Study)

Patients (n): 25,619

Race/Ethnicity and Sex:

17.3% Black women; 11.4%

Black men; 37.8% Non-Black women; 33.4% Non-Black men

Mean age: 56.5

ASCVD event incidence: 7.54%

Fox et al. 201630Develop and validate risk prediction models for cardiovascular disease (CVD) incidence in Black adults.CVD risk prediction

Added 10 biomarkers to standard CVD risk models.

10 biomarkers: adiposity (adiponectin and leptin), neurohormonal activation (aldosterone, B-type natriuretic peptide [BNP], and cortisol), inflammation (high-sensitivity C-reactive protein [hs-CRP]), endothelial function (endothelin and homocysteine), glycemic control (glycated hemoglobin), and insulin resistance (homeostasis model assessment of insulin)

Modeled comparison of models (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Data drawn from participants in the Jackson Heart Study who had their first examination between September 2000 and March 2004

Patients (n): 3689

Race/Ethnicity: 100% Black

Mean age (SD): 53 (11)

Sex: 65% female

Gutiérrez et al. 202231Evaluate how removal of race coefficient from eGFR affects prediction of risk of kidney failure with replacement therapy and mortality.

eGFR

(CKD-EPI)

Removed raceRetrospective cohortData drawn from the Chronic Kidney Disease Prognosis Consortium, 1998 to 2018

Patients (n): 62,011

Race/Ethnicity: 33.5% Black; 66.5% Non-Black

Mean age: 63

Sex: 53% female

Hammond et al. 202032Evaluate effect of including social determinants of health (SDOH) in algorithm to predict health care use, costs, and death.Regression models using sex, age, comorbid conditions, and 7 SDOH domains: rural vs urban; alcohol abuse; access to care; economic status; financial strain; marital status; and educationUse of SDOH in addition to or to replace other variables. Race was not a specific component of any algorithm.Modeled potential effect of algorithms based on sex + age; sex + age + comorbid conditions; all of these + SDOH; or SDOH alone (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Medicare Current Beneficiary Survey, 2016–17

Patients (n): 3614

Race/Ethnicity: 9.4% Black or Hispanic

Mean age: 73

Sex: 56% female

Hoenig et al. 202133Examine whether the change in use of eGFR without the Black coefficient changed access to transplant listing at our center (quality improvement project).

eGFR

(MDRD)

Removed raceRetrospective cohort study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Patients at Beth Israel Deaconess Medical Center registered on the national UNOS waiting list for preemptive kidney transplant from January 1, 2010, to December 31, 2020.

Patients (n): 567

Race/Ethnicity: 67.7% White; 32.3% Black

Mean age of patients listed preemptively: White 54.7; Black 52.1

Mean age of patients listed on dialysis: White 53.1; Black 51.4

Sex of patients listed preemptively: White 66.3% male, 33.7% female; Black 54.3% male, 45.7% female

Sex of patients listed on dialysis: White 66.2% male, 33.8% female; Black 75.5% male, 24.5% female

Huang et al. 202234Evaluate how removal of race from eGFR affects prediction of risk of acute kidney injury after percutaneous coronary intervention.

eGFR

(MDRD)

Removed raceModeled potential effect of MDRD formula with and without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)American College of Cardiology’s National Cardiovascular Data Registry; data collected between 2009 and 2017

Test cohort

Patients (n): 947,091 procedures

Race/Ethnicity: 7.9% Black

Mean age: 64.8

Sex: 32.8% female

Validation cohort

Patients (n): 3,063,853 procedures

Race/Ethnicity: 8.5% Black

Mean age: 65.3

Sex: 31.9% female

Inker et al. 202135Examine an alternative approach to estimating GFR without using race through inclusion of 2 biomarkers.eGFRReplaced race with 4 potential components: creatinine, cystatin-C, beta-trace protein, beta2-microglobulinModeling of a cross-sectional comparison of eGFR based on 4 components without race with variations of those components with race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Derived from 7 studies, validated in 7 different studies; specific studies not identified

Patients (n): 2245

Race/Ethnicity: 24% Black

Mean age (SD): 52.8 (12.8)

Sex: 29% female

Inker et al. 202136Evaluate how removal of race coefficient from eGFR, and addition of cystatin C, affects diagnosis of kidney disease.eGFR
1)

Removed race

2)

Replace race with cystatin C

Modeled potential effect with and without race and cystatin C (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Data drawn from 12 studies used to validate CKD-EPI formula

Patients (n): 4050

Race/Ethnicity: 100% Black

Julian et al. 201737Replace race with relevant genotype to improve the Kidney Donor Risk Index.Kidney Donor Risk Index (KDRI)Replaced race with apolipoprotein L1 genotypeModeled comparison of models with and without race and genotype (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Data acquired from 3 kidney transplant centers (Wake Forest University, Emory University, University of Alabama at Birmingham) and from a published study that evaluated samples from 9 organ procurement organizations

Patients (n): 622 kidney donors and 1,149 recipients

Race/Ethnicity: All donors were Black

Kabra et al. 201638Add race to existing algorithm to improve prediction of stroke risk in patients with atrial fibrillation.CHA2DS2-VASc score for stroke riskAdded “African-American ethnicity” to the CHA2DS2-VASc score, which previously included the following: congestive heart failure, hypertension, age ≥75 years, diabetes, prior stroke, vascular disease, age 65 to 74, and female sexComparison of models with and without race added to the scoring algorithm (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Data drawn from CMS claims files, 2009–2012

Patients(n): 460,417

Race/Ethnicity: 7% Black

Mean age: 79

Kimmel et al 201339Evaluate effect of using genotype data in warfarin dosing algorithm.Warfarin dosingAddition of genotype data to standard warfarin dosing algorithmsRCTPatients enrolled at 18 U.S. study sites

Patients (n): 1015

Race/Ethnicity: 27% Black

Median age: 59 (genotype group); 57 (standard group)

Sex: 49% female

Landy et al. 202140

Examine whether USPSTF-2020 guidelines reduced racial and ethnic disparities compared with USPSTF-2013 guidelines and whether using an individualized prediction model for life-years gained from screening could reduce racial and ethnic disparities by identifying high-benefit individuals ineligible under USPSTF-2020 guidelines.

*Only USPSTF-2020 and USPSTF-2020 plus LYFS-CT included for this report.

USPSTF 2020Added an individualized prediction model, life-years from screening-computed tomography (LYFS-CT)Modeled potential effect of LYFS_CT as an addition to USPSTF criteria (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)2015 U.S. National Health Interview Survey

USPSTF-2020

Patients (n): 14,508,450

Race/Ethnicity: 85.4% White; 8.0% African American; 1.9% Asian American; 4.7% Hispanic American

Age range: 50 to 79

Sex: 43.9% female

2020 plus LYFS-CT

Patients (n): 17,977,980

Race/Ethnicity: 82.7% White; 10.6% African American; 1.9% Asian American; 4.8% Hispanic American

Age range: 50 to 79

Sex: 44.2% female

Limdi et al. 201541Evaluate role of clinical vs genetic factors in warfarin dosing algorithms.Warfarin dosingUse of race-stratified analysis of predictive algorithms for warfarin dosing rather than use of race-combined and adjusted algorithmsProspective cohortPatients enrolled at study sites at academic medical centers

Patients (n): 1357

Race/Ethnicity: 44% African-American

Mean age (SD): 61 (15.8)

Sex: 49% female

Lindley et al. 202242Evaluate whether including information on the CYP2C9*5 variant in warfarin dosing algorithms improves warfarin dose prediction.Warfarin dosingAddition of genotype data to standard warfarin dosing algorithmsRetrospective cohortPatients enrolled at 7 U.S. study sites

Patients (n): 2298

Race/Ethnicity: 63.1% White, 35.2% Black, 1.6% Other, 1.7% Hispanic

Sex: 48.8% female

Mahmud et al. 202143Evaluate how removal of race coefficient from eGFR affects association between eGFR and acute kidney injury (AKI) events in patients with cirrhosis.

eGFR

(MDRD-4, MDRD-6, CKD-EPI)

Removed raceRetrospective cohort (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Veterans’ Health Administration data from the Veterans Outcomes and Costs Associated with Liver Disease, 2008–2015

Patients (n): 72,267 patients with cirrhosis

Race/Ethnicity: 57.8% White; 19.7% Black; 7.2% Hispanic; 1.3% Asian; 13.9% Other

Median Age (IQR): 61 (57 to 66)

Sex: 2.7% female

*study reports data for Black patients only

Meeusen et al. 202244Evaluate how removal of race coefficient from eGFR affects diagnosis of kidney failure and chronic kidney disease.

eGFR

(CKD-EPI)

Removed raceRetrospective cohortOutpatients treated at Mayo Clinic (Rochester, MN) between 2006 and 2021 for whom GFR was estimated by serum creatinine and measured by iothalmate renal clearance

Patients (n): 25,512

Race/Ethnicity: 2.5% Black

Age (SD):* Black patients 50.6 (13.9); Non-Black patients 55.5 (14.0)

*Study does not report whether age is mean or median

Miller et al 202245Estimate changes that would occur in donor KDRI and in the proportion of donors classified as high risk (KDPI > 85%) if KDRI and KDPI were calculated from models without vs with the Black race predictor.Original KDRIRemoved Black raceRetrospective cohort study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Scientific Registry for Transplant RecipientsPatients (n): 69 244 adults
Miller et al. 202146Investigate the impact of removing the race coefficient from the CKD-EPI equation on renal dosage adjustment recommendations in a predominantly Black patient population.

eGFR

(CKD-EPI, Deindexed CKD-EPI, CG CrCl)

Removed raceRetrospective cohort (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Patients hospitalized between October 2019 and December 2019 at Einstein Medical Center, Philadelphia, PA.

Patients (n): 210

Race/Ethnicity: 84.3% Black; 15.7% White

Median Age: Black 60.2; White 64.9

Median age higher among Whites compared to Blacks, p=0.001)

Sex: 42.9% female

Muiru et al. 202347Evaluate how removal of race from eGFR affects progression of chronic kidney disease in patients with human immunodeficiency virus.

eGFR

(CKD-EPI 2021 and CKD-EPI 2009)

Removed raceModeled potential effect of 2021 CKD-EPI formula without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)North American AIDS Cohort Collaboration on Research and Design between 2005 and 2014

Patients (n): 69,135

Race/Ethnicity: 45% Black; 40% White, 11.6% Hispanic

Mean age (SD): Black 44.4 (11.6); White 45.4 (11.3)

Sex: Black 21.8% female; White 8.2% female

Obermeyer et al. 20197

*Also addressed KQ1

Quantify racial disparities in health care resource allocation produced by a widely used commercial risk prediction algorithm.The algorithm is used to predict complex health needs in primary care patients.Replaced outcomesRetrospective cohort study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Development dataset for original algorithm is not specified. In this study, the data source is all primary care patients enrolled in risk-based contracts at a large academic hospital from 2013 to 2015 and self-identifying as either Black or as white without another race or ethnicity.

Patients (n): 49,618

Race/Ethnicity: 87.7% white, 12.3% Black

Mean Age: 51.3 (white), 48.6 (Black)

Sex: 62% female (white), 69% female (Black)

Panchal et al. 202248Evaluate how removal of race coefficient from eGFR affects eligibility for simultaneous liver-kidney transplantation and waitlist outcomes.

eGFR

(MDRD-4, CKD-EPI)

Removed raceRetrospective cohort (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)United Network for Organ Sharing national transplant registry data, 2002–2019

Patients (n): 7937 patients eligible for liver transplantation

Race/Ethnicity: 100% Black

Median Age (IQR): No waitlist CKD 55 (46 to 61); Waitlist CKD 58 (53 to 62)

*difference in age between groups, p<0.001

Sex: No-waitlist CKD 39.6% female; Waitlist CKD 47.8% female

*difference in sex between groups, p<0.001

Park et al. 202149Compare 3 techniques for mitigating bias in algorithms predicting postpartum depression.Prediction of postpartum depression diagnosis and treatment

Recalibrated through reweighing key groups during model training

1)

Removed race

Added a regularization term that adjusts the algorithm to limit the effect of race-based variables

Modeled potential effect using each technique (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Data drawn from IBM MarketScan Medicaid Database, 2014–18

Patients (n): 532,802

Race/Ethnicity: 38% Black

Mean age (SD): 26 (5.4)

Sex: 100% female

Schmeusser et al. 202250Evaluate how removal of race coefficient from eGFR affects eligibility for cancer clinical trials.

eGFR

(CKD-EPI, MDRD)

Removed raceModeled potential effect of CKD-EPI and MDRD formulas with and without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Patients who underwent nephrectomy at Emory University Hospital between 2009 and 2021

Patients (n): 459

Race/Ethnicity: 100% Black

Median age (SD): 60 (8)

Sex: 41% female

Stage 3 or 4 cancer: 29.4%

Shi et al. 202151Evaluate how removal of race coefficient from eGFR affects diagnosis of kidney disease.eGFRRemoved raceModeled potential effect of CKD-EPI and MDRD formulas with and without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Data drawn from records of patients treated at University of Washington Medicine

Patients (n): 241,760

Race/Ethnicity: 69% White; 10% Asian; 9% Black

Median age: 53

Sex: 51% female

Shores et al. 201352Develop and validate an alternative Donor Risk Index for liver transplant that is specific to Black recipients with Hepatitis C.Donor Risk Index for liver transplantationModified the Donor Risk Index with data drawn from Black patients. The original Index was derived from a diverse population and included Black race as 1 of 7 components indicating higher risk of graft failure. The revised Index was derived from a population of Black recipients with Hepatitis C. The revised Index has 3 components and includes non-Black race.Modeled comparison of revised Index to original version (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Data drawn from United Network for Organ Sharing (UNOS) registry

Patients (n): 294 patients with hepatitis-C

Race/Ethnicity: 100% Black

Thompson et al. 202113

*Also addressed KQ1

Assess fairness and bias of a previously validated machine learning opioid misuse classifier.Natural language opioid misuse classifier using a convolutional neural network.Retrospective cohort study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Development dataset consisted of adult hospital encounters from the EHR between 2007 and 2017 at a U.S. hospital and tertiary academic center. Opioid-related hospitalizations were oversampled. “The final dataset …consisted of 367 manually labeled cases, age- and sex-matched with controls that had no indications of opioid misuse.”

The external validation dataset came from EHR at a different tertiary care academic center. The dataset included “all unplanned adult inpatient encounters …who were screened between October 23, 2017, and December 31, 2019 (n = 53,974).”

Appears that race/ethnicity was self-reported.

Patients (n): 53,794

Race/Ethnicity: White 23,345; Black 17,541; Hispanic/Latinx 9,252); Other 3,836

Topel et al. 201853Compare race-specific atherosclerosis in cardiovascular disease (ASCVD) formula to non-race-specific Framingham Risk Score for predicting subclinical vascular disease.ASCVDAdded race and presence of diabetes to Framingham Risk Score to calculate ASCVDModeling of a cross-sectional comparison of ASCVD score to Framingham Risk score (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)Derived from participants in 2 studies: Morehouse and Emory Team Up to Eliminate Health Disparities (META-Health) Study; and the Emory-Georgia Tech Center for Health Discovery and Well-Being (CHDWB) Study

Patients (n): 1231

Race/Ethnicity: 37% Black

Mean age (SD): 53 (7)

Sex: 59% female

Tsai et al. 202154Evaluate how removal of race coefficient from eGFR affects diagnosis and treatment of kidney disease.eGFRRemoved raceModeled potential effect of MDRD formula with and without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)NHANES, 2015–18

Patients (n): 2401

Race/Ethnicity: 100% Black

Weale et al. 202155Evaluate effect of adding polygenic risk scores to ASCVD for estimating risk of cardiovascular disease.ASCVDAdded polygenic risk scores to ASCVDModeled potential effect of ASCVD with and without polygenic risk scores (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)

Atherosclerosis Risk in Communities (ARIC) study;

Multi-ethnic Study of Atherosclerosis (MESA);

United Kingdom Biobank

Patients (n): 18,961

Race/Ethnicity: 31% African ancestry

Sex: 55% female

Yadlowsky et al. 201856Revised the 2013 pooled cohort equations (PCEs) using newer data and statistical methods, to improve the clinical accuracy of cardiovascular risk.ASCVD PCEs

Added data from Jackson Heart Study and MESA to better reflect racial and ethnic populations;

Adjusted statistical methods to reduce model overfitting by using elastic net regularization; removed race-based subgroups

Pre-post comparison of risk predicted by PCE derived from updated data and statistical methods to original PCE-based risk

Derived from 6 cohort studies:

Atherosclerosis Risk in Communities Study (ARIC), Cardiovascular Health Study, Coronary Artery Risk Development in Young Adults Study, Framingham Health Study offspring cohort, Jackson Heart Study (JHS), MESA

Patients (n): 26,689

Race/Ethnicity: 29% Black

Mean age: 57

Sex: 56% female

Yap et al. 202157Evaluate how removal of race coefficient from eGFR affects classification of disease severity.eGFRRemoved raceModeled potential effect of CKD-EPI and MDRD formulas with and without race (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)EHRs of a large, urban academic medical center

Patients (n): 327

Race/Ethnicity: 100% Black

Mean age (SD): 62 (14.2)

Sex: 60% female

90% had diagnosis of hypertension

Zelnick et al. 202158Evaluate how removal of race coefficient from eGFR affects accuracy of GFR estimation and time to eligibility for kidney transplant.eGFRRemoved raceRetrospective cohort study (modeling using real-world data – source data for calculation of algorithmic scores is real world data and outcomes that would have resulted from using the algorithm are simulated)National Institute of Diabetes and Digestive and Kidney Diseases public repository data for Chronic Renal Insufficiency Cohort study.

Patients (n): 1658

Race/Ethnicity: 100% Black

Mean Age (SD): 58 (11)

Sex: 51% female

Abbreviations: ARIC = Atherosclerosis Risk in Communities study; ASCVD = atherosclerosis in cardiovascular disease; CKD-EPI = Chronic Kidney Disease Epidemiology Collaboration; COPD = chronic obstructive pulmonary disease; eGFR = estimated glomerular filtration rate; FRA = Framingham Risk Score; IQR = interquartile range; JHS = Jackson Heart Study; KDRI = Kidney Donor Risk Index; MESA = Multi-ethnic Study of Atherosclerosis; MDRD = Modification of Diet in Renal Disease study; NHANES = National health and nutrition examination survey; PCE = pooled cohort equations; SD = standard deviation; SDOH = social determinants of health