2Methodology

Recruitment of Technical Experts and Peer Reviewers

We recruited technical experts to provide input during the project, four of whom were experts from the Johns Hopkins University and had expertise in public health, quality improvement, physician-patient communication, and nursing. We recruited five external technical experts who had a special interest in improving minority healthcare quality and represented different perspectives including academic medical centers, professional societies and foundations (see Appendix D). We requested specific feedback from the partner (NQF) and from the internal and external technical experts for key decisions, such as selection and refinement of the questions.

We also sought comprehensive feedback on the draft evidence report from the partner, technical experts, and other peer reviewers. Similar to the technical experts, the other peer reviewers were recruited from a variety of organizations and included those based in universities, professional societies and foundations. Experts and peer reviewers were identified by team members in consultation with internal experts and AHRQ. (See Appendix D for a list of experts and peer reviewers.)

Questions

The original questions were refined through team discussions, input from internal experts, and review and feedback from the external technical experts. Listed below are the questions addressed in this report.

1. What strategies targeted at healthcare providers or organizations have been shown to improve minority healthcare quality?

   1. Which of these strategies have been shown to be effective in reducing disparities in health or in healthcare between minority and white populations?
   2. What are the costs of these strategies?
2. What strategies have been shown to improve the cultural competence of healthcare providers or organizations?

1. What are the costs of these strategies?

Components of these questions were further defined for use in our review. Minority was defined as all non-Caucasian or non-white racial and ethnic categories, including, but not limited to, African American, Hispanic, American Indian/Alaska Native, and Asian/Pacific Islander. All clinicians were considered healthcare providers. This category included dentists, dental assistants, nurses, nurse assistants, physicians, physician assistants, pharmacists, mental health workers, community healthcare workers, social workers, and others such as alternative healers. For the purposes of this review, our research questions were meant to include any health professional or healthcare organization that provides health services to patients.

Analytic Framework

We used a conceptual model developed by Cooper and colleagues to create an analytic framework for our research questions.⁸ Below, we describe in detail the elements of the model (Figure 1) and its use in the report.

In 1993, the Institute of Medicine's (IOM) Committee on Monitoring Access to Personal Health Services set out to resolve many conceptual problems in the definitions of equitable access to health care. The Committee developed a model that provided a useful starting point for the conceptual framework that is used in this report.² Indicators in this model are grouped according to barriers (personal, structural, and financial) that cause underuse of services and mediators (such as appropriateness or efficacy of treatment received, quality of provider skills, or patient adherence) that affect health outcomes and equity of services.

Cooper and colleagues modified the Institute of Medicine's access model to provide more specific directions for designing and implementing effective interventions to eliminate healthcare disparities.⁸ They expanded the scope of personal and structural barriers, specified utilization measures to include the type of setting, provider, and procedure, incorporated provider communication skills and cultural competence as measures of the quality of providers (a mediator in the original IOM model), and included patient views of care or patient-centeredness (a component of healthcare quality from Crossing the Quality Chasm) as important outcome measures.⁹

Specifically, they included additional personal barriers/facilitators documented in recent research on disparities to differentiate between the quality of healthcare received by patient race/ethnicity or to determine differences on the use of health services or in health outcomes for whites and ethnic minorities. These variables include family structure, patient preferences and expectations of treatment, patient involvement in medical decision-making, personal health behaviors, beliefs about health and disease, and health literacy.²

Cooper and colleagues also included structural barriers/facilitators within the system in their refined version of the IOM model. For example, in addition to the availability of care, how care is organized, and transportation, they included level of difficulty in getting any appointments at all with primary care physicians and specialists and the timeliness of appointments.¹⁰ A rationale for including these structural barriers or facilitators to health service utilization is provided by recent work showing that minority patients seen in primary care settings report more difficulty getting an appointment and waiting longer during appointments, even after adjustment for sociodemographic and health status characteristics.¹¹

The IOM's access model included a category for mediators. A mediator is a variable (intermediate, contingent, intervening, causal) that occurs in a causal pathway from an independent to a dependent variable. It causes variation in the dependent variable (outcomes), and it is also caused to vary by the independent variable (barriers and facilitators). Health service use and quality of care variables are mediators between barriers/facilitators and health outcomes. Because studies of healthcare disparities document that ethnic minority patients are often cared for by physicians with poorer indicators of technical quality (such as lower procedure volume rates and higher risk-adjusted mortality rates) and that interpersonal care, including patientprovider communication, differs by patient ethnicity and by ethnic concordance in the patientprovider relationship,¹² Cooper and colleagues expanded the quality of providers (a mediator between barriers and outcomes of care) to include technical skills,interpersonal/communication skills, medical knowledge, and cultural and linguistic competence.

Appropriateness of care, one of the categories of mediators, was conceptualized as the degree to which the care delivered to patients is consistent with current standards of care (for example, beta-blocker use for acute myocardial infarction, or guideline-concordant care for major depression). Efficacy of treatment, in contrast, was conceptualized as the degree to which a specific intervention, procedure, regimen, service, or treatment produces beneficial results under ideal circumstances.¹³ For example, patient knowledge about injury prevention might be considered a measure of the efficacy of a provider intervention targeting patient education and counseling skills regarding injury prevention. Patient adherence to recommended treatment (e.g., medication refills, health behavior modification, appointment-keeping) is another healthcare process measure. We included all mediators from Cooper's model in a broad category of healthcare system processes. Finally, in addition to health status and equity of services, Cooper and colleagues included patient views about healthcare, including their attitudes toward and experiences with care and satisfaction, since these have emerged as important outcomes that may differ by race, ethnicity, social class, and language.¹⁴

Interventions to improve quality of care and to eliminate racial and ethnic disparities in healthcare might address a number of personal, structural, or financial barriers/facilitators and healthcare system processes from our conceptual model. Ideally, the intervention should target factors known to contribute to disparities in healthcare quality. For example, an intervention to eliminate racial disparities in cardiovascular procedure use might focus on patient preferences, patient-provider communication, and provider knowledge of treatment guidelines. An intervention to eliminate racial disparities in mental health care might target patient attitudes, such as stigma or fear of medications, primary care provider skills in recognition of mental health problems, or structural barriers such as the availability of case managers to improve coordination of care between primary care and mental health treatment settings.

Our conceptualization of cultural competence deserves further attention, since Question 2 in this report specifically addresses the state of the evidence regarding interventions targeting cultural and linguistic competence. No single definition of cultural competence is universally accepted. However, several definitions currently in use share the requirement that healthcare professionals adjust and recognize their own culture in order to understand the culture of the patient.¹⁵ Lack of cultural and linguistic competence can be conceptualized in terms of organizational, structural, and clinical (interpersonal) barriers to care.⁵ The Office of Minority Health defines cultural competence as the ability of healthcare providers and healthcare organizations to understand and respond effectively to the cultural and linguistic needs of patients.¹⁶ At the patient-provider level, cultural competence may be defined as the ability of individuals to establish effective interpersonal and working relationships that overcome cultural differences.¹² The Liaison Committee on Medical Education includes the need for medical students to recognize and address personal biases in their interactions with patients among their objectives for cultural competence training.¹⁷ Medical educators have defined eight content areas (general cultural concepts, racism and stereotyping, physician-patient relationships, language, specific cultural content, access issues, socioeconomic status, and gender roles and sexuality) that are taught within a commonly accepted rubric of cross-cultural education curricula.¹⁸ We conceptualized cultural competence interventions as those targeting the relevant provider knowledge, attitudes, and skills (healthcare system mediators in our conceptual model).

In addressing our research questions, we acknowledge the potential for a conceptual overlap in interventions targeting quality of care broadly and those targeting cultural competence specifically. There may also be an overlap in interventions that are targeting an organization broadly and those that are targeting providers specifically. One example of this overlap would be an intervention that incorporates interpreter services. While one might consider interpreter services to be an organizational quality improvement strategy that targets structural barriers to care, this type of intervention also affects healthcare system mediators at the provider level, including patient-provider communication and provider cultural competence.

Figure 1 shows the elements of the model addressed by the studies included in the systematic review. We circled the major categories of healthcare system processes that were targeted by the interventions included in the articles that were eligible for our systematic review.

Literature Search Methods

The process of searching the literature included identifying reference sources, formulating a search strategy for each source, and executing and documenting each search.

Abstract Review

Specific inclusion and exclusion criteria were applied at each of the three levels of review. Criteria became more stringent as the process moved from searching, to reviewing abstracts, to reviewing full-text articles. After identifying a citation, two team members independently reviewed the title and abstract, and articles were included or excluded from the article review according to the criteria described below.

Article Review

The purpose of the article review was to confirm the relevance of each article to the research questions, to determine methodological characteristics pertaining to study quality, and to collect evidence pertinent to the research questions.

Grading of the Evidence

After all articles were reviewed, the quality of the evidence supporting each question was graded on the basis of its quality, quantity, and consistency (see Table 1). In terms of quality, the articles were examined by two criteria: study design and the presence of an objective assessment of outcomes. To meet the quality criteria for Grade A, there must have been at least one randomized controlled trial and at least 75 percent of the studies must have used an objective assessment method. To meet the quality criteria for Grade B, there must have been at least one controlled trial (not necessarily randomized) AND at least 50 percent of studies must have used an objective assessment method. To meet the quality criteria for Grade C or D, there did not need to be any controlled trials and less than 50 percent of studies could have used an objective assessment to measure outcomes.

In terms of quantity, there had to be at least four studies to meet criteria for Grade A, three to meet the criteria for Grade B, two to meet the criteria for Grade C, or fewer than two studies to meet the criteria for Grade D. In terms of consistency, the results of the studies had to be consistent to meet the criteria for Grade A, reasonably consistent to meet the criteria for Grade B, and inconsistent to meet the criteria for Grade C. Where there where too few studies to judge the consistency the article was assigned Grade D. The grading of the evidence was discussed at a team meeting and consensus was reached on each criterion. The evidence received a final “grade” that reflected the lowest rank on each of the four criteria (two for quality and one each for quantity and consistency).

Peer Review

Throughout the project, feedback was sought from the technical experts through formal and ad hoc requests for guidance. A draft of the completed report was sent to the technical experts, as well as to the partner (NQF), AHRQ and other peer reviewers. Substantive comments were catalogued and entered into a database. Revisions were made to the evidence report as warranted, and a summary of the comments and their disposition was submitted to AHRQ with the final report.