Key Findings and Strength of Evidence

KQ 2. Overall Effectiveness

Of the 44 studies included in this review, we rated 16 as high risk of bias overall; that is, concerns about randomization failure, confounding, or overall attrition increased the risk of bias for all outcomes. In addition, we rated some studies that were otherwise of low or medium risk of bias as high for individual outcomes, chiefly because of measurement or detection bias related to the specific outcome. These instances are specified in the relevant section the Results chapter

We rated the strength of evidence for each outcome from low- or medium risk-of-bias studies when available. MTM significantly improved some measures of medication adherence, medication appropriateness assessed in general and medication dosing (Table 82). However, we did not find evidence of benefit for any other intermediate outcomes on which we had data. No studies addressed either goals of therapy or patient engagement.

Table 82

Summary of findings and strength of evidence for intermediate outcomes of MTM interventions.

Similarly, we did not have evidence of benefit for most patient-centered outcomes, including adverse drug events or mortality (Table 83). MTM did not improve most measures of health-related quality of life (low strength of evidence for no benefit). We graded the “vitality” and “emotional role functioning” domains of the Medical Outcomes Study Short-Form (SF36) questionnaire as insufficient for this domain. For the SF-36, neither the other six domains nor the two component scores (physical health, mental health) showed significant benefit from MTM interventions. The various patient satisfaction items also showed no impact from MTM programs (low strength of evidence for no benefit). We found no evidence on activities of daily living, work or school absenteeism, and patient and caregiver participation in medical care and decisionmaking.

Table 83

Summary of findings and strength of evidence for patient-centered outcomes of MTM interventions.

Outcomes related to using health resources were similarly not much influenced by MTM interventions (Table 84). Two exceptions may merit attention: (1) health plan expenditures on medication costs and (2) the proportion and costs of hospitalization for patient with diabetes. In both instances, MTM interventions improved outcomes. MTM trials implemented in settings with a broad range of patients did not show a consistent signal of reduction in the number of hospitalizations but a single cohort study that partially addressed confounding inherent in studies of refusers and acceptors found a lower mean number of inpatient visits for patients accepting MTM when compared with patients refusing MTM. Overall, we judge the strength of evidence for this outcome to be insufficient owing to lack of consistency.

Table 84

Summary of findings and strength of evidence for resource-utilization outcomes of MTM interventions.

Over all three categories of outcomes, each of which had a substantial number of individual measures, MTM improved outcomes in only a few instances. Study limitations, lack of consistency, and lack of precision of the estimates of effects limited the strength of evidence considerably. As discussed later, even the minimal findings of effectiveness are at best only narrowly applicable.

Findings in Relation to What Is Already Known

Our findings contrast with conclusions that Chisholm-Burns and colleagues reached in a recent systematic review.⁹⁵ In that review, the authors concluded that “Pharmacist-provided direct patient care has favorable effects across various patient outcomes, health care settings, and disease states.”^{95, p. 923} Several differences between the Chisholm-Burns review and the current review may account for the discrepant conclusions. First, the Chisholm-Burns review included all studies that cited evidence of pharmacist involvement in direct patient care. The interventions examined included chronic disease management and prospective and retrospective drug utilization review; we excluded these types of efforts because of the clinical heterogeneity those interventions would have introduced into the review. Notably, the Chisholm-Burns review did not use the term “medication therapy management” to categorize the interventions in the articles they reviewed. Second, approximately 30 percent of the papers in the Chisholm-Burns review were conducted entirely in institutional settings. In contrast, we did not identify any studies within institutional settings that met our MTM intervention definition criteria. Third, the Chisholm-Burns review included a total of 298 articles and did not omit from their analyses studies with a high risk of bias; by contrast, we based our strength-of-evidence grades in this review on only those studies with no more than medium risk of bias. Thus, a direct comparison of findings between these two reviews would be ill advised.

The striking differences between the conclusions reached in these two reviews emphasize two important needs for efforts to systematically review MTM programs. The first is for researchers to specify the MTM intervention based on existing definitions, taxonomies, or service models. The second is to develop consensus guidelines for describing intervention features and fidelity of intervention delivery in publications reporting findings from evaluation studies. Progress on these two steps would enable systematic reviews to differentiate better between different types of services and avoid the problem of overgeneralizing review results.

Applicability of the Findings

This body of evidence has significant clinical and methodological heterogeneity, which limits the ability to make any universal statements about effectiveness.³⁶ However, the range of study designs, which includes RCTs, nonrandomized trials, and cohort studies, enhances the applicability of findings for real-world settings. Included studies ranged from relatively small interventions in single clinics provided by a single interventionist to evaluations of MTM services delivered on a large scale through integrated health systems or health plans as a Medicare Part D or other drug plan benefit. This diversity of studies enhanced the applicability of findings to a wide variety of settings, including outpatient clinics, community pharmacies, and centralized pharmacy call centers. A few studies conducted outside the United States included MTM as part of a home visits program; findings from this model may not be directly applicable within the United States.

The studies in this review are broadly applicable to a range of chronically ill, adult patient populations. The majority of interventions were directed at populations with multiple and common chronic conditions, such as diabetes, chronic heart failure, and hypertension. Several specifically targeted adults aged 65 years or older. Few studies reported sociodemographic characteristics beyond age and sex; thus, the applicability of findings to specific populations (e.g., rural, low socioeconomic status, cognitively impaired, uninsured) is unknown. The nature of the MTM intervention, which includes involving patients as active participants in the process, limits the extent to which findings can be generalized beyond patients who agreed to participate in such interventions. Patients who agree to participate may be systematically different from those who decline to be in such a program. For that reason, the impact of such interventions at a population or health-plan level may be limited by the degree of uptake among interested patients.

The intervention used across most studies can be characterized as complex and moderately resource intensive. Components involve identifying applicable patients; initially assessing patients; providing counseling, education, and care coordination; and following patients over time. These services were provided per protocol in some studies and as needed or ad hoc in others. Most studies described intervention components in terms of “pharmaceutical care model” components or Medicare Part D MTM program criteria, but few provided detailed descriptions or measurement of implementation fidelity.

All studies included comparator arms with usual medical care or pharmacy care. Usual care does not typically include distinct MTM services by health care providers other than prescribing providers (not common for the time period covered by most of the studies). Models of collaborative health care delivery are evolving, and the changing roles and training of pharmacists increase the potential applicability of MTM interventions in future models of health care.

The broad sets of outcomes evaluated across this body of evidence spanned a substantial range of both intermediate and health outcomes as well as outcomes related to resource use. Proximal and intermediate outcomes included number of drugs, identification of drug therapy problems, appropriateness of medication prescribing, and laboratory or biometric markers of disease control (e.g., hypertension, hemoglobin A1c, low-density lipoprotein cholesterol). Patient-centered outcomes focused on numerous measures of quality of life as well as adverse drug events. Many studies also reported outcomes involving health care resource use and expenditures (e.g., number and costs of hospitalizations, emergency department visits, outpatient visits).

Most studies did not, however, clearly indicate the expected, desired, or intended direction of effect on most resource use outcomes, making the applicability of using these interventions to reduce drug-related or overall health care costs or expenditures difficult to assess. For example, whether one should expect the number of medications prescribed or drug costs for heart failure to increase or decrease under the careful scrutiny of an MTM intervention is not clear because the desired impact will be based on the goal of therapy for each individual.

The focus of outcome measurement in many studies was the short-term identification and characterization of drug therapy problems and their resolution; these endpoints are thought to be the outcomes most sensitive to change as a result of receiving MTM services. However, by design, because identification of drug therapy problems is a part of the MTM intervention itself, differences between the nature of the intervention and that of the control group mean that measuring these outcomes cannot be as rigorous in a usual care comparison group as it is in the intervention group. In fact, many studies were able to measure only changes in this outcome in the intervention group. Hence, many studies failed to demonstrate a direct analytic link between the resolution of drug therapy problems as a result of MTM and impact on intermediate outcomes, patient-centered outcomes, and resource utilization. Thus, the applicability of studies that demonstrate an impact on the resolution of drug therapy problems is limited.

Implications for Clinical Practice and Policymakers

Although we found the evidence insufficient in general to draw definitive conclusions about the comparative effectiveness of MTM for most outcomes that we evaluated, our findings do suggest some implications for practice and policy. MTM is already in widespread practice and is now shaped in the United States largely by Medicare Part D policy: this presents both challenges and opportunities. MTM programs sponsored and administered by Part D drug benefit plans are often centrally administered and delivered primarily by phone and may be less integrated into routine health care than some of the interventions included in our review. MTM programs of the future have the potential to be more integrated into routine health care through participation in accountable care organizations or patient-centered medical home models. We were unable to answer definitively whether level of integration matters for effectiveness, but policymakers may need to consider expectations about the impact that MTM might have on patient-centered outcomes and resource use in the context of other health care delivery transformation activities or quality improvement initiatives that are also occurring. More integration of MTM services with other activities may be effective; however, the more integrated MTM becomes within routine medical care, the more difficult it becomes to isolate it as a discrete intervention for evaluation.

Policymakers could thus consider whether MTM services should be positioned as a contributor to overall improvement in processes of care, health status, and costs or positioned as an intervention to which effects can be discretely attributed. As noted earlier, improvements in medication appropriateness or drug therapy regimens may not always translate into improvements in health or costs, and even if they do, secular trends in related quality improvement (e.g., medication adherence interventions, regulatory requirements for medication reconciliation, electronic health record meaningful use incentives) may make measuring outcomes attributable to MTM very challenging.

Future training of MTM providers would benefit from a better understanding of which MTM components really matter. At the moment, such information is lacking. Policymakers and funders who wish to understand the comparative effectiveness of different MTM components could encourage rigorous program evaluation designs that fit within the context of the real-world implementation of these programs. For example, positive deviance analyses⁹⁶ with rigorous measurement of implementation features or stepped wedge trial designs⁹⁷ may be useful approaches.

A typical approach for evaluating complex interventions is to identify the “core” components for standardization, while allowing for flexibility for peripheral components or variations in implementation. In complex practice-based innovations, such flexibility may reflect desirable (or unavoidable) adaptations to local circumstances. Policy governing MTM programs may warrant modifications to permit investigators to conduct rigorous and innovative evaluative designs to identify core components or effectiveness-enhancing modifications. As future research and evaluation elucidates these components or enhancements, policy will need to evolve to keep pace with best practices.

Finally, considering both patients' and prescribers' perspectives in future design and delivery of MTM services may be needed. In our current analytic framework, MTM interventions require a significant element of engagement by both patients and prescribers if the interventions are to have a reasonable likelihood of improving outcomes. Although “opt in” strategies may increase the reach of such interventions, keeping patients (and their prescribing providers) engaged in the intervention over a reasonable amount of time may be the key to translating the potential of MTM interventions into actual improvements. Further refinement of eligibility criteria based on evidence to provide interventions to those most at risk from drug-related problems and therefore most likely to benefit may also be warranted.

Limitations of the Comparative Effectiveness Review Process

The constraints for populations, interventions, and settings that we imposed on this systematic review may limit its applicability as discussed above. During topic refinement and based on technical expert panel inputs and public comment, we expanded the scope by removing an exclusion criterion that would have required MTM interventions to have been directed at a patient population with two or more chronic conditions. As a result, we did include studies that focused on one chronic condition. Because of the prevalence of certain chronic conditions in the adult population, and particularly among Medicare beneficiaries, we think this decision was sensible and permitted us to examine a broader evidence base than would otherwise have been the case.

Although we tried to distinguish MTM from disease or case management interventions, making this distinction was challenging. We created a threshold for what intervention components were required to be present for this distinction. Specifically, we elected to emphasize whether the intervention entailed a comprehensive review of all medications; for that reason, we did not constrain studies of interest to those that targeted a single medication or drug regimen or that focused on a single condition such as diabetes or hypertension.

As described in the Methods chapter, when we were unable to determine which medications the interventionist had reviewed, we wrote to the authors for additional information. We chose to pursue authors in an effort to permit us to use studies that had been designed as MTM but did not describe the comprehensive medication review component in detail.

Our approach may have been overly inclusive because it led us to include studies that addressed a single disease, as long as the pharmacist reviewed all medications. For example, 10 of the 44 studies were relatively narrowly focused; two of these addressed patients with chronic heart failure and two addressed patients with either hypertension or hypertension and diabetes. The remaining six studies focused on, patients with diabetes, HIV, glucocorticoid-induced osteoporosis, or hemodialysis. The fact that did not require patients to have more than one clinical condition resulted in an approach that was inclusive of these more narrowly focused (albeit often termed “MTM”) studies and may render our results less applicable to MTM interventions targeted to patients with a wide range of chronic conditions.

Also based on feedback during the process of setting out the scope of this review, we chose to include interventions that were broader than the Medicare Part D MTM-defined interventions. Put another way, we broadened our view of patient populations and intervention criteria, and we allowed studies not conducted in the United States into the evidence base. This decision led us to include interventions described as “pharmaceutical care,” which were generally based on the pharmaceutical care model principles;¹⁰ it also permitted us to examine investigations with elements of pharmaceutical care or MTM that did not specifically label the intervention as either MTM or pharmaceutical care. These studies were often described as “clinical pharmacist interventions.”

Furthermore, all the non-U.S. studies involved interventions within single-payer health systems. Hence, the interventions in this review constitute a more heterogeneous group than if we had allowed only those labeled as Medicare Part D MTM programs. This is both a limitation and a strength. Although our approach makes results more challenging to interpret, it enhances our ability not to miss interventions that include MTM components but lack the descriptor term MTM.

Studies did not often explicitly describe certain MTM components. In cases when we could not determine whether investigators had provided certain MTM components (such as patient education and counseling, or coordination with other health care providers), we contacted the authors to gain additional information that would allow us to make an informed decision. We were fairly permissive in interpreting the presence of the MTM intervention components other than comprehensive medication review. The main reason is that we recognized that terms describing some components have evolved over time and may have been absent from the lexicon in earlier years or implicitly conveyed by authors by simply using the terms “MTM” or pharmaceutical care to describe their intervention.

Our approach to categorizing interventions for KQ 1 relied primarily on the short descriptions in published manuscripts and those we were able to obtain via email inquiries. Their similarities or differences substituted for any overarching taxonomy, because none that we considered seemed to fit our purpose. Thus, we have introduced intervention labels that, admittedly, do not fully describe or account for clinical heterogeneity among interventions. This approach limits our ability to make definitive statements about the effectiveness of various intervention components. We believe that the clusters and categorizations we used are useful heuristics, but some may regard them more as hypothesis generating than as reflecting settled principles of classification.

Finally, our search process was complicated by having to ensure coverage of all terms that could be used to describe MTM interventions over time. Adding to this challenge was our effort to examine the gray literature, where we thought we might find studies tilted toward effectiveness and real-world program evaluation. As it turned out, studies of these types of interventions were not indexed similarly; for that reason, we needed to rely heavily on hand searches of citation lists from key background articles to identify possibly relevant studies for inclusion. Thus, we may have missed some studies that might have qualified for inclusion. Given the considerable diversity in the evidence base we did have, however, we do not think that any potentially missed studies would have changed our conclusions in any material way. No meta-analyses included more than five studies; as a result, we did not examine included studies for publication bias quantitatively.

Limitations of the Evidence

As a body of evidence, the MTM literature evaluated in this review has measured numerous outcomes. As indicated in previous sections, very few outcomes, with the exception of harms, remain completely unexamined. Of the 44 studies in this review, we rated 28 as having medium, low, or mixed risk of bias. The 44 studies included 21 trials and 4 nonrandomized controlled studies. In other words, the literature on this topic is not marked by failure to consider important outcomes, universally high risk of bias, or pervasively weak designs.

Despite these advantages, we were unable to identify sufficient evidence on the majority of hypothesized outcomes of MTM. In several instances, our inability to rate evidence as higher than insufficient came from indirect, inconsistent, and imprecise evidence. The choice of outcome measures in this body of evidence limited our ability to come to conclusions in some instances. For example, some studies did not focus on changes that proponents might expect MTM services to produce. Because effective MTM can either increase or decrease expenditures or use of services based on the needs of the patient, studies that did not prespecify the expected direction of change had no way to interpret their results as an appropriate change. Studies that demonstrated inconsistent results in direction of change (i.e., some showing an increase in resource use and others showing a decrease) may well have been consistent in terms of appropriate change, but because they generally failed to establish a priori the direction in which they expected to find an effect, we rated such evidence as indirect and inconsistent.

Similarly, studies often used nonstandardized or idiosyncratic measures for outcomes such as adverse events, adherence, and expenditures or costs; this tendency limited our ability to meta-analyze results. When studies focused on specific outcomes, they were often significantly underpowered to detect differences between groups (that is, they did not meet optimal information size criteria). As a result, we rated several studies as imprecise.

MTM intervention studies are largely practice based and incorporate substantial heterogeneity in specific intervention elements and in patient populations targeted. Yet the evidence is sharply constrained in its ability to inform questions of the effectiveness of specific MTM components or intervention features (KQ 3 in our review) because study designs did not often capitalize on variants in MTM programs for a prospective evaluation of outcomes by those variants. Neither did they measure fidelity to intended MTM elements for post-hoc evaluation. Similarly, the relatively untargeted nature of the MTM interventions meant that, in many studies, only small numbers of patients had any one specific condition, and most studies did not measure patient characteristics beyond age and sex, thus limiting our ability to address KQ 4 in our review. For this reason, the evidence we identified for this review was most relevant for KQ 2.

Research Gaps

In many bodies of research, questions regarding the comparative effectiveness of specific intervention components or implementation features are best answered after clear evidence of the effectiveness of the intervention relative to usual care has been established. Our review largely indicates insufficient evidence on the primary question of effectiveness relative to usual care. By definition, this limited what we could say about comparative effectiveness.

Nonetheless, the widespread implementation of MTM coexists with the urgent need for actionable information for policy, program policies, and training. This clinical and policy environment means that new research cannot afford to address causal claims relative to usual care first, followed by comparative effectiveness of the intervention elements in a relatively controlled environment, and finally, program evaluation of real-world implementation, all in sequential order.

In prioritizing among various research goals, therefore, funders may wish to consider the relative value of new evidence on overall effectiveness, effectiveness of implementation features, and program implementation and accountability. Trial research in narrow clinical settings can address questions of effectiveness but may lack applicability to real-world implementation. Likewise, evaluations of real-world programs with variable fidelity to interventions can answer questions about process and implementation, but they offer limited information on effectiveness. Research prioritization exercises will also need to account for already commissioned MTM intervention studies.

For new studies focusing on causal claims, a critical gap relates to the failure to specify the expected direction of effect. New research requires a strong theoretical foundation to help specify causal mechanisms and hypothesized effects. Without such an edifice, future research will continue to produce inconsistent and uninterpretable results.

Heightened attention to causal mechanisms will also help researchers convey their understanding of what outcomes these types of interventions are likely to influence. For instance, how should researchers wishing to establish direct causal links between MTM programs and outcomes evaluate distal outcomes such as patient-centered outcomes and resource utilization? This effort requires a better understanding of the relationship between proximal outcomes like “drug therapy problems identified and resolved” and distal outcomes. For instance, MTM may reduce outpatient visits to address side effects. MTM may also result in the need for further testing and evaluation for some patients, which could, in turn, result in more rather than fewer outpatient visits. Unless the nature of change resulting from MTM is specified in relation to goals of drug therapy, studies cannot assert benefit or harm. Further, drug therapy problems are diverse and may not all have the same causal relationship to health, quality of life, patient satisfaction, or resource use outcomes. Furthermore, a causal model of these distal outcomes may need to take into account the competing or complementary contributions of MTM, new models of health care delivery (e.g., patient-centered medical homes), and other quality improvement interventions.

Investigators embarking on new studies focusing on causal links between MTM and outcomes may wish to consider the limitations of studies based on secondary data from existing MTM programs that use opt-in/opt-out patient enrollment mechanisms. Although these studies may provide invaluable information on process measures such as patient engagement, underlying issues of confounding severely limit the validity of causal claims from such studies.

Regardless of the goal of their future research, investigators should consider issues of sample size to ensure precision of their results. This issue is particularly relevant when evaluating outcomes likely to occur in smaller subgroups defined by patient risk, context, or highly adapted intervention features. Innovative designs (e.g., stepped wedge trials, statistical process control, time series analysis, simulations, and factorial experiments) may permit both rigor and adequate sample size within the context of real-world implementation. With careful attention to fidelity, new studies may also inform questions of the effectiveness of intervention components and implementation features. Mixed-methods approaches may allow more information on variations in context and implementation. Such designs may also help inform our understanding of critical training elements for MTM service providers.

Regarding research gaps for specific outcomes such as patient satisfaction, measures specific to the types of services provided through MTM (e.g., patient education about medications) or to the proximal outcomes that MTM is intended to achieve (e.g., reduced medication side effects, improved disease control) may offer better insights into the effects of MTM. Similarly, a medication-related instrument may better measure patients' concerns that are directly related to medication use (e.g., experience of side-effects, intrusiveness of the medication regimen) than generic tools.

Conclusions

We included 44 eligible studies (21 randomized controlled trials, 4 controlled clinical trials, and 19 cohort studies) reported in 61 articles, described in detail in the report (KQ 1). Evidence was insufficient on the effect of MTM on most outcomes (KQ 2). In a few instances, described below, the evidence led us to conclude with a low strength of evidence either a benefit or lack of benefit. Specifically, we found evidence that MTM results in improvement when compared with usual care for some measures of medication adherence and appropriateness, medication dosing, health plan expenditures on medication costs, and the proportion and costs of hospitalization for patient with diabetes. Similarly, we conclude based on a low strength of evidence that MTM conferred no benefit for patient satisfaction and most measures of health-related quality of life. We found evidence on five intervention components and intervention features (KQ 3): one study provided information on each feature and yielded insufficient evidence for most outcomes with two exceptions. MTM programs with pharmacist access to brief clinical summaries from the medical record reduced the mean number of adverse drug events when compared with basic MTM programs without such access (low strength of evidence). Community pharmacists increase the generic dispensing ratio more than call-center–based pharmacists (low strength of evidence). We found no relevant studies on patient characteristics moderating the effect of MTM interventions (KQ 4). Similarly, the evidence on harms associated with MTM was limited to one study on inconvenience and was rated as insufficient (KQ 5).

The evidence base offers low evidence of benefit for a limited number of intermediate and health utilization outcomes. We graded the evidence for most outcomes as insufficient because of inconsistency in direction, magnitude, and precision, rather than lack of evidence. Wide variations in populations and interventions, both within and across studies, likely explain these inconsistencies. Given the widespread implementation of MTM and urgent need for actionable information, optimal investments in new research require a process of research prioritization in which the value of information from each proposed study is carefully considered. Studies designed to identify causal relationships between MTM interventions and their outcomes require adequate controls for confounding but may offer limited information on what explains program success or failure. Studies designed to explore the reasons for program success or failure using qualitative or single-arm designs may offer hypotheses-generating rather than hypotheses-confirming insights on MTM effectiveness. New research, regardless of specific focus, will likely continue to find inconsistent results until underlying sources of heterogeneity are accounted for.