Details of Study Selection

Data Extraction

We created and pilot tested standardized forms for data extraction. Each Web resource or article underwent double review by the study investigators for data abstraction. The second reviewer confirmed the first reviewer’s abstracted data for completeness and accuracy. A third reviewer audited a sample of articles by the first two reviewers to ensure consistency in the data abstraction of the articles.

For all articles, reviewers extracted information on general study characteristics (e.g., study design, study period, and follow-up), study participant characteristics, eligibility criteria, interventions, outcome measures and the method of ascertainment, and the results of each outcome, including measures of variability. We completed the data abstraction process using forms created in Excel (Microsoft, Redmond, WA). We used the Excel files to maintain the data and to create detailed evidence tables and summary tables.

Risk of Bias Assessment of Individual Quantitative Studies

Two reviewers independently assessed risk of bias for each quantitative study. For RCTs, we used the Cochrane Risk of Bias Tool, Version 2.³¹ For non-randomized studies, we used the Cochrane Risk of Bias Assessment Tool for Non-Randomized Studies of Interventions (ROBINS-I tool).³² Differences between reviewers were resolved through consensus.

We assessed the individual risk of bias for RCTs using five items:

• Risk of bias arising from the randomization process;
• Risk of bias due to deviations from the intended interventions: effect of assignment to intervention, and effect of adhering to intervention;
• Risk of bias due to missing outcome data;
• Risk of bias in measurement of the outcome;
• Risk of bias in selection of the reported result.

Following the ROB2 guidance, concerns were expressed only about issues that are likely to affect the ability to draw reliable conclusions from the study. In reaching final judgements, the following considerations applied: judgement of ‘High’ risk of bias for any individual domain will lead to the result being at ‘High’ risk of bias overall, and a judgement of ‘Some concerns’ for any individual domain will lead to the result being at ‘Some concerns’, etc.

We assessed the individual risk of bias for non-randomized and cohort studies using 7 items:

• Bias due to confounding;
• Bias in selection of participants into the study;
• Bias in classification of interventions;
• Bias owing to deviations from intended interventions;
• Bias owing to missing data;
• Bias in measurement of outcomes;
• Bias in selection of the reported results.

Following the ROBINS guidance, judgements were made using the following algorithm:³²

• low risk of bias: the study is judged to be at low risk of bias for all domains,
• moderate risk of bias: the study is judged to be at low or moderate risk of bias for all domains,
• serious risk of bias: the study is judged to be at serious risk of bias in at least one domain, but not at critical risk of bias in any domain,
• critical risk of bias: the study is judged to be at critical risk of bias in at least one domain,
• no information: there is no clear indication that the study is at serious or critical risk of bias and there is a lack of information in one or more key domains of bias (a judgement is required for this).

Assessment of Quality of Qualitative Studies

For qualitative studies, we conducted quality assessment, as risk of bias is not relevant. We used the Joanna Briggs Institute Quality Appraisal Checklist^{33, 34} to address elements specific to our key questions. Two reviewers independently assessed the methodological quality and resolved differences through consensus.

Data Synthesis and Analysis

For part (b) of each Key Question, we created a set of detailed evidence tables containing all information extracted from eligible studies (see Appendix D). These tables include details of what is included in the interventions; for example, for models of care, details extracted include what disciplines are involved, mode of contact, and content of the intervention. Tables also include details of implementation of the interventions as described in these studies, such as clinician training provided. We synthesized all studies qualitatively. We conducted meta-analyses for outcomes with at least three studies and the studies were sufficiently homogeneous with respect to key variables (population characteristics, study duration, and intervention). Randomized controlled trials and nonrandomized studies were analyzed separately. Statistical significance was set at a two-sided alpha of 0.05. Statistical heterogeneity among studies was evaluated using an I² statistic and anticipated statistical heterogeneity. For continuous outcomes, a standardized mean difference was calculated using a random-effects model with DerSimonian and Laird formula. All meta-analyses was conducted using STATA version 14 (College Station, TX).

For part (c) of each key question, we summarized the results of the qualitative studies into categories for each KQ, informed by discussions with our Key Informants. We conducted a review of the qualitative studies to address mechanisms and context for part (c) of each KQ where studies were identified. We based our methods on the 2017 Cochrane guidance, Qualitative and Implementation Methods Group Guidance Paper 5: Methods for integrating qualitative and implementation evidence within intervention effectiveness reviews³⁵ and Joanna Briggs Institute methods for mixed methods systematic reviews.³⁶

For parts (b) and (c) of each key question, model definitions were derived from previous work and revised based on consensus.³⁷ Once established, two researchers independently reviewed each citation to determine model type.

Finally, we completed an integrative review. The Cochrane guidance defines the integrative review as “combining the findings from different types of studies to produce a more comprehensive synthesis of the evidence on ‘what works’”, recognizing that a variety of contextual factors, such as characteristics of the local population or setting, are key to intervention implementation and effectiveness (under “real world” conditions). Through the incorporation of qualitative and mixed methods research, the integrative review process can incorporate the patient and caregiver perspective, which is critical for palliative care, and the practicing clinician and health system perspective, which is critical for the integration of palliative care in the ambulatory setting. We completed integration by juxtaposing the findings from the grey literature (part (a) in each question) with the systematic review (part (b) in each question) with the identified categories from the review of qualitative studies (part (c) of each question). We focused particularly on KQ3 and KQ5 where studies were identified across all parts. We integrated categories of what is available (e.g., components of what is included in integrated palliative care interventions) from qualitative studies with evidence from effectiveness studies. We used categories informed by models of what is included in integrated ambulatory palliative care³⁸, the Consolidated Framework for Implementation Research (CFIR) adapted for complex interventions³⁸, a prior AHRQ project on key implementation factors for quality and safety studies³⁹, and refined through Key Informant input. This process helped address, in particular, the elements of the part (c) questions on why and how some types of interventions may be effective and others are not, when and which patients may benefit from these interventions, and how palliative care approaches can best be integrated into ambulatory care.

Grading the Strength of the Body of Evidence

At the completion of our systematic review, we graded the strength of evidence on critical outcomes for quantitative studies by using the grading scheme recommended by the Methods Guide for Conducting Comparative Effectiveness Reviews. We defined the critical outcomes as those most important for making decisions; we identified these a priori with input from the Technical Expert Panel.

The critical outcomes include:

• Patient health-related quality of life
• Patient symptom burden
• Patient symptoms of depression
• Patient satisfaction
• Caregiver satisfaction
• Advance directive documentation

Following this standard EPC approach, for each critical outcome, we assessed the number of studies, their study designs, the study limitations (i.e., risk of bias and overall methodological quality), the directness of the evidence to the Key Questions, the consistency of study results, the precision of any estimates of effect, the likelihood of reporting bias, and the overall findings across studies. Based on these assessments, we assigned a strength of evidence rating as being either high, moderate, or low, or insufficient evidence to estimate an effect or draw a conclusion (Table 5). Investigators writing each section completed the strength of evidence grading. The team members reviewed the assigned grade and conflicts were resolved through consensus. We used the grading scheme recommended in the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews (Methods Guide). We considered the following domains: study limitations, directness, consistency, and precision.⁴⁰

We classified the strength of evidence pertaining to the KQs into four categories:

• High (high confidence that the evidence reflects the true effect and further research is very unlikely to change our confidence in the estimate of effect)
  One or more RCTs
  Low study limitations
  Direct, consistent, and precise
• Moderate (moderate confidence that the evidence reflects the true effect, and further research may change our confidence in the estimate of effect)
  One or more RCTs
  Low study limitations, and some concerns
  Direct, consistent, and precise
• Low (low confidence that the evidence reflects the true effect and further research is likely to change our confidence in the estimate of the effect and is likely to change the effect estimate)
  One or no RCT
  High study limitations or some concerns
  At least two of the following: indirect, inconsistent, or imprecise
• Insufficient (evidence is unavailable or insufficient to assess with any confidence).
  One or no RCT
  High study limitations for RCTs or serious or critical study limitations for a cohort study
  At least two of the following: indirect, inconsistent, or imprecise

Peer Review and Public Commentary

We invited experts in palliative care and individuals representing stakeholder and user communities to provide external peer review of this review; AHRQ and an associate editor also provided comments. We posed the revised draft report on the AHRQ website for four weeks to elicit public comment (posted 5 August 2020). Reviewer comments were addressed, revising the report as appropriate. A disposition of comments table of peer and public comments was posted on the EHC website three months after the Agency posted the final review.

Definition of Terms

The following definitions are used in this report.

Ambulatory settings

Includes settings such as hospital outpatient departments and clinicians’ offices, particularly primary care, but also including geriatrics, nephrology, pulmonology, cardiology and neurology

Chronic illness

An illness that lasts one year or more and requires ongoing medical attention and/or limits activities of daily living.

Clinician

A healthcare professional qualified in the clinical practice of medicine, such as physicians, nurses, pharmacists, social workers, or other allied health professionals.⁴¹

Consultative care model

An approach to care delivery where a clinician serves in a consultant role with provision of palliative advice and does not necessarily assume primary responsibility of care.⁴²

Educational materials and resources

Include pamphlets, curricula, Web sources, and videos designed to provide information about integrating palliative care and palliative care options in ambulatory care.

Guidelines and position statements

Clinical practice guidelines and position statements from key U.S. health care professional and other organizations specifically relevant to integrating palliative care into serious illness chronic care.

Integrative review

This method allows for the combination of diverse methodologies.⁴³ We use this approach to examine qualitative and process evaluation literature (such as interviews with patients and families and implementation studies) to address how interventions work and evidence for how they should best be included in care, and to integrate this with the effectiveness literature. Combining the findings from different types of studies to produce a more comprehensive synthesis of the evidence on ‘what works’ and how.³⁵

Multimodal interventions

For the purposes of this review, combinations of the different types of included specific interventions: identification of patients, education for patients and caregivers, shared decision-making tools, and/or clinician education.

Models

Care delivery structures.

Palliative care

Care, services, or programs for patients with serious life-threatening illness and conditions and their caregivers, with the primary intent of relieving suffering and improving health-related quality of life, including dimensions of physical, psychological/emotional, social, and spiritual well-being.³⁶ Note that other terms, such as supportive care, may be similarly used. Hospice care is a type of palliative care but is not included in this review as it is not delivered in ambulatory care.

Patient education

This can be conducted either individually or as part of a group or community, including through methods such as in-person, telephone, online or other electronic, print or audio-visual educational materials.³⁷

Prediction models

Modeling of patient and illness factors to predict the likelihood of patient outcomes, such as hospitalizations.

Primary palliative care

Care in palliative care domains for relevant populations provided by non-palliative care specialists, such as by primary care clinicians.⁴⁴

Process evaluation (also a type of implementation study)

Research focusing on mechanisms (how and why something can be successfully implemented) and contextual issues (population, setting, barriers and facilitators).³⁵ Process evaluation studies include process studies that report on why and how interventions work with similar interventions, health conditions and contexts.⁴⁵ They may be:

• conducted alongside effectiveness studies
• conducted after the effectiveness study on the same groups
• unrelated to effectiveness studies

Provider education

Used to describe a variety of interventions including educational workshops, meetings (e.g., traditional Continuing Medical Education [CME]), lectures (in-person or computer-based), educational outreach visits (by a trained representative who meets with providers in their practice settings to disseminate information with the intent of changing the providers’ practice). The same term also is used to describe the distribution of educational materials (electronically published or printed clinical practice guidelines and audio-visual materials).⁴⁶ This review focuses on materials that include education about integrating palliative care into ambulatory care.

Shared care model

An approach to care delivery where there is joint participation of non-palliative clinicians and palliative care clinicians working together in relation to an individual’s care. Shared care models may also include systematic cooperation where different systems work together with various levels and disciplines of clinicians.⁴⁷

Shared decision-making tools

These are patient-facing and/or clinician-facing tools to help make decisions that reflect medical evidence and patient goals for care relevant to palliative care, such as advance care planning tools to aid with decisions about treatment options and preferences for future care.⁴⁸ For the purposes of this review, we focused on tools for serious illnesses and conditions in ambulatory care.

Triggers

Also known as screening criteria; indicators that someone may benefit from palliative care services. These may include patient or disease characteristics, palliative care needs, functional status decline or persistent or worsening symptoms, or high health care needs.

Website

A collection of Web pages which are grouped together and connected.

Webpage

Document which can be displayed in a Web browser.

Web resource

Specific resource listed on a Web page.

References

1.

DistillerSR. Evidence Partners; 2020. https://www​.evidencepartners.com/. Accessed on April 14, 2020.

2.

Aslakson RA, Dy SM, Wilson RF, et al. Patient- and Caregiver-Reported Assessment Tools for Palliative Care: Summary of the 2017 Agency for Healthcare Research and Quality Technical Brief. J Pain Symptom Manage. 2017 Dec;54(6):961–72.e16. doi: 10.1016/j.jpainsymman.2017.04.022. PMID: 28818633. [PubMed: 28818633] [CrossRef]

3.

Butler J, Khan MS, Mori C, et al. Minimal clinically important difference in quality of life scores for patients with heart failure and reduced ejection fraction. Eur J Heart Fail. 2020 Apr 2doi: 10.1002/ejhf.1810. PMID: 32239794. [PubMed: 32239794] [CrossRef]

4.

Spertus J, Peterson E, Conard MW, et al. Monitoring clinical changes in patients with heart failure: a comparison of methods. Am Heart J. 2005 Oct;150(4):707–15. doi: 10.1016/j.ahj.2004.12.010. PMID: 16209970. [PubMed: 16209970] [CrossRef]

5.

Astrup GL, Hofso K, Bjordal K, et al. Patient factors and quality of life outcomes differ among four subgroups of oncology patients based on symptom occurrence. Acta Oncol. 2017 Mar;56(3):462–70. doi: 10.1080/0284186x.2016.1273546. PMID: 28077018. [PubMed: 28077018] [CrossRef]

6.

Lyons KD, Bakitas M, Hegel MT, et al. Reliability and validity of the Functional Assessment of Chronic Illness Therapy-Palliative care (FACIT-Pal) scale. J Pain Symptom Manage. 2009 Jan;37(1):23–32. doi: 10.1016/j.jpainsymman.2007.12.015. PMID: 18504093. [PMC free article: PMC2746408] [PubMed: 18504093] [CrossRef]

7.

Gonzalez-Saenz de Tejada M, Bilbao A, Ansola L, et al. Responsiveness and minimal clinically important difference of the Minnesota living with heart failure questionnaire. Health Qual Life Outcomes. 2019 Feb 14;17(1):36. doi: 10.1186/s12955-019-1104-2. PMID: 30764842. [PMC free article: PMC6376687] [PubMed: 30764842] [CrossRef]

8.

Holden SK, Koljack CE, Prizer LP, et al. Measuring quality of life in palliative care for Parkinson’s disease: A clinimetric comparison. Parkinsonism Relat Disord. 2019 Aug;65:172–7. doi: 10.1016/j.parkreldis.2019.06.018. PMID: 31253494. [PMC free article: PMC6774894] [PubMed: 31253494] [CrossRef]

9.

Naglie G, Hogan DB, Krahn M, et al. Predictors of patient self-ratings of quality of life in Alzheimer disease: cross-sectional results from the Canadian Alzheimer’s Disease Quality of Life Study. Am J Geriatr Psychiatry. 2011 Oct;19(10):881–90. doi: 10.1097/JGP.0b013e3182006a67. PMID: 21946804. [PMC free article: PMC3267777] [PubMed: 21946804] [CrossRef]

10.

Kroenke K, Strine TW, Spitzer RL, et al. The PHQ-8 as a measure of current depression in the general population. J Affect Disord. 2009 Apr;114(1–3):163–73. doi: 10.1016/j.jad.2008.06.026. PMID: 18752852. [PubMed: 18752852] [CrossRef]

11.

Lowe B, Unutzer J, Callahan CM, et al. Monitoring depression treatment outcomes with the patient health questionnaire-9. Med Care. 2004 Dec;42(12):1194–201. doi: 10.1097/00005650-200412000-00006. PMID: 15550799. [PubMed: 15550799] [CrossRef]

12.

Williams JW, Jr., Slubicki MN, Tweedy DS, et al. VA Evidence-based Synthesis Program Reports. Evidence Synthesis for Determining the Responsiveness of Depression Questionnaires and Optimal Treatment Duration for Antidepressant Medications. Washington (DC): Department of Veterans Affairs (US); 2009. [PubMed: 21155203]

13.

Bedard G, Zeng L, Zhang L, et al. Minimal clinically important differences in the Edmonton symptom assessment system in patients with advanced cancer. J Pain Symptom Manage. 2013 Aug;46(2):192–200. doi: 10.1016/j.jpainsymman.2012.07.022. PMID: 23177724. [PubMed: 23177724] [CrossRef]

14.

Hui D, Shamieh O, Paiva CE, et al. Minimal clinically important differences in the Edmonton Symptom Assessment Scale in cancer patients: A prospective, multicenter study. Cancer. 2015 Sep 1;121(17):3027–35. doi: 10.1002/cncr.29437. PMID: 26059846. [PMC free article: PMC4595042] [PubMed: 26059846] [CrossRef]

15.

Irwin M, Artin KH, Oxman MN. Screening for depression in the older adult: criterion validity of the 10-item Center for Epidemiological Studies Depression Scale (CES-D). Arch Intern Med. 1999 Aug 9–23;159(15):1701–4. doi: 10.1001/archinte.159.15.1701. PMID: 10448771. [PubMed: 10448771] [CrossRef]

16.

Amtmann D, Kim J, Chung H, et al. Comparing CESD-10, PHQ-9, and PROMIS depression instruments in individuals with multiple sclerosis. Rehabil Psychol. 2014 May;59(2):220–9. doi: 10.1037/a0035919. PMID: 24661030. [PMC free article: PMC4059037] [PubMed: 24661030] [CrossRef]

17.

Haase I, Winkeler M, Imgart H. [Anchor-based ascertaining of meaningful changes in depressive symptoms using the example of the German short form of the CES-D]. Neuropsychiatr. 2016 Jun;30(2):82–91. doi: 10.1007/s40211-016-0184-z. PMID: 27300327. [PubMed: 27300327] [CrossRef]

18.

Lemay KR, Tulloch HE, Pipe AL, et al. Establishing the Minimal Clinically Important Difference for the Hospital Anxiety and Depression Scale in Patients With Cardiovascular Disease. J Cardiopulm Rehabil Prev. 2019 Nov;39(6):E6–e11. doi: 10.1097/hcr.0000000000000379. PMID: 30489438. [PubMed: 30489438] [CrossRef]

19.

Puhan MA, Frey M, Buchi S, et al. The minimal important difference of the hospital anxiety and depression scale in patients with chronic obstructive pulmonary disease. Health Qual Life Outcomes. 2008 Jul 2;6:46. doi: 10.1186/1477-7525-6-46. PMID: 18597689. [PMC free article: PMC2459149] [PubMed: 18597689] [CrossRef]

20.

Smid DE, Franssen FM, Houben-Wilke S, et al. Responsiveness and MCID Estimates for CAT, CCQ, and HADS in Patients With COPD Undergoing Pulmonary Rehabilitation: A Prospective Analysis. J Am Med Dir Assoc. 2017 Jan;18(1):53–8. doi: 10.1016/j.jamda.2016.08.002. PMID: 27624705. [PubMed: 27624705] [CrossRef]

21.

Kroenke K, Baye F, Lourens SG. Comparative Responsiveness and Minimally Important Difference of Common Anxiety Measures. Med Care. 2019 Nov;57(11):890–7. doi: 10.1097/mlr.0000000000001185. PMID: 31415337. [PubMed: 31415337] [CrossRef]

22.

Toussaint A, Husing P, Gumz A, et al. Sensitivity to change and minimal clinically important difference of the 7-item Generalized Anxiety Disorder Questionnaire (GAD-7). J Affect Disord. 2020 Mar 15;265:395–401. doi: 10.1016/j.jad.2020.01.032. PMID: 32090765. [PubMed: 32090765] [CrossRef]

23.

Munoz AR, Salsman JM, Stein KD, et al. Reference values of the Functional Assessment of Chronic Illness Therapy-Spiritual Well-Being: a report from the American Cancer Society’s studies of cancer survivors. Cancer. 2015 Jun 1;121(11):1838–44. doi: 10.1002/cncr.29286. PMID: 25712603. [PMC free article: PMC4441564] [PubMed: 25712603] [CrossRef]

24.

Dworkin RH, Turk DC, Wyrwich KW, et al. Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations. J Pain. 2008 Feb;9(2):105–21. doi: 10.1016/j.jpain.2007.09.005. PMID: 18055266. [PubMed: 18055266] [CrossRef]

25.

Oxberry SG, Bland JM, Clark AL, et al. Minimally clinically important difference in chronic breathlessness: every little helps. Am Heart J. 2012 Aug;164(2):229–35. doi: 10.1016/j.ahj.2012.05.003. PMID: 22877809. [PubMed: 22877809] [CrossRef]

26.

Horita N, Miyazawa N, Morita S, et al. Small, moderate, and large changes, and the minimum clinically important difference in the University of California, San Diego Shortness of Breath Questionnaire. Copd. 2014 Feb;11(1):26–32. doi: 10.3109/15412555.2013.808615. PMID: 23886071. [PubMed: 23886071] [CrossRef]

27.

Kupferberg DH, Kaplan RM, Slymen DJ, et al. Minimal clinically important difference for the UCSD Shortness of Breath Questionnaire. J Cardiopulm Rehabil. 2005 Nov–Dec;25(6):370–7. doi: 10.1097/00008483-200511000-00011. PMID: 16327533. [PubMed: 16327533] [CrossRef]

28.

Yost KJ, Eton DT, Garcia SF, et al. Minimally important differences were estimated for six Patient-Reported Outcomes Measurement Information System-Cancer scales in advanced-stage cancer patients. J Clin Epidemiol. 2011 May;64(5):507–16. doi: 10.1016/j.jclinepi.2010.11.018. PMID: 21447427. [PMC free article: PMC3076200] [PubMed: 21447427] [CrossRef]

29.

Kim OD, Cantave I, Schlesinger PK. Esophageal involvement by cutaneous T-cell lymphoma, mycosis fungoides type: diagnosis by endoscopic biopsy. J Clin Gastroenterol. 1990 Apr;12(2):178–82. doi: 10.1097/00004836-199004000-00013. PMID: 2324481. [PubMed: 2324481] [CrossRef]

30.

Ampalam P, Gunturu S, Padma V. A comparative study of caregiver burden in psychiatric illness and chronic medical illness. Indian J Psychiatry. 2012 Jul;54(3):239–43. doi: 10.4103/0019-5545.102423. PMID: 23226847. [PMC free article: PMC3512360] [PubMed: 23226847] [CrossRef]

31.

Risk of Bias 2 (RoB 2) tool. Cochrane Methods; 2020. https://www​.riskofbias.info/. Accessed on April 8, 2020.

32.

ROBINS-I tool. Cochrane Methods; 2020. https://www​.riskofbias​.info/welcome/home​/current-version-of-robins-i. Accessed on April 8, 2020.

33.

Majid U, Vanstone M. Appraising Qualitative Research for Evidence Syntheses: A Compendium of Quality Appraisal Tools. Qual Health Res. 2018 Nov;28(13):2115–31. doi: 10.1177/1049732318785358. PMID: 30047306. [PubMed: 30047306] [CrossRef]

34.

Checklist for Qualitative Research Joanna Briggs Institute. 2017. https:​//joannabriggs​.org/sites/default/files​/2019-05/JBI_Critical​_Appraisal-Checklist​_for_Qualitative_Research2017_0.pdf

35.

Harden A, Thomas J, Cargo M, et al. Cochrane Qualitative and Implementation Methods Group guidance series-paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018 May;97:70–8. doi: 10.1016/j.jclinepi.2017.11.029. PMID: 29242095. [PubMed: 29242095] [CrossRef]

36.

Lizarondo L, Stern C, Carrier J, et al. Chapter 8: Mixed methods systematic reviews. In: Aromataris E, Munn Z (Editors). Joanna Briggs Institute Reviewer’s Manual The Joanna Briggs Institute. 2017. https:​//reviewersmanual​.joannabriggs.org/

37.

Luckett T, Phillips J, Agar M, et al. Elements of effective palliative care models: a rapid review. BMC Health Serv Res. 2014 Mar 26;14:136. doi: 10.1186/1472-6963-14-136. PMID: 24670065. [PMC free article: PMC3986907] [PubMed: 24670065] [CrossRef]

38.

Yoong J, Park ER, Greer JA, et al. Early palliative care in advanced lung cancer: a qualitative study. JAMA Intern Med. 2013 Feb 25;173(4):283–90. doi: 10.1001/jamainternmed.2013.1874. PMID: 23358690. [PubMed: 23358690] [CrossRef]

39.

Taylor SL, Dy S, Foy R, et al. What context features might be important determinants of the effectiveness of patient safety practice interventions? BMJ Qual Saf. 2011 Jul;20(7):611–7. doi: 10.1136/bmjqs.2010.049379. PMID: 21617166. [PubMed: 21617166] [CrossRef]

40.

Agency for Healthcare Research and Quality. Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews Agency for Healthcare Research and Quality. Rockville, MD: 2007.

41.

Quality Measures & You: Clinicians. Centers for Medicare and Medicaid Services; 2019. https://www​.cms.gov/Medicare​/Quality-Initiatives-Patient-Assessment-Instruments​/MMS​/QMY-Clinicians2020.

42.

Luckett T, Phillips J, Agar M, et al. Elements of effective palliative care models: a rapid review. BMC Health Services Research. 2014;14(1):136-. doi: 10.1186/1472-6963-14-136. PMID: 103815400. Language: English. Entry Date: 20150123. Revision Date: 20190110. Publication Type: journal article. [CrossRef]

43.

Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005 Dec;52(5):546–53. doi: 10.1111/j.1365-2648.2005.03621.x. PMID: 16268861. [PubMed: 16268861] [CrossRef]

44.

Quill TE, Abernethy AP. Generalist plus specialist palliative care--creating a more sustainable model. N Engl J Med. 2013 Mar 28;368(13):1173–5. doi: 10.1056/NEJMp1215620. PMID: 23465068. [PubMed: 23465068] [CrossRef]

45.

Cargo M, Harris J, Pantoja T, et al. Cochrane Qualitative and Implementation Methods Group guidance series-paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018 May;97:59–69. doi: 10.1016/j.jclinepi.2017.11.028. PMID: 29223325. [PubMed: 29223325] [CrossRef]

46.

AHRQ Technical Reviews. In: Shojania KG, McDonald KM, Wachter RM, Owens DK, eds. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies (Vol. 1: Series Overview and Methodology). Rockville (MD): Agency for Healthcare Research and Quality (US); 2004. [PubMed: 20734525]

47.

Paquette-Warren J, Vingilis E, Greenslade J, et al. What do practitioners think? A qualitative study of a shared care mental health and nutrition primary care program. Int J Integr Care. 2006 Oct 9;6:e18. doi: 10.5334/ijic.164. PMID: 17041680. [PMC free article: PMC1602056] [PubMed: 17041680] [CrossRef]

48.

Legare F, Ratte S, Gravel K, et al. Barriers and facilitators to implementing shared decision-making in clinical practice: update of a systematic review of health professionals’ perceptions. Patient Educ Couns. 2008 Dec;73(3):526–35. doi: 10.1016/j.pec.2008.07.018. PMID: 18752915. [PubMed: 18752915] [CrossRef]