Results

Annette Totten; Dana M. Womack; Marian S. McDonagh; Cynthia Davis-O’Reilly; Jessica C. Griffin; Ian Blazina; Sara Grusing; Nancy Elder

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Totten A, Womack DM, McDonagh MS, et al. Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2022 Dec. (Comparative Effectiveness Review, No. 254.)

Cover of Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication

Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication [Internet].

Show details

Contents

< Prev Next >

Appendix BResults

Results of Literature Searches

Figure B-1 is a literature flow diagram that shows 6,329 records were identified through database searching and other sources. After these 6,329 abstracts were screened for inclusion, 5,305 records were excluded and 1,024 full-text articles were assessed for eligibility. Of these 1,024 articles, 845 were excluded. 181 were excluded for an ineligible population; 179 were excluded for an ineligible intervention; 44 were excluded for an ineligible outcome; 41 were excluded for ineligible comparator; 92 were excluded for an ineligible study design for Key Question; 170 were excluded due to not being a study; 2 was excluded for not being English language but possibly relevant; 13 papers were excluded due to being published prior to the cut-off date; 62 papers were systematic reviews or meta-analyses used as source documents only to identify individual stuides; and 60 were excluded for an ineligible setting. The other 179 full-text articles encompassed 166 included studies in the 179 publications. Of the 179 included publications, 7 studies addressed Key Question 1; 97 studies in 106 publications addressed Key Questions 2 and 4; and 67 stuides in 71 publications addressed Key Question 3. A note on the figure states that five papers were included for more than one Key Question.

Figure B-1Literature flow diagram

Note: Five papers are included for more than one Key Question

A total of 6,329 references were identified from electronic database searches. After dual review of abstracts, 1,024 articles were evaluated for inclusion. Search results and selection of studies are summarized in the literature flow diagram above (Figure B-1). A total of 166 studies (in 179 publications) were included for at least one key question.⁶^–¹⁸⁴ Seven studies were included for Key Question 1, 97 studies in 106 publications were included for Key Question 2, and 67 studies in 71 publications were included for Key Question 3. A list of included studies appears in Appendix C and excluded studies with reason for exclusion in Appendix G.

Description of Included Studies

Key Question 1

The systematic review protocol and a request for unpublished information was posted by AHRQ on the Federal Register Supplemental Evidence and Data (SEADs) webpage. Additionally, we sent emails requesting information to individual federal agencies as well as non governmental organizations involved in telehealth and experts familiar with telehealth practices and policy. Specific program offices contacted included FedTel, the U.S. Federal government working group on Teleheath, the Telehealth Focused Rural Health Research Center Program of the Health Resources and Services Administration (HRSA), and the COVID-19 Telehealth Impact Study organized by the COVID-19 Healthcare Coalition Telehealth Impact Study Work Group with leadership from Mayo Clinic and the MITRE Corporation.

We also explored the possibility of identifying trends through claims data. Although the Centers for Medicare & Medicaid Services has approved Common Procedure Terminology (CPT) codes 99446–99449, 99451 and 99452 for interprofessional electronic assessment and management and referral services provided by a consultative physician, or other qualified health care professional (QHP) and by a patient’s treating/requesting physician/QHP (in the case of 99452), there is anecdotal evidence that use of these billing codes is very low (personal communications).¹⁸⁵ It is very likely that informal interprofessional consultations are occurring in a non-compensated manner, but such interactions would not be included in billing records, and literature describing the frequency of informal interprofessional consultations is not currently available.

We did not receive any additional unpublished evidence on provider-to-provider telehealth in the rural U.S. usable for this report. While use of telehealth for patient and provider interactions has been documented, particularly the increase as part of the response to the COVID-19 pandemic,¹⁸⁶^, ¹⁸⁷ trends in provider-to-provider telehealth have not yet been documented to the same extent. Details can be found in Appendix Table D-1.

Table B-1Characteristics of included studies for Key Question 1

Characteristic	Categories	Number of Articles - 6	Percentage of Articles	References
Location	Nationwide	6	86%	⁸^,³⁰^,⁵⁰^,¹⁴⁹^,¹⁸⁰^,¹⁸²
Location	Regional- New England	1	14%	¹⁸¹
Adoption Category	Use	7	100%	⁸^,³⁰^,⁵⁰^,¹⁴⁹^,¹⁸⁰^–¹⁸²
	Spread	2	29%	¹⁴⁹^,¹⁸²
	Mental Health	2	29%	⁵⁰^,¹⁴⁹
	Emergency Care	2	29%	¹⁸⁰^,¹⁸¹
	Stroke	3	43%	⁸^,³⁰^,¹⁸²

Key Questions 2 and 4

Study details can be found in Appendix Tables D-2, D-3, D-4, D-5, D-6, D-7, D-8, D-9, E-1, E-2, E-3, and E-4.

Table B-2Characteristics of included studies for Key Question 2

Characteristic	Categories	Number of Articles - 106	Percentage of Articles	References
Geographic Location	United States	63	59%	¹⁰^,¹¹^,¹⁸^,²¹^,²³^,²⁹^,³⁶^,³⁸^–⁴¹^,⁴⁴^–⁴⁹^,⁵¹^,⁵⁵^,⁵⁸^–⁶⁰^,⁶⁶^,⁷²^,⁷⁵^,⁷⁸^,⁸⁴^,⁹²^,⁹³^,⁹⁶^,⁹⁸^,¹⁰⁰^,¹⁰¹^,¹⁰³^–¹⁰⁷^,¹¹⁰^,¹¹⁴^,¹¹⁷^,¹²⁰^,¹³⁰^–¹³²^,¹³⁹^,¹⁴²^,¹⁴³^,¹⁵⁰^,¹⁵³^,¹⁵⁴^,¹⁵⁸^,¹⁵⁹^,¹⁶⁴^,¹⁶⁷^,¹⁷³^,¹⁷⁵^,¹⁷⁶^,¹⁷⁸^,¹⁸²^,¹⁸⁴^,³⁵^,¹⁷²
	Australia	15	14%	¹³^,¹⁹^,²⁰^,²⁷^,³⁴^,⁴²^,⁵²^,⁷⁰^,⁹⁷^,⁹⁹^,¹¹¹^,¹²²^,¹²⁸^,¹³⁶^,¹⁵⁶
	Canada	5	5%	⁶⁵^,⁸²^,⁸⁵^,¹⁴⁵^,¹⁴⁸^,¹⁵⁵
	United Kingdom	3	3%	⁸¹^,¹⁵²^,¹⁶⁵
	Korea	3	3%	³¹^,⁷⁶^,⁷⁹
	Italy	2	2%	²⁵^,²⁶
	Countries with a single study^*	14	13%	¹²^,¹⁶^,²²^,³⁷^,⁵³^,⁶¹^,⁸¹^,⁸⁶^,¹¹²^,¹²⁴^,¹³⁸^,¹⁴¹^,¹⁵¹^,¹⁷⁹
Study Design	RCT	23	22%	¹⁹^,²²^,²⁹^,³¹^,³⁵^,³⁶^,³⁸^,⁴⁵^,⁴⁷^–⁴⁹^,⁵³^,⁶¹^,⁸²^,⁸⁶^,¹¹⁰^,¹³¹^,¹³²^,¹⁵⁵^,¹⁷²^,¹⁷⁹^,¹⁸⁴
	Observational- before/after	25	24%	¹³^,¹⁶^,²⁰^,²⁷^,⁴¹^,⁴²^,⁴⁴^,⁵²^,⁵⁵^,⁷⁵^,⁷⁸^,⁸⁷^,¹⁰⁵^,¹¹¹^,¹¹²^,¹¹⁴^,¹²⁰^,¹²²^,¹²⁸^,¹⁴³^,¹⁵⁰^,¹⁵²^,¹⁵⁴^,¹⁵⁹^,¹⁷⁸
	Observational- pre/post	18	17%	¹⁰^,¹⁸^,²³^,⁶⁶^,⁷⁹^,⁸⁴^,⁸⁵^,⁹⁶^,⁹⁸^,⁹⁹^,¹⁰⁷^,¹¹⁷^,¹³⁰^,¹³⁶^,¹⁴⁵^,¹⁴⁸^,¹⁶⁷^,¹⁷⁵
	Observational- prospective cohort	21	20%	¹¹^,¹²^,²⁵^,²⁶^,³⁴^,³⁷^,⁵⁹^,⁶⁰^,⁶⁵^,⁷⁰^,⁷⁶^,⁹³^,¹⁰⁰^,¹⁰³^,¹⁰⁶^,¹²⁴^,¹³⁸^,¹⁴¹^,¹⁴²^,¹⁵³^,¹⁷⁶
	Observational- retrospective cohort	19	18%	²¹^,³⁹^,⁴⁰^,⁵¹^,⁵⁸^,⁷²^,⁸¹^,⁹²^,⁹⁷^,¹⁰¹^,¹⁰⁴^,¹³⁹^,¹⁵¹^,¹⁵⁶^,¹⁵⁸^,¹⁶⁴^,¹⁶⁵^,¹⁷³^,¹⁸²
Risk of Bias	Low	5	5%	³⁸^,⁸²^,¹²⁰^,¹³²^,¹⁴⁸
	Medium	75	71%	¹⁰^–¹²^,¹⁶^,¹⁹^–²¹^,²⁵^–²⁷^,²⁹^,³¹^,³⁴^–³⁷^,³⁹^–⁴¹^,⁴⁴^–⁴⁹^,⁵¹^,⁵³^,⁵⁸^–⁶¹^,⁶⁵^,⁷⁰^,⁷²^,⁷⁶^,⁸⁵^,⁸⁶^,⁹²^,⁹³^,¹⁰⁰^,¹⁰¹^,¹⁰³^–¹⁰⁵^,¹¹⁰^–¹¹²^,¹¹⁴^,¹¹⁷^,¹²²^,¹²⁴^,¹²⁸^,¹³¹^,¹³⁸^,¹³⁹^,¹⁴¹^–¹⁴³^,¹⁴⁵^,¹⁵⁰^,¹⁵¹^,¹⁵³^–¹⁵⁶^,¹⁵⁸^,¹⁵⁹^,¹⁶⁴^,¹⁷²^,¹⁷³^,¹⁷⁵^,¹⁷⁶^,¹⁷⁹^,¹⁸²^,¹⁸⁴
	High	26	25%	¹³^,¹⁸^,²²^,²³^,⁴²^,⁵²^,⁵⁵^,⁶⁶^,⁷⁵^,⁷⁸^,⁷⁹^,⁸¹^,⁸⁴^,⁸⁷^,⁹⁶^–⁹⁹^,¹⁰⁶^,¹⁰⁷^,¹³⁰^,¹³⁶^,¹⁵²^,¹⁶⁵^,¹⁶⁷^,¹⁷⁸
Sample Size	Under 100	30	28%	¹⁸^,¹⁹^,²²^,²³^,²⁵^,³¹^,³⁷^,⁵²^,⁶⁶^,⁷⁰^,⁸¹^,⁸⁴^,⁸⁶^,⁹⁶^–⁹⁹^,¹¹¹^,¹¹²^,¹¹⁷^,¹³⁶^,¹³⁹^,¹⁴²^,¹⁴⁵^,¹⁵²^,¹⁵⁵^,¹⁶⁵^,¹⁶⁷^,¹⁷⁶^,¹⁸⁴
	100–500	48	45%	¹¹^,¹²^,¹⁶^,²⁰^,²⁶^,²⁹^,³⁵^,³⁶^,³⁸^–⁴⁰^,⁴²^,⁴⁴^–⁴⁹^,⁵⁸^,⁵⁹^,⁶¹^,⁶⁵^,⁷²^,⁷⁶^,⁷⁸^,⁷⁹^,⁸²^,⁸⁵^,⁹²^,⁹³^,¹⁰¹^,¹¹⁰^,¹²⁰^,¹²⁴^,¹³⁰^–¹³²^,¹³⁸^,¹⁴³^,¹⁴⁸^,¹⁵¹^,¹⁵⁴^,¹⁵⁶^,¹⁵⁹^,¹⁶⁴^,¹⁷²^,¹⁷³^,¹⁷⁸
	501–1000	6	6%	¹⁰^,⁵³^,⁷⁵^,¹⁰³^,¹²²^,¹⁵³
	1001–10,000	11	10%	²¹^,²⁷^,³⁴^,⁵¹^,¹⁰⁰^,¹⁰⁴^–¹⁰⁷^,¹⁴¹^,¹⁷⁹
	10,000+	7	7%	⁴¹^,⁵³^,⁵⁵^,⁸⁷^,¹¹⁴^,¹⁵⁰^,¹⁵⁸
	Not reported/unclear	2	2%	¹³^,¹⁷⁵^,¹⁸²
Mode of Telehealth	Video	79	75%	¹¹^,¹³^,¹⁶^,¹⁸^,²⁰^–²³^,²⁷^,³¹^,³⁴^–³⁶^,³⁸^–⁴²^,⁴⁴^,⁴⁷^,⁵¹^,⁵²^,⁵⁵^,⁵⁸^–⁶¹^,⁶⁵^,⁶⁶^,⁷⁰^,⁷²^,⁷⁵^,⁷⁶^,⁷⁸^,⁷⁹^,⁸¹^,⁸⁴^–⁸⁷^,⁹²^,⁹³^,⁹⁶^–¹⁰¹^,¹⁰³^–¹⁰⁷^,¹¹¹^,¹¹⁴^,¹²⁰^,¹²²^,¹²⁴^,¹²⁸^,¹³²^,¹³⁹^,¹⁴¹^–¹⁴³^,¹⁴⁸^,¹⁵⁰^,¹⁵³^,¹⁵⁵^,¹⁵⁶^,¹⁵⁸^,¹⁵⁹^,¹⁶⁵^,¹⁶⁷^,¹⁷⁵^,¹⁷⁶^,¹⁷⁸^,¹⁸²^,¹⁸⁴
	Data store and forward	4	4%	³⁷^,⁸²^,¹⁷²^,¹⁷⁹
	Electronic chart/record review	3	3%	²⁹^,⁴⁵^,¹⁵²
	Mixed modalities	10	12%	¹⁹^,⁴⁶^,⁴⁸^,⁴⁹^,¹¹⁰^,¹¹²^,¹³¹^,¹³⁸^,¹⁴⁵^,¹⁵⁴
	Data streaming	1	1%	²⁵
	Telephone	2	2%	¹⁰^,²⁵
	Whats App	1	1%	¹²
	SMS Based	1	1%	⁵³
	Online Module	3	3%	¹¹⁷^,¹³⁰^,¹³⁶
	NR/Unclear	2	2%	¹⁶⁴^,¹⁷³
Clinical category	Inpatient	18	17%	¹³^,²¹^,²⁷^,⁴¹^,⁵²^,⁵⁵^,⁵⁹^,⁶⁰^,⁷²^,⁷⁵^,⁷⁸^,⁹²^,⁹³^,¹¹¹^,¹²²^,¹²⁸^,¹⁵⁴^,¹⁷⁸
	Outpatient	37	35%	¹⁰^,²²^,²⁹^,³¹^,³⁴^–³⁷^,⁴⁵^–⁴⁹^,⁷⁰^,⁷⁶^,⁷⁹^,⁸¹^,⁸²^,⁸⁶^,⁸⁷^,⁹⁷^,⁹⁸^,¹¹⁰^,¹²⁰^,¹³¹^,¹³²^,¹³⁸^,¹³⁹^,¹⁴⁵^,¹⁵¹^,¹⁵²^,¹⁵⁵^,¹⁵⁶^,¹⁶⁵^,¹⁷²^,¹⁷⁶^,¹⁷⁹
	EMS/ED	28	26%	¹²^,²⁰^,²⁵^,²⁶^,³⁸^–⁴⁰^,⁴⁴^,⁵⁸^,⁶¹^,⁶⁵^,¹⁰⁰^,¹⁰¹^,¹⁰³^–¹⁰⁶^,¹¹¹^,¹¹²^,¹¹⁴^,¹²⁴^,¹⁴¹^,¹⁵⁰^,¹⁵³^,¹⁵⁹^,¹⁶⁴^,¹⁷³^,¹⁸²
	Education/mentoring	23	22%	¹¹^,¹⁹^,²³^,⁴²^,⁵¹^,⁵³^,⁶⁶^,⁸⁴^,⁸⁵^,⁹⁶^,⁹⁹^,¹⁰⁷^,¹¹⁷^,¹²⁸^,¹³⁰^,¹³⁶^,¹⁴²^,¹⁴³^,¹⁴⁸^,¹⁵⁸^,¹⁶⁷^,¹⁷⁵^,¹⁸⁴
Outcome categories	Patient	71	67%	¹⁰^–¹³^,¹⁶^,²⁰^–²²^,²⁵^–²⁷^,²⁹^,³¹^,³⁴^,³⁶^–³⁸^,⁴⁰^,⁴⁴^,⁴⁶^,⁴⁷^,⁴⁹^,⁵²^,⁵⁵^,⁵⁸^–⁶¹^,⁶⁵^,⁷⁰^,⁷²^,⁷⁵^,⁷⁶^,⁷⁸^,⁷⁹^,⁸¹^,⁸²^,⁸⁶^,⁸⁷^,⁹²^,⁹³^,⁹⁸^,¹⁰⁰^,¹⁰⁴^–¹⁰⁶^,¹¹⁰^–¹¹²^,¹¹⁴^,¹²²^,¹²⁴^,¹²⁸^,¹³⁹^,¹⁴¹^,¹⁴³^,¹⁴⁵^,¹⁵¹^–¹⁵⁵^,¹⁵⁹^,¹⁶⁴^,¹⁶⁷^,¹⁷²^,¹⁷⁵^,¹⁷⁶^,¹⁷⁸^,¹⁷⁹^,¹⁸²
	Provider	32	30%	¹⁸^,¹⁹^,²³^,³⁸^–⁴²^,⁵¹^–⁵³^,⁵⁸^,⁶⁶^,⁷⁸^,⁸⁴^,⁸⁵^,⁹⁶^,⁹⁹^,¹⁰¹^,¹⁰³^,¹⁰⁷^,¹¹⁷^,¹³⁰^,¹³⁶^,¹⁴²^,¹⁴³^,¹⁴⁸^,¹⁵⁰^,¹⁵⁸^,¹⁶⁵^,¹⁷⁵^,¹⁸⁴
	Payer	13	12%	³⁵^,³⁷^,⁴⁵^,⁷⁰^,⁹⁷^,¹²⁰^,¹³¹^,¹³²^,¹⁵²^,¹⁵⁶^,¹⁷³^,¹⁸²^,¹⁶

*: China, Denmark, Scotland, Finland, New Zealand, Spain, Germany, Chile, Turkey, Japan, Sweden, Taiwan, Vietnam

Key Question 3

Additional study details can be found in Appendix Table D-10.

Table B-3Characteristics of included studies for Key Question 3

Characteristic	Categories	Number of Articles (71 total)	Percentage of Articles	References
Geographic Location	United States	35	49%	⁶^,⁷^,⁹^,¹⁷^,²⁴^,³⁰^,³³^,⁵⁶^,⁵⁷^,⁶³^,⁶⁴^,⁶⁷^,⁷³^,⁷⁷^,⁸⁰^,⁹⁰^,⁹¹^,¹⁰⁸^,¹¹⁵^,¹²⁶^,¹²⁷^,¹²⁹^,¹³⁴^,¹³⁵^,¹⁴⁴^,¹⁴⁶^,¹⁴⁷^,¹⁵⁰^,¹⁵⁷^,¹⁶⁶^,¹⁶⁹^,¹⁷⁰^,¹⁷⁴^,¹⁸⁰^,¹⁸³^,¹⁸⁵
	Sweden	1	1%	⁷¹
	Norway	2	3%	¹⁶⁰^,¹⁶¹
	Germany	2	3%	⁹⁵^,¹¹⁹
	Australia	18	25%	¹⁴^,¹⁵^,²⁸^,⁴³^,⁶⁹^,⁷⁴^,⁸³^,⁸⁹^,¹⁰²^,¹¹³^,¹¹⁶^,¹²¹^,¹²³^,¹²⁵^,¹⁴⁰^,¹⁶²^,¹⁶³^,¹⁶⁸
	Canada	5	7%	³²^,⁵⁴^,⁶²^,⁶⁸^,¹⁷⁷
	New Zealand	1	1%	⁸⁸
	Scotland	3	4%	¹⁶^,⁹⁴^,¹⁷¹
	South Africa	1	1%	¹¹⁸
	Multiple	2	2%	¹³³^,¹³⁷
	Countries with a single study*	1	7%	¹⁰⁹
Method	Program statistics	1	1%	⁹¹
	Program records	6	8%	⁵⁴^,⁶³^,⁷⁴^,¹²⁶^,¹⁶²^,¹⁶³
	Program review	3	4%	¹¹³^,¹¹⁶^,¹²³
	Program reporting	1	1%	¹²⁵
	Program manager observations	1	1%	⁹
	Patient records	3	4%	⁵⁴^,⁶⁸^,¹⁴⁰
	Registries	3	4%	⁶^,⁷^,¹⁴⁶
	Hospital records	1	1%	¹⁵⁰
	Administrative data	4	6%	⁶^,⁷^,¹⁴⁶^,¹⁷⁴
	Financial data	1	1%	⁹⁰
	EHR data	1	1%	¹²⁵
	EMR data	1	1%	¹⁶⁶
	Pre-questionnaire	1	1%	⁹
	Survey	20	28%	¹⁷^,³⁰^,³²^,⁴³^,⁵⁶^,⁷³^,⁸⁰^,⁹¹^,¹⁰⁸^,¹⁰⁹^,¹¹⁵^,¹¹⁸^,¹²⁵^,¹²⁷^,¹³⁴^,¹³⁵^,¹³⁷^,¹⁵⁰^,¹⁷⁴^,¹⁸⁰
	Pilot tests	1	1%	⁶⁴
	Comparison of two models	2	3%	¹⁶⁹^,¹⁷⁰
	Interview/Focus groups	28	39%	¹⁴^–¹⁶^,²⁴^,²⁸^,⁶⁷^,⁶⁹^,⁷¹^,⁷³^,⁸⁰^,⁸³^,⁸⁸^–⁹⁰^,⁹⁴^,⁹⁵^,¹⁰²^,¹¹⁹^,¹²¹^,¹²⁹^,¹³⁵^,¹⁴⁴^,¹⁶⁰^–¹⁶³^,¹⁶⁶^,¹⁶⁸^,¹⁷¹^,¹⁷⁷^,¹⁸³
	Exit interviews focused on case presentation	1	1%	⁹
	Chart review	5	7%	³²^,⁶²^,⁹¹^,¹⁶²^,¹⁶³
	Case study	4	6%	¹⁴^,²⁸^,³³^,¹⁴⁷
	Case reports	2	3%	¹⁴^,²⁸
	Case review	3	4%	²⁴^,¹¹⁵^,¹⁵⁷
	Site visits	4	6%	⁵⁷^,⁹⁰^,¹³⁵^,¹⁶⁶
	Patient and staff evaluations	1	1%	³²
	Review of state statutes and regulations	1	1%	⁷⁷
	Document analysis	2	3%	¹³³^,¹³⁵
Clinical category	Inpatient	8	11%	¹⁶^,¹¹⁸^,¹³³
	Outpatient	30	42%	²⁴^,³²^,⁴³^,⁵⁴^,⁶²^–⁶⁴^,⁶⁹^,⁷¹^,⁷⁴^,⁸⁰^,⁹¹^,¹⁰²^,¹¹³^,¹¹⁶^,¹¹⁹^,¹²⁵^,¹²⁶^,¹³⁵^,¹⁴⁰^,¹⁶²^,¹⁶³^,¹⁶⁸^,¹⁷¹
	Telestroke and Emergency Care	20	28%	⁶^,⁷^,¹⁴^,²⁸^,³³^,⁷⁷^,⁸⁸^,⁹⁰^,⁹⁴^,¹³⁴^,¹⁴⁶^,¹⁴⁷^,¹⁵⁰^,¹⁶⁰^,¹⁶¹^,¹⁶⁶^,¹⁶⁹^,¹⁷⁰^,¹⁸⁰
	Education/mentoring	13	18%	⁹^,⁵⁶^,⁶⁸^,⁷³^,¹¹⁵^,¹²¹^,¹²⁷^,¹⁴⁴^,¹⁵⁷^,¹⁷⁴^,¹⁸³
Outcome categories	Facilitators	55	77%	⁶^,⁷^,⁹^,¹⁶^,²⁴^,³²^,³³^,⁴³^,⁵⁴^,⁵⁶^,⁶²^–⁶⁴^,⁶⁸^,⁶⁹^,⁷¹^,⁷³^,⁷⁴^,⁷⁷^,⁸⁰^,⁸⁸^,⁹⁰^,⁹¹^,⁹⁴^,¹⁰²^,¹¹³^,¹¹⁵^,¹¹⁶^,¹¹⁸^,¹¹⁹^,¹²¹^,¹²⁵^–¹²⁷^,¹³³^–¹³⁵^,¹⁴⁰^,¹⁴⁴^,¹⁴⁶^,¹⁴⁷^,¹⁵⁰^,¹⁵⁷^,¹⁶⁰^–¹⁶³^,¹⁶⁶^,¹⁶⁸^–¹⁷¹^,¹⁷⁴^,¹⁸⁰^,¹⁸³
Outcome categories	Barriers	51	72%	⁶^,⁷^,⁹^,¹⁶^,²⁴^,³²^,³³^,⁵⁴^,⁵⁶^,⁶²^–⁶⁴^,⁶⁸^,⁶⁹^,⁷¹^,⁷³^,⁷⁴^,⁷⁷^,⁸⁰^,⁸⁸^,⁹⁰^,⁹¹^,⁹⁴^,¹⁰²^,¹¹³^,¹¹⁵^,¹¹⁶^,¹¹⁸^,¹¹⁹^,¹²¹^,¹²⁵^–¹²⁷^,¹³³^–¹³⁵^,¹⁴⁰^,¹⁴⁴^,¹⁴⁶^,¹⁴⁷^,¹⁵⁰^,¹⁵⁷^,¹⁶²^,¹⁶³^,¹⁶⁶^,¹⁶⁸^,¹⁷⁰^,¹⁷¹^,¹⁷⁴^,¹⁸⁰^,¹⁸³

Table B-4 repeats the number of times a construct was mentioned and adds the number of publications and the number of settings out of the four possible settings (inpatient, outpatient, EMS/ED, or Education/Mentoring) in which these studies were conducted. This examination demonstrates that the constructs are relevant in all or most of the settings.

We also summarized facilitators and barriers by health care setting (inpatient, outpatient, emergency, and education/mentoring) in two ways. First, Table B-5 reports the number of barriers and facilitators by setting. Included studies of provider-to-provider telehealth for EMS/ED and education/mentoring reported more faciliators the barriers. This was reversed infor inpatient studies and the number of reports were about equal for outpatient care. Next, we created tables by setting and clinical indication similar to how the results are organized for Key Question 2. These tables provide the number of studies we identified for each clinical indication with a brief description of the telehealth interventions; basic information about the studies for each topic, including the method, size and location; implementation facilitators and barriers identified in the study as well as the impact cited as an indicator of successful implementation or motivation for sustainment. Not all studies sought to identify all three so the number of facilitators, barriers, and indicators of impact varies by topic. Finally, the studies we identified that compared strategies or interventions we described these in the narrative text for each setting.

Table B-4Distribution of CFIR constructs

Barrier or Facilitator	CFIR Contructs	# Settinqs	# Mentions	# Publications
Facilitators	Leadership Engagement	4	13	10
	Implementation Climate	4	13	11
	Patient Needs & Resources	4	32	25
	Planning	4	11	9
	Compatibility	4	33	23
	External Policy & Incentives	4	18	12
	Adaptability	4	9	9
	Knowledge & Beliefs about the Intervention	4	16	11
	Available Resources	4	60	40
	Reflecting & Evaluating	4	12	11
	Access to Knowledge & Information	4	57	36
	Networks & Communications	4	37	30
	Engaging	4	23	18
Barriers	Cost	3	15	13
	Readiness for Implementation	3	17	12
	Formally Appointed Internal Implementation Leaders	3	7	7
	Executing	3	19	12
	Relative Priority	3	8	8
	Complexity	2	11	8

Table B-5Facilitators and barriers by topic area

Topic	Facilitator or Barrier	# Mentions	# Publications
Inpatient	Barrier	45	9
Inpatient	Facilitator	24	8
Outpatient	Barrier	95	26
Outpatient	Facilitator	90	25
ED/EMS	Barrier	28	7
ED/EMS	Facilitator	65	19
Education/Mentoring	Barrier	24	7
Education/Mentoring	Facilitator	40	13

Inpatient

We identified eight assessments of implementation of provider-to-provider telehealth in rural areas that addressed inpatient care including intensive care, use of anesthesia, stroke rehabilitation, teletrauma, multidisciplinary specialty consultation, and telerobotics (Table B-6, Appendix Table D-10). One study compared facilitators and barriers in ICU programs using centralized monitoring (CM) versus virtual consult (VC) models,¹³³ one study used surveys to evaluate phone support consultation in South Africa,¹¹⁸ and one study used focus groups in Scotland to describe user experiences with video team consultations.¹⁶

One study of remote ICUs directly compared the facilitators and barriers for CM, which uses a hub with intensivists and hardwired data transfer and VC that uses portable equipment to connect local providers to relevant specialists This study analyzed documents collected as part of a systematic review of effectiveness of remote ICU programs that use CM or VC.¹³³ The structural differences in the models drove the differences in barriers and facilitators such as lower cost and faster start-up for VC compared to the CM, but VC required more effort to integrate into workflows (Table B-6). Based on surveys of rural physicians who were trained in person and then offered telephone support when they needed to use anesthesia, Ngala et al. identified that an important barrier was remote consultants lacked understanding of the rural environment (Table B-6).¹¹⁸ The study of stroke rehabilitation video team consultations reported that lack of technological issues supported implementation and that the consultation increased the patient representative’s confidence in the care provided locally (Table B-6).¹⁶

Table B-6Findings of inpatient implementation studies

Topic Number of Studies Intervention	Method N^* Location	Facilitators	Barriers	Impact
Remote ICU 1 CM compared to VC	Document analysis N=91 documents Varied¹³³	Lower cost, faster start-up (VC) Evidence supporting efficacy (CM) No legal issues (both models) Provided clinical information support (CM)	Higher fixed costs (CM) Longer start up time (CM) More reactive, requiring initiation or scheduling (VC) Lack of evidence of clinical efficacy (VC)	Not reported
Anesthesia 1 Phone support following in person training	Survey N=17 rural physicians South Africa¹¹⁸	In person training prior to implementation Perception that good advice was available	Inadequate training Lack of consultant understanding of environment No occasion to provide service/no need	Not reported
Stroke rehabilitation 1 Specialist participation via video in remote team meeting	Focus Group N=12 people; different roles in program Scotland¹⁶	No staff resistance; clear rational for use Technology training and IT involvement Reliable equipment	Delay of not having specialist immediately accessible	Improved decision making Increased confidence in care (patient representative)
Teletrauma 1	Interview N=14 stakeholders Canada¹⁷⁷	Flexibility of the technology to receive clinical input Interprofessional relationships Seamless integration of technology	Complicated nature of use Familiarity with the technology Workflow changes	Increased personal and professional support for rural clinicians
Robotic Telemedicine 1	Survey N=38 health care institutions United States, Canada, Ireland¹³⁷	Not reported	Equipment Cost Executive administration and leadership hesitancy regarding adoption of robotic telemedicine Lack of effective leadership Lack of exposure to robotic telemedicine Lack of understanding of robotic telemedicine Patients, Physicians and Nurse hesitancy regarding adoption of robotic telemedicine Physician lack of incentives to use robotic telemedicine Potential impact on quality of care Regulatory barriers (out-of-state licensing, malpractice liability, credentialing, government and nongovernment reimbursement, DEA licensing) Robotic telemedicine seen as a local threat Technology issues (usability, reliability, internet connectivity, remote data access, technical support, documentation and billing)	Not reported
Multidisciplinary Specialty Consultation 3	Interview N=63 hospitals Australia¹⁵ Secondary analysis national survey data N= 4,608 hospitals U.S. 50 States³⁰ Method= N=8 hospitals U.S. Montana, Nevada, North Dakota⁵⁷	Benefits of telehealth well communicated Investment in services to support telehealth delivery Specific service agreements Practitioner Champion Technical assistance Training on work flow and infrastructure	Billing system Clinician resistance Confusion regarding policy for out-of-catchment Cost of using existing computers to provide telehealth Credentialing process and agreeing to be credentialed Fragmentation of information technology between tertiary and primary care Software difficulties Health Information Exchange capabilities Cost Practitioner reluctance Securing properly credentialed practitioners	Not reported

*: N is used here to represent the unit of analysis, which may be number of individual participants or may be number of health care sites or systems.
: Abbreviations: CM = centralized monitoring; ICU = intensive care unit; VC = virtual consult.

Table B-7 provides the barriers and facilitators from studies of provider-to-provider telehealth for inpatient studies standardized by CFIR constructs. While there are fewer studies of inpatient care, the barriers and facilitators are not repeated in multiple studies. The most frequently repeated are complexity cited seven times as a barrier and available resources cited as a facilitator six times.

Table B-7Inpatient: barriers and facilitators by CFIR constructs

Type	Facilitator or Barrier Name	Facilitator or Barrier Number of Mentions	Reference Number(s)
Barrier	Access to Knowledge & Information^*	2	¹⁵^,¹¹⁸
	Adaptability^†	1	¹¹⁸
	Available Resources^‡	1	¹¹⁸
	Compatibility	3	¹⁵^,⁵⁷^,¹⁷⁷
	Complexity^§	7	¹⁵^,³⁰^,¹³³^,¹⁷⁷
	Cost^∥	4	¹⁵^,⁵⁷^,¹³³^,¹³⁷
	Engaging	2	⁵⁷^,¹³⁷
	Executing	2	¹³⁷^,¹⁷⁷
	External Policy & Incentives	1	¹³⁷
	Knowledge & Beliefs about the Innovation^¶	3	¹¹⁸^,¹³³^,¹³⁷
	Implementation Climate	2	¹³⁷
	Leadership Engagement	2	¹³⁷
	Patient Needs & Resources ^#	3	¹⁶^,¹²⁹^,¹³⁷
	Networks & Communications^**	1	¹¹⁸
	Planning	2	¹⁵
	Reflecting & Evaluating^††	1	¹¹⁸
Facilitator	Access to Knowledge & Information^*	2	¹⁶^,¹¹⁸
	Adaptability	1	¹⁷⁷
	Available Resources^‡	6	¹⁵^,¹⁶^,⁵⁷^,¹³³
	Cost^∥	1	¹³³
	Executing	1	¹⁷⁷
	External Policy & Incentives	1	¹⁵
	Formally Appointed Internal Implementation Leaders	1	⁵⁷
	Knowledge & Beliefs about the Innovation^¶	1	¹⁶
	Patient Needs & Resources^#	1	¹¹⁸
	Networks & Communications^**	3	¹⁵^,¹¹⁸^,¹⁷⁷
	Planning	1	⁵⁷
	Readiness for Implementation^‡‡	1	¹³³

*: Access to digestible information and knowledge about the innovation and how to incorporate it into work tasks.
†: Degree to which an innovation can be adapted, tailored, refined, or reinvented to meet local needs.
‡: Level of resources organizational dedicated for implementation and on-going operations including physical space and time.
§: Perceived difficulty of the innovation, reflected by duration, scope, radicalness, disruptiveness, centrality, and intricacy and number of steps required to implement.
∥: Costs of the innovation and costs associated with implementing the innovation including investment, supply, and opportunity costs.
¶: Individuals’ attitudes toward and value placed on the innovation, as well as familiarity with facts, truths, and principles related to the innovation.
#: Extent to which patient needs, as well as barriers and facilitators to meet those needs, are accurately known and prioritized by the organization.
**: Nature and quality of webs of social networks, and the nature and quality of formal and informal communications within an organization.
††: Quantitative and qualitative feedback about the progress and quality of implementation accompanied with regular personal and team debriefing about progress and experience.
‡‡: Tangible and immediate indicators of organizational commitment to its decision to implement an innovation.

Outpatient

We identified 30 studies of implementation of provider-to-provider telehealth for rural outpatient care. These studies all assessed consultations in which one provider, often a specialist, contributed to the diagnosis or management of a patient by another provider, often a primary care physician, nurse or someone lacking specialist certification or extensive experience with the condition or treatment. The barriers and facilitators are grouped and organized by clinical indication in Table B-8, and additional details can be found in Appendix Table D-10.

Five studies of multi-specialty programs included two statewide programs⁸⁰^, ¹³⁵ and three programs serving a small group of clinics or a single health system.⁷¹^, ¹⁰²^, ¹¹⁹ Psychiatric consultations were the subject of five studies of services that provided expert advice on a range of mental health issues including: medication therapy for opioid use disorder in a group of community clinics that are part of the Veteran Health Administration;²⁴ advice on medications and treatment for children in a state Medicaid program,⁶³ and programs to help diagnose adults and identify and arrange appropriate services.³²^, ⁶²^, ⁹¹ Five studies were of programs that provided consultations for different aspects of care related to long term services and supports including assessment of whether nursing home residents should be transferred to hospitals,⁶⁴ oral health screening and teledentistry,¹⁶²^, ¹⁶³ wound care,⁷⁴ outpatient geriatric assessment and management,¹²⁶ and pediatric hospice care.¹⁶⁸ The remaining studies each evaluated consultations related to evaluating or managing patients with chronic conditions including cancer,⁶⁹^, ¹⁴⁰ gastroenterology,⁵⁴ dermatology,¹²⁵ cardiology,¹¹⁶ nephrology,¹¹³ occupational screening of miners,⁴³ and support for midwifes managing pre-eclampsia.¹⁷¹

Studies of multispecialty programs included assessments of how one program evolved from a pilot test to a statewide program over years with a mixture of sources of funding.⁸⁰ An evaluation of a 10-year, multi-site initiative to increase access to care in rural areas in California through telehealth reported that organizational barriers contributed to a lack of networking across programs and lower uptake than expected of telehealth services.¹³⁵ Both of these studies demonstrate how implementation and spread requires sustained efforts and commitments from multiple stakeholders and suggests that statewide or regional efforts can be effective. Questions among rural clinicians about whether telehealth was truly patient-centered were a barrier for teleconsultations in rural Sweden as some providers felt it may be easier to send patients to the hospital directly rather than delay hospitalization for a consult.⁷¹ Concerns cited in United States studies were echoed in studies in other countries. A program in Germany cited concerns about time, financing and changes to established workflows as barriers that could be addressed if systems were more usable and training provided.¹¹⁹ A program in Australia illustrated time concerns by documenting that teledermatology consultations take twice as long as in-person assessments and payment does not include this extra time. This program addressed this and other barriers by adding a telehealth coordinator who reduced the need for clinician time and by assuring technical support was available.¹⁰²

Telehealth is often proposed as one solution to the shortage of mental and behavioral health providers and programs in rural areas. Some of the telehealth programs address specific treatments, such as the use of Buprenorphine for opioid use disorder in VA clinics in one state,²⁴ while others are more general. The evaluation telehealth supported Buprenorphine was one of the few that used an implementation science framework to assess their experience and then translate this experience into an implementation tool kit that could be used by others to replicate the program. Another telehealth consult program provided medication review and treatment recommendations for children in a state Medicaid program.63 These programs had to overcome specific barriers including legal concerns related to prescribing and the need for consultants to understand resource availability in other locations. Another program used a continuous quality improvement approach to identify and make workflow adjustments to assure success.⁹¹ Psychiatric teleconsultation services in Ontario, Canada, one for adults⁶² and one focused on geriatric psychiatry³² identified fundamental gaps in organization and culture as barriers, such a lack of integration of the telehealth consultation with telephone and in person visits with the patient⁶² and a concern among providers that telehealth would allow the government to justify the lack of support for increasing local, in-person services for patients.³²

The two articles on provider-to-provider telehealth for cancer care were both reports about the same program in Queensland, Australia. This program allowed chemotherapy to be administered in rural hospitals by local physicians and nurses supported by remote oncologists and chemotherapy nurses.⁶⁹^, ¹⁴⁰ Starting with a pilot to demonstrate safety, the program expanded to six sites after addressing barriers including lack of role clarity and technology restrictions. Changes included assuring the iCamera could zoom sufficiently to allow checks on chemotherapy bags and provide good visuals during physical exams; structuring the program to provide professional development opportunities for rural nurses; and financial incentives for physicians to participate.

Long term care residents often have limited access to health care services for many reasons including resident’s/patient’s difficulties traveling and the fact that specialty services are rarely available onsite in nursing homes and other residential care and home-based long-term care. In this context, telehealth consultations and programs may offer services that would not otherwise be available. For example, a multisite program was established by a health system to provide acute assessment and care planning support in order to reduce patient transfers to hospitals.⁶⁴ The program grew from 5 to 34 sites in 4 years by building on the health system’s experience with telehealth for other uses and working to change the culture from one that had defaulted to hospitalization to one that accepted treating residents in place. An oral health program provides another example in which a new service was made available. Residents who were not receiving dentistry services were screened by a technician who used a live intra oral camera to transmit images to a remote dentist who could assess what could be done on site and what required travel to a dentist.¹⁶²^, ¹⁶³ This program was able to increase compliance with guidelines and regulations while increasing staff confidence in their ability to manage oral health. Other applications included a geriatric consult service in the Veterans Health Administration that was able to increase assessments by setting up both synchronous and asynchronous consultations.¹²⁶ Implementation of a teleconsult program to support wound care by home and community providers revealed structural barriers to implementation including the need for staff computer literacy and the lack of use of standardized terminology by the home care nurses and consultants.⁷⁴ Adding telehealth consultations to a pediatric hospice program underscored tradeoffs and challenges. The program demonstrated the ability to provide multidisciplinary, timely help to supplement in person care, but found that video consults were limited in their ability to assess family distress and that the consults risk prioritizing expert views over family needs.¹⁶⁸

The remaining outpatient studies included one report each about telehealth consultations for different chronic, or not immediately acute conditions, including a regional cardiology program,¹¹⁶ chronic kidney disease consultations for an Indian Health Service clinic,¹¹³ a local gastroenterology program focused on a single condition,54 and a large dermatology program with 15 hubs in the VA.¹²⁵ All these programs were designed to increase access and timeliness of care and all faced hurdles related to lack of staff support, space, and connectivity/bandwidth. Two less common approaches included adding telehealth to a mobile clinic that provides screening for coal miners⁴³ and creating a phone app to supplement support to midwives managing pre-eclampsia in a rural area.¹⁷¹ Both of these programs had to overcome unique technical challenges, but faced common barriers related to limited connectivity.

Table B-8Findings of outpatient implementation studies

Topic Number of Studies Intervention	Method N^* Location	Facilitators	Barriers	Impact
Multi-specialty 5 Programs that make consultations from a range of specialists available over video, phone or electronic records	Stakeholder (patients and provider) surveys and interviews N=Not reported Statewide hub/spoke program in South Carolina⁸⁰ Evaluation with surveys, site visits, documentation review, interviews N=10 organizations in 22 counties California¹³⁵ Focus groups N=5 primary health-care centers; 19 health care personnel Sweden⁷¹ Interviews N=18; Physicians, administrators, medical students Germany¹¹⁹ Interviews N=10 expert providers of telehealth Australia¹⁰²	Private foundation support for start up Ongoing support from state funds and billing Purposeful stages of implementation including training and evaluation Rural providers perceived benefits were worth their investment Usability of system/training Financing plan Coordinator and tech support	Limits to pro bono consults specialists can provide, need for financial support for specialist time or other concerns about financing Low networking across programs Organization barriers and lower utilization than expected Some specialties and exams not a good fit Aversion among staff to new ways of working including technology Time demands Perception it could be easier to transport patient	Grew from 11 in 2012 to over 1300 consults in 2017 Travel/cost savings for patients
Psych/Mental Health 7 Managing opioid use disorder, mental health care planning, for adults and medications and treatment for recommendations for children	Interviews and case review N=3 Clinics; 19 interviews VA in Maine²⁴ Continuous Quality Improvement Surveys, Chart reviews and program statistics N=1 health system Illinois⁹¹ Program records N=1 state Medicaid program Washington provision to Wyoming state Medicaid⁶³ Chart reviews Interviews N=10 Ontario, Canada⁶² Chart review, patient and staff evaluations, survey to referring MDs, focus groups with community agencies GeroPsych service N=6 communities Ontario, Canada³² Focus group N=10 Psychiatrists and 4 psychologists across 3 states U.S. Washington, Michigan, Arkansas⁶⁷ Questionnaire N=8 primary care providers, 4 psychologists Chile¹⁰⁹	Mission to address issue Prior teleprescribing experience Maintenance of local control over patient Development of tool kit Ongoing evaluation and adjustment Peer relationships among clinicians Detailed recommendations and education Teamwork Communication Provider willingness to collaborate Punctuality Staff openness	Legal concerns Conflicting interests Coordination with other programs Need for team integration E consults limited to quick or simple inquiries Need for understanding of local resources Limited awareness of consults Discomfort managing mental health in primary care Perception that telehealth provides justification for not increasing in person access Ability to track clinical outcomes and cost (2234) Clinic investment Communication Credentialing Electronic record process Mobilizing resources Scheduling support Staff Turnover Training Access to computer and suitable room Incomplete information on patient Technical difficulties	Positive evaluations of ability to see and hear and usefulness by patients Physician intension to continue to use Increased ability to include family or an interpreter Reduction in outlier pediatric psychiatric medications
Cancer 2 Chemotherapy administration at remote sites	Interviews N=19⁶⁹ Patient records N=62¹⁴⁰ Australia	Opportunity for professional development for rural nurses Good communication Implementation management team that developed plan and documentation Funding support and financial incentives for physicians to participate	IT did not have capacity to zoom in to check chemotherapy bags and physical exam Lack of good electronic documentation (fixed after pilot) Lack of role clarity Turnover in management and nurses	Better continuity of care Spread from pilot to 6 sites Similar safety to in person care Project transitioned from special funding to normal financing
Long-Term Care 3 Acute illness; hospital transfer decisions	Pilot Tests N=1 health system, up to 14 sites Avera Health, Several States⁶⁴ Interview N=21 administrators and clinicians across 16 facilities U.S. Nationwide¹²⁹ Interview N=8 Clinicians Germany⁹⁵	Meetings with leadership Creation of an implementation plan with stakeholder review Establishing billing procedures Leveraging decision support tools Health system experience with telehealth Communication Creating one time slot for visits creates focus, information getting lost Transmitting patient information 1 day ahead	Need for bandwidth Space constraints requiring mobile equipment Culture change to accept treating in place Building relationships Cannot see or hear as well Clinician reluctance Difficulty working with cognitively impaired patients Increased burden on staff Lack of training Technology challenges, including internet and connectivity Workflow changes	Increase from 5 to 34 sites in 4 years
Oral Health in Long-Term Care residencies 2	Chart review and program records, interviews and focus group N=250 charts, 9 facilities Australian¹⁶²^, ¹⁶³	Feedback on compliance Documentation of cost savings	Not enough staff time for program management Oral health staff lack of experience with dementia	Minimized disruption to residents Increase compliance Increased staff confidence in managing oral health
Geriatrics 1	Program records N=12 hubs U.S. Veterans Health Administration¹²⁶	Prior relationships between hubs and rural clinics	Multiple contacts needed to establish new relationships Need to develop case finding approaches to identify patients who could benefit from consultation.	Increase from 4 to 12 hubs in 4 years
Wound Care 1	Program records N=4 home and community health providers Australia⁷⁴	Train the trainer model Staff commitment	Need for computers and staff computer training/literacy Need for web access Need for standardized terminology	Not reported
Pediatrics 2	Interview N=15 hospice nurses 1 Midwestern U.S. state¹⁶⁸ Interview N=1 hub, 7 community health centers Australia⁸⁹	Ability to involve multiple disciplines and family members Timely goals of care discussions Networking and collaboration	Difficult to assess and address distress Family perceptions could be usurped by experts Enough rooms Funding Technical issues, being able to tell who was speaking and being able to hear	Not reported
Chronic Conditions 1 Gastroenterology, care for inflammatory bowel disease	Patient and program records N=99 patients Ontario, Canada⁵⁴	Multidisciplinary team	Limited availability of remote telehealth sites Lack of awareness of program	Travel cost savings Reduction in wait times
Dermatology 2 Remote assessment and diagnosis	EHR data, program reporting, online survey N=15 hubs U.S. Veterans Administration¹²⁵ Survey N=34 Primary care providers U.S. Mississippi¹⁰⁸	Strong stakeholder support Use of residents to reduce burden on dermatologists Communications with primary care	Understaffing at rural spokes Lack of space and equipment Restrictions on funding Bureaucracy Concern regarding possibility of misdiagnosis due to poor image quality Concerns regarding possible loss of patient confidentiality Insurance coverage Investment in time needed to master technology Misconceptions about teledermatology Time required to submit consult and response time	Not reported
Cardiology 1 Case review and remote exam	Program review N=5 sites Minnesota, Wisconsin¹¹⁶	Using local nurses to prepare patient and chart	Difference in reimbursement rules for locations Need for patient and scheduler education	Able to expand from 1 to 5 sites, one site dropped out
Nephrology 1 Care review and remote patient/provider appointment	Program review N=1 site Zuni Pueblo, Indian Health Services¹¹³	Nurse care manager and ancillary staff are key EHR access Access to specialist between scheduled clinics Periodic in-person visits to build rapport and trust	Does not address need patients with acute needs Technical issues Communication components (eye contact, emotional support)	Not reported
Screening 1 Mobile clinic: mining related exposure and general health	Surveys N=278 (62%) of 4511 mobile clinic with telehealth for miners New Mexico⁴³	Understanding of occupation risks Specialist mentor non-specialists Financially sustainable; most patients have insurance	Not reported	Expansion from New Mexico to Wyoming High percentage return to clinic
Midwifery 1 Using phone app to managing pre-eclampsia	Focus groups, N=18 midwives Scotland¹⁷¹	Access to up to data information on a relative rare event	Concerns about using technology Need for system that works without an internet connection	Not reported

*: N is used here to represent the unit of analysis, which may be number of individual participants or may be number of health care sites or systems.
: Abbreviations: EHR = electronic health record; IT = information technology; MD = medical doctor; U.S. = United States; VA = U.S. Department of Veteran’s Affairs.

Table B-9 provides a summary of the barriers and facilitators identified in studies of outpatient provider-to-provider telehealth for rural populations by the standardized constructs. As almost half of the studies included for Key Question 3 involved outpatient care, the counts of facilitators and barrier are higher. However, unlike inpatient care some constructs were identified much more frequently than others. Available resources was the most frequent barrier, cited 23 times. But others also mapped to higher numbers of cited barriers including compatibility (14) and access to knowledge and information (9). Available resources (15) and access to knowledge and information (15) were also cited as common facilitators (11 times), but other important facilitators included networks & communications(16) and patient needs & resource(9)s. In the case of this last category, a facilitator for use of telehealth was often patients’ needs for expertise and services that were not otherwise available without telehealth.

Table B-9Outpatient: barriers and facilitators by CFIR constructs

Type	Facilitator or Barrier Name	Facilitator or Barrier Number of Mentions	Reference Number(s)
Barrier	Access to Knowledge & Information^*	9	⁶²^,⁶⁷^,⁶⁹^,⁷⁴^,¹⁰²^,¹²⁹^,¹⁶²
	Adaptability^†	2	⁶²^,⁶⁷
	Available Resources^‡	23	²⁴^,³²^,⁴⁹^,⁶⁴^,⁶⁷^,⁶⁹^,⁷¹^,⁷⁴^,⁸⁹^,⁹¹^,¹⁰²^,¹⁰⁹^,¹¹³^,¹²⁵^,¹⁴⁰^,¹⁶²^,¹⁷¹
	Compatibility^§	14	⁶⁴^,⁶⁷^,⁶⁹^,⁷⁴^,⁸⁰^,¹⁰²^,¹⁰⁸^,¹¹³^,¹¹⁹^,¹²⁹^,¹³⁵
	Complexity^∥	3	⁶⁷^,⁷⁴^,¹⁷¹
	Cost^¶	3	⁸⁰^,¹¹⁶^,¹¹⁹
	Engaging^#	5	²⁴^,⁵⁴^,⁷¹^,¹¹⁶^,¹³⁵
	Executing^**	10	³²^,⁸⁹^,¹⁰⁸^,¹⁰⁹^,¹¹⁹^,¹²⁹^,¹⁴⁰
	External Policy & Incentives^††	5	⁸⁹^,¹⁰²^,¹⁰⁸^,¹²⁵^,¹³⁵
	Implementation Climate^‡‡	1	⁶²
	Knowledge & Beliefs about the Innovation^§§	4	³²^,¹¹⁹^,¹³⁵
	Leadership Engagement	1	¹⁰⁸
	Networks & Communications^¶¶	6	²⁴^,⁶⁷^,¹¹³^,¹²⁵^,¹²⁶^,¹²⁹
	Patient Needs & Resources^∥∥	8	⁵⁴^,⁶²^,⁷¹^,¹²⁹^,¹⁶⁸
	Planning^##	1	¹³⁵
	Readiness for Implementation^***	3	²⁴^,⁶⁷^,¹¹⁹
	Reflecting & Evaluating^†††	2	⁶³^,¹³⁵
	Relative Priority^‡‡‡	3	²⁴^,³²^,¹⁰²
Facilitator	Access to Knowledge & Information^*	15	²⁴^,³²^,⁴³^,⁶²^,⁶⁴^,⁶⁹^,⁷¹^,⁷⁴^,⁹¹^,¹²⁵^,¹²⁶^,¹³⁵
	Adaptability^†	1	¹²⁵
	Available Resources^‡	11	³²^,⁶²^,⁶⁴^,¹¹³^,¹¹⁶^,¹¹⁹^,¹²⁵^,¹⁴⁰^,¹⁶²
	Compatibility	1	¹⁰⁹
	Complexity^∥	1	¹¹⁹
	Cost^¶	4	⁴³^,⁹¹^,¹¹⁹^,¹⁶³
	Engaging^#	7	⁵⁴^,⁶²^,⁸⁰^,¹¹⁹^,¹⁴⁰^,¹⁶⁸
	External Policy & Incentives^††	3	⁸⁰^,¹⁴⁰
	Executing	2	⁹⁵^,¹⁰⁹
	Formally Appointed Internal Implementation Leaders^§§§	3	⁵⁴^,¹¹³^,¹³⁵
	Knowledge & Beliefs about the Innovation^§§	1	³²
	Leadership Engagement^∥∥∥	4	²⁴^,¹²⁵^,¹⁴⁰
	Patient Needs & Resources ^∥∥	9	³²^,⁴³^,¹⁰⁸^,¹¹³^,¹¹⁹^,¹²⁶^,¹⁶⁸^,¹⁷¹
	Networks & Communications^¶¶	16	³²^,⁶²^,⁶⁷^,⁶⁹^,⁸⁰^,⁸⁹^,⁹⁵^,¹⁰⁹^,¹¹³^,¹²⁵^,¹²⁶^,¹²⁹^,¹³⁵^,¹⁴⁰^,¹⁶²
	Planning^##	4	⁶⁴^,¹¹⁹^,¹³⁵^,¹⁴⁰
	Readiness for Implementation^***	5	²⁴^,⁶⁴^,⁸⁰
	Reflecting & Evaluating^†††	4	⁶⁴^,⁹¹^,¹⁴⁰^,¹⁶³
	Relative Priority^‡‡‡	1	⁷⁴

*: Access to digestible information and knowledge about the innovation and how to incorporate it into work tasks.
†: Degree to which an innovation can be adapted, tailored, refined, or reinvented to meet local needs.
‡: Level of resources organizational dedicated for implementation and on-going operations including physical space and time.
§: Degree of tangible fit between meaning and values attached to the innovation by involved individuals, how those align with individuals’ own norms, values, and perceived risks and needs, and how the innovation fits with existing workflows and systems.
∥: Perceived difficulty of the innovation, reflected by duration, scope, radicalness, disruptiveness, centrality, and intricacy and number of steps required to implement.
¶: Costs of the innovation and costs associated with implementing the innovation including investment, supply, and opportunity costs.
#: Attracting and involving appropriate individuals in the implementation and use of the innovation through a combined strategy of social marketing, education, role modeling, training, and other similar activities.
**: Carrying out or accomplishing the implementation according to plan.
††: External strategies to spread innovations including policy and regulations (governmental or other central entity), external mandates, recommendations and guidelines, pay-for-performance, collaboratives, and public or benchmark reporting.
‡‡: Absorptive capacity for change, shared receptivity of involved individuals to an innovation, and the extent to which use of that innovation will be rewarded, supported, and expected within their organization.
§§: Individuals’ attitudes toward and value placed on the innovation, as well as familiarity with facts, truths, and principles related to the innovation.
∥∥: Extent to which patient needs, as well as barriers and facilitators to meet those needs, are accurately known and prioritized by the organization.
¶¶: Nature and quality of webs of social networks, and the nature and quality of formal and informal communications within an organization.
##: Degree to which a scheme or method of behavior and tasks for implementing an innovation are developed in advance, and the quality of those schemes or methods.
***: Tangible and immediate indicators of organizational commitment to its decision to implement an innovation.
†††: Quantitative and qualitative feedback about the progress and quality of implementation accompanied with regular personal and team debriefing about progress and experience.
‡‡‡: Individuals’ shared perception of the importance of the implementation within the organization.
§§§: Individuals from within the organization who have been formally appointed with responsibility for implementing an innovation as coordinator, project manager, team leader, or other similar role.
∥∥∥: Commitment, involvement, and accountability of leaders and managers with the implementation of the innovation.

Telestroke and Emergency Care

One of the most commonly studied applications of provider-to-provider telehealth in rural areas is the diagnosis and management of stroke due to the higher prevalence of stroke and stroke risk factors in rural areas and treatments that require accurate diagnosis and timely administration.¹⁸⁸^, ¹⁸⁹ The telestroke programs described in this section bridge ED and inpatient care as they include consultations as part of initial assessment and triage as well treatment decisions and care delivery. We did not identify studies of the implementation of EMS telestroke programs in which consultations focus on prehospital triage and decisions made in the field about where the patient should be transported.

Table B-10 provides an overview of seven studies (reported in eight articles)⁶^, ⁷^, ¹⁴^, ²⁸^, ³³^, ⁷⁷^, ¹⁴⁶ of facilitators, barriers and impact related to the implementation of telestroke programs (Additional details in Appendix Table D-10). All of the projects studied were hub-spoke models in which one or more hubs where specialists were located were connected with rural hospitals. Evaluations of these programs included studies of statewide programs in West Virginia⁶ and South Carolina,⁷^, ¹⁴⁶ case studies of networks around a single hub,³³ a comparison of the early implementation of a network in South Carolina to one in Georgia¹⁴⁷ and an assessment of a regional program in Australia.¹⁴^, ²⁸ One evaluation reviewed state laws and regulations in the United States.⁷⁷

The telestroke implementation studies were of successful programs and the evaluations focused on the factors that supported this success. A case study comparing the early (1991) implementation in two networks reported that the networks had not integrated the technology into their care delivery processes and identified enablers which continue to be called out in other, more recent studies including: resource needs, the key role of performance monitoring and continuous improvement; the importance of a champion and dedicated coordinator at spokes, stakeholder involvement, and tangible goals such as stroke center certification¹⁴⁷ A frequently cited approach included stepped or phased implementation that started with preliminary needs and workflow assessments to inform pilot tests; diversity of engagement and funding, including private and government support; the need for staff to support the program, and training; and the importance of ongoing evaluation and program improvement. Barriers were less frequently cited but included lack of sufficient IT support, lack of integration of records and patient data, and the prohibition on fees or additional reimbursement for telehealth infrastructure in some states. Reports on telestroke implementation also focused on the impact on care and organizational outcomes, citing fewer transfers and certification as a stroke center as motivation to continue to sustain and improve the programs.

We identified studies of implementation of provider-to-provider consultations for general emergency care, pediatric emergencies and psychiatric emergencies in addition to telestroke. Five studies were of models in which a remote specialist or emergency physician advises a generalist physician, nurse practitioner or nurse who staff a rural emergency room.⁸⁸^, ⁹⁰^, ¹⁵⁰^, ¹⁶⁶^, ¹⁸⁰ Follow-up with rural EDs that did and did not use telehealth based on a United States national survey, found that 67% of nonusers had considered implementing telehealth but reported that the single most important reason the ED is not using telehealth was cost (37% of respondents), followed by technologic concerns and the assessment that telehealth is not needed to meet patient’s needs (11% each). Despite these barriers, six percent reported they had started to use telehealth since the original survey.¹⁸⁰ Costs cited in this and other studies included the cost of technology but also the cost of the subscription services that provide access to the consultants. Additional barriers included rural providers lack of understanding of telehealth, lack of perception that telehealth will address a clear problem and perceptions that the motivation is to save money, particularly on personnel. For this particular use the identified facilitators were general satisfaction and having a telehealth coordinator who could handle scheduling and technology.

Two studies assessed implementation of telehealth specifically for pediatric emergencies.¹³⁴^, ¹⁷⁰ One is an example of the few studies that compared different models; one model that provided only pediatric specialty consultations and one in which pediatrics was one of several specialties provided as part of a system wide consult service.¹⁷⁰ This study found that both models were considered successful, but produced very different results as they served different populations. The specialist only model was used for more critical cases while in the other, pediatric consults were used less, but used for both high and low risk cases. As a result, perceptions and measures of the systems differed. These studies identified specific factors that were not emphasized in other studies, such as the need to test technology that is not in frequent use and the importance of building rapport and assuring telehealth fits in the culture of the practice.

Two studies in three articles assessed telehealth psychiatric consultations for patients presenting in EDs. One study evaluated two different network approaches to emergency psychiatric telehealth: a regional network, with assessments and consults available as part of a system that provides multispecialty consults for MI, stroke and other acute illnesses as well as behavior health, by pressing a button compared to a smaller, local system in which a behavioral health specialist was paged when needed.¹⁶⁹ The assessment focused on patient characteristics and confirmed that both models increase access to inpatient care, but the evaluation did not explore detailed implementation differences in the systems. A study in northern Norway ¹⁶⁰^, ¹⁶¹ evaluated a system that made consultation available 24/7 by telephone and video; the study reported that using well established technology and having a safety net system supported the implementation and use of telehealth consults.

We identified one study that specifically addressed implementation of telehealth in prehospital care by EMS. A small study in Scotland⁹⁴ reported several major barriers to the use of remotely guided ultrasound in prehospital care. These included a lack of evidence and lack of documented need for remote guided ultrasound as well as different perceptions of EMS personnel and consulting physicians about skills and priorities.

Table B-10Findings of telestroke and emergency care implementation studies

Topic Number of Studies Intervention	Method N^* Location	Facilitators	Barriers	Impact
Telestroke 7 Hub:Spoke	Registries and administrative Data N=2 statewide studies West Virginia⁶ South Carolina⁷^, ¹⁴⁶ Case study N=1 program South Carolina³³ Case studies N=2 networks in South Carolina/ Georgia¹⁴⁷ Case study including reports and interviews N=16 stakeholder reports, 13 funder reports, 10 protocols, 3 collaborative agreements, 93 meeting minutes Australia ¹⁴^, ²⁸ Review of state statutes and regulations N=50 states United States⁷⁷	Stepped implementation: pre, pilot, full, sustainability Engagement Diverse support and funding (governmental health and non-health, philanthropic) Site champions Pre-implementation clinician surveys State legislation Shadowing at hubs Training for all roles with technology and guidelines Sustainable technology Local site coordinator to support start-up at spokes Expansion locations based on data Feedback on performance; ongoing evaluation and promotion of success Focus on application of telehealth that is sustainable and improved care Licensing and allowing practice across state lines Same reimbursement for in-person or additional technology fee	Video conferencing, imaging and clinical records not integrated IT support not available 24/7 Prohibition of fees or reimbursement in some states related to telehealth infrastructure	Increase in consult use Appropriate triage and treatment (tPA) Fewer transfers Certification of additional stroke centers Diffusion of knowledge Expansion from pilots to regional or multisite programs
Emergency Care: Non specific 6 Remote consultation Rural: Primary care provider, Nurse practitioner, nurse or generalist Consultant: Specialists or ED physician or nurse.	Survey of rural EDs N=153/177 telehealth users;375/453 non users U.S.¹⁸⁰ Surveys and hospital records Pre/Post implementation N=9 hospitals Mississippi¹⁵⁰ EMR data, interviews, and site visits N=85 administrator at 26 rural hospitals South Dakota Avera Health¹⁶⁶ Financial data, interviews, site visits N=1 emergency system; 49 rural hospitals, same interviews as Ward above South Dakota Avera Health⁹⁰ Interviews N=12 New Zealand⁸⁸ Program evaluation N = 206 patient records Australia¹²³	Rural provider satisfaction Having a telehealth coordinator Communication Executive Sponsorship	Cost of technology or subscription Lack of understanding of telehealth No relationship or experience with distance provider Perception it is about saving money, not improving care Lack of a clear problem telehealth will solve Technology issues include lack of bandwidth and power cuts	Increase in rural hospital admissions (sterling) -Increase in volume -Decrease in inappropriate admissions Low rate of utilization, but used when expert needed Facilitates transfers, documentation, urgent critical care when physicians not in ED Maybe profitable if hospital is able to increase revenue admissions and save on local physician back up for NP/PA staffing ED High level of use (67% of NP used system every shift)
Emergency Care: Pediatrics 2 Pediatrics as part of multispecialty consults service; Dedicated pediatric service	Two models (University of California Davis-specialty/hub vs. Advera -general ED including pediatrics) N=30 hospitals; 15 each¹⁷⁰ Survey based on themes from interviews N=7 hospitals, 48 interviews, surveys 5 hospitals 104 (34%) of 306 clinicians invited University of Pittsburgh¹³⁴	Weekly test calls to preemptively solve connection issues Hub providers build rapport by attending remote ED staff meetings and collaborating on quality initiatives Perception that telehealth is useful	Initially use not part of culture or care processes Lack of provider comfort with telehealth Technology issues Lack of time; negative impact on workflows	Unclear when use is appropriate and whether goal is to reduce transfers and/or discharges
Emergency Care: Psychiatrics 2 studies (3 articles)	Compare 2 ED Behavioral Health Models N=19 spoke hospitals; 2 networks) U.S. Midwest¹⁶⁹ Interviews N=29 Norway¹⁶⁰^, ¹⁶¹	Well established technology Ability to confirm initial assessment and collaborative solve problems System provides safety net	Not reported	Increase in admissions to inpatient facilities (interpreted as an increase in access) Immediacy of assessment Engagement with patient Access to specialist/MD that is not available locally Reduced uncertainty
EMS: Ultrasound 1	Interviews N=12 Scotland⁹⁴	Willingness to collaborate in training and care	Difference in EMS and MD perceptions of utility and skills needed Unclear evidence of need and benefits Perception it could result in delay in transport	Not reported

*: N is used here to represent the unit of analysis, which may be number of individual participants or may be number of health care sites or systems.
: Abbreviations: ED = emergency department; EMS = emergency medical services; EMR = electronic medical record; IT = information technology; NP = nurse practitioner; PA = physician’s assistant; tPA = tissue plasminogen activator; U.S. = United States.

These barriers and facilitators are presented according to the CFIR constructs in Table B-11. The barriers are distributed across constructs, with mosted cited one to three times. The most frequently identified category of barrier was knowledge & beliefs about the innovation with this cited five times. The facilitators were more concentrated in categories that were also frequent in in- and out-patient studies; access to knowledge and information (11), available resources (9), and patient needs & resources (7). One construct that was more frequent in emergency care was engaging (7) which represents including the right people in the implementation, which may respresent the need for emergency care to coordinate activities and processes across organizations such as EMS, multiple hospitals and outpatient care.

Table B-11Emergency care: barriers and facilitators by CFIR constructs

Type	Facilitator or Barrier Name	Facilitator or Barrier Number of Mentions	Reference Number(s)
Barrier	Access to Knowledge & Information^*	2	²⁸^,⁸⁸
	Adaptability^†	1	¹⁷⁰
	Available Resources^‡	4	⁸⁸^,¹³⁴^,¹⁷⁰
	Compatibility^§	4	⁹⁴^,¹³⁴^,¹⁸⁰
	Cost^∥	1	¹⁸⁰
	Executing^¶	2	²⁸
	External Policy & Incentives^#	1	⁷⁷
	Implementation Climate^**	1	⁸⁸
	Knowledge & Beliefs about the Innovation^††	5	⁹⁴^,¹⁷⁰^,¹⁸⁰
	Networks & Communications^‡‡	2	⁸⁸^,¹²³
	Readiness for Implementation^§§	3	⁸⁸^,¹⁸⁰
	Relative Priority^∥∥	3	²⁸^,⁹⁴^,¹⁸⁰
Facilitators	Access to Knowledge & Information^*	11	⁷^,¹⁴^,²⁸^,³³^,⁸⁸^,¹³⁴^,¹⁴⁶^,¹⁴⁷
	Available Resources^‡	9	⁷^,¹⁴^,³³^,⁸⁸^,¹⁴⁷^,¹⁶⁶^,¹⁶⁹^,¹⁷⁰
	Cost^∥	2	⁷⁷^,⁹⁰
	Engaging^¶¶	7	¹⁴^,²⁸^,³³^,¹⁴⁷
	Executing^¶	1	³³
	External Policy & Incentives^#	4	¹⁴^,⁷⁷
	Formally Appointed Internal Implementation Leaders^##	3	¹⁴^,³³^,¹⁴⁷
	Implementation Climate^**	3	²⁸^,⁹⁴^,¹⁴⁷
	Knowledge & Beliefs about the Innovation^††	1	¹⁷⁰
	Leadership Engagement^***	4	²⁸^,¹²³^,¹⁴⁷
	Patient Needs & Resources^†††	7	⁶^,⁷^,⁸⁸^,¹⁵⁰^,¹⁶⁰^,¹⁶¹
	Networks & Communications^‡‡	4	⁶^,⁹⁴^,¹⁷⁰
	Planning^‡‡‡	2	²⁸^,¹⁴⁷
	Readiness for Implementation^§§	5	⁷^,¹⁴^,³³^,¹⁴⁷
	Reflecting & Evaluating^§§§	4	¹⁴^,¹⁴⁶^,¹⁴⁷

*: Access to digestible information and knowledge about the innovation and how to incorporate it into work tasks.
†: Degree to which an innovation can be adapted, tailored, refined, or reinvented to meet local needs.
‡: Level of resources organizational dedicated for implementation and on-going operations including physical space and time.
§: Degree of tangible fit between meaning and values attached to the innovation by involved individuals, how those align with individuals’ own norms, values, and perceived risks and needs, and how the innovation fits with existing workflows and systems.
∥: Costs of the innovation and costs associated with implementing the innovation including investment, supply, and opportunity costs.
¶: Carrying out or accomplishing the implementation according to plan.
#: External strategies to spread innovations including policy and regulations (governmental or other central entity), external mandates, recommendations and guidelines, pay-for-performance, collaboratives, and public or benchmark reporting.
**: Absorptive capacity for change, shared receptivity of involved individuals to an innovation, and the extent to which use of that innovation will be rewarded, supported, and expected within their organization.
††: Individuals’ attitudes toward and value placed on the innovation, as well as familiarity with facts, truths, and principles related to the innovation.
‡‡: Nature and quality of webs of social networks, and the nature and quality of formal and informal communications within an organization.
§§: Tangible and immediate indicators of organizational commitment to its decision to implement an innovation.
∥∥: Individuals’ shared perception of the importance of the implementation within the organization.
¶¶: Attracting and involving appropriate individuals in the implementation and use of the innovation through a combined strategy of social marketing, education, role modeling, training, and other similar activities.
##: Individuals from within the organization who have been formally appointed with responsibility for implementing an innovation as coordinator, project manager, team leader, or other similar role.
***: Commitment, involvement, and accountability of leaders and managers with the implementation of the innovation.
†††: Extent to which patient needs, as well as barriers and facilitators to meet those needs, are accurately known and prioritized by the organization.
‡‡‡: Degree to which a scheme or method of behavior and tasks for implementing an innovation are developed in advance, and the quality of those schemes or methods.
§§§: Quantitative and qualitative feedback about the progress and quality of implementation accompanied with regular personal and team debriefing about progress and experience.

Education/Mentoring

Table B-12 includes a summary of information from 13 studies or reports on the use of telehealth for training and mentoring health care providers in rural areas (Additional details in Appendix D-10).

Nine of these studies are assessments of ECHO programs. ECHO combines didactic training, case presentations, virtual clinics and peer support to increase capacity and quality of care.⁹^, ⁵⁶^, ⁷³^, ¹¹⁵^, ¹²⁷^, ¹⁴⁴^, ¹⁵⁷^, ¹⁷⁴ The ECHO programs evaluated in these articles address different topics with four focused on pain or opioids,⁵⁶^, ⁷³^, ¹⁴⁴^, ¹⁵⁷ one each about Hepatitis C,¹²⁷ Multiple Sclerosis,⁹ and HIV in pregnancy,¹¹⁵ and one program that implemented ECHO as part of a larger expansion of specialist care.¹⁷⁴ While the subject matter and scale of these programs differed, facilitators, challenges and impact were similar. A frequently cited barrier was lack of clinician time to attend sessions or issues with scheduling. The evaluations also acknowledged that while ECHO could increase provider knowledge and skills it could not address all policy and practice barriers to practice change. Most ECHO program evaluations report a high level of stakeholder support and that the programs address rural participants needs for peer interaction, current knowledge, and access to experts. Evaluations of the impact of ECHO programs have documented that participants have changed practice, managed patients they would have referred, engaged in consultations with the expert faculty outside of the ECHO program, and become resources for other providers in their communities.

The three additional evaluations all address the rural clinicians’ need for training in emergency care. One evaluation documented how training could be incorporated into consultations, reporting how this assured training was useful and relevant.¹⁸³ An experimental study tested whether training medical students using simulations for relatively rare emergency procedures could be managed by a remote expert trainer and documented that this was feasible and produced similar educational outcomes.⁶⁸ An assessment of remote training for emergency care in Australia documented that training could reduce professional isolation, but that sometimes topics were not relevant to rural working conditions.¹²¹

Table B-12Findings of education and mentoring implementation studies

Topic

Number of studies

Intervention

Method

Location

Facilitators

Barriers

Impact

ECHO

Program with remote training, case reviews/clinics and peer interaction ECHO as part of a comprehensive program including e-consults

Hepatitis C

Survey to participants and non

N=32 of 72 contacted; 15 facilities; Indian Health Service¹²⁷

Multiple Sclerosis

Pre questionnaire, program manager observations, exit interviews focused on case presentations

N=8 of 24 clinicians, 13 practice sites; Mississippi, Washington State, Alaska, Montana, Idaho⁹

HIV in pregnancy

Survey and review of cases

N=41 of 53 surveys, 11 cases

Perinatal HIV; Washington State, Alaska, Montana, Idaho, Oregon, Colorado¹¹⁵

Chronic Pain

Content analysis

N=Random selection of 25 of 67 cases; 406 data units; U.S. Connecticut. Single federally qualified health center¹⁵⁷

Buprenorphine

Interviews

N=20; U.S. North Carolina¹⁴⁴

Pain and Opioid Management with 2–3 in person supplemental training

Questionnaires and focus groups

N=38 participants; 2 workshops; New Mexico Indian Health Service⁷³

Cancer Pain Survey of participants and non-participants

N=24(46%) who attended education; 32 (34%) who attended case conferences; U.S. New Mexico⁵⁶

Multiple Topics

Surveys of clinician leads, administrative data

N=180 from 87 sites; U.S. VA¹⁷⁴

Melanoma

Survey

N = 10 Primary care providers; U.S. Missouri¹⁷

Preference or need for
- Collegial discussion with peers
- Being well-informed
- Access to experts
Willingness to present cases. conferences
High level of stakeholder support
Adaptability to local needs
Compatibility with existing workflows and systems
Feedback on progress
Quality of networks and communications
Leadership engagement
Access to Knowledge & Information
Networks & Communications

Lack of time to attend
Scheduling/time of day
Sessions seemed too long
Lack of local leadership support
Negative staff attitudes/stigma toward topic (Opioid use disorder)
ECHO cannot address policy issues (e.g., regulations and payment policy)
Assessment and screening skills require in-person add on

Increased ability to manage and treat, knowledge and confidence in existing treatment or changed care for case patient
Changed practice for other patients
Managed patients who would have been referred
Content consistent with guidelines
Participants engaged in additional consultations with faculty experts
Participants scored higher on competence than non-participants
Participants serve as resource for other providers

Telehealth Training

Training incorporated as part of the consultation¹⁸³

Simulation training lead by remote expert⁶⁸

Training needs assessment¹²¹

Emergency Care

Interviews

N=18 hospitals

Kansas, Minnesota, Nebraska, North Dakota, South Dakota Avera tele-training during consultation¹⁸³

High-Acuity Low-Occurrence

Procedures

Assessments

N=69 medical students

Randomized to telehealth, in person and no training

Newfoundland, Canada⁶⁸

Emergency Care

Interviews

N=20 rural physician

Australia¹²¹

Topics of formal training are useful
Real-time training can include exactly what is needed
Respect and supportive teaching from consultants
Interactive sessions preferred over didactic
Ability to reduce professional isolation
Broadcast live procedures
Technology that functions well.

Timing of live sessions
Workload constraints
Limited relevance to current working conditions

Asynchronous training courses included in program were valued
Training increased confidence and improved performance
Simulation training is feasible from a distance; results in similar education outcomes

Structured Dentist Network

Dentistry

Interview and focus group

N= 2 Dental specialists and 8 General dentists

Australia⁸³

Networks & Communications

Access to Knowledge & Information
Planning
Available Resources

Lower proportion of patients treated at the Special Needs Unit, reflecting increased treatment at local facilities

: Abbreviations: ECHO = Extension for Community Healthcare Outcomes; HIV = human immunodeficiency virus; U.S. = United States.

While telehealth interventions for education and mentoring differ in format and outcomes from those that are targeted to care of a specific patient, barriers and facilitators were able to be mapped to CFIR constructs as the framework was designed to be applicable across different types of interventions. Table B-13 summarizes these. The most frequently cited barrier was compatibility (8), being when education did not correspond to the need or environment of the trainees. Not surprisingly, the most common facilator was access to knowledge and information (11) as the telehealth education and mentoring programs were more likely to be implemented when they provided accessible information that could be directly incorporated into the trainees tasks and environment.

Table B-13Education/mentoring: barriers and facilitators by CFIR constructs

Type	Facilitator or Barrier Name	Facilitator or Barrier Number of Mentions	Article Reference IDs
Barrier	Access to Knowledge & Information^*	3	⁷³^,⁸³
	Adaptability^*	1	¹²¹
	Available Resources^‡	4	⁵⁶^,⁷³^,⁸³^,¹²⁷
	Compatibility^§	8	¹²¹^,¹²⁷^,¹⁴⁴^,¹⁸³
	External Policy & Incentives^∥	2	¹⁴⁴
	Implementation Climate^¶	3	¹²¹^,¹²⁷
	Knowledge & Beliefs about the Innovation^#	1	¹⁴⁴
	Leadership Engagement^**	1	¹⁴⁴
	Planning	1	⁸³
Facilitator	Access to Knowledge & Information^*	12	¹⁷^,¹²¹^,¹²⁷^,¹⁴⁴^,¹⁷⁴^,¹⁸³
	Adaptability^*	2	⁵⁶^,¹⁷⁴
	Available Resources^‡	2	⁶⁸^,¹²¹
	Compatibility^§	2	¹²¹^,¹⁷⁴
	Engaging^††	3	¹⁵⁷^,¹⁷⁴
	Implementation Climate^¶	3	⁷³^,¹⁴⁴^,¹⁷⁴
	Leadership Engagement^**	2	¹⁷⁴
	Patient Needs & Resources^‡‡	5	⁹^,¹¹⁵^,¹²¹^,¹²⁷
	Networks & Communications^§§	6	¹⁷^,⁸³^,¹²⁷^,¹⁵⁷^,¹⁷⁴^,¹⁸³
	Reflecting & Evaluating^∥∥	1	¹⁷⁴
	Relative Priority^¶¶	1	¹⁷⁴

*: Access to digestible information and knowledge about the innovation and how to incorporate it into work tasks.
†: Degree to which an innovation can be adapted, tailored, refined, or reinvented to meet local needs.
‡: Level of resources organizational dedicated for implementation and on-going operations including physical space and time.
§: Degree of tangible fit between meaning and values attached to the innovation by involved individuals, how those align with individuals’ own norms, values, and perceived risks and needs, and how the innovation fits with existing workflows and systems.
∥: External strategies to spread innovations including policy and regulations (governmental or other central entity), external mandates, recommendations and guidelines, pay-for-performance, collaboratives, and public or benchmark reporting.
¶: Absorptive capacity for change, shared receptivity of involved individuals to an innovation, and the extent to which use of that innovation will be rewarded, supported, and expected within their organization.
#: Individuals’ attitudes toward and value placed on the innovation, as well as familiarity with facts, truths, and principles related to the innovation.
**: Commitment, involvement, and accountability of leaders and managers with the implementation of the innovation.
††: Attracting and involving appropriate individuals in the implementation and use of the innovation through a combined strategy of social marketing, education, role modeling, training, and other similar activities.
‡‡: Extent to which patient needs, as well as barriers and facilitators to meet those needs, are accurately known and prioritized by the organization.
§§: Nature and quality of webs of social networks, and the nature and quality of formal and informal communications within an organization.
∥∥: Quantitative and qualitative feedback about the progress and quality of implementation accompanied with regular personal and team debriefing about progress and experience.
¶¶: Individuals’ shared perception of the importance of the implementation within the organization.

Bookshelf ID: NBK588286

Contents

< Prev Next >

PubReader
Print View
Cite this Page
Totten A, Womack DM, McDonagh MS, et al. Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2022 Dec. (Comparative Effectiveness Review, No. 254.) Appendix B, Results.
PDF version of this title (2.7M)

Results - Improving Rural Health Through Telehealth-Guided Provider-to-Provider ...
Results - Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication
PCORI Methodology Standards Checklist - Treatment of Stages I-III Squamous Cell ...
PCORI Methodology Standards Checklist - Treatment of Stages I-III Squamous Cell Anal Cancer: A Systematic Review
Sample Abstract and Full-Text Review Forms - Evaluation and Treatment of Cryptor...
Sample Abstract and Full-Text Review Forms - Evaluation and Treatment of Cryptorchidism
Physiologic Predictors of Severe Injury: Systematic Review
Physiologic Predictors of Severe Injury: Systematic Review
Diagnosis of Gout
Diagnosis of Gout

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Bookshelf

Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication [Internet].

Appendix BResults

Results of Literature Searches

Figure B-1Literature flow diagram

Description of Included Studies

Key Question 1

Table B-1Characteristics of included studies for Key Question 1

Key Questions 2 and 4

Table B-2Characteristics of included studies for Key Question 2

Key Question 3

Table B-3Characteristics of included studies for Key Question 3

Table B-4Distribution of CFIR constructs

Table B-5Facilitators and barriers by topic area

Inpatient

Table B-6Findings of inpatient implementation studies

Table B-7Inpatient: barriers and facilitators by CFIR constructs

Outpatient

Table B-8Findings of outpatient implementation studies

Table B-9Outpatient: barriers and facilitators by CFIR constructs

Telestroke and Emergency Care

Table B-10Findings of telestroke and emergency care implementation studies

Table B-11Emergency care: barriers and facilitators by CFIR constructs

Education/Mentoring

Table B-12Findings of education and mentoring implementation studies

Table B-13Education/mentoring: barriers and facilitators by CFIR constructs

Views

In this Page

Other titles in this collection

Recent Activity