Identification and Recruitment of Stakeholders

We considered several different categories of types of stakeholders who might be interested in future research in this area when identifying potential individuals/organizations to invite. Categories included patient advocates, researchers, research funders, providers/practitioners, and policymakers; both federal and nonfederal representation was sought. We aimed to include individuals from both the respiratory and complementary and alternative medicine fields given the topic. Because the categories were not necessarily mutually exclusive, we aimed to identify individuals who had several roles and could represent multiple viewpoints. For instance, some practitioners were also members of professional societies and organizations who make guidelines for their field. Patient advocacy groups may not only represent the needs of patients, but might also advocate for and direct specific policy decisions.

In order to identify potential stakeholders, we started by identifying individuals who was considered for and/or participated as a Key Informant (KI) during topic refinement (n=11) or a Technical Expert Panel (TEP) member during the review process (n=15), and those who were invited or provided peer review (n=18) or public comments (n=3) on the draft CER (40 unique individuals). We identified an additional eight individuals in areas that we felt were underrepresented on the original list including individuals from federal agencies, professional societies, patient advocacy groups, and academic research institutions. Several of these individuals also served on the National Heart, Lung, and Blood Institute's National Asthma Education and Prevention Program Coordinating Committee. We also identified five investigators from the included studies in the CER that we felt might provide relevant insight into the current state of the research. Lastly, we asked the CER lead author for nominations and recommendations based on her experience working with the KIs, TEP members, and in addressing peer and public comment and we asked the stakeholders to refer colleagues where appropriate. We also identified seven individuals for back-up should specific stakeholders not be available. The resulting list included 60 potential stakeholders.

We narrowed the list of potential stakeholders based on our previous experience with individual stakeholders and in an attempt to ensure that an adequate mix of stakeholder categories was represented. In total, we invited 22 individuals to participate as stakeholders for this project. As described further, 12 individuals were invited to help identify and refine the research gaps and an additional 10 individuals were invited to help prioritize the final list of research questions. As part of the invitation, we provided potential stakeholders with a brief description of the project, including their role and the amount of time we expected them to participate, along with a copy of the original CER's executive summary. In the event that a potential stakeholder declined, we contacted an alternative stakeholder from our list or asked individuals to refer us to someone else from their organization and/or others working in the field that they thought would be a good fit. After agreeing to participate, each stakeholder was asked to complete a standard AHRQ Conflict of Interest (COI) disclosure form.

Identification of Evidence Gaps

We developed a preliminary list of evidence gaps by reviewing the CER and discussing the evidence gaps and research needs with the lead author of the CER. We sought to identify areas with insufficient evidence or where future research was warranted. We organized the evidence gaps and research needs according to the most relevant element of the PICO (Population, Intervention, Comparators, and Outcomes) framework.

Upon organizing the evidence needs and receiving COI disclosure forms, we scheduled 30 to 45 minute one-on-one interviews with the stakeholders. We sent them a copy of the preliminary list of evidence gaps and used a semi-structured interview guide (Appendix A) to facilitate the discussions. Stakeholders were asked to comment on whether or not they felt that the list of evidence needs was comprehensive and if there were better ways to express any of the research topics. We also asked if there were specific questions or topics that seemed out of place or did not belong on the list in terms of their importance to the field. Among stakeholders who specifically conducted research in this area, we asked them to describe, in general, what study they would conduct next if given funding to initiate a large study.

Based on input from the one-on-one interviews, the project team revised the original list of research needs, reframed them as research questions, and organized them according to the specific PICO they addressed. We also listed any methodological issues that were identified in the original CER or through our discussions for consideration in the final report. To ensure that we had adequately addressed all of the stakeholders' comments, we sent a follow-up e-mail with the revised list of research needs to all of the stakeholders who participated in a one-on-one interview and asked them to let us know if any of the questions were not clear or did not adequately represent our conversation with them. We also welcomed them to add additional research questions if they felt we had missed something.

Prioritization of Evidence Gaps

Once the stakeholders had an opportunity to review and edit the revised list of research needs we developed a web-based questionnaire using prioritization software developed by the Research Triangle Institute International and University of North Carolina EPC (Appendix B). The questionnaire listed 37 research questions organized according to the PICO element most relevant to the question (non-rank ordered within group). Of the 37 research questions, 16 were considered to be within the scope of the original CER while 21 were considered to be out-of-scope of the original CER. We asked stakeholders to prioritize among all of the research questions as to not diminish the questions that they felt were important yet not within the specific scope of this topic. The questionnaire used a forced prioritization exercise where we assigned each stakeholder a total of 20 “stars” or votes to assign amongst the 37 questions. Individual stakeholders could assign a maximum of five stars to any one research question. Respondents were asked to consider the following criteria when prioritizing the questions. These criteria were adapted from AHRQ's Effective Health Care Program and our recent experience in identifying and prioritizing CER questions within our organization.⁶

1. Impact: New research related to this question could potentially lead to improved health care quality, efficiency, or equity (i.e., better health outcomes, reduced variation in quality of care, cost improvement, or reduced health disparities).
2. Information Gap: More research on this topic is needed since it is not adequately answered by existing research.
3. Variation: Addressing this issue would contribute to solving important variation in research or clinical care, or controversy in what constitutes optimal care.
4. Uncertainty: Addressing the issue would contribute to solving important uncertainty for decision makers.

Data from the surveys were imported into Microsoft Excel^© (Microsoft, Seattle, WA) for analysis. In order to determine the most highly prioritized research questions, we examined the total number of votes assigned to each research question considered within the scope of the original CER. We also evaluated the level of consensus for each question (i.e., the number of participants who assigned at least one star to each question). Based on the distribution of votes, research questions that received a total of six or more stars (out of a possible 30 stars) were considered high priority, those with four to five stars were considered medium priority, and those with zero to three stars were considered low priority. Questions were considered to have a high level of consensus if more than half of the respondents (i.e., four or more) assigned at least one star to it.

Identification and Selection of Ongoing Studies and Funding Opportunities

After identifying the highly prioritized research questions we searched for ongoing studies and funding opportunities that were relevant to these needs.⁷ To identify ongoing or recently published research, we searched MEDLINE, Embase, Cumulative Index to Nursing and Allied Health Literature, Physiotherapy Evidence Database, Cochrane Central Register of Controlled Trials, AltHealthWatch, and Indian Medical Journals. In addition, we conducted searches of regulatory documents (e.g., the Authorized Medicine for the European Union and the U.S. Food and Drug Administration Medical and Statistical Reviews), clinical trial registries (e.g., National Institute of Health Research Portfolio Online Reporting Tools, ClinicalTrials.gov, the World Health Organization's International Clinical Trials Registry Platform, and International Standard Randomized Controlled Trial Number registry [also known as Current Controlled Trials]), and conference abstracts (e.g., CSA's Conference Paper Index, Scopus conference papers, ProceedingsFirst and PapersFirst). We searched these sources for relevant literature from the last search date of the CER (December 8, 2011) to June 25, 2012. The literature searches were not restricted to the English language. We replicated the original CER search in 16 databases (Appendix C). We reviewed titles and abstracts to identify relevant studies; only those that were not already included in the CER are included in this report.

To identify relevant funding opportunities, we searched for current funding opportunities posted by the National Institutes of Health (e.g., grants.gov) and organizations specific to the topic (e.g., American Lung Association and the Breathing Center [formerly Buteyko Center U.S.A.]) using keywords “asthma” and “breathing” where applicable. We also examined the funding sources of the included studies from the CER to identify additional organizations that may provide additional funding opportunities. Of 22 included studies, nine potential funding sources were identified: National Heart Lung and Blood Institute, Australian Association of Asthma Foundations, British Lung Foundation, National Asthma Campaign, Medical Capital Corporation, Centers for Disease Control and Prevention, Asthma UK, Royal College of General Practitioners, and the Central Council for Research in Yoga and Naturopathy. We searched for funding opportunities in June 2012 and we reviewed announcement details determine if the relevance of the funding to the highly prioritized research questions.

Approach to Research Design Considerations

We evaluated potential study designs to address each of the highest-ranked research questions considering several factors presented in an AHRQ methods paper.⁸ We considered factors such as advantages of the study design to produce a valid result; resource use, size, and duration; potential social, legal, and ethical issues; and availability of data or ability to recruit participants. It is clear that more than one study design may be appropriate to address a given research question. Likewise, specific study designs and study details are likely to apply to more than one research question. We imagine that the study design chosen is dependent on the study setting, administrative feasibility, and funding availability. Thus, this discussion is meant to provide considerations to potential researchers and funders for future research in this area, rather than being prescriptive or definitive.