Identification of Evidence Gaps

As the original authors of the CER, we generated the initial list of FRN topics based on the Research Needs section of the report, and then organized the list of evidence gaps according to Key Questions and PICOD (Population, Intervention, Comparator, Outcomes, and study Design) elements. We then used an iterative process to identify additional FRN topics through Webinars and emails with a stakeholder panel (see Appendix A). The nominated topics were evaluated by EPC program staff to determine whether the nominated topics were related to one of the Key Questions of the CER. In general, nominated topics were deemed out-of-scope if the state of the evidence was not rigorously assessed as part of the CER. These out-of-scope topics were not entered to the final list of potential topics, but listed in the Discussion of this report. We asked the stakeholder panel to prioritize the FRN topics following a formalized schema of prioritization criteria.

Criteria for Prioritization

Stakeholders (described in the next section) were asked to consider four dimensions of need. These four dimensions are outlined in the Effective Health Care Program Selection Criteria (Appendix B). Briefly, they are:

• Importance
• Desirability of Research/Avoidance of Unnecessary Duplication
• Feasibility
• Potential Impact

The fifth dimension, Appropriateness, was not evaluated by the stakeholders, because the initial FRN topics based on the Research Needs section of the CER were already deemed to adequately meet this criterion by the Agency for Healthcare Research and Quality (AHRQ).

To further inform the selection criteria of Desirability of Research/Avoidance of Unnecessary Duplication, we also searched the National Library of Medicine Clinical Trial Registry (www.ClinicalTrials.gov) to identify ongoing or recently completed trials relevant to the CER questions (Appendix C). Relevant recently completed or registered studies identified in the searches were compared against the nominated FRN topics to assess if they would make future research on any nominated topic redundant, but none were judged to do so.

Engagement of Stakeholders, Researchers, and Funders

Although researchers and funders of research are the primary audience for FRN documents, the EPC solicits input from other stakeholders as well when identifying high-priority research gaps and FRN. Stakeholders are selected to provide broad expertise and a breadth of perspectives, as well as input on the kind of information that is helpful in health care decisionmaking. These stakeholders are engaged throughout the future research process. Their role is to (1) review the preliminary list of evidence gaps and possible future research topics derived from them, (2) to nominate additional topics to the list, (3) to discuss topic nominations, and (4) to participate in prioritization of the FRN topics. Stakeholders are not involved in translating the gaps into research questions and study designs, or composing or reviewing the report. The final FRN document will be released for public comment. Stakeholders who participated in the FRN identification process are invited to provide comments on this report during the public posting. Public input may be incorporated into or otherwise reflected in the final report.

To form the panel, we adapted a Tufts-developed “7Ps” model of stakeholder engagement,⁴⁴ which identifies seven primary stakeholder categories. The stakeholder categories of the 7Ps model consist of:

1. Patients and the Public. This group represents current and potential consumers of patient-centered health care and population-focused public health programs. This group also includes caregivers, family members, and patient advocacy organizations, all of whom represent the interests of consumers or patients.
2. Providers. This group includes individuals (e.g., nurses, physicians, and other providers of care and support services) and organizations (e.g., hospitals, clinics, community health centers, pharmacies, emergency medical services agencies, schools) that provide care to patients and populations.
3. Purchasers. This group includes employers; the self-insured; Federal, state, and local governments; and other entities responsible for underwriting the costs of health care.
4. Payers. This group represents private insurers, government insurers (e.g., Medicare, Medicaid, the Veterans Administration), and others responsible for reimbursement for care.
5. Policymakers. This group includes entities such as the legislative and executive branches of the Federal and state governments, professional associations, and other intermediary groups that collect and distribute information to policymakers.
6. Principal investigators. This group consists primarily of researchers, and research funders.
7. Product makers. This group consists primarily of manufacturers and device makers.

These categories are not necessarily mutually exclusive, and one stakeholder may belong to more than one category.

For this project, we recruited stakeholders representing Patients and the Public, Providers, Payers, Policymakers, and Principal investigators and asked them to provide input foremost according to their designated stakeholder category. Product makers were not included because it was not possible to include all product makers (particularly since there are many laboratory-based tests available in individual labs), and selected participation might bias the FRN discussions. Purchasers (employers who purchase insurance policies) were not included because they were considered to share the payer perspective for the diagnostic tests in question.

We identified individuals to serve on the stakeholder panel through several means. We invited some individuals who had previously served in advisory roles on the original CER. We contacted the National Kidney Foundation Patient and Family Council to invite CKD patient/patient advocate stakeholders. We used professional contacts to identify public and private payers and a policy maker. Individuals were selected based on their particular stakeholder perspective. Our predetermined goal was to assemble a representative panel of 10 stakeholders across the appropriate stakeholder categories (Table 3).

Table 3

List of predetermined target stakeholders.

All individuals involved in the project were required to submit a standard disclosure of interest form. Participation was only confirmed after review of the disclosure form. Stakeholders were asked to disclose any financial conflicts of interest greater than $10,000, and any other relevant business or professional conflicts of interest.

Individuals who met the criteria to participate as a stakeholder were contacted by email with a brief description of the project and its purpose, a formal invitation to serve as a stakeholder, and the executive summary of the original CER. Once their participation was confirmed, we sent additional materials to orient stakeholders, including the Future Research Needs section of the original CER and a proposed outline for this project.

The first round of Webinars was held in June 2012. All stakeholders attended. (Four duplicate Webinars were held to maximize participation.) During these Webinars, we reviewed the purpose and processes of this project. Subsequently stakeholders were asked to self-identify as a representative of a particular stakeholder category and directed to provide feedback chiefly with respect to their primary stakeholder category. We also directed stakeholders to restrict their FRN nominations to topics within the scope of the CER, specifically delineating out-of-scope topics such as pediatric CKD patient and cost effectiveness and cost utility of tests (based on the topic and study eligibility criteria of the CER). We then reviewed the research gaps identified in the CER and explained the criteria for prioritization. After this orientation, the stakeholders were asked to identify additional FRN topics of interest to them and to discuss their or others’ suggestions, their supporting rationale, or related research challenges. Stakeholders were also given the option of nominating additional topics by email afterwards.

Topics nominated by stakeholders were incorporated into the topic list along with supporting rationale, which we condensed from the discussion and subsequent emails. We combined duplicate or similar FRN topics together and disseminated the revised list of topics, along with minutes from all the webinars, with an invitation to comment as to whether the nominated topics and supporting rationales were appropriately recorded and accounted for.

We held a second round of Webinars in late June and early July 2012, during which we reviewed the refined list of nominated topics. Again all stakeholders attended the second Webinar (held in duplicate.) Stakeholders were asked if their nominations were appropriately captured and accounted for, and were provided the opportunity for further discussion.

Following this second round, we further edited the topic list based on stakeholder input. The finalized topic prioritization form and the minutes from the second set of webinars were sent to each stakeholder. Stakeholders were asked to review the minutes for accuracy and in order to hear the perspectives of others, and then to identify their top five priority topics with corresponding justification for each topic based upon AHRQ’s Effective Health Care Program Selection Criteria (Appendix B). The five topics with the highest number of stakeholder endorsements were designated as the prioritized FRN topics.

Use of the Effectiveness Health Care Program prioritization criteria was repeatedly emphasized throughout the prioritization process, including during discussion, nomination, and final topic selection. Upon the close of stakeholder prioritization, we identified the top five topics as those most frequently endorsed by stakeholders in their top five selections.

Research Question Development and Research Design Considerations

We transformed the final list of FRN topics into research questions using standard PICOD criteria. (Stakeholders were not utilized during this process.) We discussed various alternatives for future research efforts aimed at answering each question, specifically considering the feasibility of addressing the potential research questions with respect to sample size, the time required, recruitment, and possible ethical issues.