Introduction
What is a Clinical Guideline?
Guidelines are recommendations for the care of individuals in specific clinical conditions or circumstances – from prevention and self-care through to primary and secondary care and on to more specialised services. NICE clinical guidelines are based on the best available evidence of clinical and cost effectiveness, and are produced to help healthcare professionals and patients make informed choices about appropriate healthcare. While guidelines assist the practice of healthcare professionals, they do not replace their knowledge and skills.
Clinical guidelines for the NHS in England, Wales and Northern Ireland are produced as a response to a request from the Department of Health (DH). They approve topics for guideline development. Before deciding whether to refer a particular topic to the National Institute for Health and Clinical Excellence (NICE) they consult with the relevant patient bodies, professional organisations and companies. Once a topic is referred, NICE then commissions one of four National Collaborating Centres (NCCs) to produce a guideline. The Collaborating Centres are independent of government and comprise partnerships between a variety of academic institutions, health profession bodies and patient groups. The National Collaborating Centre for Cancer (NCC-C) was referred the topic of metastatic malignant disease of unknown primary origin in April 2007 as part of NICE’s fourteenth wave work programme. However, the guideline development process began officially on 8 May 2008 when sufficient capacity became available at the NCC-C.
Who is the Guideline Intended For?
This guideline does not include recommendations covering every detail of the diagnosis and management of metastic malignant disease of unknown primary origin. Instead this guideline has tried to focus on those areas of clinical practice (i) that are known to be controversial or uncertain; (ii) where there is identifiable practice variation; (iii) where there is a lack of high quality evidence; or (iv) where NICE guidelines are likely to have most impact. More detail on how this was achieved is presented later in the section on ‘Developing Clinical Evidence Based Questions’.
This guideline is relevant to all healthcare professionals who come into contact with patients with metastic malignant disease of unknown primary origin, as well as to the patients themselves and their carers. It is also expected that the guideline will be of value to those involved in clinical governance in both primary and secondary care to help ensure that arrangements are in place to deliver appropriate care to this group of patients.
The Remit of the Guideline
Guideline topics selected by the DH identify the main areas to be covered by the guideline in a specific remit. The following remit for this guideline was received as part of NICE’s fourteenth wave programme of work:
‘To prepare a clinical guideline on the diagnosis and management of metastatic malignant disease of unknown primary origin, including service delivery where appropriate.’
What the Guideline Covers - The Scope
The remit was then translated into a scope document by the Guideline Development Group (GDG) Chair and Lead Clinician and staff at the NCC-C. The purpose of the scope was to:
provide an overview of what the guideline would include and exclude
identify the key aspects of care that must be included
set the boundaries of the development work and provide a clear framework to enable work to stay within the priorities agreed by NICE and the NCC-C and the remit set by the DH
inform the development of the clinical questions and search strategy
inform professionals and the public about the expected content of the guideline.
Prior to the commencement of the guideline development process, the scope was subject to a four week stakeholder consultation in accordance with processes established by NICE in the ‘NICE guidelines manual’ (NICE 2007). The full scope is shown in Appendix 5. During the consultation period, the scope was posted on the NICE website (www.nice.org.uk). Comments were invited from registered stakeholder organisations and the NICE Guideline Review Panel (GRP). Further information about the GRP can also be found on the NICE website. The NCC-C and NICE reviewed the scope in light of comments received, and the revised scope was reviewed by the GRP, signed off by NICE and posted on the NICE website.
Involvement of Stakeholders
Key to the development of all NICE guidance are the involvement of relevant professional and patient/carer organisations that register as stakeholders. Details of this process can be found on the NICE website or in the ‘NICE guidelines manual’ (NICE 2007). In brief, their contribution involves commenting on the draft scope, submitting relevant evidence and commenting on the draft version of the guideline during the end consultation period. A full list of all stakeholder organisations who registered for the metastatic malignant disease of unknown primary origin guideline can be found in Appendix 7.2.
Needs Assessment
As part of the guideline development process the NCC-C invited a specialist registrar, with the support of the GDG, to undertake a needs assessment (see Appendix 7.3). The needs assessment aims to describe the burden of disease and current service provision for patients with carcinoma of unknown primary in England and Wales, which informed the development of the guideline.
Assessment of the effectiveness of interventions is not included in the needs assessment, and was undertaken separately by researchers in the NCC-C as part of the guideline development process.
The information included in the needs assessment document was presented to the GDG. Most of the information was presented in the early stages of guideline development, and other information was included to meet the evolving information needs of the GDG during the course of guideline development.
The Process of Guideline Development – Who Develops the Guideline?
Overview
The development of this guideline was based upon methods outlined in the ‘NICE guidelines manual’ (NICE 2007). A team of health professionals, lay representatives and technical experts known as the Guideline Development Group (GDG) (see Appendix 7.1), with support from the NCC-C staff, undertook the development of this clinical guideline. The basic steps in the process of developing a guideline are listed and discussed below:
using the remit, define the scope which sets the parameters of the guideline
forming the GDG
developing clinical questions
systematically searching for the evidence
critically appraising the evidence
incorporating health economic evidence
distilling and synthesising the evidence and writing recommendations
agreeing the recommendations
structuring and writing the guideline
updating the guideline.
The Guideline Development Group (GDG)
The CUP GDG was recruited in line with the existing NICE protocol as set out in the ‘NICE guidelines manual’ (NICE 2007). The first step was to appoint a Chair and a Lead Clinician. Advertisements were placed for both posts and candidates were interviewed prior to being offered the role. The NCC-C Director, GDG Chair and Lead Clinician identified a list of specialties that needed to be represented on the GDG. Requests for applications were sent to the main stakeholder organisations and patient organisations/charities (see Appendix 7.2). Individual GDG members were selected by the NCC-C Director, GDG Chair and Lead Clinician, based on their application forms. The guideline development process was supported by staff from the NCC-C, who undertook the clinical and health economics literature searches, reviewed and presented the evidence to the GDG, managed the process and contributed to drafting the guideline. At the start of the guideline development process all GDG members’ interests were recorded on a standard declaration form that covered consultancies, fee-paid work, share-holdings, fellowships and support from the healthcare industry. At all subsequent GDG meetings, members declared new, arising conflicts of interest which were always recorded (see Appendix 7.1).
Guideline Development Group Meetings
Thirteen GDG meetings were held between 8 May 2008 and 5 October 2009. During each GDG meeting (either held over one or two days) clinical questions and clinical and economic evidence were reviewed, assessed and recommendations formulated. At each meeting patient/carer and service-user concerns were routinely discussed as part of a standing agenda item.
NCC-C project managers divided the GDG workload by allocating specific clinical questions, relevant to their area of clinical practice, to small sub-groups of the GDG in order to simplify and speed up the guideline development process. These groups considered the evidence, as reviewed by the researcher, and synthesised it into draft recommendations prior to presenting it to the GDG as a whole. Each clinical question was led by a GDG member with expert knowledge of the clinical area (usually one of the healthcare professionals). The GDG subgroups often helped refine the clinical questions and the clinical definitions of treatments. They also assisted the NCC-C team in drafting the section of the guideline relevant to their specific topic.
Patient/Carer Members
Individuals with direct experience of carcinoma of unknown primary services gave an integral user focus to the GDG and the guideline development process. The GDG included four patient/carer members. They contributed as full GDG members to writing the clinical questions, helping to ensure that the evidence addressed their views and preferences, highlighting sensitive issues and terminology relevant to the guideline and bringing service-user research to the attention of the GDG.
Sadly during development of the guideline two of the patient members of the group passed away. An additional patient member was recruited.
Expert Advisers
During the development phase of the guideline the GDG identified areas where there was a requirement for expert input on particular specialist clinical questions. The clinical questions were addressed by formal presentations by a recognised expert who had been identified by the GDG. A full list of recognised experts who contributed to the guideline can be found in Appendix 7.4.
Developing Clinical Evidence-Based Questions
Background
The scope, as described in Appendix 5, needs to be very clear about which patient groups are included and which areas of clinical care should be considered. However within these boundaries it does not usually specify which topics are considered a priority.
It was recognised by the NCC-C at an early stage that in order to complete the guideline development work to an appropriate standard the GDG needed to restrict its work to approximately 20 clinical questions. Previously this prioritisation would have been carried out by the GDG at its first two meetings but it was clear from some guidelines already published that this approach had resulted in a much larger number of questions than 20 being addressed.
Clinical guidelines should be aimed at improving clinical practice and should avoid ending up as ‘evidence-based textbooks’ or making recommendations on topics where there is already agreed clinical practice. It was therefore felt important that the 20 clinical questions should be prioritised into areas that were known to be controversial or uncertain, where there was identifiable practice variation, or where NICE guidelines were likely to have most impact.
Method
An extensive list of potential topics for the guideline to investigate was compiled by the NCC-C Director and GDG Chair and Lead Clinician. This list was incorporated into a questionnaire which asked respondents to rate each topic on a three point Likert scale ranging from 0 (low priority) to 2 (high priority). It was made clear that respondents would be rating the priority for each topic to be included in a clinical guideline to be published in two years’ time. The questionnaire also asked respondents to suggest any additional topics they would like included with an equivalent assessment of their priority.
Questionnaires were subsequently sent to the GDG in advance of the first GDG meeting.
The scores from each completed questionnaire were aggregated by NCC-C staff and ranked. These results together with information on identifiable practice variation (see needs assessment) were presented to the GDG at its first meeting. The list of prioritised topics produced via the questionnaire survey was in no way definitive and the GDG used these results to agree their final priorities for the clinical questions.
For clinical questions about interventions, the PICO framework was used. This structured approach divides each question into four components: the patients (the population under study – P), the interventions (what is being done - I), the comparisons (other main treatment options – C) and the outcomes (the measures of how effective the interventions have been – O). Where appropriate, the clinical questions were refined once the evidence had been searched and, where necessary, sub-questions were generated.
The final list of clinical questions can be found in Appendix 6.
Care Pathway
Early in the development process the GDG drafted an outline care pathway (or algorithm) in order to explore how patients with CUP might best access and be treated by the NHS.
Review of Clinical Literature
At the beginning of the development phase, initial scoping searches were carried out to identify any relevant guidelines (local, national or international) produced by other groups or institutions. Additionally, stakeholder organisations were invited to submit evidence for consideration by the GDG, provided it was relevant to the agreed list of clinical questions.
In order to answer each question the NCC-C information specialist developed a search strategy to identify relevant published evidence for both clinical and cost effectiveness. Key words and terms for the search were agreed in collaboration with the GDG. When required, the health economist searched for supplementary papers to inform detailed health economic work, for example modelling (see section on ‘Incorporating Health Economic Evidence’).
Papers that were published or accepted for publication in peer-reviewed journals were considered as evidence. Search filters, such as those to identify systematic reviews (SRs) and randomised controlled trials (RCTs) were applied to the search strategies when there was a wealth of these types of studies. No language restrictions were applied to the search; however, foreign language papers were not requested or reviewed (unless of particular importance to that question).
The following databases were included in the literature search:
The Cochrane Library
Medline and Premedline 1950 onwards
Excerpta Medica (Embase) 1980 onwards
Cumulative Index to Nursing and Allied Health Literature (Cinahl) 1982 onwards
Allied & Complementary Medicine (AMED) 1985 onwards
British Nursing Index (BNI) 1994 onwards
Psychinfo 1806 onwards
Web of Science 1970 onwards. [specifically Science Citation Index Expanded
(SCI-EXPANDED) and Social Sciences Citation Index (SSCI)]
System for Information on Grey Literature In Europe (SIGLE) 1980–2005
Biomed Central 1997 onwards
National Research Register (NRR)
Current Controlled Trials.
From this list the information specialist sifted and removed any irrelevant material based on the title or abstract before passing to the researcher. All the remaining articles were then stored in a Reference Manager electronic library.
Searches were updated and re-run 6–8 weeks before the stakeholder consultation, thereby ensuring that the latest relevant published evidence was included in the database. Any evidence published after this date was not included. For the purposes of updating this guideline, 9 October 2009 should be considered the starting point for searching for new evidence.
Further details of the search strategies, including the methodological filters used, are provided in the evidence review (and appear on the CD-ROM accompanying this guideline).
Critical Appraisal
Following the literature search one researcher independently scanned the titles and abstracts of every article for each question, and full publications were obtained for any studies considered relevant or where there was insufficient information from the title and abstract to make a decision. The researcher then individually applied the inclusion/exclusion criteria to determine which studies would be relevant for inclusion and subsequent appraisal. Lists of excluded papers were generated for each question and the rationale for the exclusion was presented to the GDG when required.
The researcher then critically appraised the full papers. Critical appraisal checklists were compiled for each paper and one researcher undertook the critical appraisal and data extraction.
For all the relevant appraised studies for a particular question, data on the type of population, intervention, comparator and outcomes (PICO) was recorded in evidence tables and an accompanying evidence summary prepared for the GDG (see evidence review). All the evidence was considered carefully by the GDG for accuracy and completeness.
All procedures were fully compliant with NICE methodology as detailed in the ‘NICE guidelines manual’ (NICE 2007). In general, no formal contact was made with authors; however, there were ad hoc occasions when this was required in order to clarify specific details.
Incorporating Health Economics Evidence
The aim of providing economic input into the development of the guideline was to inform the GDG of potential economic issues relating to CUP. It is important to investigate whether health services are both clinically effective and cost effective, i.e. are they ‘value for money’.
Prioritising topics for economic analysis
In addition to the review of the relevant clinical evidence, the GDG were required to determine whether or not the cost-effectiveness of each of the individual clinical questions should or could be investigated. After the clinical questions were decided, and with the help of the health economist, the GDG agreed which of the clinical questions were an economic priority for analysis. These ‘economic priorities’ were chosen on the basis of the following criteria, in broad accordance with the NICE guidelines manual (NICE 2007).
Overall relevance of the topic
The number of patients affected: interventions affecting relatively large numbers of patients were given a higher economic priority than those affecting fewer patients
The health benefits to the patient: interventions that that were considered to have a potentially significant impact on both survival and quality of life were given a higher economic priority
The per patient cost: interventions with potentially high financial (cost/savings) implications were given high priority compared to interventions expected to have lower financial implications
Likelihood of changing clinical practice: priority was given to topics that were considered likely to represent a significant change to existing clinical practice.
Uncertainty
High level of existing uncertainty: higher economic priority was given to clinical questions in which further economic analysis was considered likely to reduce current uncertainty over cost-effectiveness. Low priority was given to clinical questions when the current literature implied a clearly ‘attractive’ or ‘unattractive’ incremental cost-effectiveness ratio, which was regarded as generalisable to a UK healthcare setting
Likelihood of reducing uncertainty with further analyses (feasibility issues): when there was poor evidence for the clinical effectiveness of an intervention, there was considered to be less justification for an economic analysis to be undertaken.
For each topic that was prioritised for economic analysis a comprehensive systematic review of the economic literature was conducted. Where published economic evaluation studies were identified that addressed the economic issues for a clinical question, these are presented alongside the clinical evidence wherever possible. For those clinical areas reviewed, the information specialists used a similar search strategy as used for the review of clinical evidence but with the inclusion of a health economics and quality of life filter. Each search strategy was designed to find any applied study estimating the cost or cost effectiveness of the topic under consideration. A health economist reviewed abstracts and relevant papers were ordered for appraisal.
Published economic evidence was obtained from a variety of sources:
Economic Modelling
Once the priority topics for economic analysis had been agreed by the GDG, the health economist investigated whether or not a cost-effectiveness analysis of each topic could be carried out. Cost-effectiveness evaluations require evidence on numerous parameters, including treatment effects, health-related preferences (utilities), healthcare resource use and costs. However, high quality evidence on all relevant parameters within an economic analysis is not always available. If the evidence base used to inform a cost-effectiveness analysis is poor, decisions based upon such an analysis may be subject to a high degree of uncertainty and therefore cost effectiveness analysis would not be appropriate.
Expected Value of Perfect Information (EVPI)
Given the scarcity of high quality data to inform a cost effectiveness analysis in the metastatic malignant disease of unknown primary origin guideline the GDG agreed instead to assess the expected value of perfect information (EVPI) on one of the prioritised topics in the guideline (see Appendix 1).
EVPI is a decision analytical approach that allows health economists to estimate the cost of existing uncertainty within a particular clinical area (Briggs et al. 2006). It also enables the health economist to prioritise future research by identifying areas where collection of additional data will lead to a reduction in that current level of uncertainty. EVPI is calculated as the difference between the expected value of the decision made with perfect information and the decision made with current information.
Once the GDG had agreed to this approach the next task was to perform a systematic review of the literature. When relevant published evidence was identified and considered to be of sufficient quality, this information was used to inform the economic analysis. Assumptions and designs of the economic analysis were explained to and agreed by the GDG members during meetings, and they commented on subsequent revisions.
The details of the model are presented in the evidence review and Appendix 1. During the analysis the following general principles were adhered to:
the GDG Chair and Clinical Lead were consulted during the construction and interpretation of the analysis
assumptions were reported fully and transparently
the results were subject to thorough sensitivity analysis and limitations discussed
costs were calculated from a health services perspective.
Agreeing the Recommendations
For each clinical question the GDG were presented with a summary of the clinical evidence, and where appropriate economic evidence, derived from the studies reviewed and appraised. From this information the GDG were able to derive the guideline recommendations. The link between the evidence and the view of the GDG in making each recommendation is made explicit in the accompanying qualifying statement.
Qualifying Statements
As clinical guidelines are currently formatted, there is limited scope for expressing how and why a GDG made a particular recommendation from the evidence of clinical and cost effectiveness. To make this process more transparent to the reader, the NCC-C felt the need for an explicit, easily understood and consistent way of expressing the reasons for making each recommendation.
The way we have chosen to do this is by writing a ‘qualifying statement’ to accompany every recommendation and usually covering:
the strength of evidence about benefits and harms for the intervention being considered
the degree of consensus within the GDG
the costs and cost-effectiveness of an intervention (if formally assessed by the health economics team).
Where evidence was weak or lacking the GDG agreed the final recommendations through informal consensus. Shortly before the consultation period, ten key priorities and five key research recommendations were selected by the GDG for implementation and the patient algorithms were agreed. To avoid giving the impression that higher grade recommendations are of higher priority for implementation, NICE no longer assigns grades to recommendations.
Consultation and Validation of the Guideline
The draft of the guideline was prepared by NCC-C staff in partnership with the GDG Chair and Lead Clinician. This was then discussed and agreed with the GDG and subsequently forwarded to NICE for consultation with stakeholders.
Registered stakeholders (see Appendix 7.2) had one opportunity to comment on the draft guideline which was posted on the NICE website between 2 December 2009 and 2 February 2010 in line with NICE methodology (NICE 2009). The Guideline Review Panel also reviewed the guideline and checked that stakeholder comments had been addressed.
The pre-publication check process
Following stakeholder consultation and subsequent revision, the draft guideline was then subject to a pre-publication check (NICE 2009). The pre-publication check provides registered stakeholders with the opportunity to raise any concerns about factual errors and inaccuracies that may exist in the revised guideline after consultation.
During the pre-publication check the full guideline was posted on the NICE website for 15 working days, together with the guideline consultation table that listed comments received during consultation from stakeholders and responses from the NCC-C and GDG.
All stakeholders were invited to report factual errors using a standard proforma. NICE, the NCC and the GDG Chair and Lead Clinician considered the reported errors and responded only to those related to factual errors. A list of all corrected errors and the revised guideline were submitted to NICE, and the revised guideline was then signed off by Guidance Executive. The list of reported errors from the pre-publication check and the responses from the NCC-C were subsequently published on the NICE website.
The final document was then submitted to NICE for publication on their website. The other versions of the guideline (see below) were also discussed and approved by the GDG and published at the same time.
Updating the Guideline
Literature searches were repeated for all of the clinical questions at the end of the GDG development process, allowing any relevant papers published before 9 October 2009 to be considered. Future guideline updates will consider evidence published after this cut-off date.
Three years after publication of the guideline, NICE will commission a National Collaborating Centre to determine whether the evidence base has progressed significantly to alter the guideline recommendations and warrant an early update.
Disclaimer
The GDG assumes that healthcare professionals will use clinical judgment, knowledge and expertise when deciding whether it is appropriate to apply these guidelines. The recommendations cited here are a guide and may not be appropriate for use in all situations. The decision to adopt any of the recommendations cited here must be made by the practitioner in light of individual patient circumstances, the wishes of the patient and clinical expertise.
The NCC-C disclaims any responsibility for damages arising out of the use or non-use of these guidelines and the literature used in support of these guidelines.
