3.3. THE GUIDELINE DEVELOPMENT GROUP
The GDG was made up of professionals in psychiatry, clinical psychology, nursing, and general practice; academic experts in psychiatry and psychology; two former service users and a carer. The guideline development process was supported by staff from the NCCMH, who undertook the clinical and health economics literature searches, reviewed and presented the evidence to the GDG, managed the process, and contributed to drafting the guideline.
3.3.1. Guideline Development Group meetings
Seventeen GDG meetings were held between January 2007 and September 2008. During each day-long GDG meeting, in a plenary session, clinical questions and clinical and economic evidence were reviewed and assessed, and recommendations formulated. At each meeting, all GDG members declared any potential conflicts of interest, and service user and carer concerns were routinely discussed as part of a standing agenda.
3.3.2. Topic groups
The GDG divided its workload along clinically relevant lines to simplify the guideline development process, and GDG members formed smaller topic groups to undertake guideline work in that area of clinical practice. Topic group 1 covered questions relating to pharmacological interventions; topic group 2 covered psychological therapies (with a sub-group covering therapeutic communities); topic group 3 covered services; topic group 4 covered young people; and topic group 5 covered service user and family/carer issues. These groups were designed to manage the appraisal of the evidence more efficiently before presenting it to the GDG as a whole. Each topic group was chaired by a GDG member with expert knowledge of the topic area (one of the healthcare professionals or service users as appropriate). Topic groups refined the clinical questions, refined the clinical definitions of treatment interventions, reviewed and prepared the evidence with NCCMH staff before presenting it to the GDG as a whole, and also helped the GDG to identify further expertise in the topic. Topic group leaders reported the status of the group’s work as part of the standing agenda. They also introduced and led the GDG discussion of the evidence review for that topic and assisted the GDG Chair in drafting the section of the guideline relevant to the work of each topic group.
3.3.3. Service users and families/carers
Individuals with direct experience of services gave an integral service user/carer focus to the GDG and the guideline. The GDG included two former service users and one carer. They contributed as full GDG members to writing the clinical questions, helping to ensure that the evidence addressed their views and preferences, highlighting sensitive issues and terminology relevant to the guideline, and bringing service user/carer research to the attention of the GDG. In drafting the guideline, they contributed to writing a chapter on service user and family/carer issues for the full guideline, and to formulating recommendations from the service user and family/carer perspective.
3.3.4. Special advisors
Special advisors, who had specific expertise in one or more aspects of treatment and management relevant to the guideline, assisted the GDG, commenting on specific aspects of the developing guideline, including attending topic group meetings and teleconferences if appropriate. Appendix 3 lists those who acted as special advisors.
3.3.5. Researchers contacted for unpublished studies
National and international experts in the area under review were identified through the literature search and through the experience of the GDG members. These experts were contacted to recommend unpublished or soon-to-be published studies in order to ensure up-to-date evidence was included in the development of the guideline. They informed the group about completed trials at the pre-publication stage, systematic reviews in the process of being published, studies relating to the cost effectiveness of treatment and trial data if the GDG could be provided with full access to the complete trial report. Appendix 5 lists researchers who were contacted.
3.3.6. Peer reviewers
Peer reviewers were identified by the GDG to review the guideline during the consultation phase, as well as stakeholders. In addition, the review of pharmacological treatments was sent to international experts for peer review during the guideline development process because this section of the guideline was completed ahead of time and the draft recommendations were potentially controversial because they contradicted current clinical opinion. Therefore peer reviewers who were leaders in the field were appointed; they were named as special advisers to ensure that confidentiality was maintained (see Appendix 3). Their comments and responses from the GDG are presented in Appendix 11.
3.4. CLINICAL QUESTIONS
Clinical questions were used to guide the identification and interrogation of the evidence base relevant to the topic of the guideline. Before the first GDG meeting, draft questions were prepared by NCCMH staff based on the scope. They were then discussed by the GDG at their first two meetings and a final list drawn up. Where appropriate, the questions were refined once the evidence had been searched and, where necessary, sub-questions were generated. The final list of clinical questions can be found in Appendix 6.
For questions about interventions, the PICO (patient, intervention, comparison and outcome) framework was used. This structured approach divides each question into four components: the patients (the population under study), the interventions (what is being done), the comparisons (other main treatment options) and the outcomes (the measures of how effective the interventions have been) (see ).
Features of a well-formulated question on effectiveness intervention – the PICO guide.
To help facilitate the literature review, a note was made of the best study design to answer each question. There are four main types of clinical question of relevance to NICE guidelines. These are listed in . For each type of question, the best primary study design varies, where ‘best’ is interpreted as ‘least likely to give misleading answers to the question’. However, in all cases, a well-conducted systematic review of the appropriate type of study is likely to yield a better answer than a single study. Deciding on the best design type to answer a specific clinical or public health question does not mean that studies of different design types addressing the same question were discarded.
Best study design to answer each type of question.
3.5. SYSTEMATIC CLINICAL LITERATURE REVIEW
The aim of the clinical literature review was to systematically identify and synthesise relevant evidence from the literature in order to answer the specific clinical questions developed by the GDG. Thus, clinical practice recommendations are evidence-based, where possible. If evidence was not available, informal consensus methods were used (see Section 3.5.9) and the need for future research specified.
3.5.1. Methodology
A stepwise, hierarchical approach was taken to locating and presenting evidence to the GDG. The NCCMH developed this process based on methods set out in The Guidelines Manual3 (NICE, 2006b) and after considering recommendations from a range of other sources. These included:
Clinical Policy and Practice Program of the New South Wales Department of Health (Australia)
Clinical Evidence online
The Cochrane Collaboration
New Zealand Guidelines Group
NHS Centre for Reviews and Dissemination
Oxford Centre for Evidence-Based Medicine
Scottish Intercollegiate Guidelines Network (
SIGN)
United States Agency for Healthcare Research and Quality
Oxford Systematic Review Development Programme
Grading of Recommendations: Assessment, Development and Evaluation (
GRADE) Working Group.
3.5.2. The review process
After the scope was finalised, a more extensive search for existing systematic reviews and published guidelines was undertaken to inform the review process. The review team, in conjunction with the GDG, assessed the available existing systematic reviews for relevance to the clinical questions. This helped to assess the quantity and likely quality of available primary research. The initial approach taken to locating primary-level studies depended on the type of clinical question and availability of evidence.
The GDG then decided which questions were best addressed by good practice based on expert opinion, which questions were likely to have a good evidence base and which questions were likely to have little or no directly relevant evidence. Recommendations based on good practice were developed by informal consensus of the GDG. For questions with a good evidence base, the review process depended on the type of key question. For questions that were unlikely to have a good evidence base, a brief descriptive review was initially undertaken by a member of the GDG.
Searches
The standard mental health related bibliographic databases were searched including EMBASE, MEDLINE, PsycINFO, and Central, together with the grey literature database HMIC. Search filters developed by the review team consisted of a combination of subject heading and free-text phrases. Specific filters were developed for the guideline topic and, where necessary, for individual clinical questions (see relevant chapters for details). The topic-specific filters were combined with appropriate research design filters developed for systematic reviews, RCTs and other appropriate research designs (Appendix 7).
The review team also scanned the reference lists of included studies and existing systematic reviews for additional references, together with evidence submitted by stakeholders. Unpublished evidence was also sought (see below). In addition, the tables of contents of appropriate journals were checked regularly for relevant studies. Searches for evidence were re-run every 6 months during the guideline development process with the final search undertaken between 6 and 8 weeks before submission of the consultation drafts. After this point, studies were included only if they were judged by the GDG to be exceptional (for example, the evidence was likely to change a recommendation).
The search process for questions concerning interventions
For questions related to interventions, the initial evidence base was formed from well-conducted RCTs that addressed at least one of the clinical questions (the review process is illustrated in Flowchart 1). Although there are a number of difficulties with the use of RCTs in the evaluation of interventions in mental health, the RCT remains the most important method for establishing treatment efficacy (this is discussed in more detail in the appropriate clinical evidence chapters). For other clinical questions, searches were for the appropriate study design (see above).
Since it was known from a review of existing systematic reviews in this area that the evidence base for borderline personality disorder was relatively small, a search for all RCTs for this topic area was undertaken together regardless of intervention.
After the initial search results were scanned liberally to exclude irrelevant papers, the review team used a purpose-built study information database to manage both the included and the excluded studies (eligibility criteria were developed after consultation with the GDG). For questions without good-quality evidence (after the initial search), a decision was made by the GDG about whether to (a) repeat the search using subject-specific databases (for example, CINAHL, AMED, SIGLE or PILOTS), (b) conduct a new search for lower levels of evidence, or (c) adopt a consensus process (see Section 3.5.9).
Study selection
All primary-level studies included after the first scan of citations were acquired in full and re-evaluated for eligibility at the time they were being entered into the study information database. Appendix 8 lists the standard inclusion and exclusion criteria. More specific eligibility criteria were developed for each clinical question and are described in the relevant clinical evidence chapters. Studies were critically appraised for methodological quality (see Appendix 9 and Appendix 16). The eligibility of each study was confirmed by at least one member of the appropriate topic group.
For some clinical questions, it was necessary to prioritise the evidence with respect to the UK context (that is, external validity). To make this process explicit, the topic groups took into account the following factors when assessing the evidence:
participant factors (for example, comorbid diagnoses and setting)
provider factors (for example, model fidelity, the conditions under which the intervention was performed and the availability of experienced staff to undertake the procedure)
cultural factors (for example, differences in standard care and differences in the welfare system).
It was the responsibility of each topic group to decide which prioritisation factors were relevant to each clinical question in light of the UK context and then decide how they should modify their recommendations.
Unpublished evidence
The GDG used a number of criteria when deciding whether or not to accept unpublished data. First, the evidence must have been accompanied by a trial report containing sufficient detail to properly assess the quality of the data. Second, the evidence must have been submitted with the understanding that data from the study and a summary of the study’s characteristics would be published in the full guideline. Therefore, the GDG did not accept evidence submitted as commercial in confidence. However, the GDG recognised that unpublished evidence submitted by investigators might later be retracted by those investigators if the inclusion of such data would jeopardise publication of their research.
Flowchart 1. Guideline review process (PDF, 66K)
3.5.3. Outcomes
Outcome measurement on borderline personality disorder is problematic, partly because of the nature of the disorder and partly because of the relative immaturity of intervention research in this field. Since diagnosis of the disorder is based on the presence of five symptoms out of a possible total of nine with no requirement for the presence of particular symptoms (based on DSM-IV which is used by most treatment studies), trialists usually measure outcomes on all or some of these symptoms. In addition, more than one outcome measure has been developed for most areas of psychopathology caused by the disorder as well as psychosocial functioning affected by the disorder.
In order to deal with the plethora of outcomes reported by the trials forming the guideline’s evidence base, the GDG appointed a special advisor with expertise in this area (see Appendix 3). A list of outcomes reported by the studies considered by the GDG is in Appendix 10, together with information on which were used and which were not. For a rating scale to be considered a validation study had to be published in a peer-reviewed journal. In order to increase the power of the meta-analyses, scales reporting the same outcome were examined in detail to assess whether they could be combined. However, self-report and clinical-rated scales were not combined.
3.5.4. Data extraction
Outcome data were extracted from all eligible studies, which met the quality criteria, using a standardised form (see Appendix 8). Study characteristics were also extracted into an Access database. Full study characteristics are in Appendix 16 with summary tables in the evidence chapters.
For a given outcome (continuous and dichotomous), where more than 50% of the number randomised to any group were not accounted for4 by trial authors, the data were excluded from the review because of the risk of bias. However, where possible, dichotomous efficacy outcomes were calculated on an intention-to-treat (ITT) basis (that is, a ‘once-randomised-always-analyse’ basis). This assumes that those participants who ceased to engage in the study—from whatever group—had an unfavourable outcome. This meant that the 50% rule was not applied to dichotomous outcomes where there was good evidence that those participants who ceased to engage in the study were likely to have an unfavourable outcome (in this case, early withdrawals were included in both the numerator and denominator). Adverse effects were entered into Review Manager 4.2.8 (Cochrane Collaboration, 2005) as reported by the study authors because it was usually not possible to determine whether early withdrawals had an unfavourable outcome. For the outcome ‘leaving the study early for any reason’, the denominator was the number randomised.
Where some of the studies failed to report standard deviations (for a continuous outcome), and where an estimate of the variance could not be computed from other reported data or obtained from the study author, the following approach was taken5:
When the number of studies with missing standard deviations was small and when the total number of studies was large, the pooled standard deviation from all the other available studies in the same meta-analysis was used. In this case, the appropriateness of the imputation was made by comparing the standardised mean differences (SMDs) of those trials that had reported standard deviations against the hypothetical SMDs of the same trials based on the imputed standard deviations. If they converged, the meta-analytical results were considered to be reliable.
When the number of studies with missing standard deviations was large or when the total number of studies was small, standard deviations were taken from a previous systematic review (where available), because the small sample size may allow unexpected deviation due to chance. In this case, the results were considered to be less reliable.
Consultation was used to overcome difficulties with coding. Data from studies included in existing systematic reviews were extracted independently by one reviewer and cross-checked with the existing dataset. Where possible, two independent reviewers extracted data from new studies. Where double data extraction was not possible, data extracted by one reviewer was checked by the second reviewer. Disagreements were resolved with discussion. Where consensus could not be reached, a third reviewer resolved the disagreement. Masked assessment (that is, blind to the journal from which the article comes, the authors, the institution and the magnitude of the effect) was not used since it is unclear that doing so reduces bias (Jadad et al., 1996; Berlin, 1997).
3.5.5. Synthesising the evidence
Where possible, meta-analysis was used to synthesise the evidence using Review Manager 4.2.8 (Cochrane Collaboration, 2005). If necessary, reanalyses of the data or sub-analyses were used to answer clinical questions not addressed in the original studies or reviews.
Dichotomous outcomes were analysed as relative risks (RR) with the associated 95% confidence interval (CI) (for an example, see ). A relative risk (also called a risk ratio) is the ratio of the treatment event rate to the control event rate. An RR of 1 indicates no difference between treatment and control. In , the overall RR of 0.73 indicates that the event rate (that is, non-remission rate) associated with intervention A is about three quarters of that with the control intervention or, in other words, the relative risk reduction is 27%.
Example of a forest plot displaying dichotomous data.
The CI shows with 95% certainty the range within which the true treatment effect should lie and can be used to determine statistical significance. If the CI does not cross the ‘line of no effect’, the effect is statistically significant at the 5% significance level.
Continuous outcomes were analysed as weighted mean differences (WMD), or as a standardised mean difference (SMD) when different measures were used in different studies to estimate the same underlying effect (for an example, see ). If provided, ITT data, using a method such as ‘last observation carried forward’, were preferred over data from completers.
Example of a forest plot displaying continuous data.
To check for consistency between studies, both the I2 test of heterogeneity and a visual inspection of the forest plots were used. The I2 statistic describes the proportion of total variation in study estimates that is due to heterogeneity (Higgins & Thompson, 2002). The I2 statistic was interpreted in the following way:
>50%: notable heterogeneity (an attempt was made to explain the variation, for example outliers were removed from the analysis or sub-analyses were conducted to examine the possibility of moderators. If studies with heterogeneous results were found to be comparable, a random-effects model was used to summarise the results [
DerSimonian & Laird, 1986]. In the random-effects analysis, heterogeneity is accounted for both in the width of CIs and in the estimate of the treatment effect. With decreasing heterogeneity the random-effects approach moves asymptotically towards a fixed-effects model).
30 to 50%: moderate heterogeneity (both the chi-squared test of heterogeneity and a visual inspection of the forest plot were used to decide between a fixed and random-effects model)
<30%: mild heterogeneity (a fixed-effects model was used to synthesise the results).
To explore the possibility that the results entered into each meta-analysis suffered from publication bias, data from included studies were entered, where there were sufficient data, into a funnel plot. Asymmetry of the plot was taken to indicate possible publication bias and investigated further.
An estimate of the proportion of eligible data that were missing (because some studies did not include all relevant outcomes) was calculated for each analysis.
The number needed to treat–benefit (NNTB) or the number needed to treat–harm (NNTH) was reported for each outcome where the baseline risk (that is, control group event rate) was similar across studies. In addition, NNTs calculated at follow-up were only reported where the length of follow-up was similar across studies. When the length of follow-up or baseline risk varies (especially with low risk), the NNT is a poor summary of the treatment effect (Deeks, 2002). The percentage with the event in question was reported for each treatment group.
Included/excluded studies tables, generated automatically from the study information database, were used to summarise general information about each study (see Appendix 16). Where meta-analysis was not appropriate and/or possible, the reported results from each primary-level study were also presented in the included studies table (and included, where appropriate, in a narrative review).
Skewed data
Continuous data reported by the trials may not be normally distributed. While this is not so much of a problem in larger trials, effect sizes calculated from data from smaller trials should be treated with caution. Given that many of the trials reviewed for this guideline used relatively small populations, skewedness was assessed based on the definition that the mean is greater than two times the standard deviation. Evidence was downgraded where skewed data existed (see section on evidence profile tables below). All effect sizes calculated with skewed data are marked with an asterisk and should therefore be interpreted cautiously.
3.5.6. Presenting the data to the GDG
Summary characteristics tables and, where appropriate, forest plots generated with Review Manager 4.2.8 (Cochrane Collaboration, 2005) were presented to the GDG in order to prepare an evidence profile for each review and to develop recommendations.
Evidence profile tables
An evidence profile table was used to summarise both the quality of the evidence and the results of the evidence synthesis (see for an example of an evidence profile table). Each table included details about the quality assessment of each outcome: quality of the included studies based on the SIGN grade (see Appendix 9 for checklist), number of studies, and limitations, information about the consistency of the evidence (see below for how consistency was measured), directness of the evidence (that is, how closely the outcome measures, interventions and participants match those of interest) and any other considerations (for example, effect sizes with wide CIs would be described as imprecise data). Each evidence profile also included a summary of the findings: number of patients included in each group, an estimate of the magnitude of the effect, quality of the evidence, and the importance of the evidence. The quality of the evidence was based on the quality assessment components (study design, limitations to study quality, consistency, directness and any other considerations) and graded using the following definitions:
Example evidence profile for brief psychological interventions.
High = Further research is very unlikely to change confidence in the estimate of the effect
Moderate = Further research is likely to have an important impact on confidence in the estimate of the effect and may change the estimate
Low = Further research is very likely to have an important impact on confidence in the estimate of the effect and is likely to change the estimate
Very low = Any estimate of effect is very uncertain.
For further information about the process and the rationale of producing an evidence profile table, see GRADE Working Group (2004). Full evidence profiles are in Appendix 18 and summary profiles are included in the evidence chapters.
Forest plots
Each forest plot displayed the effect size and CI for each study as well as the overall summary statistic. The graphs were organised so that the display of data in the area to the left of the ‘line of no effect’ indicated a ‘favourable’ outcome for the treatment in question. Forest plots are in Appendix 17.
3.5.7. Determining clinical significance
In order to facilitate consistency in generating and drafting the clinical summaries, a decision tree was used to help determine, for each comparison, the likelihood of the effect being clinically significant (see Figure 3). The decision tree was designed to be used as one step in the interpretation of the evidence (primarily to separate clinically important from clinical negligible effects) and was not designed to replace clinical judgement. For each comparison, the GDG defined a priori a clinically significant threshold, taking into account both the comparison group and the outcome.
Figure 3. Decision tree for helping to judge the likelihood of clinical significance (PDF, 47K)
As shown in Figure 3, the review team first classified the point estimate of the effect as clinically significant or not. For example, if an RR of 0.75 was considered to be the threshold, then a point estimate of 0.73 (as can be seen in ), would meet the criteria for clinical significance. Where heterogeneity between studies was judged problematic, in the first instance an attempt was made to explain the cause of the heterogeneity (for example, outliers were removed from the analysis or sub-analyses were conducted to examine the possibility of moderators). Where homogeneity could not be achieved, a random-effects model was used.
Where the point estimate of the effect exceeded the threshold, a further consideration was made about the precision of the evidence by examining the range of estimates defined by the CI. Where the effect size was judged clinically significant for the full range of plausible estimates, the result was described as very likely to be clinically significant (that is, CS1). In situations where the CI included clinically unimportant values, but the point estimate was both clinically and statistically significant, the result was described as likely to be clinically significant (CS2). However, if the CI crossed the line of no effect (that is, the result was not statistically significant), the result was described as inconclusive (CS4).
Where the point estimate did not meet the criteria for clinical significance and the CI completely excluded clinically significant values, the result was described as unlikely to be clinically significant (CS3). Alternatively, if the CI included both clinically significant and clinically unimportant values, the result was described as inconclusive (CS4). In all cases described as inconclusive, the GDG used clinical judgement to interpret the results.
3.5.8. Forming the clinical summaries and recommendations
Once the evidence profile tables relating to a particular clinical question were completed, summary tables incorporating important information from the evidence profile and an assessment of the clinical significance of the evidence were produced (these tables are presented in the evidence chapters). Finally, the systematic reviewer along with the topic group lead produced a clinical summary. Once the evidence profile tables and clinical summaries were finalised and agreed by the GDG, the associated recommendations were produced, taking into account the trade-off between the benefits and risks as well as other important factors. These included economic considerations, values of the development group and society, and the GDG’s awareness of practical issues (Eccles et al., 1998).
3.5.9. Method used to answer a clinical question in the absence of appropriately designed, high-quality research
In the absence of level I evidence (or a level that is appropriate to the question), or where the GDG were of the opinion (on the basis of previous searches or their knowledge of the literature) that there were unlikely to be such evidence, either an informal or formal consensus process was adopted. This process focused on those questions that the GDG considered a priority.
Informal consensus
The starting point for the process of informal consensus was that a member of the topic group identified, with help from the systematic reviewer, a narrative review that most directly addressed the clinical question. Where this was not possible, a brief review of the recent literature was initiated.
This existing narrative review or new review was used as a basis for beginning an iterative process to identify lower levels of evidence relevant to the clinical question and to lead to written statements for the guideline. The process involved a number of steps:
A description of what is known about the issues concerning the clinical question was written by one of the topic group members.
Evidence from the existing review or new review was then presented in narrative form to the
GDG and further comments were sought about the evidence and its perceived relevance to the clinical question.
Based on the feedback from the
GDG, additional information was sought and added to the information collected. This may include studies that did not directly address the clinical question but were thought to contain relevant data.
If, during the course of preparing the report, a significant body of primary-level studies (of appropriate design to answer the question) were identified, a full systematic review was undertaken.
At this time, subject possibly to further reviews of the evidence, a series of statements that directly addressed the clinical question were developed.
Following this, on occasions and as deemed appropriate by the development group, the report was then sent to appointed experts outside the
GDG for peer review and comment. The information from this process was then fed back to the GDG for further discussion of the statements.
Recommendations were then developed.
After this final stage of comment, the statements and recommendations were again reviewed and agreed upon by the
GDG.
3.6. HEALTH ECONOMICS REVIEW STRATEGIES
The aim of the health economics was to contribute to the guideline’s development by providing evidence on the cost effectiveness of interventions for people with borderline personality disorder covered in the guideline. For this reason, a systematic literature review of existing economic evidence in this area was conducted.
3.6.1. Search strategy
For the systematic review of economic evidence the standard mental-health-related bibliographic databases (EMBASE, MEDLINE, CINAHL and PsycINFO) were searched. For these databases, a health economics search filter adapted from the Centre for Reviews and Dissemination at the University of York was used in combination with the general strategy for borderline personality disorder. Additional searches were performed in specific health economics databases (NHS EED, OHE HEED), as well as in the HTA database. For the HTA and NHS EED databases, the general strategy for borderline personality disorder was used. OHE HEED was searched using a shorter, database-specific strategy. Initial searches were performed in January 2007. The searches were updated regularly, with the final search performed in May 2008. Details on the search strategies adopted for the systematic review of economic evidence are provided in Appendix 12.
In parallel to searches of electronic databases, reference lists of eligible studies and relevant reviews were searched by hand. Studies included in the clinical evidence review were also screened for economic evidence.
The systematic search of the literature resulted in 3,656 references in total. Publications that were clearly not relevant to the topic (that is, did not provide any information on the economics of borderline personality disorder) were excluded first. The abstracts of all potentially relevant publications (58 papers) were then assessed against a set of inclusion criteria by the health economist. Full texts of the studies potentially meeting the inclusion criteria (including those for which eligibility was not clear from the abstract) were obtained. At this stage, 30 studies had been selected. Studies that did not meet the inclusion criteria, were duplicates, were secondary publications of one study, or had been updated in more recent publications were subsequently excluded. Finally, 18 studies that provided information on the economics of borderline disorder were selected. Of these, ten were cost-of-illness studies or studies that reported data on healthcare resource use associated with borderline personality disorder in general, and eight were economic evaluations of specific interventions for people with borderline personality disorder covered in this guideline. All economic evaluations eligible for inclusion in the systematic review of economic literature were critically appraised according to the checklists used by the British Medical Journal to assist referees in appraising full and partial economic analyses (Drummond & Jefferson, 1996) (Appendix 13).
3.6.2. Inclusion/exclusion criteria
The following inclusion/exclusion criteria were applied to select studies identified by the economic searches for further analysis:
No restriction was placed on language or publication status of the papers.
Studies published from 1996 onwards were included. This date restriction was imposed in order to obtain data relevant to current healthcare settings and costs.
Only studies from Organisation for Economic Co-operation and Development countries were included, as the aim of the review was to identify economic information transferable to the UK context.
Selection criteria based on types of clinical conditions and patients were identical to the clinical literature review; the intention was to include studies that provided data exclusively on people with borderline personality disorder; however, when no studies answering a specific economic question met this criterion, the criterion was relaxed and economic studies considering a wider study population relevant to people with borderline personality disorder were included in the review, following consensus of the
GDG.
Studies were included provided that sufficient details regarding methods and results were available to enable the methodological quality of the study to be assessed, and provided that the study’s data and results were extractable; poster presentations or abstracts were excluded from the review.
Full economic evaluations that compared two or more relevant options and considered both costs and consequences (that is, cost–consequence analyses, cost-effectiveness analyses, cost–utility analyses or cost–benefit analyses) as well as partial economic evaluations (that is, costing analyses) were included in the systematic review.
Economic studies that omitted intervention costs from the analysis were excluded from the review because their results were considered potentially misleading.
3.6.3. Data extraction
Data were extracted by the health economist using a standard economic data extraction form (Appendix 14).
3.6.4. Presentation of economic evidence
The economic evidence identified by the health economics systematic review is summarised in the respective chapters of the guideline, following presentation of the clinical evidence. The characteristics and results of all economic studies included in the review are provided in the form of evidence tables in Appendix 15.
3.6.5. Economic modelling
Formal decision-analytic economic modelling was not undertaken, owing to lack of appropriate data. Overall, availability of clinical data was limited; clinical studies examined different study populations and reported a large number of outcomes, mainly expressed as scores in rating scales, which could not be pooled together and subsequently converted into a meaningful outcome for economic analysis, for example quality adjusted life years (QALYs). In addition, a well-defined treatment pathway that would form the basis for the structure of an economic model does not exist in the area of borderline personality disorder. A recent Health Technology Assessment (HTA) (Brazier et al., 2006) identified the same problems in attempting to undertake formal economic modelling; instead, the authors used an alternative approach and carried out separate economic analyses for each of the RCTs reviewed in their report. These analyses by Brazier and colleagues (2006) are described in the respective sections of this guideline (in Chapter 5) and have been considered by the GDG when formulating recommendations. However, they are characterised by strong limitations, as acknowledged by the authors of the report (details on the methods adopted by Brazier and colleagues [2006] are provided in Chapter 5). The GDG estimated that adopting the same approach for the additional RCTs included in this guideline that were not covered by Brazier and colleagues (2006) would suffer from the same strong limitations and therefore would not add substantial information that would be useful in decision making. For this reason, no extra economic modelling was undertaken for this guideline and economic considerations were based exclusively on previously published economic evidence.
