Study design

A parallel two-armed pragmatic multisite single-blind randomised control trial (RCT) was conducted with blinding of outcome assessors. A process evaluation was also included utilising parent and service manager qualitative interviews to enhance understanding of the appropriateness and feasibility of the intervention. The study was planned and implemented following the Consolidated Standards of Reporting Trials (CONSORT) extension to compare the cost-effectiveness of the combination of SSTP plus TAU, versus TAU alone in reducing behaviours that challenge at 12 months post randomisation. The trial design is summarised in Figure 1.

FIGURE 1

The CONSORT flow diagram for the EPICC-ID RCT.

Randomisation

Participants were randomised into the intervention arm using the 3 : 2 ratio (SSTP vs. TAU) using randomly permuted blocks of different block sizes and stratification by site and the level of intellectual disability (moderate and severe). Eligible participants were allocated online to the next available treatment code in the relevant randomisation list. All randomisation and data management were provided by an internet-based site called Sealed Envelope.

Allocation concealment and implementation

At the end of the baseline assessment, the researchers entered the results on a web-based case report form (CRF) form. Parents and therapists were informed about the allocation status and arranged the commencing of the group sessions. Researchers were based in a different location from the staff involved in the delivery of the level 4 SSTP. Therapists were not involved in any treatment of the families that were allocated to TAU.

Blinding

Since parents and therapists were aware of the treatment allocation, it was not possible to ensure a completely blind trial. All research assistants who were collecting the data remained blinded to treatment allocation throughout the study. Parents were reminded not to disclose any information about their treatment allocation to the research assistant during assessments.

The researcher entered the results of all study assessments on web-based electronic case report forms (eCRFs). They did not have access to eCRFs that may pose a risk of unblinding them [e.g. Child and Adolescent Service Use Schedule (CA-SUS) at 4 and 12 months, Client Satisfaction Questionnaire, serious adverse event (SAE) form]. During coronavirus disease 2019 (COVID-19), CA-SUS forms were collected remotely, and the question relating to parenting groups was omitted and collected later.

The lead study statistician remained blinded to arm allocation throughout and the subsidiary trial statistician attended the Data Safety and Monitoring Board (DSMB) meetings. The lead statistician attended the Trial Management Group (TMG) and Trial Steering Committee (TSC) meetings during the study.

Primary outcome

Secondary outcomes

Revised Family Observation Schedule (FOS-RIII)

Direct observations were carried out by researchers using the FOS-RIII, a measure of parent–child interaction.⁴⁴ This measure is commonly used in studies that investigate SSTP. Parents are asked to interact with their child while being filmed for 20 minutes carrying out various activities that reflect tasks that likely happen in their daily life at home. Overall, there are four 5-minute tasks: (1) child’s free play; (2) a structured block task; (3) parent and child in the same room but each involved in separate activities; and (4) cleaning up after play. The FOS-RIII codes 10-second segments and computes four scores that indicate positive and negative child behaviours, and positive and negative parent behaviours (see Table 1 for further details). A senior developmental psychologist scored the video observations. The FOS-RIII has demonstrated reliability and discriminant validity and is sensitive to the effects of behavioural interventions.⁴⁵

TABLE 1

Categories of positive and negative child and parent behaviours assessed by the FOS-RIII measure

Other measures

Sample size

The original sample size calculation required a sample of 258 children in an allocation ratio of 3 : 2 (155 vs. 103) to detect a low to a moderate (standardised) effect size of 0.40 for the primary outcome CBCL at 12 months at the 5% significance level with 90% power; the equivalent to detecting a clinically important difference of 8 points, assuming a standard deviation (SD) of 20. This was calculated as follows: A standard calculation based on analysis of covariance (ANCOVA) leads to a sample size of 99 children per arm, assuming a correlation of 0.5 between baseline and follow-up measurements. An equivalent calculation (same power) based on an allocation ratio of 1.15 : 1 gives group sizes of 107 and 93. Increasing the SSTP arm (only) to allow for therapist clustering leads to 139 children in the SSTP arm, assuming an intraclass correlation of 0.05 and an average therapist group size of 7 (design effect = 1.3). An adjustment for the anticipated dropout of 10% leads to 155 children in the SSTP group and 103 in the control group.

Eligibility criteria

The inclusion criteria for the study were as follows:

1. parents aged 18 years or over;
2. child aged 30–59 months at identification;
3. child had a moderate to severe intellectual disability, measured by parent-reported Adaptive Behaviour Assessment Schedule (ABAS) General Adaptive Functioning score between 40 and 69;
4. reported child’s behaviours that challenge over a 6-month period prior to the study, but no less than 2 months;
5. obtain a written consent by parent/caregiver.

Exclusion criteria

Participants were not included in the study if:

1. child had a mild, profound or no intellectual disability on parent-reported ABAS form;
2. parent/carer had insufficient English language skills to complete the study questionnaires;
3. another sibling was taking part in the study.

Recruitment

Participants were recruited from various community services including Participant Identification Centres (PIC) in four main areas in England: Blackpool (and surrounding areas; Site 1), North London (Site 2), South London (Site 3) and Newcastle (and surrounding areas; Site 4). The services included National Health Services (NHS) settings (e.g. Child Development Teams), Child and Adolescent Mental Health Services (CAMHS), education (e.g. nursery, preschool) and third sector (e.g. caregiver groups). Health or social care professionals identified eligible participants through new referrals or existing cases. Children were assessed for developmental delay through the Healthy Child Programme, which aims to identify families that require further support with their children, and children aged 0–5 years who are at risk of ‘poor outcomes’. Flyers were put up at local parent groups, nurseries, special schools and general practitioners (GPs) to promote the study among families. We followed a multisource referral strategy facilitated by the clinical research networks, our national, clinical and third-sector contacts and social media. We opened five PIC adjacent to Site 4, three PIC sites adjacent to Site 2 and two PIC sites adjacent to Site 3 to reach our planned recruitment target at the sites. Several recruitment difficulties were addressed throughout the study. This included parents declining or not being eligible to take part in the study due to various reasons, such as time commitments, child-care issues, health problems, language barriers or parents not thinking that child has an intellectual disability or behaviours that challenge. Table 3 presents the overview of recruitment at each study site.

TABLE 3

Recruitment overview by site

Intervention arm

SSTP is part of Triple P (Positive Parenting Program), a psycho-educational parenting and family intervention developed for families who have children with behavioural or emotional problems.⁵⁸ SSTP was adapted specifically for parents of children with intellectual disabilities. This trial provided the SSTP level 4 manualised course, which offers six group sessions and three individual telephone or face-to-face contacts with the parent over a period of 9 weeks. Each group session lasted approximately 2.5 hours and individual sessions took about 30 minutes. The combination of group and individual sessions allowed parents to share experiences and build on skills gained while providing personalised support. The group sessions aimed to educate and actively train parents in skills and behaviour management. The individual consultations aimed to facilitate independent problem-solving and allowed parents to pick strategies most relevant to their child’s difficulties. All parents also received a course book that covered topics from each session. Those who missed a session were contacted by the therapist to discuss their progress and were encouraged to attend the next sessions. The group sessions were delivered in person until March 2020. Afterwards, all group sessions were delivered online via Zoom or Microsoft Teams. Table 4 provides the outline of the SSTP sessions.

TABLE 4

Outline of SSTP sessions

Treatment as usual arm

Treatment as usual was available to participants in both arms of the trial. Parents allocated to both arms also received a list of national resources and the Contact charity’s guide for managing behaviours that challenge with recommendations about social and health care support. We identified a variety of services and support offered to families with children with intellectual disabilities and autism in each of the study’s local areas. This included individual help from speech and language therapists, occupational therapists, child psychotherapists, clinical psychologists, child psychiatrists and family therapists. There were numerous children’s centres offering support with parenting advice, local child-care options and access to specialist services for families and their children with intellectual disabilities and parenting classes (e.g. Camden MOSAIC centre in Site 2; SLAM Child and Family Service in Site 3, etc.). Parenting programmes were available in each of the study local areas (N = 3 in Site1; N = 8 in Site 2; N = 9 in Site 3; N = 9 in Site 4). For example, the Early Bird parenting programme was available in all four study areas. This is a 3-month programme for parents providing general information about autism and advice about looking after children with autism. It combines group training sessions for parents with individual home visits where video feedback is used to help parents apply what they have learnt with their child. Parenting programmes that focus on conduct problems in children without an intellectual disability were also available, such as The Incredible Years (Site 2, Site 3 and Site 4), The Family Links Course (Site 2) and Triple P (Site 3). Other support included third-sector organisations, which provided wide-ranging help through workshops, home visits, child care, information and advice and time-limited interventions. Most of the available programmes focused on autism awareness and general information for parents about autism or intellectual disabilities, providing limited advice on parenting strategies and techniques for addressing behaviours that challenge displayed by disabled children.

Participant identification

The study population was parents of preschool children with moderate to severe intellectual disability, who were concerned about their child’s behaviour living in the community in the four study sites. We excluded children with mild intellectual disabilities, as they are likely to access other interventions more suitable for their needs. Similarly, children with profound intellectual disabilities were excluded as they were unlikely to be able to follow the observation tasks and the psychometric assessments. Eligible participants were identified by local community paediatric or CAMHS teams through new referrals or existing cases. Members of the clinical teams screened and reviewed identifiable personal information of potential participants. Parents who were approached by a member of clinical staff or a clinical study officer were given a description of the study, the study Patient Information Sheet and an Expression of Interest form to complete. Those who were eligible and interested in taking part completed the ‘Expression of Interest’ form, which was passed to the researchers, who then contacted the parents for further screening.

Screening process

The eligibility of each participant was confirmed during the screening assessment with a member of the research team. During the screening, the parent rated the child’s level of functional abilities on the ABAS form. Children who had a General Adaptive Functioning score between 40 and 69 were considered to have moderate to severe intellectual disability, and thus were eligible to participate. Parents also had to confirm that their child displayed behaviours that challenge continually over the 2 months prior to the screening assessment. If the result of the screening assessment indicated the child’s adaptive functioning to be outside of our inclusion criteria, parents were given the reason for non-inclusion, were thanked for their cooperation and time, and no further contact was made with the family. Those who were eligible after the screening assessment were contacted to schedule the baseline assessment. Following this, the participants were randomised into the intervention arm (SSTP plus TAU) or TAU.

Informed consent procedure

Informed consent was taken in two stages. First, the person who had parental responsibility for the child completed the consent form at the beginning of the screening assessment prior to completing the ABAS form. Second, if the child was eligible, the family was invited to enter the study and complete the consent form prior to their randomisation. Parents were given a minimum of 24 hours to decide whether they wanted to enter the study and were given contact details of the research assistant to ask further questions. For both stages, parents provided written or audio-recorded verbal informed consent. The right to withdraw at any time without giving a reason or affecting further treatment was explained to each participant. Each participant was given a copy of their informed consent form, and the signed original was retained at the study site.

The researchers were inducted into the study procedures about obtaining informed consent and completed the online Good Clinical Practice (GCP) course.

For the process evaluation, parents signed a consent form and were provided with a copy to keep if interviews were done in person. For telephone interviews, the consent process was audio-recorded and stored securely in Data Safe Haven (DSH), a secure online platform to store confidential data at University College London (UCL).

Data collection methods and handling

Data management in this study complied with the UK Data Protection Act 1998, PRIMENT Standard Operating Procedures (SOPs) and GCP. Researchers uploaded and stored all data into DSH, which is a secure system for storing sensitive information. Audio and video recordings were uploaded to the system, and researchers deleted all other versions of the data from digital machines where they were originally recorded.

The investigator was responsible for ensuring the accuracy of all data entered in the CRFs. Data verification checks were completed on 100% of CBCLs (primary outcome measure), and 10% of all secondary outcome measures. The original application stipulated that 5% of all secondary outcome measures would be checked, but due to researcher turnover, it was deemed necessary to carry out a higher percentage of checks for quality assurance in three study sites (Site 2, 3 and 4), where 10% checks were carried out. All study staff were also reminded of the importance of careful data entry to minimise errors.

Confidentiality

All personal data collected about the participant was managed following the PRIMENT SOP for Managing Personal Data. Each participant was given a unique identification number at randomisation. The participant’s initials, date of birth and unique randomisation number were used for identification. All other personal or identifiable information about the participants was stored separately and securely in the UCL DSH system. The CRFs did not contain identifiable information. We used a delegation log to track personnel responsible for data management.

Trial database

All data from the participants were entered into an online clinical data management system called Sealed Envelope. PRIMENT approved the quality management, software development and security of this system. The original paper forms of outcome measures were stored in secured locked cabinets in the office at each trial site. PRIMENT SOP Database Lock, Unlock and Closure was followed at the end of the trial.

Data entry

Baseline data for 261 randomised participants from all four sites were entered into the Sealed Envelope database. The primary outcome measure (CBCL) was available in the database for 261 participants and the CRFs for Source Data Verification (SDV) checking were available for all participants.

Four-month follow-up data for 229 participants from four sites have been entered into the database. The primary outcome measure was available for 223 participants.

Approval

Approval for the study was given by the London – Camden and Kings Cross REC (reference: 17/LO/0659).

Substantial amendments

Ethics and Health Research Authority (HRA) approval for the study was given on 19 May 2017. Overall, there were six substantial amendments approved by the REC as well as nine non-substantial amendments to the study protocol.

• Substantial amendment 1: added participant recruitment via third-sector organisations, such as ‘Contact’ (co-applicant) and added PIC sites to the study.
• Substantial amendment 2: extended the screening and baseline visit window from 2 to 4 weeks, approved giving the brochure and list of resources to all participants in both arms of the trial.
• Substantial amendment 3: included minor updates to the participant information sheets and the study poster, created treatment allocation letters for the participants and created a letter for clinicians, teachers or third-sector organisations to send the study information sheet and expression of interest form to potential participants.
• Substantial amendment 4: added an independent reviewer to code the video interviews, included the possibility of verbal consent and the possibility of conducting qualitative interviews for process evaluation over the phone, and added sections of the transparency wording from the HRA website in line with the new General Data Protection Regulation (GDPR), added intervention survey, created a letter for parents from the intervention group that did not attend any of the sessions, provided minor amendments to information sheets, consent forms and topic guide for interviewing service managers.
• Substantial amendment 5: approval to invite parents not taking part in the study to attend the SSTP groups to ensure viable group sizes, created a ‘Thank you’ card for parents to boost attendance at follow-up assessments, added a minor update to the information sheet.
• Substantial amendment 6: approved use of external transcription service to transcribe recorded audio files and to include some participants from the TAU group in qualitative interviews, including information regarding the 10-month costed extension approved by the study funders.

Deviations from the study protocol

There were 37 protocol deviations at 4 sites. Most of these deviations concerned baseline or follow-up assessments being completed outside of the assessment window as specified in the protocol. The assessment window between screening and baseline was extended to 4 weeks as part of substantial amendment 2. Other deviations included issues with the consent forms (e.g. a participant signed a consent form with the wrong site logo). These consent forms were corrected, or the correct forms were completed retrospectively. Protocol deviations relating to the COVID-19 pandemic are described later in the report under the heading ‘COVID-19 Adaptations’.

Recruitment during the COVID-19 pandemic

In total, 51 out of 261 families in the trial were randomised on or after 16 March 2020. The non-substantial amendments also simplified the process for Clinical Study Officers to pass on expressions of interest to the study team and updated the study protocol to reflect all changes. Study recruitment was halted between 18 March 2020 and 18 May 2020, and the timescale for re-opening was determined independently by each site. Sites reopened gradually between May and August 2020. Study follow-ups continued throughout this period, although child−parent observations and cognitive neuropsychological assessments (i.e. Mullen Scales of Early Learning)⁵⁷ that required face-to-face contact were omitted from the study, as we were unable to find alternative ways to conduct the assessments (max n = 51). Wet signatures for logs were replaced with online signatures, and verbal recorded consent was obtained rather than written consent.

Observing government guidance on the spread of COVID-19 led to the study team working remotely. Before the pandemic, the CA-SUS forms at 4 and 12 months were completed by parents during visits and then handed to the Trial Manager for data entry. As this was no longer possible, research assistants completed the CA-SUS assessment over the phone but did not ask questions about attendance at parenting groups that would unblind them. These data were collected once the participant had completed the final follow-up. Moreover, the issues associated with the pandemic caused further difficulties with completing the study assessments with participants. For example, some parents found it challenging to find the time to complete all questionnaires over the phone, therefore many only completed the primary outcome measure (CBCL). Since all work, appointments, therapies and school responsibilities became remote, many assessments were re-scheduled and this increased the number of missing assessments and those completed outside of the assessment window.

The restrictions from the COVID-19 pandemic caused difficulties with obtaining the secondary outcome measure (C-TRF), which was meant to be completed by the child’s teacher. Most of the schools were closed and many parents did not have other relatives or people who would know their child’s behaviours well enough to complete this measure.

Parents from the PAG reflected on their own experiences during the pandemic, including big routine changes, which could affect the availability of parents to complete assessments. In particular, single parents may not have had the time to dedicate to the study. One parent reflected that during the pandemic situation, it was difficult to make phone calls without interruptions, but as the situation progressed, the family started to settle into a routine and this increased their capacity to take phone calls and allocate time to more responsibilities. They said this may also be similar for other parents. Another parent from the PAG noted that many of these families may not have education health care plans yet, which was allowing children to continue attending school during lockdowns. For those without a plan, routines are being disrupted, which may have been extremely difficult for both children and their families. Overall, the COVID-19 pandemic brought challenges not only to the delivery of SSTP and other services in TAU but also to the daily lives of every individual. This likely resulted in a deterioration in children’s behaviour patterns, increased parental stress and poorer parental ability to manage behaviours. These disruptions likely challenged many parents’ ability to comply with study directives.

Treatment as usual during the COVID-19 pandemic

Treatment as usual could not have been delivered in the conventional way as most of the services available in local areas were interrupted or completely suspended from March 2020. As such, families that were recruited in this study from this time had little to no help or support available to them.

During the qualitative interviews conducted for the process evaluation, parents shared their views about the COVID-19 pandemic and the national lockdowns. They described this time as extremely challenging as the children’s challenging behaviours worsened due to lack of routine and limited access to school. Parents were trying to balance working from home and caring for their child with no additional support from services or school. As such, parents felt even more abandoned during the times when they felt the most need to be supported.

The behaviour during the three months of lockdown took its toll. It escalated to where it was, like, unmanageable most days. Um, but there wasn’t a lot we could do about that.

One of the biggest challenges I’ve ever had to face is being in three months lockdown with a child with needs … It was really, really unfair to do I think, what they done originally, was said children who have EHCP plans would be taken care of. We’d be taken into consideration, the needs, and it wasn’t.

Stepping Stones Triple P delivery during the COVID-19 pandemic

From March 2020, the intervention was adapted to follow the COVID-19 safety precautions and regulations. At the start of the pandemic, no online version of SSTP was available from the Triple P group. The therapists continued with the same session plans and content that was delivered before the COVID-19 pandemic and delivered this remotely. The final group combined participants from all sites to ensure a sufficient group size. The Triple P held a panel discussion on remote delivery in practice on 21 July 2020. This was attended by the Trial Manager. The Triple P developers granted permission for us to share the advice provided in this session, including:

• Using video conferencing software, which allows for better participation than the telephone as parents can see the therapist when skills are being demonstrated.
• Allowing space for discussing progress and being able to problem-solve technical challenges.
• Acknowledging the stressors parents are facing and normalising the feelings they are experiencing. Offer hope that with the plans and strategies to be discussed, they will feel more confident in how to tackle this new normal, reduce stress and enjoy their parent–child interaction.
• Maintain contact with fellow Triple P practitioners, including Peer-Assisted Supervision and Support (PASS) sessions, to support each other in learning this new format and maintain fidelity.
• For parents with children at home, consider proposing that parents use a Planned Activities Routine to occupy the children so parents can participate fully in the session.
• Consider sending parents text reminders of their upcoming session.
• Resources can be available to pick up from an office (if open) or mailed to families.
• Consider planning how to use behavioural rehearsal based on your chosen delivery format (on videoconference platforms, it can be done as usual).
• When parents or practitioners are stressed or anxious, it may be more difficult to come up with ideas. Practitioners may need to think about giving more examples or offering more specific prompts sooner if they realise that parents are struggling with idea generation.
• Provide an opportunity for informal communication between parents before or after the sessions.
• Show DVDs by sharing your screen or demonstrate live the strategies. The DVD could also be played in the background, sharing the audio only (additional commentary may be required), or therapists should read the relevant section of the Positive Parenting Booklet.
• Consider if the parent has safe activities for the children while they are taking part in the intervention.
• Content can be fully delivered remotely, and the process/strategies should not change.

Data on attendance at parenting groups were collected once the participant has completed the study. Forms were sent to the Trial Manager for data entry.

Assessment of baseline characteristics

Summary measures for the baseline characteristics of each group are presented as mean and SDs for continuous, symmetric variables, medians and interquartile ranges (IQR) for continuous, skewed variables and frequencies and percentages for categorical variables. We compared baseline characteristics visually to assess whether the balance had been achieved. Any notable imbalances prompted additional adjusted analyses (see later).

Adherence to allocated programme and attrition

Some loss to follow-up was expected over 12 months. The proportion of participants missing was summarised for each outcome measure, in each arm, and at each time point. Potential bias due to missing data was initially investigated by comparing the baseline characteristics of the trial participants who have (analysable) primary outcome data to those who do not, using descriptive comparisons, t-tests, chi-squared tests and Fisher’s exact tests as appropriate. Reasons for withdrawal from the programme were summarised. Participants’ adherence with SSTP is defined as attendance to most of the planned group and individual sessions, that is participation in at least 4 (out of 6) group sessions and 2 (out of 3) individual sessions. Participants who engaged with SSTP were compared descriptively with those who did not in terms of their baseline characteristics.

Analysis of primary outcome

The primary outcome is the CBCL total score at 12 months. The primary analysis used mixed models to perform an individual-level analysis and followed Roberts and Roberts⁶⁰ in adjusting for therapist clustering in the intervention arm only (random coefficient model). This model also adjusted for baseline total CBCL score and randomisation stratification factors (centre, level of intellectual disability) using fixed effects. This was a complete case analysis. The only ‘imputation’ performed was done by following guidance from the CBCL scoring manual. That is, missing values for CBCL were scored as zero unless more than eight items were missing, in which case the participant was excluded. The presentation of all findings is in accordance with the latest CONSORT statement. The model assumes that the residuals are normally distributed and homoscedastic, which was checked using residual plots (e.g. normal Q−Q plots). Where substantial departures from normality occurred, a transformation of the outcome variable was considered.

Analysis of the secondary and other outcomes

In addition to the analysis of the total CBCL score, we analysed the internal and external scores separately using the same approach. In addition, analyses were performed for each of the secondary outcomes. Continuous outcomes were analysed using the same approach as that described for the primary outcome. For binary outcomes, we used analogous logistic mixed models,⁶¹ although without adjustment for baseline outcome scores. All analyses of secondary and other outcomes should be considered supportive analyses. Missing values in the outcomes were handled, where possible, using guidance from the corresponding manual.

Sensitivity analysis

The following additional sensitivity analyses were performed. We repeated the primary analysis with additional adjustments to see if any notable baseline imbalances were encountered (due to chance or missing data). We used a mixed model to analyse both the 4- and 12-month CBCL outcomes simultaneously. We repeated the primary analysis after imputing outcome data using multiple imputations using chained equations. Specifically, we imputed component items of the CBCL score (i.e. not the total score) using item information from the baseline, 4- and 12-month CBCL scores and other variables (as appropriate). Missing baseline data were imputed using single imputation methods where multiple imputation was not successful. Per-protocol and complier-average causal effect (CACE) analyses were performed as participant adherence was relatively low.

Coronavirus disease 2019 considerations and additional analyses

The following analyses were carried out to explore the effect of the COVID-19 pandemic on the trial findings. The baseline characteristics of children and parents were summarised by randomised group before and after 16 March 2020. In addition, adherence to the programme, attrition and adverse events were summarised before and after this time point. We performed a subgroup analysis to investigate whether the intervention effect differed depending on whether recruitment was before or after 16 March 2020. This was achieved using the primary analysis model with additional indicator and interaction terms. Finally, we considered whether the effect of the intervention depended on the size of session groups. Separate analyses considered: (1) the overall group size and (2) the average group size in sessions attended. These analyses were achieved using the primary analysis model with additional indicator and interaction terms. Similar health economics analyses were performed to assess whether the cost of the intervention was affected when intervention sessions were moved online. We also assessed whether the COVID-19 pandemic had an impact on children and carers’ mental health and HRQoL during this period.

Valuation of economic outcomes

The primary economic outcome measure was the QALYs derived from utility scores obtained using the PedsQL™ General Core Scales (GCS). Mapped EQ-5D-Y utility scores algorithm⁶² provided an empirical basis for estimating health utilities. For parents/caregivers utility scores were derived from responses to the EuroQol-5 Dimensions, five-level version (EQ-5D-5L) using valuations obtained from an English population.⁶³ These were used to form QALYs over the 12-month period, adjusting for any imbalances in baseline scores.⁶⁴ Measurements have been recorded at baseline, 4 and 12 months.

Valuation of resource use

Study records of the number of therapists attending training sessions were used to track resources used in the delivery of the training programmes including trainee and trainer time (and preparation time), travel costs, attendance incentives and course materials to calculate the fixed cost of training. For the intervention delivery, we recorded the number of sessions delivered, the time each therapist spent with a family, and also any materials provided to parents/caregivers.

For the NHS and PSS analysis, data were collected on the use of health services in primary and community care, investigations and prescribed medication, hospital admission and outpatient attendance, ambulance use and social care. Data on health and social care resource use were collected at baseline (for the past 6 months), 4 months (for the past 4 months) and 12 months (for the past 6 months) post intervention.

For the analysis from the parent/caregiver perspective, we additionally collected data on out-of-pocket expenses. Expenditure on private use of treatments and therapies was captured in the CA-SUS.

Health and social care resource use was costed using unit costs from the most recent Unit Costs of Health and Social Care published by the Personal Social Services Research Unit (PSSRU)⁶⁵ and NHS reference costs. The costs of medications were estimated from the British National Formulary. The cost of each resource item was calculated by multiplying the number of resource units used by the unit cost. The total cost for each participant was estimated as the sum of the cost of resource use items consumed. The primary analysis included only health and social care data collected as part of the trial and hence covered only 10 months of the trial (missing months 4–6). We have projected costs from 4- and 12-month follow-ups to estimate the 12-month health and social care resource use as part of sensitivity analyses. All costs are reported in 2019–20 Great British pounds. No discounts were applied, as trial follow-up did not exceed 12 months.

The overall economic evaluation

All analyses were conducted using intention-to-treat (ITT) principles, comparing the two groups as randomised and including all participants in the analysis. Analyses were compliant with the accepted economic evaluation methods.⁶⁶

• Cost-effectiveness analysis: mean incremental cost from the NHS and PSS perspective per change in CBCL. Incremental cost-effectiveness ratios (ICER) were reported, and uncertainty was explored using cost-effectiveness acceptability curves (CEACs).^67,68
• Exploratory analysis of quality of life using PedsQL™ to predict utility scores. The mean cost per participant for the intervention and TAU was reported by type of service use. We calculated the mean cost per QALY using the mapped EQ-5D-Y. Mean QALY per participant was calculated as the area under the curve for the duration of the trial, adjusting for baseline values.

Cost−benefit analysis of the impact on the parents/caregivers: Responses to EQ-5D-5L were used to calculate QALYs in a standard format and valued as willingness to pay (WTP) for a QALY gained. We calculated the mean cost (including out-of-pocket expenditure) per QALYs. Mean QALY per participant was calculated as the area under the curve for the duration of the trial, adjusting for baseline values.

Missing data

Missing data were explored to determine any patterns, extent and association with participant characteristics. The primary analysis included all participants using multiple imputation to predict missing costs and outcomes.⁶⁹

Analysis of relative costs and outcomes

Cost and QALY data were combined to calculate an ICER. Uncertainty in the point estimate of the cost per QALY was quantified using bootstrapping methods to calculate confidence intervals around the ICER.

The results of the non-parametric bootstrap are presented on a cost-effectiveness plane (CEP). CEACs, showing the percentage of cases where the intervention is cost-effective, over a range of values of WTP for a QALY gained, were constructed using the bootstrap data from a range of values of WTP for a QALY gained for each different costing perspective and for the different methods of calculating QALYs. The probability that the intervention is cost-effective compared to TAU at a WTP for a QALY gained of £20,000 and £30,000, and £13,000 as a measure of opportunity cost were reported.⁷⁰

Sensitivity analyses

One-way sensitivity analyses were used to judge the potential impact of sources of uncertainty:

• complete case analysis;
• given that training costs may differ between the trial and implementation of the intervention due to learning or being delivered to a larger patient group, we tested the impact of varying training costs (particularly because of larger patient numbers per staff member trained) on the mean incremental cost per QALY gained.