Co-produce (with stakeholders) an intervention that was acceptable to families and feasible to deliver in the NHS and in the community with support from health and non-health professionals

Chapter 2 of this report provides definitive evidence that we successfully co-developed a standardised intervention called Young SMILES (including staff and service user manuals and resources) for CAPRI. We did this in collaboration with service users, NHS and non-NHS stakeholders at every stage of the process. We also developed training for staff facilitators that was tailored to the differing needs of NHS, Barnardo’s and NSPCC staff. This training was standardised and included a manual for practitioners delivering the intervention and a compendium of resources, including craft materials for activities and relevant literature.

Young SMILES broadened the scope and content of the existing Family SMILES NSPCC intervention to make it:

• specific to families whose parents have SMI – schizophrenia, bipolar affective disorder or severe depression or personality disorder
• in line with current NHS priorities and service structures
• deliverable in different practice settings by a varied staff skill mix, including NHS IAPT practitioners and NSPCC third-sector providers
• age appropriate to a wide age range of children
• accessible to all CAPRI and not only those with identified risk of maltreatment/neglect or childhood mental health problems.

The early stages of our qualitative evaluation prioritised already established frameworks of acceptability and implementation for intervention development rather than identifying critical change mechanisms. Shaping our intervention using our intervention theory of change would have been an alternative approach; however, our qualitative studies evidenced multiple examples of the change mechanisms postulated at the start of the feasibility study.

Strongly endorsed theory of change mechanisms (see Appendix 2, Figure 2) included improving parent–child communication, normalising mental health and improving children’s social networks. Others will require further consideration at intervention refinement stages, including practising mindfulness.

Determine the rates of intervention uptake and adherence, and of completed follow-up measures

One-fifth of eligible families who were approached by their care co-ordinators showed an interest in participating. Out of the total 49 families who were assessed by telephone (via the NHS, n = 31; via the NSPCC, n = 18) for eligibility, 35 (71%) met inclusion criteria and received a face-to-face assessment and subsequent randomisation; nearly one-third of those (n = 12) were recruited in the NSPCC sites and two-thirds (n = 23) in the NHS.

Children typically had very high adherence to the intervention, completing seven out of the eight offered sessions. The overwhelming majority of recruited parents were mothers, although there was a balance of sexes among participating CAPRI.

Quantitative data revealed that CYP had a high adherence to the intervention, attending seven out of the eight offered sessions (see Chapter 4). Treatment uptake by parents mirrored previous experience for Family SMILES,¹⁴⁶ which similarly found it more difficult to recruit parents than: Family SMILES included three or four assessments with parents prior to the intervention and, in general, was significantly parent focused compared with our child-centred approach. This is notable given that NSPCC facilitators and referrers raised particular concerns (see Chapters 5 and 6) about the relative lack of parent assessment within Young SMILES and attributed parent difficulties in engaging with the process to this. A key misunderstanding among these parents related to the parental aspect of the intervention and the fact that their needs were not the focus of sessions, which many had difficulty with and were not used to.

Qualitative feedback reported in Chapter 5 was provided by 17 children (from different families) and by 14 parents who underwent Young SMILES. Feedback from CAPRI randomised to Young SMILES was primarily from younger children, equally divided between girls and boys. Both parents and children initially expressed reservations about what the intervention was for and about transport difficulties. However, feedback from both was largely positive with key areas for refinement/adaptation. However, there was no clear pattern of positive/negative views by demographic group (e.g. about the nature of the intervention and their experience of attending the sessions). Children highlighted the benefits of having learnt more about mental health and what some of the experiences of their parents were like. They also valued making new friends and the fact that the Young SMILES intervention acknowledged their roles in supporting their parents, which they felt needed more attention from services. Children said that attending the Young SMILES sessions provided learning and subsequent understanding that they felt could affect their parent and family directly rather than changing their QoL directly.

Despite initial apprehension, parents were motivated to attend for the benefit of their child’s well-being. For many, the Young SMILES intervention offered a unique opportunity compared with the support that their family had previously received, instilling a sense of hope through recognition of improvements they observed in their relationship with their children and facilitating communication and connection with their children and with other parents in similar situations. They expressed difficulties with attendance focusing on their anxieties about exposing their difficulties to professionals and about practical aspects of arranging transport or having the time to attend. Notwithstanding this, parents all asked for more sessions (i.e. the same number of sessions as their children) and for more preparation for groups (akin to Family SMILES¹⁴⁶). In other words, parental feedback about adherence was contradictory.

Only three of the older children provided feedback. They expressed concerns about a ‘one size fits all’ approach in sessions. In spite of this, participation and adherence among older children was uniformly excellent.

Chapter 6 reflected facilitator and referrer feedback about if and how the Young SMILES intervention met the expectations and needs of the target group. It was clear that it provided a much needed approach for a group of children who these stakeholders viewed as vulnerable, in need and significant in number with one referrer commenting ‘I can’t believe that you’ve not been inundated . . . it’s what we’ve been looking for’ (WR1). However, they identified the need for further development to create efficient, safe referral pathways with better mechanisms for working across organisations.

Combining evidence from CAPRI, parents, facilitators and referrers, we propose that Young SMILES represents a standardised intervention that does indeed meet many, if not all, the varied and complex needs of participating families. We also conclude that the Young SMILES intervention needs to be tailored and adapted to address the criticisms described in this report. Some of these changes include:

• more training about the randomisation process for referrers and services
• developing an orientation for parents and children prior to their involvement to clarify the purpose of the Young SMILES intervention
• using simplified packs that we developed for referrers and parents
• describing elements of the content more clearly for facilitators with more resources and exemplars in the manuals
• further development of the Young SMILES intervention for the older age groups and to meet the diverse needs of different families and children.

Overall, participants’ perceptions following attendance suggested that the intervention aligned well with the principal aims of the Young SMILES intervention. In particular, narratives highlighted specific improvements gained by children. Post-intervention improvements in children’s well-being and behaviour were particularly noted through narrative accounts by a range of stakeholders and children, who said that they had gained insight into parental mental illness and improved their mental health literacy.

The NSPCC practitioners had a high level of skill and experience of working with children in group work settings, and the NSPCC assessment for suitability, completed on receipt of the referral, provided essential information to help the practitioners respond to the child’s needs within the group and on an individual basis. NSPCC practitioners were also experienced in delivering and achieving intervention outcomes via creative activities.

Out of the 35 randomised families, 33 families (Young SMILES group, n = 18; TAU group, n = 15) provided baseline data. Of those families, 28 gave follow-up data at the primary end point (4 months post baseline): 15 out of 20 (75%) in the intervention group and 13 out of 15 in the control group (87%). Across both intervention group and control group, once randomised, there was a high rate of > 80% completion of follow-up measures. Complete or near-complete outcome measures were collected from CYP at baseline and the two follow-up time points (i.e. 4 and 6 months) for the primary outcomes of HRQoL and for our secondary outcomes. Researchers reported the ease with which they collected measures from CYP, who, in the main, demonstrated considerable enthusiasm when engaging with the research process. For parents, there were slightly more missing data by the 6-month follow-up as more parents had dropped out. This reflects not only the experience of the Family SMILES intervention in earlier studies, but also reflects better engagement and adherence by CAPRI compared with their parents overall.

Identify appropriate outcome measures and estimate their data missingness

From a battery of standardised questionnaires, we identified two primary outcome measures for child HRQoL: the PedsQL¹²⁷ and the KIDSCREEN.¹²⁶ We explored each measure’s sensitivity to change and ability to capture key areas of HRQoL for CAPRI. Outcome measures also included the SDQ,¹¹⁰ which is routinely used by IAPT-CYP and NSPCC services, and the RCADS.¹¹¹ We also measured parenting skills and child–parent relationships using the Arnold–O’Leary parenting scale¹¹⁴ and the PSI-SF.¹¹⁵

We noted high completion rates (> 80%) of the child-reported outcomes measures at the primary end point (4 months). Parent-completed questionnaires had more data missing (up to one-third) for child-proxy outcomes and for parent outcomes (up to one-quarter missing). We were not able to capture any change or a sense of direction for change in the measured outcomes because of the sample’s small size and large heterogeneity.

Our systematic review, which underpinned the original HTA 14/29 call, found that there were no interventions to improve HRQoL aimed at recent populations of exposed children.^6,68 Similarly, there is little or no evidence available about the HRQoL of these children. However, accumulating high-quality evidence reports that children living with parental mental illness in the UK in 2020 are more likely to live in the lowest socioeconomic quintile and, therefore, more likely to be exposed to multiple deprivation and experience significant unmet needs. Furthermore, although population data suggest that the majority of CAPRI do not develop mental illness, or other adverse outcomes, their relative risk across almost all outcomes (e.g. mortality, injuries, broader physical health, mental health, health-care use, vaccination uptake, as well as school leaving grades and employment) suggests that they do relatively badly compared with their peers without parental mental illness. Taken together, we would expect more unmet needs, as well as worse QoL in CAPRI. This was not picked up in our small sample using the available, age-appropriate, HRQoL measures developed for the general population.

Our view about this is as follows: either future research needs to develop better ways of measuring QoL specific to the experiences of this population and QoL is likely to fluctuate over time and any such outcomes need to reflect this fluctuation, or QoL may not be the most appropriate construct to use as a measure of the vulnerabilities experienced by this group of young people and is, therefore, inappropriate to target in any future interventions to improve their life outcomes. With this is mind, we have reflected that assessment of unmet needs across a broad array of experiences in the home, their social lives and school may be a more sensitive guide to the repeatedly evidenced vulnerability of children living with parental mental illness. We are also of the view that such needs are likely to vary over time and to depend on circumstances within and outside the family (e.g. exams or bullying) not just the state of the parent’s health.

Bearing these challenges in mind, it is our view that improving the lives of CAPRI requires a public health approach (because the vulnerabilities evidenced are described across the population of exposed offspring) that is scalable, affordable and responsive to changes in individual needs across time. We believe that such an intervention is likely to be digital and our preliminary discussions with stakeholders suggest that many of the valued elements of the Young SMILES intervention can be repurposed to this end.

Develop a child resource utilisation questionnaire and estimate its data missingness

We developed a resource use data collection tool. We did this by adapting the CA-SUS¹³⁰ for our study setting in consultation with Professor Sarah Byford, the designer of the original CA-SUS questionnaire. This adaptation involved removing the sections on out-of-pocket expenses and employment, removing the question on education type, removing the follow-on questions asking name of hospital for the hospital service use questions, removing complementary therapist (e.g. homeopath) from the list of community services, adding NHS walk-in services and NHS Direct to the list of community services, and simplifying the questions in the criminal justice services section.

We then piloted the collection of resource utilisation data using this adapted version of the CA-SUS. Resource utilisation was collected retrospectively during this piloting using participant recall.

From this piloting exercise we found that participants were willing and able to respond to questions from the CA-SUS regarding accommodation, education, hospital service utilisation and community services utilisation. However, of the 27 community services listed in the CA-SUS questionnaire, participants reported no utilisation of nine of these services at all three time points. To minimise participant burden it may therefore be advisable to remove some or all of these services from the list of community services questions in a future trial.

Response rates were zero for questions regarding medication prescriptions, and very low for questions regarding interactions with the criminal justice system. For a future trial, a decision must be made about the likely importance of these categories for this patient group. If it is hypothesised that the intervention is likely to have an impact on resource use in these areas then alternative methods of data collection may be necessary. The importance of criminal justice service use will also depend on the perspective of any future evaluation, and may be excluded if taking only a national health and personal social services perspective.

Capture the experiences of children and parents who participated in the intervention and of professionals who referred or supported families

Content co-development was within the limitations of the target population. The aim was not to co-design from the start. The outcome of the intervention (i.e. HRQoL) was determined a priori by previous literature identifying evidence gaps. Thus, intervention development may be more closely aligned with stakeholder co-refinement and co-development of format and content, rather than co-production across the piece. The aim of the qualitative aspect of the feasibility trial was to seek further CYP input for amendments and adaptations in future iterations of the intervention.

Although children were unable to contribute to intervention content directly, their views were gathered through qualitative work in Phase I and represented during presentations at the consensus exercise. Professionals involved were not restricted to those working in health, allowing for varied views from different capacities and organisations to support the development of the Young SMILES intervention.

Children and young people who provided their views (and who subsequently took part in the Young SMILES intervention) lacked representation of older CAPRI (12–16 years). This is discussed as a limitation in Chapter 9.

We assessed acceptability as per the TFA framework criteria detailed in Table 23. A key misunderstanding among parents related to the parental aspect of the intervention. There were uncertainties about the reasons for attending and the way in which the parental element would be delivered with a clear tension between child and parental needs and outcomes evident. In comparison to previous support, Young SMILES was favourable with improvements to family environment, relationships and cohesion recognised as important outcomes.

The views of CAPRI were generally favourable towards the Young SMILES intervention: atmosphere, environment and facilitator personality were all reported as important contributors to a positive experience. Some concern was expressed about the ability of the Young SMILES intervention to support such a broad age range. In any future interventions, work may be needed to ensure the successful support and integration of older children to reduce alienation. Increasing opportunities for developing social networks may also enhance the experiences of young people.

Individuals referring in to and delivering the intervention recognised the opportunity for the Young SMILES intervention to fill an identified gap in service provision for CAPRI. The Young SMILES intervention was felt to be well aligned with organisational aims and objectives but needs to be adaptable to individual needs. In the current model, this depends on experienced facilitators being able to draw on others’ experiences in a collaborative approach, and when this was achieved in the groups it was valued. Strategies to implement the Young SMILES intervention in co-delivered sites requires exploration, and adequate resourcing would be necessary for the successful implementation and sustainability of the Young SMILES intervention. Parental readiness was identified as a factor that could influence the implementation and success of the Young SMILES intervention; further work is needed to maximise young people’s outcomes.

In the children’s groups, facilitators introduced self-esteem and confidence-building work as this was judged necessary for their group; this was not included in the Young SMILES programme. The imaginary family was a useful tool that the group did use, but sometimes the group was more comfortable referring to their own family’s experiences. Most of the exercises in the manual had two versions, one for the younger children and one for the older children, but, at times, facilitators had to pitch the exercises somewhere in between. In the parents’ groups, self-esteem work was also incorporated, as the facilitators felt that the parents needed this to enable them to engage with the group. The facilitators also allowed additional time for parents to talk about the impact that their mental health has on their children, as this was an emotive and important issue for them. Parents dealing with guilt and shame needed extra time to discuss and reflect on this during and after the sessions.

Some of the issues identified in the qualitative work, specifically parental readiness, highlight that further exploration of this issue is necessary to add to the current existing knowledge gap; if the Young SMILES intervention becomes routine, this potentially will have a larger impact. Families would need to be assessed to identify when they are ‘ready’ to engage in the intervention to ensure that child outcomes are maximised.

One adverse event was recorded throughout the entire study. A mother reportedly experienced distress after a Young SMILES facilitator took her comments about the welfare of her family too literally, contacting social services and reporting it as a safeguarding concern. The parent made a formal complaint to the research team, which was handled appropriately and responded to by the NSPCC after an investigation was conducted. The mother accepted the verdict of investigation, which saw evidence of malpractice.