Introduction
Context: a realist process evaluation
Process evaluation promises insight into fidelity and quality of implementation, provides information about causal mechanisms and contextual factors, and supports an understanding of why an intervention fails or has unexpected consequences.25 The more recent development of the Medical Research Council (MRC) guidelines has stressed the importance of theory in this.25
Realist evaluation is one such theoretical approach. Based on the work of Pawson and Tilley,62–65 realist evaluation demonstrates a concern for causality and change mechanisms, postulating the ‘contextually contingent nature of these’ and challenging how ‘hypothesised causal chains play out in the implementation of a complex intervention’.25 Put more simply, a realist approach considers the GtACH programme to be a resource that enables change to happen. The GtACH programme in itself does not reduce falls but rather provokes a response or creates mechanisms for change to happen. Change may come (for example) in the form of individual knowledge, awareness, confidence or organisational structures and it is these changes that lead to difference in falls outcome.
A key tenet of realist evaluation is that different mechanisms will be triggered in different contexts. The GtACH programme may not work in all places and, if it does work, it may work in different ways depending on which mechanisms (organisational structure, individual knowledge, self-confidence, etc.) are triggered. Understanding the contextual circumstances of GtACH programme delivery and identifying those mechanisms that are triggered in different settings is key to understanding how the GtACH programme might be implemented in the future.
Aims and objectives
The aim of this realist evaluation was to generate detailed insight into the delivery of the GtACH programme to (1) identify those contexts in which it is easily adopted and (2) recognise those mechanisms that lead to positive outcomes.
The evaluation considered consistency of the GtACH programme’s use within and across care home settings and illustrated the views and opinions of key stakeholders about the adoption of the GtACH programme. Specifically, it assessed:
fidelity in GtACH training
fidelity in the use of the GtACH screening and assessment paper tool
acceptability of the GtACH programme (training and tool)
impact of the GtACH programme on falls rate.
Background
Care homes pose a distinct challenge for the introduction of complex interventions: they vary in size, funding, workforce and culture, and house vulnerable individuals with far-reaching health and social care needs. This heterogeneity of organisational context and uncertainty of individual need is an inherent (and unavoidable) barrier to effective innovation.24,66–70 Although the delivery of programmes such as the GtACH programme is intended to be consistent, with justifiable variation only, it may be that the needs of local residents and the preference of local staff create situated and specific variations in how programmes are delivered.
Our starting point in this (as with all realist evaluation) was to consider how the GtACH programme was intended to work. In the realist tenet, programmes are not simply treatments or interventions, but rather ‘Programmes are “theories incarnate”. Every programme has a theoretical underpinning, whether it is made explicit or not’.65 A programme such as GtACH rests on some theorised causal relationship that has a broader reach and application than its specific components; previous realist research illuminates the type of programme theories that might underpin initiatives in care homes.71
In a recent realist review,24 three broad programme theories were recognised in the delivery of health care to care home residents. Incentives, targets and sanctions might motivate GPs to engage more routinely in the delivery of health care to this group; greater involvement of experts might make for more appropriate provision of elderly health care; and health outcomes might be improved by better relational working that spans care home staff and external health professionals. (Relational working may already exist; this is more about improving relational working.)
More recent research by the same authors66 identified similar programme theories. Improved relational working might underpin better outcomes in the delivery of health care; dedicated (financial) investment can trigger more appropriate provision; and wrap-around care, manifest in referral networks for external services, can support care home staff in accessing appropriate specialist care. The benefit of dementia-specialist services is a final focus for improved care home health care.
The PEACH (Proactive hEAlthcare of older people in Care Homes) programme67 looked to programme theories derived from the quality improvement collaborative literature to examine the implementation of comprehensive geriatric assessment (CGA) in care homes.
The FIRE (Facilitating Implementation of Research Evidence) evaluation68 speculated that the implementation of incontinence recommendations is mediated by organisational context, a pertinent theory of action, and staff support for change.
The GtACH programme had its own programme theories (see Chapter 5, Initial programme theories), but these examples demonstrate how broader theories and causal relationships might underpin the specific components of an intervention. They highlight that it is not (simply) the adequacy of the incontinence recommendations, the appropriateness of the CGA approach or the health care delivered, but rather that it might be incentives, better relational working,24,66 sharing best practice67 or the fit with organisational context68 that govern the impact of an intervention.
Initial programme theories
In accordance with this approach, this evaluation looked beyond the individual elements of the GtACH programme to identify initial programme theories that could be tested in the evaluation. These programme theories were derived from previous published work relating to the creation of the GtACH programme27,28 and its early testing,26 and were verified by the FinCH trial TMG (February 2017).
Programme theory 1: connecting falls risk to remedial actions
In care homes, falls are a constant risk because of the complex mix of individual, organisational and environmental factors; this complexity and variety has made falls management difficult. Prior approaches to falls management have stressed the inter-relationship of different risk factors and have sought to quantify an individual’s risk of falling. Prior strategies have often focused on generating combined risk scores for individual care home residents, with less concern for the measures that might be taken to limit falls risk.
The GtACH screening and assessment paper tool isolates and disaggregates individual risk factors and connects them with specific actions to reduce risk. The GtACH programme is based on the value of considering each category of falls risk independently and the importance of generating solutions as well as understanding risks.
Programme theory 2: supporting all staff in falls risk management
Care home staff are heterogeneous in expertise, experience, training and skills; they will be more or less aware of falls risk and those measures that might reduce it. Consequently, care home staff may be more or less able and prepared to utilise the GtACH paper tool, which has implications for the effective delivery of the GtACH programme.
Specialist staff (local falls champions and regional NHS falls leads) are intended to support implementation by offering direct support to individual staff and by providing greater knowledge and expertise about falls risk management. Specialist staff are intended to ensure that there is consistent and appropriate delivery of the GtACH programme.
Methods
Study design
This was a multimethod process evaluation run concurrently with (but independently of) the main trial. It was informed by the principles of realist evaluation62–65 and was characterised by a concern for testing the programme theories described in Initial programme theories.
The evaluation incorporated a number of distinct but inter-related stages: (1) the formulation of initial programme theories (see Initial programme theories), (2) theoretical sampling to identify the most appropriate environments to test these theories, (3) the adaptation of these theories or creation of new programme theories, (4) recognising patterns in these revisions and (5) identifying a mid-range theory that explains these patterns.
Review and revision of the initial programme theory takes the form of context–mechanism–outcome (CMO) configurations that are the mainstay of the realist approach.62,63,65 Context here relates to those individual or organisational features that predate the introduction of the GtACH programme, including (among a variety of care home-specific details) the size and ownership of the care home, the nature of its provision (residential and/or nursing) and will also include characteristics of both staff and residents. Outcome describes that which results from the introduction of the GtACH programme, including a concern for fidelity and acceptability, as well as any measurable change in the frequency or consequence of resident falls.
Mechanism is perhaps the most complex element of this equation64 and is seen here as a mediating factor that illuminates the causal relationship between the introduction of a programme into a specific context and the precise outcomes that result therein. Mechanisms are more than the resources introduced; they also encapsulate the individual or organisational reaction to or reasoning about the resources invested.62 As this suggests, a realist approach acknowledges that mechanism might just as equally be a subjective response to the GtACH programme as it might be an objective change to practice.
Multiple CMO configurations are the likely output of any realist evaluation and it is evidence of recurrent patterns across these CMOs that is suggestive of more certain causal relationships.72 Although such demi-regularities do not represent undeniable causality, they do offer a layer of explanatory power that aids understanding of the outcomes achieved. Reflecting on these demi-regularities completes the evaluation cycle and it is in these recurrent patterns that the strengths and shortcomings of the initial programme theories are made explicit. Further explanatory power might be achieved in the application of some more general mid-range theory to unpick these patterns.
Participants
Only those care homes randomised to the GtACH programme were included in this process evaluation and consent for the evaluation was taken independently of that for the main trial.
Care homes were selected purposively73 from those that expressed a willingness to participate. Selection was driven by the realist agenda of testing programme theories against a range of contextual features that initially were considered pertinent to delivery of the GtACH programme. This included the size of the care home, its ownership and the presence or absence of nursing staff.
Where the care home manager consented, all pertinent staff were approached and those willing were consented to the process evaluation. Residents who were identified as able were also approached and consented when they were willing.
Care homes were recruited in different geographic areas so as to capture any variation in local practice/policy, and regional NHS falls leads (who delivered GtACH training) were also involved in the process evaluation.
Data collection
Finally, for each care home involved, falls rate data were also included. This necessarily includes falls data for those residents who were not able (or unwilling) to consent to the process evaluation, but this was collected as part of the main trial and was not specifically collected for the process evaluation.
Data were collected using a combination of interviews, focus groups, fidelity observations, documentary review and falls rate reviews. Data were primarily collected during a 3-month period following the introduction of the GtACH programme, with an additional care home visit made 6 months after the introduction of the GtACH programme.
GtACH training was observed in each care home using a checklist to assess fidelity with the training protocol (see Report Supplementary Material 6). The primary training session was observed by two researchers, with additional sessions observed by at least one researcher.
The implementation of the GtACH paper tool was observed in each care home using a checklist to assess fidelity with the GtACH programme protocol (for the fidelity checklist, see Report Supplementary Material 6). In each care home, multiple researchers would record this process with a number of different residents. Evidence of use of the GtACH tool was also sought in a review of care home records.
A staff focus group was held immediately post GtACH training that considered their experience of training and their expectations of the GtACH programme. At the 6-month visit, a second focus group was organised that reflected on staff experiences and their thoughts about future use of the GtACH programme (for focus group guides, see Report Supplementary Material 7).
During the evaluation, a number of key stakeholders were interviewed. This included care home staff, the care home manager, the care home falls champion and the regional NHS falls lead; where possible, care home residents were also interviewed. Interviews focused on the local experience of the GtACH programme (for interview topic guides, see Report Supplementary Material 7). All interviews were recorded using digital audio equipment.
Falls data for care homes included in the process evaluation were sought to inform the outcome element of our CMO configurations. Practicalities of trial management (maintaining blinding, etc.) meant that this was provided as a single data set after all other process evaluation data had been collected, and it did not include adjustments informed by HES data about resident hospitalisation.
Data analysis
All interview and focus group data were transcribed in full, anonymised and handled using the NVivo (QSR International, Warrington, UK) software package. All data were coded by at least two researchers and the organisation of themes and the structure of the coding book was agreed by the process evaluation team. Initial coding was also verified by the FinCH trial PPI group. Fidelity checklists were reviewed by at least two researchers.
The focus of analysis in this realist evaluation was on the iterative development of those programme theories that aid understanding of the implementation and impact of the GtACH programme; it is focused through the lens of three conceptual tools: context, mechanism and outcome.
Thematic analysis74 of interview and focus group data added specific detail to the context by exposing existing practice, process and priorities. Baseline falls data also supplemented our understanding of the context.
Thematic analysis74 exposed those mechanisms triggered by the introduction of the GtACH programme that were manifest in stakeholders’ descriptions of its delivery.
Descriptive statistics for each process evaluation care home, for all intervention care homes and for all control care homes were produced to illuminate the outcome of the GtACH programme. These findings were mapped to the main trial baseline and primary data outcome time points. Thematic analysis and fidelity checks also aided our understanding of the outcome, highlighting stakeholder assessment of acceptability and demonstrating fidelity of use.
Data were synthesised in the form of multiple, specific CMO configurations for each care home. In each configuration, context and outcome were considered fixed, with the mechanism ascribed the causal power to explain why/how specific outcomes emerged in a context. CMO configurations were reviewed by the process evaluation team and recurrent patterns across multiple care homes (demi-regularities) were identified.
Results
Care home characteristics
Six care homes were recruited to the process evaluation from different parts of the country (to reflect different local practice); they ranged in size, included both residential and nursing homes and demonstrated different models of ownership and management ().
Care homes and participants in process evaluation
Across these settings, 88 participants consented to take part in the evaluation, 44 stakeholders were interviewed and 11 focus groups took place. Overall, 7 managers, 4 deputy managers, 1 nurse, 3 falls champions, 1 floor manager, 22 senior caring staff, 38 caring staff, 6 residents and 6 NHS falls leads took part in the evaluation.
Falls rate data
A total of 194 independent codes were identified in the data. These were organised within a simple thematic structure (consisting of 14 broad themes) that reflects a pragmatic concern for delivering the GtACH programme. Themes included the GtACH tool, the falls champion, GtACH training and GtACH programme implementation. The complete code book is presented in Appendix 6.
The data presented in Chapter 3 suggest that the GtACH programme (training and tool) offered benefit to those care homes in which it was introduced. The process evaluation was completed in six of the intervention care homes, with the falls rates for these homes shown in , alongside the average from all the homes.
Summary of falls and falls rate data for process evaluation
Counts of the number of falls in each setting (unadjusted for the size of the care home or the nature of the care offered) suggest a distinct trend, which we describe here to allow the data from the process evaluation to be compared with the whole sample. At baseline (90 days prior to randomisation), the number of falls recorded in participants from all homes ranged from 0 to 25 falls. For the period 91–180 days post randomisation, the number of falls ranged from 1 to 116 in control settings and from 0 to 28 in those homes where the GtACH programme was introduced. No care home that received the GtACH programme recorded > 30 falls in a 90-day period (at either baseline or post randomisation); in the control arm, three care homes recorded > 50 falls in the period 91–180 days post randomisation. An unadjusted count of falls in all care homes is presented in Appendix 7, Table 30. In the control arm, the rate of falls increased by 4.53 falls per 1000 resident-days at 91–180 days post randomisation (see ); in the GtACH arm the increase was less pronounced, at only 1.32 more falls per 1000 resident-days.
Of those care homes included in the process evaluation, the falls rates decreased in care homes B and E; in care homes C and F, the rate of falls increased at a slower rate than in the control arm; and in care homes A and D the rate of falls increased at a greater rate than in the control arm.
Care home experiences
Results for each home are described with a full list of CMO configurations for all care homes in Appendix 8, Table 31, and summarised in .
The CMOs for six care homes
Care home A (0803)
Context
This was a 71-bed care home providing residential dementia care. The home was corporately owned and was part of a large national chain.
Outcome
GtACH training was delivered in accordance with the training guidelines but implementation of the GtACH screening and assessment paper tool was poor (only six GtACH tools were completed for the 18 recruited participants). Only senior staff used the GtACH tool correctly, as per the training and intervention manual; carers rarely used it independently (only doing so when observed by a researcher) and did not complete it in full. The GtACH tool was not being used at the end of the 6-month period.
Both the number of falls and the falls rate increased during the observation period: the number of falls increased from 12 in the baseline period to 19 in the primary outcome period, and the falls rate increased from 7.41 falls per 1000 resident-days to 12.44 falls per 1000 resident-days.
Commentary
The staff reported that falls prevention provision was well established before the study, and staff felt knowledgeable about falls and confident in their management. Staff were reluctant to adopt new ways of working alongside (and in addition to) the home’s existing systems. Managerial changes during the study had a negative impact on the implementation of the GtACH programme and change was not driven by either senior managers or care home staff. The home did not actively instigate the falls champion role.
Care home B (0703)
Context
This was a 48-bed, dual-registered, nursing-led home providing residential, dementia and nursing care. The home was corporately owned by a small chain.
Outcome
The GtACH training was delivered in accordance with the training guidelines. Implementation fidelity was poor. A number of GtACH assessments were completed by a single member of nursing staff, but only in anticipation of a process evaluation interview. Otherwise, the tool was not used during or after the process evaluation observation period.
Both the number of falls and the falls rate decreased during the observation period: the number of falls decreased from six falls (baseline) to four falls (primary outcome period), and the falls rate decreased from 4.21 falls per 1000 resident-days to 3.53 falls per 1000 resident-days.
Commentary
A change of manager at the outset of the study was marked by a reluctance to introduce new systems at a time of change. The new manager would not sanction additional paperwork alongside existing home systems and processes. By contrast, the GtACH training was well received and valued by staff and management alike. Staff described feeling more aware of falls risk and more confident in addressing them; management described changes to staff behaviour, with staff becoming more proactive with falls management.
Care home C (0402)
Context
This was a 46-bed residential home, which was part of a small local chain run by a charity that specialised in supporting people with sight loss and/or dementia.
Outcome
The GtACH training was delivered in accordance with the training guidelines. Twenty-four GtACH screening and assessment paper tools were completed during the observation period, although most of these (n = 14) were completed when researchers were present. Few of those observed were actually completed correctly. It was considered unlikely that the GtACH programme would be continued post study.
The number of falls and the falls rate increased during the observation period: the number of falls increased from 14 falls (baseline) to 20 falls (primary outcome period), and the falls rate decreased from 5.42 falls per 1000 resident-days to 8.85 falls per 1000 resident-days.
Commentary
An enthusiastic falls champion involved all grades of staff in the completion of the GtACH programme and staff reported that it was more in-depth than the home’s own documentation. Despite (or, perhaps, because of) this, care staff in this setting were uncomfortable and lacked confidence when faced with the GtACH programme; some of them did not consider ‘paperwork’ to be part of their job and some were anxious about their ability to complete the tool correctly. Longer term, it was felt it was unlikely that the GtACH programme would be used, as any change in paperwork had to be adopted by all homes in the chain.
Care home D (0302)
Context
This was a 40-bed residential home with a high number of residents with dementia. Only residents with dementia were recruited into the trial.
Outcome
Training was delivered in accordance with the GtACH training guidelines. Implementation fidelity was poor. Only one observation was completed as a result of cancelled visits, and fidelity was assessed using filed tools – in all cases completion of the GtACH tool was judged to be poor. The GtACH programme was not continued post study period.
Both the number of falls and the falls rate increased during the observation period: the numbers of falls increased from 8 falls (baseline) to 16 falls (primary outcome period), and the falls rate increased from 8.97 falls per 1000 resident-days to 18.29 falls per 1000 resident-days.
Commentary
This home was part of a very large national chain. The manager had previously worked as the falls awareness trainer for the chain and had trained the staff in falls prevention. Staff felt knowledgeable and confident in falls management. It was reported that most falls occurred in the evenings and that this may be attributed to increased confusion as a result of dementia. The staff perceived the GtACH programme as a useful prompt, but felt that it could not be used as a standalone tool without the entire chain changing its practice and procedures.
Care home E (0209)
Context
This was a small independent residential home with 17 beds and 19 staff.
Outcome
Training was delivered in accordance with the GtACH training guidelines. Implementation fidelity was poor – three tools were completed during the observation period, but none of the completed GtACH tools was judged to have met fidelity standards. It was reported that it would be unlikely that use of the GtACH tools would be continued post study.
Both the number of falls and the falls rate decreased during the observation period: the number of falls decreased from nine falls (baseline) to three falls (primary outcome period), and the falls rate decreased from 8.33 falls per 1000 resident-days to 3.03 falls per 1000 resident-days.
Commentary
In contrast to the other homes, residents here were more physically able and independent in their day-to-day lives; some residents were observed leaving the home to walk around a local park. This home was not registered for dementia care. Few of the residents were considered to be at high risk of falling.
This independent care home had previously received in-house falls prevention training only, and the external training provided as part of the FinCH trial was received with enthusiasm. By contrast, perhaps because residents were more mobile and independent, the GtACH tools were not considered appropriate for the residents’ needs.
Care home F (0107)
Context
This was a 53-bed residential home, which was part of a small, family-operated chain providing care across five homes.
Outcome
Training was not observed in this setting, so it is not possible to comment on the fidelity of the training. Some negative feedback about the training was received and it should be noted that the NHS falls lead did not participate in the in-depth training, but was introduced to the GtACH programme at the site initiation visit. The GtACH tool was not inserted into residents’ notes until the later end of the process evaluation period and, consequently, the implementation of the GtACH tools could not be observed.
Both the number of falls and falls rate increased during the observation period: the number of falls increased from 12 falls (baseline) to 20 falls (primary outcome period), and the falls rate increased from 3.19 falls per 1000 resident-days to 6.93 falls per 1000 resident-days.
Commentary
There was a change of management in this home, with the new (temporary) management having little knowledge of the FinCH trial. This meant that the falls champion role was not adopted and the implementation of the GtACH programme was delayed. Previous training in this home had been largely limited to in-house training. Staff were reported to be keen to attend falls awareness training. However, there was a lack of staff confidence around completing the GtACH programme.
Recurring patterns (demi-regularities)
The effectiveness of the GtACH programme is predicated on two notions: (1) that falls risks are better managed when they are identified and specifically rectified (rather than simply quantified); and (2) that care home staff may benefit from training and peer support in managing falls. Here, we introduce five recurring patterns that illuminate the extent to which these notions are fulfilled in the data generated here.
The relevance of prior practice
All settings included in this evaluation demonstrated the existence of falls management systems prior to the introduction of the GtACH programme, and no setting totally adopted the GtACH tool and removed their own process. The tool was more often used by the evaluation team rather than being adopted into routine practice. Care home staff indicated satisfaction and familiarity with their existing systems, which meant that they were not motivated to adapt their practice and incorporate a new tool:
I don’t think I’d feel any better or, I don’t feel I’d do my job any better filling this in every time. The form we’ve got is adequate.
Senior carer 0209 417
Staff pointed to capacity issues (and to the duplication of effort) associated with the implementation of the GtACH tool:
It’d be the time element, we wouldn’t be able to fill one out three times because it’d be three times for the same thing. We wouldn’t have the time to do that because we’ve already got the action tools to fill out, then we’ve got the 24-hour obs[ervation] to fill out. Erm, so realistically, you know, we wouldn’t be filling that out.
Carer 0803 502
Because our paperwork, as it is, takes a lot of our day up, especially when it comes to a fall, you know . . . to then have to fill out more paperwork, and to duplicate it however many times it happens, it can be time consuming to us, and it takes us away from doing the rest of our work, you know, that’s the concern for me.
Senior carer 0302 413
In accordance with programme theory 1, staff did recognise the value of identifying and specifically rectifying falls risks, but felt that their existing systems already achieved this without the need for new tools.
The relevance of training
The benefits of the GtACH training were recognised across all settings, with a clear recognition among more experienced staff that its benefits reached beyond and are distinct from using the GtACH tool:
. . . I liked the training. It was a refresher for myself and the other qualified [staff] . . . I think, again, it made us look a bit beyond what, why, you know, what medication are they on, have they got an infection? I think we pretty much do that anyway. But there was factors on there that I perhaps didn’t think of myself. You know, because it does tell you through the list of other things to look for. I think, we have struggled filling the paperwork in but the knowledge has stayed in our head. I don’t know if that’s the right or wrong thing to say but the knowledge is certainly there and we do talk and look at why people are falling, but I think some of the care staff struggled with the paperwork.
Falls champion 0703
Training was considered particularly beneficial in those settings where prior training had been lacking or had been internally delivered (e.g. care homes B, E and F) and in those settings where parts of the staff group had not previously managed falls (e.g. care home B). In these settings, training generated greater knowledge of and confidence regarding falls management, and more acceptance of shared responsibility for managing residents’ falls.
Comments in care home F expressed a disappointment that more had not been covered in the training:
Thought it was very good, really. I thought it was going to be a bit more about falls in general, rather than just the form [tool], it seemed to just cover the form [tool], and I thought it was, well, it was just kinda sold to us wrong as staff.
Deputy manager 0107 203
It is manifest in other aspects of the evaluation that the GtACH training did not sufficiently encourage a broad use of the GtACH tool (as, perhaps, was intended). However, it acted as a refresher for experienced staff and provided new knowledge for the less experienced; training encouraged engagement with falls management, if not with the GtACH tool itself.
Staff roles
Where not all staff manage falls (see ), the GtACH programme potentially brings changes to staff roles and responsibilities – falls are no longer the domain of nursing staff or senior staff alone, but become a concern for all. Training might encourage carers to engage in falls management, but this process is more effectively cemented when local staff take ownership of the GtACH programme and support its use. In care home B, where the falls rate decreased, it was nursing staff who acted as advocates for the GtACH programme; in care home C, the falls champion sought to engage all staff in its use:
. . . because the carers care for the people, and they know them more than what we probably do, and what their daily living is, that’s why we’re getting involved with the carers with this as well . . .
Falls champion 0402 0602
When (less experienced) staff indicated that they might become engaged with falls, they often made a distinction between providing care and completing paperwork:
. . . we want to provide practical care and support, etc., and, unfortunately, it’s like in the hospitals, there’s more and more going in, on to, you know, the computer, on to paper, and it’s time-consuming, it does take you away from looking after the ladies and gentlemen.
Carer 533 0107
Anxiety about completing paperwork was communicated in all care homes and many carers felt that completing formal records was beyond their level of qualification and experience:
I think is better for someone who is more . . . higher from me. I am not confident with fill this everything. I think is better job for them, and I think, because, exactly, they have better contact with GP, doctors, everything. They know more better about like, some forms, documents, I mean.
Carer focus group 0803
The introduction of the GtACH programme challenges care home staff to review their roles and responsibilities with regard to residents’ falls. With the provision of training and the support of peers (programme theory 2), such changes seem acceptable to care staff; however, this acceptance does not extend to incorporating completing paperwork into their roles.
The significance of residents with dementia
The presence (or absence) of residents with dementia would seem significant in the implementation of the GtACH programme: care homes A (with a high proportion of residents with dementia) and D (where only those residents with dementia were recruited) demonstrated the greatest increase in falls rate; and care home E (where no residents were registered with dementia) showed the greatest reduction in falls rate. The GtACH tool was not adopted in any of these settings, but for quite different reasons.
In care homes A, D and C (with residents who have dementia and/or were visually impaired), falls were considered an inevitable consequence of residents’ health. Implementing the GtACH programme for residents with dementia could not change this underlying factor and was thus considered to be of little value with these residents:
. . . it’s silly questions to me, because I know the gentleman has got, probably, the end journey of dementia, he’s not going to be able to tell us, you know. He knows, if he gets up, he’s not aware of what’s around him, you know, and you’re asking me these questions where I’m thinking, oh my God, you know, you lot, you know, people, whatever, you know, I know him that well, he doesn’t acknowledge what time of day it is, what’s around him or anything, you’re asking me these quest– it just doesn’t help.
Carer 0402 518
This directly challenges the notion that falls risks can be mitigated and managed by appropriate actions, as proposed in programme theory 1. By contrast, the circumstances in home E led staff to make a similar assessment, but in this case it was because no resident displayed a constant and significant risk of falling:
. . . even though we’re relatively small as care homes go, we do have quite a lot of able-bodied residents, at least half or so, with capacity, so they make their own decisions around their own risks. And it’s something that we’re very keen on here, that we don’t restrain anybody with moving around the home with freedom. So we do have quite a lot of falls, we have periods where we’ll have, you know, one or two people that, for whatever reason, do have a number of falls in a short space of time . . .
Care home manager 0209 104
Although residents fell in this home, the more independent and self-caring nature of the residents meant that risks were perceived differently:
[Residents] take their own risks, and it’s something that, that we train the staff to, to support residents to explain what the risks are, but actually then make them, allow them to make that decision.
Care home manager 0209 104
Both the presence and absence of residents with dementia undermined the perceived utility of the GtACH tool; in one setting, residents were beyond assistance, while in the other they did not require assistance.
Care home ownership and operation
All bar one of the care homes were part of broader organisations: care homes A and D were part of large, national care home groups, homes B and F were part of smaller, regional groups and care home C was operated by a national charity; only care home E was independently operated. External management potentially inhibits the freedom with which a home might adapt its local practice, and/or might place restrictions on what might be changed so as to maintain consistency across a number of care settings:
As an organisation, across the four homes, because there’s four homes, if we want to change anything or do anything, we have to do it as an organisation. So it would not be sort of, if you like, correct for us to suddenly stop using what we already use, and to take on board a different tool, unless we could get that tool approved for the rest of the organisation, particularly around falls and falls prevention.
Care home manager 0402
Of more immediate impact within this study was the requirement for homes to continue using systems and paperwork; the GtACH programme might be used alongside, but not instead of, existing systems and processes. With internal processes being a requirement for all staff, their motivation to utilise the GtACH programme was somewhat diminished by the sense that it would duplicate their efforts and double their workload:
I feel like, well, first of all, we have to fill out the accident form, the legal one. Then we’ve got our own that we have to fill out. Then we have to go on to the system, and update all the care plans due to the accident, we have to write all what, everything that I’ve wrote on this, I put on the system anyway, so I just feel like I’m duplicating myself all the time.
Senior carer 0402
I know that the seniors think that it’s a lot, it’s a lot of, sort of, it’s, if they were doing one or the other, if they were doing, I think they wouldn’t mind doing it, but because they’re having to fill two lots of documentation in at the moment, they sort of do pull a face and say ‘Oh, I’ve got another one to fill in, another, more document, paperwork’.
Manager 0302
The implementation of the GtACH programme is premised on programme theories that make sense in relation to individual care settings (programme theory 1, identify and act on falls risks; programme theory 2, support staff in this), but which might be inhibited by the organisation of broader management systems.
Discussion
Key points
Several general points might be made about the care home experiences of the GtACH programme:
The impact of the GtACH programme might vary in different settings. In our evaluation, the falls rates decreased in two homes, stayed stable in two homes and increased in two homes.
It is pertinent to reflect that (1) awareness of the intervention, (2) taking part in training, (3) completing the tool and (4) taking action to reduce falls might be distinct activities that are not mutually dependent. A commitment to falls management and fidelity in training might have a positive impact on falls rates without the GtACH tool being widely used in a care home, as was evident in some settings here.
Different aspects of the programme sparked different mechanisms. Training was viewed as a refresher by some, empowering others and broadening engagement with falls management. The tool was viewed with indifference, considered a duplication of local systems and was a source of anxiety for some. In some homes, local champions encouraged innovation in practice, while in others external management inhibited local ownership of change and were a barrier to long-term integration.
Despite these variations, the initial programme theories still have broad application. Falls would seem best managed when specific systematic strategies are aligned with specified risks. However, there was a strong feeling that falls are viewed as unavoidable in people with dementia and efforts to manage these falls are considered pointless.
The evaluation also demonstrates the value of specialist and peer support to care home staff – this may not take the form of a formal falls champion, but might be more informally managed.
Interpreting the results
The final element in our realist evaluation is to utilise normalisation process theory (NPT)75,76 to reflect on our findings. NPT is a mid-range sociological theory that supports understanding of how innovation becomes normalised in everyday routines in practice. For the GtACH programme to become part of everyday practice, it needs to be understood by stakeholders (coherence) and valued by them (cognitive participation); individuals should be able to enact the work associated with the GtACH programme (collective action) and the outcome of the GtACH programme should be clear and observable (reflexive monitoring). Failure to achieve any one of these building blocks is a barrier to the GtACH programme becoming part of normal practice.
Coherence
A recurrent observation was that staff (and some managers) found it difficult to differentiate the GtACH programme from already existing falls management initiatives (evident in demi-regularities, see The relevance of prior practice and The relevance of training). GtACH training was identified as a refresher for previously undertaken training, or was viewed as disappointing and too limited in its focus on the GtACH programme (rather than on falls generically). The GtACH tool was viewed in some places as an unwelcome duplication of existing paperwork. It is important to recognise this in the future implementation of the GtACH programme.
It is notable that in care home E, where no previous external training had been received, GtACH training was recognised as novel and as improving local understanding; this setting witnessed the largest improvement in falls rate of the included homes.
In care home C, the GtACH tool was identified as more detailed than local documentation; here, despite a population at a high risk of falling, the increase in falls rate was more marginal and was better than in the control arm of the study.
Communicating the value and distinctiveness of the GtACH programme is, perhaps, an important element of any future implementation; distinguishing it from more routine falls assessment will help stakeholders to more readily accept it into local practice. Communicating that the GtACH programme is a more appropriate form of provision might also support adoption, and mirrors strategies identified in the management of health care in care homes.24
Cognitive participation
We have noted above that dementia is an important contextual feature that might challenge the underlying legitimacy of the GtACH programme (see also The significance of residents with dementia). This is most explicit in care home D where (evening) falls were directly attributed to dementia and in care home C, where carers questioned the value of using the GtACH tool with residents who had dementia. In care home E, it was the absence of dementia that challenged the value of the GtACH programme.
The pertinence of dementia-specialist services has been identified in other realist care home research66 and highlighting the GtACH programme’s place in this research might aid stakeholders in recognising its appropriateness for all care home residents.
A second area in which the legitimacy of the GtACH tools has been commonly challenged is that some staff consider forms to be a distraction from the act of caring for residents or view forms to be outside their job role [see Results, Recurring patterns (demi-regularities), Job roles]. To this we might add staff who lack confidence in completing paperwork. This was manifest across all homes and perhaps suggests that a simpler form of paperwork is required for the future implementation of the GtACH programme, or that different types of the GtACH tool are required for different levels of care staff.
Collective action
The persistence of local, organisational falls management systems after the introduction of the GtACH programme was perhaps the single most significant barrier to the GtACH programme becoming normalised in those homes. Introducing the GtACH programme alongside existing systems and paperwork (see Care home ownership and operation) undermined the contextual integration of the GtACH tool into the actual work undertaken by care home staff. Staff faced the unenviable dilemma of either duplicating their efforts (for ostensibly the same ends) or ignoring one system.
We have noted above that the distinctiveness of the GtACH programme should be stressed to help establish it. For future implementation, GtACH may have greater influence where it is adopted as a single, coherent falls management system rather than alongside other falls management approaches. Recognising the appropriateness of the GtACH approach will help staff to adopt it as normal practice.24
Although the GtACH tool did not integrate well into the work undertaken by staff, this does not mean that the knowledge gained and the increased awareness of falls made no practical difference. Such differences were most clearly manifest in care home B (where the falls rate decreased) and care home C (where the rate of falls increased at a slower rate than in the control group). In both locations, local individuals championed the GtACH programme and encouraged all staff to change their practice (see Staff roles). The role of the falls champion and other informal advocates of the GtACH programme should be stressed in future implementation; these individuals have a critical role in translating the GtACH programme into workable local practice and in supporting other staff to adapt to new ways of working.24
Reflexive monitoring
Changes in management in care homes A, B and F undermined the coherence of the implementation of the GtACH programme either in part (a reluctance to use the tool) or totally (not wishing to introduce new things at a time of managerial uncertainty). Without the commitment of and, importantly, monitoring by senior management, the delivery and impact of the GtACH programme is uncertain; changes to practice are not rewarded and benefits of new approaches are not recorded. Without locally observed evidence of the GtACH programme positively contributing to residents’ well-being, it is difficult for it to become established. The success of the future implementation of the GtACH programme rests not only in integrating new ways of working, but in the effective monitoring of its impact.
Of more general concern is the role of external management systems in governing local practice (see Care home ownership and operation). With the exception of care home E (where the most improvement was manifest), all other homes had some form of external management system that might impose its own incentives, targets and sanctions.24 For the GtACH programme to become normalised, it needs to sit within these wider systems and demonstrate value to the broader corporate group. Future implementation needs to consider how the GtACH programme maps to broader organisational priorities and stress how it serves these metrics.
Process evaluation reflections: strengths and limitations
This process evaluation offers a detailed, contextualised understanding of how the GtACH programme was delivered and illuminates the experiences and opinions of the involved stakeholders. Using different methods, it complements the clinical and economic data reported in Chapters 3 and 4 and provides a framework for others to interpret these findings. It offers more textured results than the overarching RCT and provides insight about how best to implement the GtACH programme in the future, supporting stakeholders in considering which elements of the GtACH programme might work best in their setting, what adaptations might be required, and which elements might be ignored. In this way, it demonstrates the value of a process evaluation aligned with a RCT in the trajectory of developing and evaluating complex interventions.25
In line with more recent recommendations,25 a theorised approach to evaluation is taken here, with initial programme theories being a focus for testing and sampling decisions. The adoption of a realist approach provides a distinctive flavour in this: highlighting contextual variation in how the GtACH programme might work and recognising that subjective responses can be as important as objective change in the delivery of the GtACH programme. This evaluation demonstrates a pragmatic application of the different stages of a realist evaluation: initial programme theories, sampling to test theories, revised theories (and emergent mechanisms), recurrent patterns in CMO configurations, and mid-range theories to explore these patterns.
The evaluation was delivered by a multidisciplinary team and was managed independently of the main study. This was to ensure that care home allocation remained blinded and was not revealed to other parts of the FinCH trial,31 and to ensure that early insight did not lead to any change in practice in either the control or intervention arms of the trial.77 Each home was visited by multiple researchers on multiple occasions and all data were reviewed by at least two members of the team. Researchers were flexible and responsive to the needs of the care home, accepting that caring responsibilities were more important than our research. Other practical challenges might be taken as limitations: few homes had private space where interviews or focus groups could take place; in some homes, staff could participate when on their break or off-shift only; in some, management governed which staff participated; and research visits and research activities were sometimes cancelled or curtailed at short notice because of staffing and/or resident issues in the homes.78
We should also acknowledge some limitations with the realist method as applied here.77 Resources allowed us to evaluate in one care home at a time only, recruiting sequentially, and this affected our purposive sampling strategy. Rather than being able to recruit from a broad population (the 39 care homes randomised to the GtACH arm) at any one point in time, a more restricted choice was possible: those recently recruited to the trial, who were randomised to the GtACH arm but had not yet received training and who were willing to participate in the process evaluation. The window of recruitment to GtACH training made local investigation of, for example, prior falls history or staff knowledge (which might have productively directed our purposive sampling) impossible. Consequently, our sampling used simpler and more restricted characteristics (size, ownership, nursing provision, etc.) to govern where we tested the programme theories.
Difficulties accessing outcome data also had an impact on the realist evaluation. Seeking data for specific homes risked identifying to the trial team that these homes were receiving the GtACH programme. Consequently, hard outcome data were not used in the CMO configurations; rather, these incorporated softer, process concerns for fidelity, acceptability and evidence of use. In addition, primary outcome data (falls incidence in days 90–181 post randomisation, adjusted for hospitalisations) were not available until all trial data collection had been completed; consequently, the process evaluation pragmatically used unadjusted data in reviewing care home outcomes. Both of these barriers had an impact on the completeness of data available to the process evaluation when finalising CMO configurations, prioritising mechanisms and in sampling care homes for inclusion.
Despite these issues, this evaluation demonstrates the potential of a realist approach and contributes to recent debates about the integration of realist evaluation into RCTs.68,79–81 To be explicit, for the realist evaluation to sit within the FinCH RCT, some methodological compromises were necessary, most notably (although not exclusively) in sampling and access to outcome data. However, these compromises do not undermine the insight generated here.77 A realist approach uncovered those contexts where the GtACH programme had the most impact, and identified the reasons and responses to the GtACH programme that make it successful; a realist approach has extended our understanding beyond that which would have been possible with a trial alone.
