U.S. flag

An official website of the United States government

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Underwood M, Lamb SE, Eldridge S, et al. Exercise for depression in care home residents: a randomised controlled trial with cost-effectiveness analysis (OPERA). Southampton (UK): NIHR Journals Library; 2013 May. (Health Technology Assessment, No. 17.18.)

Cover of Exercise for depression in care home residents: a randomised controlled trial with cost-effectiveness analysis (OPERA)

Exercise for depression in care home residents: a randomised controlled trial with cost-effectiveness analysis (OPERA).

Show details

Chapter 4Discussion

Overview

OPERA was a complex multifaceted study. The nature of both the intervention being studied and the population studied means that a simpler study design would not capture all of the active intervention's possible effects (positive or negative) on care home residents. In addition to obtaining data on the effects of the intervention we have also been able to investigate, in detail, both the process of running the study and of implementing a complex intervention within care homes. This has generated high-quality data on how to do research and on change implementation in this environment.

At the time we started OPERA we had serious concerns regarding whether or not the study was feasible. In particular there were concerns that care homes, which are often unfamiliar with research, might be reluctant to participate. Contrary to expectations, over half of the care homes we approached were interested in participating. Furthermore, no homes dropped out of the study after randomisation, meaning that we were able to collect ample follow-up data for our cohort analyses and for our end-of-study cross-sectional analysis we had a full complement of homes. Care homes and residents maintained their commitment to the study through what, for some, were difficult times or after many care home staff changes. For example, one care home needed to close during the study because of safety issues and residents were moved to other homes. We were able to follow up most of these residents in their temporary care homes, and after building work was completed, the home re-opened and residents returned, and we were able to complete our 12-month study assessments as planned.

The number of participants per care home was fewer than anticipated at the study design stage. This was partly because of the surprisingly high number of vacant beds in our homes and also the high proportion of residents who were too ill or too cognitively impaired to complete study assessments. For this reason, during the study we changed one of our primary outcomes to ensure that we had adequate statistical power, while still recruiting the originally planned number of homes. Our original sample size estimates were inflated to allow for clustering effects. One of the striking findings was that the intracluster correlation coefficients for participant assessments were very small or zero. These factors meant that although we recruited substantially fewer participants than originally planned (765 vs 1231, with a Geriatric Depression Scale-15 prior to randomisation) the effective number of participants contributing to our three primary analyses is substantially greater than specified in our revised sample size calculation. This means that all of our main effect estimates are much more precise than planned, giving reassurance that our findings are robust.

The overall findings of the study are clear and conclusive. We developed a high-quality intervention that was received extremely well both by staff and residents within the care homes. Uptake of the intervention was very good and was maintained throughout the 12-month intervention period. We have some evidence that homes welcomed, and were supportive of, the intervention package. There was not, however, any benefit on any of our primary or secondary outcome measures. The OR for residents being depressed at the end of the study was 0.75 (95% CI 0.52 to 1.08), the mean difference in Geriatric Depression Scale-15 in the cohort analysis at 12 months was 0.14 (95% CI −0.33 to 0.60) and for the depressed cohort at 6 months this was 0.23 (95% CI −0.50 to 0.96) (Table 35). The limits of the 95% CIs for the mean differences are less than the minimally clinically important difference specified for the study (1.2 points in the Geriatric Depression Scale-15), effectively excluding the possibility that the OPERA intervention has a meaningful effect on depressive symptoms as measured by the Geriatric Depression Scale-15. Furthermore, in our health economic evaluation the OPERA intervention was both more costly and less effective than the control intervention. We conclude that the OPERA intervention cannot be recommended as a means of reducing the burden of depression in care home residents.

OPERA care homes

The OPERA care homes were broadly representative of care homes in the two localities, except that we excluded larger homes and we recruited only one local authority-run home. Local authority homes have also been under-represented in previous studies.86 It is unlikely that the process of implementing the intervention in large or local authority-run homes would be so different from that in our included homes that the intervention would be substantially more effective. Our findings are generalisable to residents in these homes.

We included only one dementia specialist home. This was because of practical difficulties in obtaining our primary outcome data from residents in these homes. We observed substantial differences in approach to activity between dementia specialist units within OPERA care homes and the rest of the homes; these units were generally more proactive in encouraging activity. This is a difference we would expect to be seen also in dementia specialist homes. It is possible that within a dementia specialist home improved adherence to the exercise regimen might produce beneficial outcomes. Thus, although a large proportion of our participants had substantial cognitive impairment our findings are not necessarily applicable to dementia specialist homes.

OPERA participants

Recruitment

The number of participating residents per care home was fewer than originally planned; the number of eligible residents per home was 28 rather than the anticipated 32. Only 37% of eligible residents provided a baseline Geriatric Depression Scale-15 score and only 43% agreed to provide care home data. This is similar to the proportion of residents recruited to another recent UK care home study of a physiotherapy intervention, targeted just at those with mobility limitations.86

Substantial numbers of residents were excluded (524/2078, 25%) (either because their health was too poor or because the care home manager felt that it was inappropriate to approach that individual for another reason). Those residents with poor health would have been unlikely to be able to participate in the exercise groups and would be unlikely to benefit from the intervention.

The baseline characteristics of those recruited prior to randomisation were broadly similar to population values. The baseline Geriatric Depression Scale-15 scores for our population (mean Geriatric Depression Scale-15 score 4.8, SD 3.3) (Table 19) are very close to those reported for care home populations in the UK (mean Geriatric Depression Scale-15 score 5.4, SD 3.2)120 and in the USA (mean Geriatric Depression Scale-15 score 4.8, SD 3.5).122 This mean Geriatric Depression Scale-15 score also supports reports that a cut-off of 4/5 on Geriatric Depression Scale-15 has good sensitivity and specificity for detecting clinically significant depression among older people, with an expected prevalence of significant depression of 40% in a care home population.

Around half of our participants were depressed, with a mean Geriatric Depression Scale score of 4.8 (SD 3.3). Of the 374 participants depressed at baseline, only 92 (25%) had a recorded diagnosis of depression and only 126 (34%) were taking antidepressants (Table 22). This is indicative of substantial under-recognition and undertreatment of depression in care homes and concurs with findings in previous studies.24,26

The mean Mini Mental State Examination score of 18.4 indicates that our participants had substantial levels of cognitive impairment and that our findings are applicable to the many care home residents with cognitive impairment. The mean EQ-5D scores of 0.57 (self-report) and 0.45 (proxy) attest to the overall poor quality of life of our participants. UK norms for people in the community aged ≥ 75 years were 0.73 (SD 0.27).211 This is consistent with findings from previous studies. Of participants recruited in intervention homes, 16 (3%) were considered ineligible for the group sessions by the study physiotherapists. Our participants overall state of health was, as anticipated, very poor and representative of our target population of care home residents able to participate in exercise sessions.

The mean baseline Short Physical Performance Battery scores indicate poor levels of lower limb function. To our knowledge we are the first study to administer the Short Physical Performance Battery among a care home-dwelling population in the UK. Some studies in the USA have used the Short Physical Performance Battery in assisted living populations;212214 however, it is not apparent that assisted living facilities in the USA are comparable with care homes within the UK. The floor effect, demonstrated in the baseline Short Physical Performance Battery scores, limits the ability of the Short Physical Performance Battery to detect deterioration in scores at the follow-up time points. Furthermore, as the Short Physical Performance Battery was designed for community-dwelling older adults, it may not be sufficiently sensitive to detect changes in lower limb function of the OPERA participants who are considerably more frail than older people living in the community.

Baseline social engagement shows a high percentage of residents reported as being highly socially engaged (i.e. interacting with others, very involved in activities, not withdrawn) in both the intervention and control groups (62% and 60%, respectively). In contrast, an international study of nursing home residents in five countries (Denmark, Iceland, Italy, Japan and USA) reported at most 30% of residents reporting high scores. In a large sample in the USA with high social engagement, residents also had high levels of cognitive function and high levels of activity of daily living.142 This difference in apparent social activity may be related to how we used the measure. As originally designed, it was based on direct observation. For this study we used carer report, which might be subject to bias in measurement of absolute score. We would, however, expect it still to be sensitive to change.

Unusually in the OPERA study we also recruited participants after care homes were randomised to ensure that for our end-of-study cross-sectional analyses we were studying a representative population rather than just relatively healthy survivors. This analysis tells us the likely long-term effect of implementing the OPERA intervention on overall prevalence of depressive symptoms in care home residents. Again this population was broadly representative of the population of care home residents overall, ensuring that our findings are generalisable.

Follow-up

Overall we obtained good follow-up rates. As expected there was a lot of attrition due to death but loss to follow-up arising from residents moving out of an OPERA home was higher than expected. Thus although we were able to obtain a Geriatric Depression Scale-15 on around 90% of participants present in an OPERA home at each end point this represents only 61% of our population of interest for individual assessment data in the cohort analysis at 12 months in the control homes. Nevertheless, our follow-up rate was always in excess of three-quarters of those who were alive for the primary outcome – ensuring that our conclusions in the cohort analysis are robust. There was a consistent trend for more follow-up data to be collected in the intervention homes than in the control homes. Our findings did not, however, change materially in a sensitivity analysis in which missing data were imputed or when an extreme scenario sensitivity analysis was conducted (data not presented). As the study had a much more visible presence in the intervention homes it is not surprising that there was some differential follow-up. The process evaluation has identified how warmly the intervention was received in the homes. It is likely that this was translated into an extra willingness on the part of the care home staff to help follow-up assessments and, of course, an increased interest in assisting in data collection from residents and providing data on residents. The difference is small, however, and any bias introduced by this differential follow-up would be very unlikely to change our conclusions.

In this study we have made use of Secondary Uses Service data from primary care trusts to collect most health service activity. We are extremely grateful to the primary care trusts who kindly collated these data for us during a difficult time of organisational change within the National Health Service. These data became available just 12 weeks after the end of the study period. Although these data may not be as clean as a final Hospital Episodes Statistics data set, available 6 months after the end of the financial year in which the activity accrued, this has provided a very timely resource for monitoring health service costs in a standardised manner. This data set does, however, have some limitations for collecting causes of A&E attendances and outpatient appointments. Although we cannot be certain that any attendance for injury at an A&E department or at a fracture clinic is the result of a fracture, these are likely to be indicative of a significant injury and are a good marker for the safety of the OPERA intervention.

Clinical effectiveness

This is one of the larger intervention studies run in a care home environment. Surprisingly, for many of the participant-reported outcomes the intracluster correlation coefficient was zero, or close to zero. This is in contrast with findings in some other UK-based cluster randomised care home studies, which have found intracluster correlation coefficients in the range 0.37 to 0.49 for some outcome measures.86,215 These intracluster correlation coefficients were, however, for measures such as the Barthel Index and the Rivermead Mobility Index, which were collected by independent assessors rather than directly from residents. Our largest intracluster correlation coefficient for assessment data, collected directly from residents, was 0.09 for the Short Physical Performance Battery; a larger intracluster correlation coefficient for measuring an outcome that is dependent on environmental and, possibly, assessor factors is not surprising.

We collected our proxy data from care home staff. Although the intracluster correlation coefficients were large enough to affect our statistical power (0.17 for proxy EQ-5D and 0.16 for Social Engagement Scale at 12 months) they were, surprisingly, much smaller than those found by others.

As a consequence of these small intracluster correlation coefficients we have sufficient data to make quite precise estimates for most of our primary and secondary analyses for between-group differences. Although we have conducted a large number of primary and secondary analyses we have failed to show any statistically significant differences in any of our outcome measures. The large numbers of participants means that we can be confident that these are true-negative effects.

Geriatric Depression Scale-15

Surprisingly, the point estimates for benefit from the OPERA intervention on the Geriatric Depression Scale-15 in both the cohort analysis (0.14, 95% CI −0.33 to 0.60) and depressed cohort (0.23, 95% CI −0.50 to 0.96) favour the control intervention (Table 35). We originally set the clinically important difference on the Geriatric Depression Scale-15 score as 1.2. It could be argued that this was based upon a minimally clinically important change for an individual, equivalent to the effect of loss of a spouse, rather than a minimally important change for a population and that we should have chosen a smaller mean difference as clinically important.216 The limit of the 95% confidence for possible benefit from the OPERA intervention is around one-quarter of the minimally clinically important change for an individual and equates to a standardised mean difference of 0.1 (lower limit 95% CI 0.33, SD at baseline = 3.3), effectively excluding any possibility of a beneficial effect on depressive symptoms, as measured on the Geriatric Depression Scale-15, from the OPERA intervention. For the original research question set by the funders, the proportion of residents with depressive symptoms at baseline who experienced remission of depression (at 6 months), we had data on 259 participants. This is fewer than the 343 participants specified in the simple sample size from our original application, justifying our a priori decision not to consider this as a primary analysis. In this case the direction of change was in favour of the OPERA intervention. This might suggest a possibility that we are overlooking a true effect. However, this would seem unlikely, given that the direction of change for mean scores in the cohort analysis favours the control intervention.

That there was no difference in the prevalence of depressive symptoms between baseline and follow-up in the control group (mean Geriatric Depression Scale-15 at baseline and follow-up 4.7 and 4.6, respectively) (Table 35) suggests the control intervention was also ineffective and it is not the success of depression awareness training that means we failed to find a beneficial effect from the OPERA intervention. There was not a significant difference in the rates of antidepressant use between the intervention and control homes at any time point (Tables 46 and 47). Neither was there any suggestion of a difference in how many participants started or stopped antidepressants. It is therefore unlikely that changes in medication use, for example increased use of antidepressants in the control arm, masked a beneficial effect from the OPERA intervention.

We are aware of one other cluster randomised trial that has reported the effect of a reasonably similar exercise programme on Geriatric Depression Scale-15 scores delivered to a reasonably similar population in residential accommodation. Conradsson et al.95 tested a moderate intensity functional exercise, delivered 29 times over 3 months, which did not include a whole-home approach. They found no benefit on the Geriatric Depression Scale-15 score at either 3 months (mean difference 0.09, 95% CI −0.55 to 0.74) or 6 months (n = 191)95 for all participants, or for those with a Geriatric Depression Scale-15 score of > 4 at baseline (mean difference −0.59, 95% CI −1.66 to 0.48 at 3 months; mean difference, 0.12 95% CI −1.12 to 1.37 at 6 months). Meta-analysis of these data and the OPERA data, for the main cohort and the depressed cohort, provide further support for the notion that such exercise interventions are unlikely to be effective in this population (Figures 21 and 22). For these analyses we have used Conradsson's 3-month data, which was at the end of the intervention period rather than 6-month follow-up, as it is at this time that any positive effect was most likely to be evident (the direction of change does favour the intervention in one of Conradsson's analyses), and pooled this with data from our primary analyses for these two populations.

FIGURE 21. Meta-analysis of mean difference on Geriatric Depression Scale-15 in those with depressive symptoms at baseline.

FIGURE 21

Meta-analysis of mean difference on Geriatric Depression Scale-15 in those with depressive symptoms at baseline.

FIGURE 22. Meta-analysis of mean difference on Geriatric Depression Scale-15 in all participants.

FIGURE 22

Meta-analysis of mean difference on Geriatric Depression Scale-15 in all participants.

Other outcomes

The absence of any statistically significant difference in any of our other outcomes is notable. Only for the Short Physical Performance Battery does the difference approach statistical significance; this is the case on both the cohort analysis (0.30, 95% −0.05 to 0.64) (Table 41) and the cross-sectional analysis (0.24, 95% CI −0.1 to 0.57) (Tables 44 and 45). A change of 0.5 points on the Short Physical Performance Battery is considered a small but meaningful change, and 1.0 points a substantial change.217,218 We have effectively excluded the possibility that the OPERA intervention will have a substantial effect on mobility and lower limb function. We have not excluded the possibility that the OPERA intervention might have a small but meaningful effect on mobility and lower limb function.

Failing to show a positive effect on the Short Physical Performance Battery may not be a surprising result given that the structured exercise activities needed to be performed largely seated due to the poor physical health and abilities of the exercise group participants, and so were less likely to positively affect the abilities measured by the Short Physical Performance Battery. Although the whole-home intervention included efforts to increase the amount of safe walking for all residents, it appears that the main effect of this approach was improved attendance and adherence to the structured exercise programme offered by participation in the exercise groups.

There is also an emerging literature suggesting that many very frail elderly people, such as those included in OPERA, may be unable to exercise at levels sufficient to fully participate in exercise activities and exercise intensities that might activate the physiological processes hypothesised to ameliorate depressive symptoms.178,219221

Our overall measure of health-related quality of life was the EQ-5D, self-reported and proxy. A priori we decided to use the proxy values, as we were anticipating a large amount of missing data, in particular for completion of the EuroQol thermometer. Although this was the case, fewer participants satisfactorily completed this at baseline and follow-up and the difference in intracluster correlation coefficients between the self-assessed EQ-5D (0.02) (Table 37) and the proxy EQ-5D (0.17) (Table 38) mean that the precision for the estimate is similar for both proxy and self-completed EQ-5D. Neither shows any significant benefit from the intervention in any of our analyses, with point estimates at, or very close to, zero; effectively ruling out any meaningful benefit on overall health-related quality of life from the OPERA intervention.

Safety

At the time we started OPERA there were significant concerns that we might have been exposing residents to some risk of harm. This was both around the time of the group exercise sessions and at other times when mobility was being encouraged. No directly attributable serious adverse events occurred during over 30,000 individual attendances at the exercise sessions by a very frail elderly population, which is a tribute to the professionalism of the physiotherapy team. Ascertainment of fractures in the population exposed to the intervention is important. We were able to collect medical record data only on study participants and so we needed to rely on pooled data from the care homes to ascertain overall fracture rates among all residents. Because there is a statutory duty to record these they should be a complete record but they do not give us data on person-years of exposure or identify those with multiple fractures at different times. There was, however, no suggestion of a difference here. The use of routine National Health Service statistics to identify fractures has the potential to provide confirmatory information on fracture rates, based on years of exposure. For inpatient events the data are of high quality but the quality of reporting in routine data sets of events in A&E departments and fracture clinics does not allow for definite diagnosis of fractures. Notwithstanding this problem the data clearly do not show any increased risk of injury fracture in the intervention group. Thus the OPERA intervention is very safe to deliver.

Cost-effectiveness

The results show that the exercise intervention made no significant difference in the number of quality-adjusted life-years accrued by residents while increasing overall costs due to the cost of delivery of the intervention. A high cost of implementation (£322 per resident) was expected for such an intensive intervention. This cost was not offset through decreases in health services use during the 12 months of the study. The intervention did not contribute to an increase in health service use. The findings of no significant difference in mean costs between residents in the intervention and control arms were consistent across all types of health-care visits. Consequently, the incremental cost of the intervention from an National Health Service perspective was £374 (95% CI −£655 to £1404) and £366 (95% CI −£664 to £1396) from a societal perspective.

The base-case analysis found the incremental quality-adjusted life-year figure to favour the control arm of the study; the difference was negligible (0.0014) with a wide CI (−0.0728 to 0.0699). The wide CI means that we cannot formally conclude equivalence in quality-adjusted life-years, as the limits of the 95% CI include values where we might have concluded that the intervention was cost-effective. The sensitivity and secondary analyses showed similar results. In light of the small negative effect from the intervention in all analyses, the economic evaluation does not support the use of the OPERA intervention. The low probability of the OPERA intervention being cost-effective at a willingness-to-pay of £20,000 and £30,000 demonstrates fairly conclusively that this is not a cost-effective intervention.

In such an elderly and frail population, it can be difficult to influence EQ-5D scores because there may be limited potential to achieve the relatively large changes in states within a domain (e.g. from confined to bed to some problems walking about). Often interventions in these populations are only able to slow the deterioration in quality of life rather than improve it. Furthermore, quality-adjusted life-years using the EQ-5D may not be an appropriate outcome for those receiving palliative care.222 Twenty-three per cent of the residents included in the economic evaluation died during the 12-month follow-up period, indicating that many residents were effectively receiving palliative care at some point during the study. No better method currently exists to measure health utility for those at the end of life.

If there was a measure of health utility that was more suitable for use in this population, and that was more sensitive to change than the EQ-5D, then it might be easier to demonstrate cost-effectiveness of interventions in care homes. As the incremental cost of delivering the OPERA intervention is £374 per resident per year it would still need to show a net quality-adjusted life-year gain of at least 0.012 to be considered as cost-effective at a willingness to pay of £30,000 per quality-adjusted life-year.

Proxy EQ-5D data was chosen a priori for this analysis as it was more complete and collected more frequently, allowing for more residents to be included in our analysis. Over 25% of the self-reported EQ-5D data were missing at each time point. Some residents were unable to fill out the self-completed surveys because they were too unwell or cognitively impaired. Proxy data, however, has some shortcomings, as demonstrated by studies that compare self-completed quality-of-life surveys with those completed by a proxy.138140 In some homes, these surveys were completed by the care home manager due to language or literacy barriers or time constraints among the care workers working with the residents on a day-to-day basis. Although these care workers may have found it more difficult to complete the survey, they may have had a more comprehensive understanding of what life entailed for the residents. The care home managers, paradoxically, may have had a better understanding of the survey but may have had less interaction with the residents, which might have affected their interpretation of the residents' quality of life, although, particularly in smaller homes, the managers had a great deal of contact with their residents.

Process evaluation of the study

The process evaluation had some limitations. The number and spread of interviews carried out was smaller than planned, particularly with residents in the care homes. Relatives were also difficult to approach as we had no access to their contact information. Here we consider our finding under the different components of our process evaluation.

Context: aspects of the larger social political and economic environment that may influence implementation198

Our quantitative and qualitative data clearly illustrate the frailty of care home residents. A 2011 report from the British Geriatric Society concluded:223

Care homes and the residents they support have changed. They are no longer housing options for frail and financially insecure older people, as might be inferred from reading the National Assistance Act (1948) which set out in Part III the duty of local authorities to provide accommodation. They are now a major component of the welfare system's provision of care for vulnerable and clinically unstable older people. Many are now providing highly specialised services, for instance for older people with dementia.

The frailty of the residents and the high prevalence of cognitive impairment seen (see Chapter 3, Individual characteristics) led some care home managers to be pessimistic about the prospects of residents' participation at the start of the study, and doubtless impacted on the implementation of the OPERA intervention. Nevertheless, the principal message from our process evaluation is that, even in this challenging population, it was possible to recruit participants and to implement a complex, multicomponent intervention which made considerable demands on care home staff and residents.

Providing seamless health-care provision for residents in care homes within the independent sector has been identified as an important problem. For example, the OPERA physiotherapists succeeded in obtaining simple, relatively low-cost mobility aids that care homes had been either unaware of or unable to access.

The OPERA study took place across a period of increasing economic uncertainty and, latterly, in an evolving economic recession. The majority of UK care homes are part of the independent sector and one of the largest groups in the UK, Southern Cross, announced it was closing down just after the end of the study;224 four care homes from the chain took part in OPERA. Even at baseline the mean occupancy was 87% and the recruitment team reported that the number of vacant beds increased during the study. Observation and interviews in the care homes suggested that care home staff often have to work very hard and some resource-stretched care homes may have little spare capacity to engage in cultural shifts that might consume more carer time, such as promoting physical activity among frail elderly residents. Overall in England in 2009 about one-third of adult social care workers had been in their post for < 3 years, and one-third for 3–7 years (see State of the Adult Social Care Workforce 2010, Skills for Care, p. 11210).

One interesting finding from this evaluation is the difference between care homes within the OPERA study. The activity sweeps show patterns of activity that appear to be relatively consistent across time within the homes, but very different between homes, suggesting that there are huge differences across the care home sector. We do not fully understand why the culture around physical activity appears to differ so much between care homes and it seems unlikely that these differences could be completely explained by a difference in the mix of residents. The process evaluation identified some of the very different physical environments provided by different care homes but it is not fully understood how these environmental differences may impact on residents and care staff. The stability of patterns of activity across 12 months within the case study care homes suggests that changing the culture of homes, at least with regard to patterns of activity among residents, might be difficult.

Reach: the proportion of the intended target audience that participates in the intervention198

The OPERA study succeeded in recruiting a representative sample of care homes and the process evaluation suggests that care home managers and staff were enthusiastic about the intervention (even if they did initially express doubts about attendance). The evaluation suggests that it would be possible to introduce the intervention across the care home sector and it would prove popular with most homes.

The cluster randomised design meant that residents within participating homes were exposed to the intervention whether or not they participated in the evaluation of OPERA. Some residents who had neither consented nor been assented into the OPERA evaluation choose to participate in the exercise classes. Unfortunately, because we cannot collect outcome data on these residents we will not be able to measure any potential benefits or disbenefits and in this regard the true magnitude of the effect of the OPERA intervention remains partly unknown.

Dose delivered: the number or amount of intended units of each intervention or each component delivered or provided198

All 35 intervention homes and 42 out of 43 control homes received the depression awareness training session, although it was very difficult to arrange, and required considerable persistence, in a few homes. In one intervention home we were unable to deliver the intervention because the residents were too disabled to participate in the exercise group. The physiotherapists assessed all residents for eligibility and safety before they attended the exercise classes and fed this information back to the care home staff. Assessments also provided the opportunity to consider the aids a resident was using, and where necessary to make adjustments or indeed obtain suitable mobility aids/equipment. Across the intervention homes the physiotherapists delivered on average 90% of the maximum dose of exercise classes (Table 61).

We are not able to quantify the dose delivered of one aspect of this whole-home intervention, ongoing support and advice from the visiting physiotherapist to the care home staff, but it seems likely from the process evaluation that this aspect of the intervention may have differed between the two geographical locations of the study because of the different ways the physiotherapists delivering the OPERA intervention were employed at the two sites. If the intervention were to be rolled out across the care sector it is more likely that the model seen in NEL, where physiotherapists were seconded from the National Health Service to provide the intervention, would be followed. Our evaluation suggests that this model makes this aspect of the intervention more difficult to deliver.

Dose received: the extent to which participants actively engage with and interact with the recommended resources198

Just over one-third of participants attended 51 classes, our predefined estimate of an effective dose, with nearly 10% attending 41–50 classes (Table 63). In particular, attendance was good in those who contributed to our primary analyses. Attendance at the class does not, of course, mean that residents exercised to their set target level. There was little evidence of fall off in the numbers attending exercise groups across time, suggesting that ongoing group-based exercise in care home settings is viable. Only 9% of participating individuals attended no classes. This slightly disappointing attendance rate should be seen in the context of the very frail, sedentary, elderly population in the care homes and attendance rates at other exercise or other self-management interventions in community settings, which commonly have low attendance rates.

Vectors for change

Alongside the components of a process evaluation198 this evaluation was underpinned by the theory of change197 which we outlined in Chapter 2 (see Process evaluation). We identified four vectors through which activity among residents in the homes might be increased: care home staff training, exercise groups, physiotherapist contact with care home staff and improvements in the home environment.

Care home staff training took place and was positively received by those who participated. However, only around 39% of staff in intervention homes in C&W and 54% of staff in intervention homes in NEL actually attended the training. Initial feedback from both the intervention and control training suggests that there was some increase in care home staff's awareness of depression and how to deal with it, and there was some indication that intervention home staff were, with the help of the physiotherapist, more confident in promoting physical activity. However, in the very small proportion who returned a 3-month follow-up evaluation questionnaire, there was little evidence that attendance at the training had changed their practice. In contrast, interview and focus group data strongly suggested that, at least in some cases, the training of care home staff made a real difference to their awareness, ability and practice around promoting increased physical activity in residents. There was less evidence from qualitative sources that the training had influenced the awareness and recognition of depression in practice among care home staff.

The exercise groups were delivered as planned and information from the focus groups and interviews identified many positive effects of the groups and almost universal enthusiasm from the care home staff and residents for the groups, which only appeared to increase across the duration of the study.

Measuring the effect of the physiotherapists' contact with care home staff proved more difficult, and we have relied mainly on data from the observations, interviews and focus groups. Most of the comments about the physiotherapists, especially later in the study, were very positive, making it clear that they had come to be regarded as one of the team and would be sorely missed. However, it was clear that continuity was important. In cases where there were changes of physiotherapist, there were comments about the difficulties this raised. Care home managers and carers also valued the access to professional advice from the physiotherapists about their residents, which they seemed otherwise to find almost impossible to access. A further unexpected benefit of the visits of the physiotherapists was their role as an advisor or guide for care home staff negotiating access to other National Health Service services, including particularly the provision of appropriate mobility aids, but also, for example, providing advice on accessing mental health services.

Measuring changes in the home environment also proved challenging. The activity sweeps that we carried out in the case study care homes provided some insight into the variation between homes in patterns of activity (or inactivity), but failed to provide any evidence of change in activity over the duration of the intervention. Several care homes reported having regular exercise sessions at baseline but there was little evidence of such sessions in either control or intervention homes during the study. Indeed, some homes were seen to struggle to maintain a full activity programme, most commonly due to shortage of staff. Some care homes had activities co-ordinators appointed, but their effectiveness varied. Some were successful in promoting social interaction and activity, but others had many competing demands on their time (e.g. being a carer, errands) and were less effective as activities co-ordinators. Reported staff–resident ratio seems low across the day and in some cases it does seem that there were staff shortages.

There were two models of physiotherapist delivery: one used by the team in C&W and one used by the team in NEL. In C&W, the physiotherapists were employed directly by the University of Warwick to carry out the groups and usually had no other clinical commitments. This resulted in more continuity in provision and may also have meant that physiotherapists were less rushed and more likely to spend time talking to the staff in the care homes. In NEL, there was a contract with the local National Health Service physiotherapy service to deliver the groups; this resulted in less continuity and, because the physiotherapists had many other clinical commitments, may have meant that they were less able to linger to talk to staff in the care homes.

Hypotheses derived from the process evaluation

One very striking finding from all the observational work was the very large variation between homes in culture, daily activity and staffing level (Tables 15 and 16). At the time the process evaluation was being completed and before seeing the final results of the study, we hypothesised that the magnitude of any effect from the OPERA intervention is likely to vary considerably between care homes and this variability will lessen any overall effect size. This is because there is enormous variation between the care homes in the level of engagement of care home staff with the residents, and in the opportunities for activity within the care homes (including the opportunity for trips and activities outside the homes). We also hypothesised that there would be a dose effect, such that the intervention will have a greater effect on those attending a greater number of group exercise sessions. A caveat is that such a dose effect may be hard to detect if it should be found that the absence of depression is a key determinant of whether or not residents were frequent attendees at exercise classes.

Long-term follow-up

The aim of this follow-up was to explore if any beneficial changes in culture within homes brought about by having participated in the OPERA intervention appear to be persistent. It is clear that OPERA was well liked by the care homes and it also seems that it has had a lasting impact. Some care homes have adopted versions of the OPERA intervention and in some cases are still getting input from OPERA-trained physiotherapists (privately). It seems that OPERA has impacted on outcomes far beyond those measured, with reports of ‘happier’ residents and services beyond those normally available in care homes. There is some evidence that staff have benefited from the training and interactions with the OPERA team but little evidence that OPERA has changed practice other than promoting some homes to employ an activities co-ordinator and inspiring some activities co-ordinators to do more.

In terms of the factors that support beneficial change it is hard to be conclusive. Care homes that have adopted OPERA have probably increased ‘activity’ and social interaction in their homes, which should benefit the residents. Staffing and funding are barriers to sustainability of programmes such as this, as homes have to prioritise limited budgets. In light of the ineffectiveness of the OPERA intervention on depressive symptoms, it is unlikely to be prioritised by care homes for funding.

Little change was seen in control homes. Staff who were interviewed were complimentary about the training and interactions with the OPERA team. As with the intervention homes, there were control homes with lots of activities for residents and some with very few.

Ethics

Cluster randomised trials and care home managers as gatekeepers for research in care homes

Cluster randomised trials raise particular ethical considerations due to lack of individual consent from all participants in the clusters for at least some aspects of the study. National and international ethics guidelines allow for waiver of consent in specific circumstances and cluster randomised trials are recognised as one potential justification for not obtaining explicit individual consent from all research participants.225 However, the ethical justification for this must include strong scientific, practical or economic reasons as to why a cluster randomised design is necessary for the research question to be answered. This should include an explanation of the importance of the benefit of the research to the community in which the research takes place (and the individuals in that community), and that the potential harm to individuals within the cluster as a result of the research will be minimal. In addition, researchers should endeavour to seek individual consent whenever possible within the research project, and take steps to ensure appropriate cluster representation mechanisms to protect the interests of the cluster.225 The OPERA intervention was a whole-home intervention and therefore individual consent for the intervention was not possible. Individual consent (or use of a personal consultee) was obtained for data collection at the individual level, although some pooled anonymous data on fracture rates were also collected at a home level without consent. The whole-home intervention, however, included a physiotherapist-led group exercise class that was offered to individual residents. All residents, whether or not they had consented to participate in data collection, were encouraged to attend the exercise classes but a refusal to attend, or a resident's request to leave during the class, was respected in line with normal clinical or home practice for group activities. Thus agreement to participate was obtained but there was no formal research consent process for participation in the intervention. Clinical records were kept for residents attending the exercise classes in line with Good Clinical Practice but these were kept separate from the research data for residents who had not consented to data collection. The complexity of the different levels of consent and arrangements for data confidentiality in OPERA reflects the endeavours of the research team to comply with guidance on cluster randomised trials and illustrates the need for researchers to consider the ethical and practical issues of consent carefully in relation to the specifics of the particular cluster randomised trial being conducted, as recommended by Eldridge et al.98

The care home managers in OPERA had a key role as both cluster guardians, providing consent for randomisation and implementation of the interventions, and gatekeepers to individual residents for recruitment for the assessments. The literature on cluster randomised trials emphasises the role of cluster guardians in protecting the interests of cluster members. Our interview data from care home managers suggest that they took this role very seriously, choosing to participate in OPERA because they considered that the intervention was likely to benefit their residents but also monitoring the researchers and controlling access to residents and relatives. The importance of care home managers and staff in the recruitment of participants to research in care home settings has been noted previously.100 Reasons for excluding access varied and included the care home staff's views on a resident's capacity to consent, a resident's or their NOK's physical frailty, and the value they placed on their privacy. We found similar reasons in OPERA. There is a difficult ethical balance to be struck by care home managers in protecting their residents' interests while not denying them the opportunity to participate in research if they wish. Some home managers in OPERA were surprised when residents made decisions that were different to those they had predicted. On the other hand enthusiasm and a sense of ownership of the study could lead to care home staff putting implicit pressure on residents to take part, either in the intervention or the assessments. The care home staff we interviewed were aware of their position as trusted intermediary between residents and their families, and the research team, reassuring residents about the study but respecting their right to say no.

Assessment of capacity and the process of consent

A major challenge identified by the recruitment team in OPERA was the process of assessing capacity and gaining consent from individual residents for the assessments and access to medical records. The difficulty of obtaining valid consent when a substantial number of the relevant population have cognitive impairment was also identified by key informants in the ethics substudy. Notwithstanding that many OPERA participants had substantial cognitive impairment, 67% of participants were assessed as having capacity and gave consent. A similar mismatch between Mini Mental State Examination scores and capacity to consent was found by Warner et al.101 Cognitive impairment, per se, is clearly not determinative of whether or not a person has capacity to make a specific decision such as consenting or refusing to participate in a research project. This poses serious challenges for recruitment staff, placing on them the burden of assessing capacity, ensuring that potential participants have the opportunity to exercise their autonomous wishes but not accepting agreement where there is no understanding. Assessing capacity in people with cognitive impairment is not straightforward, even when using standardised capacity assessment tools. Kim et al.102 found marked variability in judgements of capacity of people with Alzheimer's disease to consent to research or to appoint a research proxy by experienced psychiatrists, including both between expert and within expert variability. An added difficulty is that capacity may fluctuate, not only over the course of a longitudinal study, which is well recognised, but also from day to day, for example between assessment of capacity and taking of consent. Given the above difficulties it may need to be accepted that judgement of capacity to consent to research in this population may have to be different to those judgements in other populations.

The Mental Capacity Act 2005 specifies retention of relevant information as a component of the capacity test but does not require indefinite retention.105 Usual practice for the research consent process is to provide participants with at least 24 hours between provision of information and obtaining consent to allow the person to consider the information at greater length and to minimise any likelihood of implicit coercion. This raises the question of whether or not the participant must retain the information over this period for them to be assessed as having capacity. There is no consensus on this issue among researchers or the research ethics community. Goodman et al.100 have described conflicting views on retention by care home managers directly affecting their recruitment of residents with dementia into a research study. The experience in OPERA was that retention for 24 hours was often not achieved but that provision of information some time before the consent process was useful as a first step in the process of information consolidation and in the establishment of a relationship between the researcher and resident.

The importance of a relationship of trust between researcher and participant for the success of the consent process was a key finding of the ethics substudy. Older people are more likely to consent to take part in research if they know and trust the person giving them the information about the study. This will be particularly true for residents of care homes who have less independence and may have cognitive impairment, contributing to feelings of vulnerability. The presence of care home staff or relatives to support, reassure and advise residents can be seen as facilitating autonomous decision making, although the risk of implicit coercion must be borne in mind. However, the idea that the trustworthiness and familiarity of the researcher encourages consent, perhaps even when understanding of the research by the participant is limited, requires further exploration. Difficulties with the formality and complexity of research paperwork and a reluctance of older people to sign documents, or preferring a relative to sign on their behalf, raise further issues for researchers in developing an appropriate consent process that respects older people's preferences and experience.

Recruiting participants who lack capacity

Many research participants in care homes lack capacity to consent to take part in the research (33% of participants in OPERA). Prior to the implementation of The Mental Capacity Act 2005,105 the legal position regarding research with adults who lacked capacity to consent was unclear for studies not covered by the Medicines for Human Use (Clinical Trials) Regulations (2004).226 The Mental Capacity Act 2005105 sets out the criteria under which such research may be lawful, which includes a requirement that researchers consult with either a personal or nominated consultee to seek advice on whether or not the person who lacks capacity would have objected to taking part in the research if they were able to consent. OPERA commenced in 2007, shortly after implementation of The Mental Capacity Act 2005,105 when there was still some confusion among researchers and research ethics committees regarding the process of involving personal and nominated consultees for people who lacked capacity to consent to research.106,107 Our recruiting staff found the process of identifying and approaching personal consultees straightforward but very time-consuming, an experience shared by other researchers. The recruitment rate through personal consultees was disappointing (48%), with 23% not responding to letters of invitation. In addition, we were unable to identify anyone willing to be a nominated consultee when no personal consultee was identified. The difficulty in engaging both personal and nominated consultees for conducting research will pose a challenge for future research. A substantial proportion of the relevant population could be excluded from participation in research studies. This could potentially affect generalisability of results, although the nature of the OPERA results is such that this would be unlikely in this case.

The difficulties of identifying appropriate proxy decision-makers for consent to research in the emergency setting has led to the recognition of waiver of consent for this type of research in some research ethics guidelines.103 However, in non-emergency settings the argument of urgency does not hold and the requirements of the relevant legislation (clinical trials regulations or The Mental Capacity Act 2005105) must be fulfilled. In many settings identification of a personal consultee will be straightforward as they are likely to be living with the person or in close contact. Care home residents, however, may not have a readily identifiable person to consult, for example if they have no close family or friends and their designated NOK is not in regular contact. Two UK studies that specified recruitment rates for care home residents through proxies or consultees reported levels of 61% and 41%, respectively.100,108 Recruitment of research participants who lack capacity by using consultees or proxy decision-makers raises a range of legal, ethical and practical issues that may be context dependent and that require further exploration.

The OPERA study met the criteria for waiver of consent in that (1) it had appropriate approval from a multicentre research ethics committee; (2) the intervention was a whole-home intervention, with the exercise classes including people who had cognitive impairment that may have affected their capacity; (3) it was not possible to provide the intervention selectively to residents with capacity and all residents had the potential to benefit without a disproportionate burden of taking part; (4) residents in the control homes, and those taking part in the assessments, were at negligible risk of harm; and (5) the research provided knowledge of the effect of the intervention on depression in residents of care homes, a condition that either affected or had the potential to affect all participants.

Strengths and weaknesses

Particular strengths of this study are that we have recruited a large sample of residents from a representative sample of care homes. This indicates that these findings are generalisable to UK care homes. Although we recruited a smaller proportion of residents than planned (38% vs 50%), the results will be applicable to the overall care home population; as the intervention was ineffective in the more capable and motivated residents who joined the study it is unlikely that it would have any positive effect on those who were too ill to participate. We achieved good follow-up rates. Within the limitations of the outcome measures used we have achieved very precise estimates of the possible effect of the OPERA intervention. However, the study is limited by the sensitivity of the measures used and it might be that other, more sensitive, outcome measures might have shown a benefit, particularly on health-related quality of life.

Any outcome measure measuring depression in a trial focusing on depression in a frail population such as ours should ideally be short, sensitive to change, have face validity, have a relative lack of directly or indirectly somatic items that are likely to be affected by physical health problems, and should have concurrent validity against a gold standard measure (expert clinician judgement). The Geriatric Depression Scale-15 has been shown to be sensitive to change.121 The full Geriatric Depression Scale–30, from which the shorter 15-item version is drawn, has been shown to demonstrate significant change in intervention among a very similar population of Australian care home residents.15 It has been used as a primary outcome measure in trials of interventions for depression among older people.95,124,227229 In Conradsson et al.95 the Geriatric Depression Scale-15 was used as the primary outcome measure in a trial of a reasonably similar intervention (exercise) and setting (residential care homes) and range of cognitive impairment. At the core of depressive disorders are low mood, reduced energy and reduced enjoyment; the Geriatric Depression Scale–15 items have demonstrated ability to distinguish older people with depressive disorder from those without and despite the atypical situation of care home residents, we believe that any intervention with antidepressant effect should be able to show a reduction on Geriatric Depression Scale-15 symptoms.

Potential alternative outcome measures for trials among people with depression include The Montgomery–Åsberg Depression Rating Scale (MADRS),230 the Hamilton Depression Rating Scale (HDRS),231 Center for Epidemiologic Studies Depression Scale (CES-D)232 and the Cornell Scale.233 Importantly, very similar outcomes have been found for the Geriatric Depression Scale-15 against alternative established measures of depression, including MADRS and CES-D,229 and against clinician ICD-10 diagnosis149 of depressive disorder.227

Against other potential measures the Geriatric Depression Scale-15 is significantly shorter than the MADRS,230 the HDRS,231 CES-D232 and the Cornell Scale.149 Of these scales, the Cornell Scale149 includes 3/19 items that are indirectly affected by physical disorder (appetite loss, weight loss and lack of energy), the MADRS230 includes items on lack of sleep and reduced appetite, the HDRS includes – within its 17 symptom areas – three directly somatic items including gastrointestinal, genitourinary and general somatic symptoms, while the CES-D232 includes one item on lack of appetite.

What was perhaps surprising in the OPERA study was the degree of frailty and cognitive impairment among potential participants, which limited the pool of potential participants who were able to complete the primary outcome measure. Nevertheless, with the substantial numbers in the analyses with full data on Geriatric Depression Scale-15, the choice of instrument was appropriate for this population and the findings are robust.

The primary hypothesis for this study was that promoting physical activity might reduce depressive symptoms. Depression rating scales are, however, only one way of measuring low mood. Since we designed this study there has been a developing interest in well-being as a health outcome. The observational work in our process evaluation suggests that although we did not change any of our objective outcomes, the OPERA intervention did effect some changes within the care homes and we had good participation in the group exercise sessions. It may be that if we had measured well-being instead of either depression or health-related quality of life we might have had a positive impact. Although measures are becoming established for the measurement of well-being in the general population, for example the Warwick–Edinburgh Mental Well-being Scale (WEMWBS),234 these are not yet well established in care home residents. The social well-being of nursing home residents (SWON-scale) is a candidate that was developed during the lifetime of the OPERA study and might be a suitable measure.235 Furthermore, we have not measured the effect that our intervention may have had on the care home staff. It was clear from our observations in the homes that the level of interest in the work and training of care home staff was greater than that which they normally experienced. The staff also particularly valued the contact with the OPERA physiotherapists. We have not been able to measure, and account for, these effects in our analyses.

We were unable to collect Short Physical Performance Battery data at the 6-month follow-up data when we changed to collecting 6-month outcomes on all participants rather than just those with depression. This was because much of this was a particularly time-consuming test taking around 15 minutes per participant; this was substantially greater than anticipated at the time we designed the study. With the help of research nurses from the primary care and mental health research networks we were able to collect these data at baseline and 12 months but adding in 6-month data would have increased the workload above capacity at a time when we were at our maximal home recruitment rate. This does mean, however, that we are not able to examine if changes in Short Physical Performance Battery at 6 months mediate any possible changes in depressive symptoms at 12 months.

We have developed a high-quality, theoretically informed intervention that was popular with care homes. We were able to deliver this satisfactorily within homes without any related serious adverse events.

Overall we believe this is a very robust study that has obtained a clear answer to the research question set and has helped to develop our understanding both of how to undertake research in a care home environment and how care is delivered in this environment.

Conclusions

Implications for health care

There is a high prevalence of depressive symptoms in those living in care homes; much of this is unrecognised. There is a clear need for research to reduce the burden of depression in this group, many of whom are extremely frail and have a very limited life expectancy. It is possible to recruit care home residents with substantial cognitive impairment to studies; however, there remain some structural problems for obtaining agreement from those whose NOK cannot be contacted to take part in research. The OPERA intervention, targeted at improving physical fitness and social interaction, could be delivered safely and to a high standard over a prolonged period. Both residents and care home staff valued participation in the study, and also the activities of the physiotherapists in promoting physical activity within the homes. There was less evidence of achieving a cultural shift in the attitude towards physical activity in care homes. In some homes, however, we were able to achieve some sustained changes that were maintained after the end of the study. The OPERA intervention had no positive effect on any of our primary or secondary outcome measures and it was dominated by the control intervention in the health economic analysis. This evidence does not support the use of moderate-intensity functional exercise programmes, such as the OPERA intervention, to reduce depression in care home residents.

Recommendations for future research

  1. A cluster randomised trial of care home staff training in basic psychological/behavioural techniques may include cognitive approaches (positive thinking/installing hope) and behavioural approaches (suggesting positive activities). This could also include case finding followed by referral for advice on optimisation of drug regimen.
  2. Development and evaluation of measures of wellness and health utility that are specific for care home populations, suitable for completion either by residents or by proxies on their behalf.
  3. Observational studies to identify modifiable factors that impact on the wellness of care home residents.
  4. Work to further develop the evidence base of best practice for recruitment of those who lack capacity, in a variety of clinical situations, to take part in clinical trials, informed both by ethical and legal analysis and empirical data from key stakeholders including trial participants or their NOK.
Copyright © Queen's Printer and Controller of HMSO 2013. This work was produced by Underwood et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Included under terms of UK Non-commercial Government License.

Bookshelf ID: NBK260575

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (9.2M)

Other titles in this collection

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...