Characterizing Interventions (Key Question 1)

Multidisciplinary postacute rehabilitation programs for impairments from moderate to severe TBI varied widely in terms of populations targeted, setting, program intensity and duration, and timing of intervention. Clear categorization of all studied interventions was not possible. However, programs based upon the comprehensive holistic day treatment model of care are the most frequently studied. These programs maintained a similar approach and mode of delivery. Individuals were enrolled in and progressed through these structured intensive day-treatment programs in small cohort groups, receiving several hours of treatments per day, several days per week. Treatment was delivered largely through group sessions, while maintaining an emphasis on addressing individual needs. Areas of focus included self-awareness of impairments and compensatory approaches to retraining, with vocational rehabilitation as a key component.

Effectiveness and Comparative Effectiveness (Key Question 2)

Our review, like others, found the currently available evidence insufficient to draw conclusions about the effectiveness or ineffectiveness of multidisciplinary postacute rehabilitation for moderate to severe TBI. While we found stronger evidence on the comparative effectiveness of different approaches to multidisciplinary postacute rehabilitation, we identified few well-designed studies that addressed comparative effectiveness and we were unable to find robust evidence for the superiority of any one approach over another. Table 17 lists summary results for comparative effectiveness. Comparative effectiveness research on complex conditions and interventions lends itself to conclusions about specific populations and interventions:

Table 17

Summary and strength of evidence of effectiveness and comparative effectiveness of multidisciplinary postacute rehabilitation for TBI.

We found that gainful employment or return to military fitness did not differ significantly at 1-year post-treatment between groups enrolled in a 6-week inpatient hospital treatment versus an 8-week limited home-based treatment (low SOE). Participants were active duty military patients with closed head injuries experiencing relatively mild impairment levels and treated within 3 months of injury.

We found that productivity did not differ significantly at 1-year post-treatment between closed head injury groups enrolled in functional-experiential versus cognitive didactic inpatient rehabilitation programs (low SOE). Both programs lasted an average of just over 1 month and were delivered in VA rehabilitation facilities. Participants began treatment within 6 months of injury.

We found that rates of return to community-based employment were higher immediately post-treatment among the group of TBI survivors with predominantly chronic impairments enrolled in the ICRP versus the group enrolled in standard rehabilitation (low SOE). These individuals were treated in civilian outpatient rehabilitation hospitals and judged to need 16 weeks of intensive treatment. The ICRP group did not achieve higher rates of community integration (low SOE).

We found that rates of return to community-based employment between these two groups equalized by 6-month post-treatment (rates in the standard rehabilitation group caught up with those of the ICRP group) (low SOE).

Effectiveness and comparative effectiveness conclusions of this review are highly specific to the populations and settings addressed by individual studies. On the face, various competing treatments appeared to produce similar effects, demonstrating no statistical differences between treatment groups 1 year after completion of multidisciplinary rehabilitation programs.

Two studies demonstrated equivalent participation results in comparison groups with regard to productivity; however, these equivalent results may be an embodiment of the context in which these studies were conducted. For instance, Salazar, et al. enrolled patients whose functional status was high enough to allow for randomization to home care.⁷² Thus, the fact that this group experienced similar improvements to those randomized to inpatient rehabilitation may be specific to their low level of impairment. Indeed, the authors’ post hoc subgroup analysis of those with more serious injuries found greater improvements from inpatient rehabilitation. A similar situation occurred in the Vanderploeg study, in which certain patient subgroups fared better with one rehabilitation approach versus the other as detected in post hoc analysis.⁷¹ Similar findings relevant to a specific subgroup are evident with regard to the CIQ.⁶¹ The prospective cohort study delivered the ICRP to a more chronically impaired group and achieved a greater rate of clinically significant improvement, suggesting that this approach might be better suited to these individuals. Although these programs achieved equivalent outcomes, the studies also indicated that perhaps different patient subgroups respond better to certain types of treatments.⁶¹ In certain studies, the timing of outcome measurement was important. For example, when Cicerone et al. measured participation outcomes at earlier timepoints, results suggested greater improvements for the groups involved in a comprehensive holistic program compared to a traditional program.⁷⁰ This distinction could appear irrelevant since outcomes equalized within 6 months post-treatment in the single study that collected followup data.⁷⁰ However, given the financial and social impact of TBI on survivors and their families, earlier participation outcomes may be important to patients and families.

Minimum Clinically Important Differences (Key Question 3)

We identified no evidence establishing minimum clinically important differences (MCIDs) for the MPAI. In their pilot study of the ICRP, Cicerone and colleagues derived a “reliable change index” of 4.2 of the total CIQ score to evaluate the incidence of clinically significant changes in community integration. The authors described the reliable change index as indicating whether individuals made positive change, no change, or negative change in community integration in a previous sample of TBI survivors, essentially an MCID concept.⁶¹ However, the later RCT evaluating the ICRP did not mention a reliable change index or any attempts to determine the incidence of clinically significant changes, nor did it explain the omission.⁷⁰

Maintenance of Outcomes (Key Question 4)

Very few eligible studies conducted followup assessments to determine maintenance of rehabilitation gains. The two studies that evaluated followup outcomes yielded highly specific conclusions:

We found a low strength of evidence that improvements in return to community-based employment and CIQ scores were sustained at 6 months post-treatment.⁷⁰

We found a low strength of evidence that rates of participation in competitive work achieved at 6 months post-treatment appear to be sustained at 12 months post-treatment.⁷³

Adverse Events (Key Question 5)

The single study that mentioned adverse events does not appear to have assessed them in a systematic manner, reporting that no adverse events were observed.⁷¹

Comparison With Previous Systematic Reviews

Our review found the currently available evidence on the comparative effectiveness of multidisciplinary postacute rehabilitation for moderate to severe TBI in adults limited, as other reviews have suggested.^{38, 39} Conclusions from these reviews report insufficient or low levels of evidence about multidisciplinary rehabilitation programs for moderate to severe TBI. However, these conclusions are inconsistent with those of some previous systematic reviews that suggested more robust evidence of effectiveness.^{14, 37, 40, 42, 85} However, these contrasting reviews differed from ours methodologically in important ways, such as by addressing research on the ABI population (which may include studies that enrolled primarily stroke patients), and by applying more lenient inclusion criteria with respect to study design or less rigorous assessments of SOE. The reviews conducted by groups specializing in systematic reviews apply a more rigorous level of scrutiny to the evidence base than has been previously applied to the literature on this topic. More rigorous scrutiny of the evidence tends to result in more conservative assessments about effectiveness.

Strength of Evidence

In many ways, the results of this review are unsatisfactory. Problems with synthesizing evidence arise from the complexity of sustained TBI impairments and the interventions to rehabilitate them. This complexity makes it challenging to achieve SOE assessments higher than low. Systematic review methodology requires the assessment of SOE at the outcome level. The specificity of the comparisons for this topic means that often, single studies were the basis for drawing conclusions and assessing SOE. Several factors impede high SOE assessments on complex interventions. First, heterogeneity among populations, interventions, and outcomes makes pooling of data impossible. Further, inconsistency in selection of outcomes as well as timing and method of outcome measurement complicates the ability to group studies for grading and interpretation. In addition to the limited number of studies within a comparison, formidable obstacles to obtaining a SOE on this topic include small sample sizes, and the difficultly in achieving a “low risk of bias” for individual studies evaluating complex interventions.

Risk of Bias

Risk of bias presented a major challenge in drawing conclusions about effectiveness. In order to earn an overall low risk of bias assessment, a body of evidence should include several well designed studies, RCTs and prospective cohort studies, of sufficient sample sizes that study similar interventions and controls in similar populations with consistent patterns across consistent outcomes measures. Further, the individual studies must have a low risk of bias. Risk of bias increases when treatment and control groups are not comparable; participants, providers, and outcomes assessors are not blinded; interventions are not well defined or implemented; outcomes measures do not have strong psychometric properties, appropriate statistical analysis is not conducted; confounding variables are not controlled for; estimates are not adjusted for multiple comparisons; and for indications of possible reporting bias.

For this topic, blinding may be the greatest hurdle. Double blinding is typically impossible in rehabilitation research, but outcome evaluators can and should be blinded. Risk of bias is higher without adequate blinding of participants, providers, and outcomes assessors. This risk is especially heightened when intervention outcomes are assessed via subjective self-report measures, which are common in rehabilitation research.

The aforementioned inadequacy of intervention definitions detracts from the internal validity of these studies. Further, the inadequate treatment definitions were often accompanied by a lack of information about measures to insure effective implementation. We looked for reports of staff training, references to treatment manuals documenting treatment components and/or algorithms, and fidelity checks assessing whether interventions were effectively implemented. The studies we reviewed rarely addressed these issues. Lastly, several outcome-related issues contribute to the higher risk of bias for individual studies on this topic.

The primary outcomes we selected appeared to have acceptable psychometric properties, but often failed to identify MCIDs. Additionally, many studies tested the effect of their interventions on many different outcome scales. While some studies identified their primary outcomes, very few adjusted estimates for multiple comparisons or provided justification for not doing so. Failure to use a Bonferroni correction or other appropriate adjustment technique when multiple comparisons are made can result in accepting statistically significant results when they occurred by chance.

Study design also affects risk of bias during SOE assessment. We recognize a difficult paradox with regard to studying postacute multidisciplinary rehabilitation for moderate to severe TBI. That is, the complexity of the topic adds significant challenge to the design, conduct, and expense of RCTs (compared to pharmaceutical intervention studies), and the resources and incentives (i.e. Federal Drug Administration approval) for conducting these trials is not well established. Yet, given the potential for selection bias and the high number of confounding and effect-modifying variables, RCTs are a superior methodology for studying the impact of these interventions. The cohort studies we reviewed typically failed to adequately select controls and/or adjust for differences between groups.

Applicability

The studies evaluated for this review may be applicable to the specific populations targeted by the examined interventions (e.g. military populations, those with significant disabilities, without other psychiatric diagnoses, chronically impaired, etc.) and the time periods in which they were studied. Even then, many of the interventions and control conditions seemed to be embodiments of their local rehabilitation systems, making replicability in other contexts challenging. This is especially evident in studies in military and VA health systems, in which rehabilitation may differ markedly from that available in civilian facilities. Because rehabilitation for TBI is a rapidly evolving field, studies conducted in the 1990s may not be applicable to the conditions in which rehabilitation is conducted today. Additionally, most studies excluded individuals with substance abuse or psychiatric diagnoses, both of which are common in the TBI population.⁸⁶ Inconsistent insurance coverage for rehabilitation services¹⁰ may limit applicability of these results. Moreover, TBI disproportionately affects males, those aged 15–24, and those with lower socioeconomic status,¹¹ groups known to have lower rates of health insurance. Knowledge of which treatments are most effective is less likely to benefit those who lack insurance coverage to receive the services.

Selected Primary Outcomes

The outcomes selected for this review reflect current views on the importance of participation as an outcome of rehabilitation. However, given the complexity of this condition, arguments can be made for the importance of other outcomes despite small changes in participation measures. Some rehabilitation programs may have specific goals related to maintaining function or preventing deterioration of functional status. To maintain or prevent deterioration in participation outcomes may also be important goals of rehabilitation. Cicerone et al. re-analyzed data from previous studies and found that preventing deterioration in these outcomes may have substantial impact.²⁰ Other patient-centered outcomes such as reduced burden of care or need for supervision may be meaningful without changes in participation measures. Other reviews have considered a wider array of outcomes than those selected here. The recent IOM review considered the outcomes of cognitive functioning, quality of life, and functional status, and reached conclusions similar to ours, and concluded that the evidence on multimodal cognitive rehabilitation was not informative.³⁸

Clinical Implications

Our inability to draw broader and more meaningful conclusions is of limited value to providers and payers seeking to identify the best possible care for those experiencing impairments from moderate to severe TBI. Ultimately, the available evidence provided little information about the overall effectiveness or comparative effectiveness of postacute multidisciplinary rehabilitation for adults with for moderate to severe TBI. However, our failure to draw broad conclusions must not be misunderstood to be evidence of ineffectiveness. This topic, like many other complex topics, merely lacks high quality conclusive evidence of effectiveness or ineffectiveness from rigorously conducted systematic reviews. This type of evidence is a high bar currently met by only a small portion of medical interventions (and an even smaller portion of rehabilitation interventions). The limited evidence on this topic stems from the complexity of the condition and treatments resulting in limited available research, and from limitations within that research to answer salient research questions about what works for which patients. In light of the attention dedicated to this topic as demonstrated by the number of recent reviews and media stories, future research to better establish the evidence base for rehabilitation interventions for the TBI population is of utmost importance.