The a priori beliefs about the data or experimental set-up that are a prerequisite for valid statistical testing and therapeutic-effect estimation.

The average (mean) of the individual treatment effects for all of the clients recruited to a given clinical trial (or from everyone in the population from which one wishes to infer effects of the therapeutic intervention).

The average (mean) of the individual treatment effects for all of the clients recruited to a given clinical trial who actually received the treatment. There is no reason to believe that this average will be the same as the average treatment effect in the untreated population. Clients who turn up and adhere to their therapy, for example, might do so because they correctly believe that it will work. Those who drop out may know that, for them, it is unlikely to be beneficial.

Information such as initial values for subsequent outcome measures or demographic information (e.g. age, gender, etc.) collected and recorded on a client at the time of the recruitment to the trial (ideally just before randomisation). The data are frequently collected because they have prognostic value and can be incorporated into an analysis in order to increase the precision of the therapeutic effect estimates. They may also be useful to examine which type of client might benefit most from the therapy (see Moderation) or non-compliance with allocated therapy, and so on. Common examples include the client’s age, gender, education, family circumstances and clinical history, and the baseline values of variables that are going to be used to measure the outcome of therapy (severity of depression and other symptoms).

The lack of independence of participants’ outcomes. This between-participant correlation is induced either by the method of treatment allocation (such as in a cluster randomised trial) or by the method of delivery of the therapy (individual therapists being responsible for the treatment of several clients; therapy being delivered in groups).

The average treatment effect in those clients (in a given trial) who comply with their actual treatment allocation and would have complied had they been allocated to any of the other treatment conditions. The intention-to-treat estimate is an unbiased estimate of the effect of treatment allocation (effectiveness) but is typically a biased (attenuated) estimate of the effect of actually receiving treatment (efficacy). This attenuation is increased as the amount of non-compliance increases. One way of adjusting for non-compliance in a simple two-arm trial (therapy vs. control) is to divide the intention-to-treat estimate of the effect of allocation on outcome by the corresponding intention-to-treat effect of allocation on the receipt of treatment (i.e. the difference between the proportions receiving therapy in the two randomised groups).

The ability to attribute an apparent effect of an intervention (e.g. therapy) to an omitted common cause. When a client decides to seek counselling or psychotherapy the decision may be in some way related to treatment-free prognosis. A general practitioner may refer a depressed patient to cognitive–behavioural therapy because he or she appears to have particular chronic problems and are not responding to antidepressants. Clients receiving counselling or psychotherapy might or might not have had a worse treatment-free outcome. If we wish to estimate and/or test the causal effect of therapy (treatment effect), then we need to eliminate confounding. It is possible that we can measure some of these omitted common causes and allow for them in our statistical analyses, but the only sure way of being confident that their effects have been eliminated is to randomise treatment allocation, as done in a randomised controlled trial. The great advantage of randomisation is that it ensures (on average, at least) that we have accounted for all confounders (omitted causes) even if we have not measured them or are not even aware of their existence.

The relation between two variables and whether or not a change in one is the reason for a change in the other. If we observe a correlation then something must have caused it. The big problem is to decide what. If we observe, for example, that the strength of the therapeutic alliance during therapy is positively correlated with improvement in clinical outcome, it is very tempting to infer that it is the good alliance that has led to the better outcomes. However, it is not as simple as that. It could be that very early improvements in clinical outcome enabled the client to develop a strong therapeutic relationship, that is the causal influence is actually the other way round. The timing of the measurements may go some way to solve this problem (temporal precedence is important for inferences concerning causality) but very rarely in a fully satisfactory way. The third possibility (and it may even be a combination of all three) is that there is confounding. Clients with a good treatment-free prognosis (their clinical improvement will be relatively better even if they do not receive therapy) may be the ones who are able to develop a strong therapeutic alliance. For the same reason, it is likely that they will have a better outcome under treatment. The common procedure of taking a cohort of treated clients (either a case series or those in the therapy arm of a clinical trial), simply correlating the outcome of therapy with the strength of the therapeutic alliance and then inferring that there is a causal influence of alliance on outcome is fundamentally flawed. This procedure, in particular, cannot distinguish the phenomenon we are looking for (the relationship of the alliance with a treatment effect) from that resulting from an omitted common cause (therapy-free prognosis).

The way in which a treatment effect changes with the dose of treatment, for example the concentration of drug in the blood. In a psychotherapy trial, we may be interested in the effect of the number of sessions attended or perhaps the overall duration of therapy. The different doses of drugs or psychotherapies may be determined by the random allocation procedure or may be simply a reflection of the level of patience adherence to the prescribed medication or psychotherapeutic intervention. In the latter case (the level of patience adherence to the prescribed medication or psychotherapeutic intervention), the estimation of a dose–response relationship is by no means straightforward (it may be subject to confounding). Dose–response relationships in a psychotherapy trial are much less straightforward. It is not always clear how dose of therapy might be measured; it may be a function of the prescribed (allocated) number of sessions, the number of therapy sessions actually attended, or the quality or intensity of the therapeutic process (quality of the therapeutic alliance or fidelity of the sessions to a given therapeutic model). This is an extremely challenging area; none of these characteristics can be measured without error and their effects on outcome are almost certainly subject to hidden confounding. It is usually best to assume that virtually all simple, naive analyses will be invalid and the corresponding conclusions unfounded.

The effect of being offered (allocated to) treatment.

The effect of receiving treatment.

Usually a secondary analysis (in addition to the primary intention-to-treat analyses, not instead of them) in which one tries to explain how a given therapeutic effect has been achieved or, alternatively, why the therapy is apparently ineffective. Such analyses may aim to evaluate treatment effect by client characteristic interactions (such as a subgroup analysis or treatment effect moderation), estimation of efficacy and dose–response relationships (particularly the effects of non-compliance), and the intervening effects of process measures such as the therapeutic alliance and treatment fidelity. It will frequently also involve the evaluation of the role of putative mediators. All these areas are fraught with difficulties. Subgroup analyses might easily find differences arising by chance (particularly if they are the result of a so-called ‘fishing expedition’). Process variables (including mediators) are subject to considerable measurement error and their effects on outcome subject to hidden confounding. It is usually best to assume that virtually all simple, naive analyses will be invalid and the corresponding conclusions unfounded.

Therapies that are delivered to groups of clients rather than to individuals. Outcomes for clients within these groups are likely to be correlated (i.e. not statistically independent). The behaviour or outcome of one client may influence that of another in the group, particularly if the group contains unco-operative or disruptive individuals), so violating one of the assumptions underlying the use of the simpler traditional significance tests and estimation procedures. Technically, the problems may be a little more difficult than in a traditional cluster-randomised trial because we may be comparing a group therapy (clustered) with treatment as usual (not clustered) or even a therapy (potentially the same one) delivered to individuals (again, not clustered).

Confounding in which we have not measured all of the relevant confounders (or even been aware of their existence) and therefore cannot allow for them in our statistical analyses.

The comparison of a particular individual’s outcome after receiving therapy with the outcome for that same individual had he or she received the control condition. Without a comparison a treatment effect is not defined. Of course, an individual treatment effect can never be observed or measured. Our only hope is to use statistical methods to estimate what it might be in groups of similar individuals.

Analysis of participants as randomised, that is an analysis that estimates and tests the effect of the treatment assignment as determined by random allocation and not by the effect of the treatment that was actually received. It estimates the effect of the offer of treatment (effectiveness) as opposed to the receipt of treatment (efficacy). This is the gold standard for the analysis of the results of clinical trials, primarily because it answers the pragmatic question: is there convincing evidence that it is worth offering this intervention in clinical practice? It is also an estimation procedure that is dependent upon the fewest unverifiable assumptions (concerning lack of confounding), is open to fewest abuses during the analysis phase of trials and, accordingly, is the one preferred by regulatory authorities. In practice, it is not always straightforward to implement, as there will almost always be trial participants in whom outcome data cannot be obtained, but the aim should be that the analysed population is as close as possible to the intention-to-treat population.

An intermediate outcome of therapy (a process variable) that, in turn, leads to changes in the clinical outcome. A cognitive therapist, for example, may have a strong a priori conceptual model that states that if the therapy is able to change a client’s beliefs (attributions) concerning his or her depression, for example, then changes in these beliefs will lead to improvements in the client’s symptoms of depression. Similarly, changes in levels of worry may lead to lower levels of psychotic symptoms in patients suffering from delusions. In these two examples, the putative mediators are attributions and levels of worry, respectively. The effect of the therapy on the clinical outcomes (depression or severity of delusions, respectively) is said to be mediated by the effects of the more immediate outcomes of the intervention (attributions or worry). A third example arises in a situation in which mediation, itself, is not the prime interest. In randomised trials of cognitive therapy for depression, for example, we might observe that it is likely that clients will adhere better to their antidepressant medication (such medication is not usually prohibited by the trials protocol, particularly in a pragmatic trial) or even ask to be prescribed medication by their doctor. The obvious question that then follows is ‘How much of the therapeutic effect of the cognitive therapy is explained by the changes in medication?’. Is there a direct effect of therapy on outcome (depression) that is not explained by the increased levels of medication? Despite the vast literature on mediation in psychology, there is very little valid evidence concerning meditational mechanisms in psychotherapy (or elsewhere, for that matter) because naive investigators consistently fail to distinguish inferences concerning association (correlation) from implications concerning causality. The putative mediator and the final clinical outcome are both therapeutic outcomes that are not under the control of the investigator. It is very likely that there are (hidden) common causes other than the therapy itself and therefore the effect of mediator on clinical outcome will be subject to (hidden) confounding. Standard potentially flawed methods of analysis of mediation are based on the almost universally unstated (and unappreciated) assumption that the effects of the supposed mediator are not subject to hidden confounding.

Modification of a treatment effect by a stable patient characteristic measured prior to treatment allocation (or, if not, a characteristic that is convincingly not influenced by treatment allocation), that is, a source of treatment effect heterogeneity. What works for whom? It is usually demonstrated by a statistically significant treatment by patient characteristic interaction. Such a finding is much more convincing; it is one of very few such moderating effects specified in a prior data analysis plan. Post-hoc searches for moderators are notoriously unreliable and very rarely reveal anything other than the play of chance. What variables might be important moderators? Candidates include a history of child abuse, length of untreated illness, chronicity of the illness, and so on. Moderation is the foundation of so-called stratified medicine (personalised therapy) in which a patient’s or client’s characteristics predict his or her optimal therapy. Some patients might respond well to pharmacotherapy, for example, and others to a psychological intervention or counselling. Biological psychiatrists are convinced that the most important moderators (stratifying factors) will be genetic/genomic characteristics but, so far, there is little valid evidence to convince us that this is the case.

The situation in which a client receives a therapy or other form of intervention not exactly as intended by the trial’s allocation procedure. She or he may not turn up for the allocated therapy, for example, or may turn up for one or two sessions and then drop out of the trial. There should not necessarily be any inference concerning ‘delinquency’ on behalf of the client; the decision to drop or to switch treatments may be made by the client’s clinician in the interests of the client’s safety or health (she or he may be prescribed some sort of rescue medication, for example, or admitted to an inpatient unit if she or he has become suicidal or is a danger to others). Accordingly, ‘adherence’ is thought by many to be better than ‘compliance’ (it is a more ‘politically correct’ description). Consider a trial to compare inpatient admission and the use of outpatient care in patients who are newly diagnosed with severe psychotic symptoms. Many of the patients will receive the care to which they have been allocated. Several allocated to inpatient admission may never actually be admitted to hospital (because there are not enough available beds) and several of those allocated to outpatient day care may, in the end, have to be admitted because they have become severely disturbed and have become a danger to either themselves or others.

An analysis of a trial’s outcomes restricted to those participants who have been seen to adhere to the original protocol (received the intervention to which they were allocated). Frequently, randomised participants do not receive the treatment or therapy that they were allocated to (see Non-compliance). Those who do receive their allocated intervention have adhered to the trial’s protocol (i.e. per protocol). The results are potentially biased, as lack of adherence does not occur simply by chance and therefore this is very likely to be confounding.

The outcomes of all potential courses of action. Consider a depressed patient deciding whether or not to seek counselling or psychotherapy. Prior to the decision and seeking help (or not) there are two potential courses of action and associated outcomes. She or he can seek help (therapy), and the severity of her or his depression (e.g. Beck Depression Inventory score) can be measured after a given follow-up period (let us label this ‘Beck Depression Inventory after therapy’). The other potential course of action is not to seek therapy (let us call it the control condition) and, again, the severity of the patient’s depression can be measured after the same period of follow-up (‘Beck Depression Inventory after control’). Once a decision has been made (e.g. the patient receives therapy) then only one of these outcomes is observed (Beck Depression Inventory after therapy) and the other one becomes a counterfactual (what might have been). Although it is impossible to observe both of these potential outcomes for any individual, they do provide us with a very powerful way of defining individual treatment effects and associated average treatment effects.

A biological measurement made before treatment to identify which patient is likely or unlikely to benefit from a particular treatment.

A biological measurement made before treatment to indicate long-term outcome for patients either untreated or receiving standard treatment.

The phenomenon whereby clients, particularly when they have been recruited into a trial at a time of crisis, will improve even without any intervention (i.e. drift back to their own typical state). For this reason it is unwise to assess a therapeutic intervention by simply observing a cohort (case series) of clients who have all received the same treatment. The data from such a study cannot distinguish regression to the mean from an effect of the therapeutic intervention.

Tests for statistical interactions that involve a comparison of the average treatment effect in, say, one subgroup of participants, with the average treatment effect in another. They are used in subgroup analysis and in the evaluation of moderation of treatment effects.

The identification and development of treatments that are effective for particular groups or subgroups of patients with distinct mechanisms of disease, or particular responses to treatments. It aims to ensure that the right patient gets the right treatment at the right time. The key underlying concept is treatment effect heterogeneity, and the search for patient characteristics (predictive markers identified through a statistical interaction) that will explain this heterogeneity and will be useful in subsequent treatment choice. Stratified medicine is also commonly referred to as personalised therapy, personalised medicine, predictive medicine, genomic medicine and pharmacogenomics, and, more recently, precision medicine.

Statistical models used to evaluate whether or not theoretical models are plausible when compared with observed data. Structural equation models are very general, and include common methods such as confirmatory factor analysis, path analysis and latent growth modelling.

A measure of the strength of the working partnership between therapist and client. It is claimed to be a vital ingredient of all successful therapy, irrespective of the theoretical underpinnings of any given therapeutic approach.

Differences in the ability of therapists to successfully treat clients all other things being equal. This might be (and is) of interest in its own right but it will also have possible implications for the design and analysis of a trial in which they deliver therapy.

The comparison between the average outcomes under one treatment condition and the average outcomes under another, conditional on the belief that allocation of participants to the alternative treatment conditions is not confounded, that is, that treatment allocation is essentially random. Comparison of the outcomes under different conditions is the vital ingredient; a treatment effect cannot be estimated validly by simply observing the outcome of a single cohort (case series) of clients receiving a given therapy.

Variation in treatment effects from one individual to another or from one group of similar individuals to another. The treatment effect may differ between those who choose to seek treatment and those who do not, or between participants who comply with their allocated treatment in a randomised trial and those who do not. This is the basis for the increasingly important area (in cancer treatment development, for example, but equally in psychotherapy and counselling) of personalised or stratified medicine, that is investigating what treatments work for whom.

The extent to which therapy, as delivered, is the same as that prescribed, for example in the treatment manual. How closely, for example, does therapy conform to the procedures specified by a given form of cognitive–behavioural therapy? It is frequently assessed within a therapy trial by audio- or video-taping a sample of therapy sessions.

In a trial in which there has been treatment switching or non-adherence to the allocated treatments, analysis of the results in terms of the treatment(s) actually received rather than those to which the client was allocated. The method is potentially flawed (subject to confounding) and the results potentially biased. It is sometimes referred to as an as-treated analysis.