Overview of acceptability evidence by study interventions

Geraldine Macdonald; Nuala Livingstone; Jennifer Hanratty; Claire McCartan; Richard Cotmore; Maria Cary; Danya Glaser; Sarah Byford; Nicky J Welton; Tania Bosqui; Lucy Bowes; Suzanne Audrey; Gill Mezey; Helen L Fisher; Wendy Riches; Rachel Churchill

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Macdonald G, Livingstone N, Hanratty J, et al. The effectiveness, acceptability and cost-effectiveness of psychosocial interventions for maltreated children and adolescents: an evidence synthesis. Southampton (UK): NIHR Journals Library; 2016 Sep. (Health Technology Assessment, No. 20.69.)

Cover of The effectiveness, acceptability and cost-effectiveness of psychosocial interventions for maltreated children and adolescents: an evidence synthesis

The effectiveness, acceptability and cost-effectiveness of psychosocial interventions for maltreated children and adolescents: an evidence synthesis.

Show details

Contents

< Prev Next >

Appendix 14Overview of acceptability evidence by study interventions

Cognitive–behavioural interventions

Study	Sample size	Data collection	Analysis	Quality overview
Barker 2005⁴⁸¹	14/67 children; 5 staff	Interviews	Summary overview	Aims of the research were clearly stated Methodology and research design were appropriate for qualitative data Data collection was not adequately described The findings not clearly described
Buschbacher 2002⁶²⁹	n = 1	Interviews with foster parent and staff members		Aims clearly described but this was a testimonial rather than a research study One of the treatment team conducted interviews with foster mother and colleagues Retrospective interview in one successful case High risk of bias
Chasson 2008⁴⁸²	n = 99	Withdrawal metrics	Multiple regression analysis	Aims were clearly stated and the methodology and research design were appropriate Data collection was adequately described Potential confounders not added to statistical model Findings were clearly described
Chasson 2013⁶³⁰	n = 134 (99 from Chasson 2008)	Withdrawal metrics	Regression analysis of trauma characteristics as predictors of attrition	Aims were clearly stated and the methodology and research design were appropriate Data collection was adequately described Potential confounders were not added to statistical model Findings clearly described
Eslinger 2014⁶³¹	n = 115	Baseline, post treatment and 3 months	Multinomial logistic regression of predictors of dropout	Study design was appropriate Sampling methods were not sufficiently clear; age range was quite wide and sample heterogeneous Data analysis was appropriate
Fraynt 2014⁶³²	n = 562	Factors predicting treatment engagement	Binominal regression; multinomial logistic regression	Study design and sampling was appropriate; however, the study did not measure the type of intervention received other than individual or group treatment, which limited the validity of the study Data collection methods and measures were appropriate for the population and the analysis was adequate for the aim of the study
Hubel 2014⁴⁹²	n = 99; n = 67 analysed	Child and parent client evaluation form		Study design was appropriate but the acceptability questionnaire lacks validity, as it relied on satisfaction ratings Findings were clearly reported
Kolko 1996¹⁰⁸	n = 25 CBT, n = 18 FT, n = 12 control	Treatment expectancy (Likert scale) Brief telephone calls to parents Post-treatment CEI		Study design appropriate and procedure were clearly described Standardised measures were used with evidence of reliability and validity Findings were reported clearly
Lange 2010⁴⁹⁷	n = 24	Client satisfaction: general, with treatment and with therapists Interviews plus 10-point rating scale (1 = high) Attrition		Study design appropriate and procedure were clearly described Sampling through the media open to bias Standardised measures were used with evidence of reliability and validity; however, acceptability was measured using subjective satisfaction ratings Findings were clearly presented
McPherson 2012⁶³³	n = 254	Treatment completion	Univariate analysis and multivariate logistic regression models	Appropriate study design and sampling Data collection based on electronic medical files which relies on accuracy of record-keeping Data analysis was appropriate and clearly described
Salloum 2014⁵⁰⁴	n = 6	Parents completed expectancy rating form; client satisfaction questionnaire plus treatment costs calculated with the time tracking system		Study design was appropriate Acceptability measure relied on satisfaction ratings, although used a questionnaire that used in other research; however, no ‘depth’ of data was captured by this measure, such as why caregivers were or were not satisfied with treatment Findings were clearly reported
San Diego 2011⁶⁸⁰	5/6 of those referred	Participants’ journals, therapy notes, and interviews	IPA	Study was well designed, and appropriate data collection methods and analysis were used Findings were clearly reported Ethical considerations were not clearly outlined
Silovsky 2007⁵⁰⁵	n = 85	CSBP preschool group satisfaction and social validity questionnaire – caregiver satisfaction		Study design was appropriate but measure of acceptability relies on satisfaction ratings Findings were clearly reported
Smith 2008⁵⁰⁶	n = 6	Client satisfaction questionnaire		Study design was appropriate but measure of acceptability relies on satisfaction ratings Findings only partially reported descriptively

: CSBP, Children with Sexual Behavior Problems; IPA, interpretative phenomenological analysis.

Acceptability of attachment-based interventions

Study	Sample size	Data collection	Analysis method	Quality overview
Cross 2013⁶³⁴	n = 1085; 60% retained	Case file data	Single predictors of study retention – logistic regression Multivariate predictors of study retention – multivariate regression remodelling	One site was excluded because it did not comply with study protocol Programme content and delivery varied across the 15 sites
Osofsky 2007⁵¹⁵	n = 346 children; n = 57 dyads	Caregivers’ and therapists’ qualitative impressions of treatment		Study design suffered from differing procedures across sites Some measures used were standardised and had evidence of reliability and validity, although impressions of treatment was measured using satisfaction ratings of ‘yes’ or ‘no’ Findings were clearly presented but lacked depth
Sudbery 2010⁶⁶³	Experiences reports of n = 45 children	Interviews; content analysis of files and organisational documents; children; surveys of social workers and others		Study was adequately designed and described and the findings were clearly reported
Powell 2010⁶⁶²	n = 5 children, n = 4 mothers and 1 grandmother	Semistructured qualitative interviews		Methodology and research design were appropriate for the research aims Data were reported clearly and thoroughly It is unclear whether or not ethical issues were considered

Acceptability of parent–child interaction therapy

Study	Sample size	Data collection	Analysis method	Quality overview
Timmer 2004⁶⁵⁶	102 kin; 157 non-kin foster carers	Treatment completion		The study design was limited by having no control group Patterns of termination of treatment were measured by clinician discharge sheets Results were reported in full and the analysis is appropriate

Acceptability of parent training interventions

Study	Sample size	Data collection	Quality overview
Ducharme 2000⁵¹⁰	n = 28 parents n = 15 children from n = 9 families	Eight mothers completed the satisfaction questionnaire	The study design was limited by not having a control group Treatment satisfaction was measured using mother’s self-report rating scales It is unclear how perceptions of treatment was measured and analysed Results were limited by not reporting variance
Golding 2004⁵¹¹	n = 44 children, n = 41 foster carers	Participant satisfaction questionnaire, qualitative evaluation, group facilitator feedback	Study design and methodology were appropriate Findings were clearly reported
Taban 2001⁶³⁵	n = 45 families health training, n = 37 home accident prevention; n = 31 PCIT	Three social validity questionnaires developed specifically for the study	Study design was appropriate Social validity questionnaire was well developed, but relied on satisfaction ratings as a measure of acceptability Findings were partially reported

Acceptability of family/systemic interventions

Study	Sample size	Data collection	Analysis method	Quality overview
Conran 1993⁶²¹	n = 2, mother and daughter	Interviews		This single case study was not adequately designed or reported
Costa 2009⁶⁷⁸	n = 28; n = 8 families interviewed	Questionnaires, children drawings, evaluation of the effects of conversations and actions in families	Qualitative epistemology	The study aims and objectives were clearly stated and the research methodology was appropriate Reporting of findings was appropriate
Danielson 2010⁵²⁶	n = 10	3–6 months post treatment Ecological functioning measure treatment satisfaction and adherence		Study was well designed, and the data collection, analysis and findings were clearly reported
Tjersland 2006⁶⁷³	n = 23 families, n = 32 children	Observational data from therapeutic sessions; interviews with mothers, children and alleged perpetrators	Descriptive coding developed by two therapists, independently rated to verify categorisation	Some relevant points about the tension that mothers feel between not wanting to talk about the abuse/suspected abuse to protect their child, but also wanting to help them Nothing useful on children’s perspectives
Woodworth 1991⁵³³	n = 16/22	N = 40 interviews with n = 13 incest victims, n = 12 offenders, n = 2 siblings		Poorly described Opportunistic sampling, insufficient methodological information to be able to properly assess the quality of the work

Acceptability of psychoeducation interventions

Study and location	Sample size; response rate	Data collection: acceptability	Analysis method	Quality overview
Barth 1994¹⁶¹	n = 15	Brief client satisfaction survey		Study design was appropriate but acceptability measures relied on satisfaction ratings with no other detail Findings were clearly reported
Boisvert 2008⁶⁵⁸	n = 116	‘Dropout’ defined as someone who agrees to therapy and participates in at least one session and then stops before the half-way point without the therapists approval or an agreement that treatment is finished (Garfield 1994⁸⁰⁴)	Univariate and multivariate analyses exploring relationships between dropout and sexual abuse, individual and family characteristics	Study design did not include a control group and compared characteristics of only treatment completers and those who dropped out Standardised measures were used but the study did not describe the treatment in any detail or measure effectiveness, which limited the conclusions of the study
Hyde 1995⁵³⁸		Interview and rating		Study procedure is unclear Measures used had evidence of reliability and validity and were age appropriate Analysis of data is unclear
Rushton 2000⁶⁶⁴	n = 58; n = 46 mothers, n = 19 foster/adoptive mothers; 14 carers	Carer interviews – 1- and 2-year follow-up Follow-up interviews with carers’ workers 1 year after start of treatment		Methodology and research design were appropriate for the research aims The research findings were reported clearly

Acceptability of group work for children

Study and location	Sample size	Data collection: acceptability	Quality overview
Ashby 1987⁶³⁶	n = 9	Acceptability and usefulness evaluation rating on numerical scale – midway and final group; school counsellor reports	The research aims were clearly described but the methodology, analysis and findings were not clearly reported
Baker 2001⁶³⁷	n = 5–14	Satisfaction ratings; evaluations conducted in 1997, 1998 and 2000 Four group evaluations and follow-up telephone survey of n = 7 terminated clients	The research aims were clearly described but the methodology, analysis and findings were not clearly reported
De Luca 1995¹⁷⁰	n = 35 intervention n = 35 control	Child-report measures	Study design was limited by no control group Treatment satisfaction was measured using self-report social validation ratings by children and parents Presentation of results was limited to percentages Unclear how themes responded to were identified
Grayston 1996⁶⁵⁹	n = 6	Child feedback and parent feedback questionnaires	Research aims were clearly defined and methodology is appropriate Data collection and ethical considerations were not described in detail Findings were reported clearly
Gustafsson 1995⁶⁷⁶	n = 19	Semistructured interviews with group therapists	Research was well designed but findings were not clearly reported
Monck 1996¹⁶⁹	n = 47	Participant rating	Study design was appropriate but the wide scope means that interventions lack specificity in terms of aims and theoretical underpinning Although randomised, small sample size Data collection lacked detail in description and appeared to lack sophistication in terms of non-standardised measures and the use of subjective rating scales Data analysis lacked complexity and included a simplistic presentation of ratings
Peled 1992⁶³⁹	n = 30	Interviews plus observations	Aims and methodology were well explained, and data collection and analysis appropriate Findings were clearly reported

Acceptability of counselling interventions

Study and location	Sample size; response rate	Data collection: acceptability	Analysis method	Quality overview
Baginsky 2001⁶⁴⁰	n = 130	Telephone interviews (Italy) Interviews, letters, telephone and questionnaires (the Netherlands) Individual and group interviews (UK)	Summary overview	Research was well designed but data collection methods were not similar across the three countries and may not have been representative Findings were clearly reported
Fowler 1992⁶⁴¹	n = 35	Seven-point Likert scale		The study design is appropriate for the research aims although is limited by a sample that includes only girls Gender preference was measured using only self-reported rating scores Results were reported in full and the analysis was appropriate
Fowler 1993⁶⁴²	n = 20	Pre- and post-treatment preference stated		Although the study design was appropriate, sampling methods may not be representative of the typical population The method of data collection is questionable, as data were collected post intervention but longer-term follow-up may have been more informative Age range was wide and may have been too heterogeneous The type of data analysis conducted was appropriate
Haight 2010¹⁷⁵	n = 15	Views from children, caregivers and community clinicians; field notes	Emic coding
Haskett 1991⁶⁵⁷	n = 129	Factors associated with treatment entry		Study design was appropriate and the sampling included all referred cases within a set time frame Data relied on parents self-reports with no use of standardised questionnaires The same therapists were also used as researchers; however, the analysis was appropriate and thorough
Kilcrease-Fleming 1992⁶⁴³	n = 20	Three counselling process rating scales of videotaped interviews		An innovative design using observer ratings but with an unclear methodology in terms of participant recruitment, intervention type Although measures were designed for the study and lack prior use or testing, clear efforts were made to improve the reliability of the measures The analysis was appropriate but the small sample size hampered generalisations
Kolko 1999⁶⁴⁴	n = 86	Children and parents completed an evaluation of the level of perceived service needs, motivation and interests, goals or expectations and obstacles; interviewed at study intake and 4–8 months after initial service	Predictors of service use computed using Pearson’s correlations or chi-squared tests	A clear study design, participant recruitment strategy and assessment schedule Measures included standardised and non-standardised measures Analysis was appropriate and findings were clearly presented
Nelson-Gardell 2001⁶³⁸	n = 34 plus five focus groups	Focus group methodology to address the issues of interest	Thematic analysis	The aims of the research were clearly defined and the methodology was appropriate to meet these aims Data analysis was not fully described Findings were reported clearly
Overlien 2011⁶⁷⁴	n = 50 women’s refuges directors, n = 22 children	Face-to-face interviews, age-appropriate schedules	Grounded theory approach	Study design was appropriate, analysis was adequate and findings were clearly reported Unclear if all ethical considerations were made
Porter 1996⁶⁴⁵	n = 27	Client Behavior System⁷⁴⁵ – verbalisation measure		Study design was appropriate overall but relies on the measurement of verbalisation within one specific session; however, attempts were made to blind therapists to the study aims and observer raters were recruited externally and trained to standardise ratings using a predetermined protocol (the Client Behavior System) Findings were clearly reported, although descriptions lack direction, such as reporting that non-abuse-related questions elicited significantly more non-abuse-related answers
Reddy 2013¹⁷⁶	n = 70	Qualitative post-treatment feedback		Study design was appropriate as was sampling. However, data collection for treatment acceptability relied on non-standardised rating scales. Findings were reported clearly
Scott 1996⁶⁷⁷	n = 10 families (n = 17 children)	In-depth interviews with parents		Research aims were not clearly stated but research design and data collection clearly described Findings were clearly reported
Deb 2011⁶⁷⁹	n = 120	Quantitative and qualitative data		The study design was appropriate for the aims of the research It is unclear how perceptions of treatment were measured and analysed Results were limited to descriptive narratives and percentages
Thompson 2011⁶⁴⁶	n = 40	Semistructured interviews developed to elicit information about mothers and youth experiences and satisfaction with mental health services – Mother Interview Guide and Youth Interview Guide		The methodology and research design were appropriate for the research aims The research findings were reported clearly

Acceptability of psychotherapy interventions

Study and location	Sample size; response rate	Data collection: acceptability	Analysis method	Quality overview
Davies 2009⁶⁶⁵	n = 4	Single time point interview of children’s experiences of participation	IPA	Aims of the research were clearly defined and the methodology was appropriate to meet these aims Findings were reported clearly
Horowitz 1997⁶⁴⁷	n = 81	Therapy experiences gathered from parents	Regression analyses	Study was part of a larger longitudinal study but the design suffers from a reliance on therapists’ own feedback, lacking child feedback and a lack of clarity on the type of intervention other than the format (individual, family or group); however, the analysis of data was appropriate and findings were clearly reported
Jensen 2010⁶⁷⁵	n = 15	Follow-up interview in child’s home	Content analysis guided by Bordin’s conceptualisation of the working alliance	Aims of the research were clearly defined and the methodology was appropriate to meet these aims Findings were reported clearly
Lippert 2008⁶⁴⁸	n = 101	Case record review Caregiver interview including ‘perceptions related to therapy’		Systematic and appropriate study design, including both quantitative and qualitative components Measures used were standardised and interviews used a predetermined protocol Data analysis was appropriate, although reported results lacked qualitative depth

: IPA, interpretative phenomenological analysis.

Acceptability of peer-mentoring interventions

Study and location	Sample size; response rate	Data collection: acceptability	Analysis method	Quality overview
Alaggia 1999⁶⁶⁰	24 families, 26 children	Interviewed a sample of parents, youths and professionals	Summary overview	Aims of the research were clearly defined and the methodology was appropriate to meet these aims Data analysis was not fully described Findings were reported clearly

Acceptability of therapeutic residential care interventions

Study and location	Sample size; response rate	Data collection: acceptability	Analysis method	Quality overview
Cunningham 2009⁶⁴⁹	n = 130	Semistructured interviews with RTC staff; school, clinical and residential treatment team questionnaires (T2 and T3); data from client case files	Inductive thematic coding; confirmatory factor analysis	Study design was adequate, and method was clearly described and appropriate Data were collected over a number of time points (weeks after admission, the approximate midpoint of the client’s stay, immediately prior to discharge and 4 months after discharge) and from multiple sources Clear thought was given to the age-appropriateness and ethics of data collection from young people Thematic analysis and factor analysis were both appropriate and reported in detail
Gallagher 2012⁶⁶⁶	16/34	Semistructured interviews	Summary overview	Study was well designed and described, and the findings were clearly reported
Leenarts 2013⁶⁷²	n = 154		Multiple linear regression for treatment motivation. Logistic regression to identify possible predictors for dropout	Appropriate study design and sampling Measures had evidence of reliability and validity Appropriate data analysis and findings were clearly reported
Shennum 1995⁶⁵⁰	n = 80	Qualitative interviews with participants	Summary overview	Appropriate study design but sampling was unclear Measures had evidence of reliability and validity Data analysis could have been more rigorous but findings were clearly reported
West 2014⁶⁵¹	n = 39	Focus group	Thematic analysis	Aims, methodology, data collection and analysis were all good quality; however, the content may not be very relevant to our review – it is more focused on the components that a good intervention for externalising behaviour in a school setting should contain

: RTC, residential treatment centre.

Acceptability of enhanced foster care interventions

Study and location	Sample size; response rate	Data collection: acceptability	Analysis	Quality overview
Biehal 2012¹⁴⁵^,¹⁴⁶	n = 219; RCT n = 34; observational n = 185 Case studies n = 20	Postal questionnaires; interviews; reports; parent records	Bivariate and multivariate analyses plus thematic analyses of interview data and more in-depth analyses of case study data	Aims and progress of the study clearly stated Criteria for case study selection clear and appropriate Data analyses appropriate and findings clearly described
Laan 2001⁶⁷¹	n = 78	Case notes and questionnaire data	Analysis of questionnaire data; thematic analysis of case notes	Study design was appropriate Data collection relied on satisfaction ratings, although bias was addressed by matching with qualitative answers when given Satisfaction ratings were clearly presented but there is limited reporting of qualitative findings
Staines 2011⁶⁶⁷	n = 450	Questionnaires – foster parents and social workers; at placement start and 1 year on	Summary overview	A well-designed study with appropriate data collection measures Questionnaires were anonymised to prevent bias Findings were clearly reported

: RCT, randomised controlled trial.

Acceptability of activity-based Interventions

Study and location	Sample size; response rate	Data collection: acceptability	Analysis	Quality overview
Bannister 1996⁶⁶⁸	n = 6	Pre- and post-treatment interviews with social worker, carer and child		Aims, methodology, analysis and findings were not clearly described
Burgon 2011⁶⁶⁹	n = 7	Qualitative interviews, field note observations	Qualitative, participative and reflexive ethnography; thematic analysis	Aims, methods and design of the study were appropriate Recruitment of sample was unclear and author was participant researcher, which may be subject to bias Reporting of findings were clear
Gilbert 1988⁶⁵²	Group attendance ranged from two to six children (average attendance n = 4)	Art qualitative feedback		Aims and methodology were clearly described but data collection and analysis were lacking in depth
Hill 2009⁶⁷⁰ UK	13 families: mothers n = 12, fathers/stepfathers n = 4 Children n = 28	Qualitative case study of 13 cases files plus in-depth interviews with therapists (n = 27), parents (n = 18) and children (n = 3)	Thematic analysis	Aims, methods and design of the study were appropriate Data analysis was adequate Reporting of findings are clear
Mishna 2012⁶⁶¹	n = 11	63 interviews were conducted at 6, 12 and 18 months with parents, teachers and therapists	Grounded theory approach	Aims of the research were clearly stated, and the methodology and research design were appropriate Data collection was adequate described and the findings were clearly described

Copyright © Queen’s Printer and Controller of HMSO 2016. This work was produced by Macdonald et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Included under terms of UK Non-commercial Government License.

Bookshelf ID: NBK385389

Contents

< Prev Next >

PubReader
Print View
Cite this Page
Macdonald G, Livingstone N, Hanratty J, et al. The effectiveness, acceptability and cost-effectiveness of psychosocial interventions for maltreated children and adolescents: an evidence synthesis. Southampton (UK): NIHR Journals Library; 2016 Sep. (Health Technology Assessment, No. 20.69.) Appendix 14, Overview of acceptability evidence by study interventions.
PDF version of this title (4.2M)

Overview of acceptability evidence by study interventions - The effectiveness, a...
Overview of acceptability evidence by study interventions - The effectiveness, acceptability and cost-effectiveness of psychosocial interventions for maltreated children and adolescents: an evidence synthesis
List of abbreviations - Palivizumab for Immunoprophylaxis of Respiratory Syncyti...
List of abbreviations - Palivizumab for Immunoprophylaxis of Respiratory Syncytial Virus (RSV) Bronchiolitis in High-Risk Infants and Young Children: Systematic Review and Additional Economic Modelling of Subgroup Analyses
Qualitative research study profiles - OPTIMA prelim: a randomised feasibility st...
Qualitative research study profiles - OPTIMA prelim: a randomised feasibility study of personalised care in the treatment of women with early breast cancer
Ongoing trials - Clinical effectiveness and cost-effectiveness of use of therape...
Ongoing trials - Clinical effectiveness and cost-effectiveness of use of therapeutic monitoring of tumour necrosis factor alpha (TNF-α) inhibitors [LISA-TRACKER® enzyme-linked immunosorbent assay (ELISA) kits, TNF-α-Blocker ELISA kits and Promonitor® ELISA kits] versus standard care in patients with Crohn’s disease: systematic reviews and economic modelling
Plain English summary - Different temperature thresholds for antipyretic interve...
Plain English summary - Different temperature thresholds for antipyretic intervention in critically ill children with fever due to infection: the FEVER feasibility RCT

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Bookshelf