Psychiatry and Behavioral Sciences
Fidelity & Quality Assurance Publications
55. Henggeler, S.W., Melton, G.B., Brondino, M.J., Scherer, D.G., & Hanley, J.H. (1997). Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65(5), 821-833.
85. Henggeler, S.W., Pickrel, S.G., & Brondino, M.J. (1999). Multisystemic treatment of substance abusing and dependent delinquents: Outcomes, treatment fidelity, and transportability. Mental Health Services Research, 1, 171-184.
91. Huey, S.J., Henggeler, S.W., & Brondino, M.J., & Pickrel, S.G. (2000). Mechanisms of change in Multisystemic therapy: Reducing delinquent behavior through therapist adherence and improved family and peer functioning. Journal of Consulting and Clinical Psychology, 68, 451-467.
95. Schoenwald, S.K., Henggeler, S.W., Brondino, M.J., Rowland, M.D. (2000). Multisystemic therapy: Monitoring treatment fidelity. Family Process, 39, 83-103.
99. Henggeler, S.W., & Schoenwald, S.K. (1999). The role of quality assurance in achieving outcomes in MST programs. Journal of Juvenile Justice and Detention Services, 14(2), 1-17.
210. Edwards, D.L., Schoenwald, S.K., Henggeler, S.W., & Strother, K.B. (2001). A multi-level perspective in the implementation of multisystemic therapy (MST): Attempting dissemination with fidelity. In G.A. Bernfield, D.P. Farrington, & A.W. Leschied (Eds.), Offender rehabilitation in practice: Implementing and evaluating effective programs (pp.97-120). London: Wiley.
232. Henggeler, S,W., & Schoenwald, S.K., (2002). Treatment manuals: necessary, but far from sufficient. Clinical Psychology: Science and Practice, 9(4), 419-420.
245. Schoenwald, S.K., & Henggeler, S.W. (2003). Current strategies for moving evidence-based interventions into clinical practice: Introductory comments. Cognitive and Behavioral Practice, 10, 275-277.
248. Ogden, T., & Halliday-Boykins, C. A. (2004). Multisystemic treatment of antisocial adolescents in Norway: Replication of clinical outcomes outside of the US. Child and Adolescent Mental Health, 9(2), 77-83.
292. Cunningham, P.B., Randall, J., Henggeler, S.W., Schoenwald, S.K. (2006). Multisystemic therapy supervision: A key component of quality assurance. In T. Kirby Neill (Eds.), Helping Others Help Children: Clinical Supervision of Child Psychotherapy (pp. 137-160). American Psychological Association: Washington, DC.
304. Ellis, D.A., Naar-King, S., Templin, T., Frey, M.A., Cunningham, P.B. (2007). Improving health outcomes among youth with poorly controlled type I diabetes: The role of treatment fidelity in a randomized clinical trial of multisystemic therapy. Journal of Family Psychology, 21(3), 363-371.
322. Chapman, J.E., Sheidow, A.J., Henggeler, S.W., Halladay-Boykins, C.A., & Cunningham, P.B. (2008). Developing a measure of therapist adherence to contingency management: An application of the many-facet rasch model. Journal of Child and Adolescent Substance Abuse, 17(3), 47-68.
333. Sheidow, A.J., Donohue, B.C., Hill, H.H., Henggeler, S.W., & Ford, J.D. (2008). Development of an audio-tape review system for supporting adherence to an evidence-based treatment. Professional Psychology: Research and Practice, 39, 553-560.
381. Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level Evidence-Based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78(4), 537-550.
397. Holth, P., Torsheim, T., Sheidow, A. J., Ogden, T., Henggeler, S. W. (2011). Intensive quality assurance of therapist adherence to behavioral interventions for adolescent substance use problems. Journal of Child and Adolescent Substance Abuse, 20, 289-313.
400. Schoenwald, S. K., Garland, A. F., Southam-Gerow, M. A., Chorpita, B. F., & Chapman, J. E. (2011). Adherence Measurement in treatments for disruptive behavior disorders: Pursuing clear vision through varied lenses. Clinical Psychology Science and Practice, 18(4), 331-341.
Fidelity and Quality Assurance Abstracts
The effects of multisystemic therapy (MST) in treating violent and chronic juvenile offenders and their families in the absence of ongoing treatment fidelity checks were examined. Across 2 public sector mental health sites, 155 youths and their families were randomly assigned to MST versus usual juvenile justice services. Although MST improved adolescent symptomology at posttreatment and decreased incarceration by 47% at a 1.7-year follow-up, findings for decreased criminal activity were not as favorable as observed on other recent trials of MST. Analyses of parent, adolescent, and therapist reports of MST treatment adherence ratings were high. These results highlight the importance of maintaining treatment fidelity when disseminating complex family-based services to community settings.
The effectiveness and transportability of multisystemic therapy (MST) were examined in a study that included 118 juvenile offenders meeting DSM-III-R criteria for substance abuse or dependence and their families. Participants were randomly assigned to receive MST versus usual community services. Outcome measures assessed drug use, criminal activity, and days in out-of-home placement at posttreatment (T2) and at a 6-month posttreatment follow-up (T3); also treatment adherence was examined from multiple perspectives (i.e., caregiver, youth, and therapist). MST reduced alcohol, marijuana, and other drug use at T2 and total days in out-of-home placement by 50% at T3. Reductions in criminal activity, however, were not as large as have been obtained previously for MST. Examinations of treatment adherence measures suggests that the modest results of MST were due, at least in part, to difficulty in transporting this complex treatment model from the direct control of its developers. Increased emphasis on quality assurance mechanisms to enhance treatment fidelity may help overcome barriers to transportability.
The mechanisms through which multisystemic therapy (MST) decreased delinquent behavior were assessed in 2 samples of juvenile offenders. Sample 1 included serious offenders who were predominately rural, male, and African American. Sample 2 included substance-abusing offenders who were predominately urban, male, and Caucasian. Therapist adherence to the MST protocol (based on multiple respondents) was associated with improved family relations (family cohesion, family functioning, and parent monitoring) and decreased delinquent peer affiliation, which, in turn, were associated with decreased delinquent behavior. Furthermore, changes in family relations and delinquent peer affiliation mediated the relationship between caregiver-rated adherence and reductions in delinquent behavior. The findings highlight the importance of identifying central change mechanisms in determining how complex treatment such as MST contribute to ultimate outcomes.
The challenges of specifying a complex and individualized treatment model and measuring fidelity thereto are described, using multisystemic therapy (MST) as an example. Relations between therapist adherence to MST principles and instrumental and ultimate outcome variables are examined, as are relations between clinical supervision and therapist adherence. The findings provide modest support for the associations between MST adherence measures and instrumental and ultimate outcomes. Results also show that adherence can be altered when clinical supervision and adherence monitoring procedures are fortified. The modest associations between adherence measures and youth outcomes argue for further refinement and validation of the MST adherence measure, especially in light of the well-established effectiveness of MST with challenging clinical populations and the increasing dissemination of MST programs.
Multisystemic therapy (MST) is a family-and community-based treatment that has proven effective at reducing long-term rates of rearrest and out-of-home placement in clinical trials with children and adolescents presenting serious problems. A key feature of the success of MST and to the dissemination of MST programs across the nation is a well-specified quality assurance system. This paper describes the manualization of the components of the MST quality assurance system and the corresponding training protocols used to promote program fidelity. In addition, empirical support for central aspects of the quality assurance system and current research on this system are presented. Ultimately, the validation of quality assurance systems will be critical for the effective transport of evidence-based interventions in the fields of juvenile justice and mental health.
This commentary suggests that the transport of evidence-based mental health treatments to field settings is a complex and multi-layered process. Although treatment manuals may represent one important component of that process, researchers are often ignoring many other critical components.
This special section presents four evidence-based practices that are currently being transported to community-based settings: Nurse-Family Partnership for infant health and development, Assertive Community Treatment for adults with serious and persistent mental illness. Oregon Therapeutic Foster Care for chronic juvenile offenders, and Parenting Wisely for caregivers of children with disruptive behavior: Although each model addresses a very different clinical population, their similarities in development, emphasis on fidelity and quality assurance, and use of dedicated organizations to support effective transport are noteworthy. Admirably, the intervention developers are taking the same care in crafting strategies to transport their models as they took in validating these clinical innovations. Such care is critical to increasing the probability that the intervention models will be transported to community settings with the fidelity needed to achieve intended outcomes.
Background: MST is an intensive home- and community-based intervention for youths with serious antisocial behavior and other serious clinical problems that has been effective at reducing out-of-home placements and producing favorable long-term clinical outcomes in the U.S. The aims of the study were to determine the degree to which these outcomes would be replicated in Norway for youths with serious behavior problems and to conduct a randomized trial of MST by an independent team of investigators. Method: Participants were 100 seriously antisocial youths in Norway who were randomly assigned to Multisystemic Therapy (MST) or usual Child Welfare Services (CS) treatment conditions. Data were gathered from youths, parents, and teachers pre- and post-treatment. Results: MST was more effective than CS at reducing youth internalizing and externalizing behaviors and out-of-home placements, as well as increasing youth social competence and family satisfaction with treatment.
Discussion: This is the first study of MST outside of the US and one of the first not conducted by the developers of MST. The findings replicate those obtained by MST's developers and demonstrate the generalisability of short-term MST effects beyond the US.
This chapter focuses on MST clinical supervision and on its functions within the broader, quality-assurance system, especially supervisor training and consultation, measurement of supervisor performance, common problems encountered by MST supervisors, and strategies for managing these problems, such as therapist drift and frustration.
The purpose of the study was to assess whether therapist treatment fidelity was a predictor of treatment outcome in a randomized clinical trial of multisystemic therapy with 10- to 16-year-old youths with chronically poorly controlled Type I diabetes (N= 40). Treatment fidelity was assessed by objective ratings of therapy sessions and questionnaires completed by caregivers and by therapists. Relationships between fidelity measures were assessed. Structural equation modeling (SEM) was used to test whether high fidelity would lead to improved regimen adherence and to improved metabolic control outcomes via regimen adherence. Objective ratings of treatment fidelity were significantly related to therapist-reported but not to caregiver-reported treatment fidelity. SEM resulted supported a completely mediated pathway between treatment fidelity and metabolic control, with regimen adherence mediating the relationship. Results suggest that conducting complex behavioral interventions with a high degree of fidelity can improve treatment outcomes among youths with chronic illnesses.
A unique application of the Many-Facet Rasch Model (MFRM) is introduced as the preferred method for evaluating the psychometric properties of a measure of therapist adherence to Contingency Management (CM) treatment of adolescent substance use. The utility of psychometric methods based in Classical Test Theory was limited by complexities of the data, including: (1) ratings provided by multiple informants (i.e., youth, caregivers, and therapists), (2) data from separate research studies, (3) repeated measurements, (4) multiple versions of the questionnaire, and (5) missing data. Two dimensions of CM adherence were supported: adherence to Cognitive Behavioral components and adherence to monitoring components. The rating scale performed differently for items in these subscales, and of 11 items evaluated, eight were found to perform well. The MFRM is presented as a highly flexible approach that can be used to overcome the limitations of traditional methods in the development of adherence measures for evidence-based practices.
In bridging the science-to-service gap, effective yet practical strategies are needed for supportig practitioner implementation of evidence-based treatments. The development and preliminary evaluation of an adherence monitoring system to support clinician fidelity to an evidence-based treatment for substance-abusing adolescents was tested for community-based practitioners. Session tapes were monitored for adherence to a family-based approach to contingency management for 27 practitioners during baseline, postworkshop, and follow-up periods. Approximately half of the practitioners were randomized to receive intensive quality assurance following a family-based contingency management workshop as part of a larger study. Findings supported the clinical feasibility of the developed system as well as the face and content validity, reliability, and concurrent validity. Future directions are discussed in light of these results, including instructions for the use of the developed system to efficiently train clinicians to adequate fidelity.