RESUMEN

Ensuring high levels of procedural fidelity during behavior-analytic interventions is a crucial component of providing effective behavior-analytic services. However, few resources are available to help guide practitioners through measuring procedural fidelity. In fact, most published behavior-analytic research on procedural fidelity analyzes a single treatment procedure, which might not completely reflect the process of monitoring and addressing the procedural fidelity of a robust treatment package that might be necessary in clinical settings. The purpose of this article is to guide behavior analysts through the process of creating and using procedural fidelity measurement systems, with a focus on direct observation of implementation as a means of fidelity data collection. This process consists of six steps: (1) task analyze treatment procedures into measurable units; (2) assign measures to each treatment component; (3) plan the direct observation; (4) collect procedural fidelity data; (5) analyze and interpret procedural fidelity data; and (6) take action to improve procedural fidelity. Each step is described and discussed in the article.

RESUMEN

Procedural fidelity refers to the degree to which procedures for an assessment or intervention (i.e., independent variables) are implemented consistent with the prescribed protocols. Procedural fidelity is an important factor in demonstrating the internal validity of an experiment and clinical treatments. Previous reviews evaluating the inclusion of procedural fidelity in published empirical articles demonstrated underreporting of procedural fidelity procedures and measures within specific journals. We conducted a systematic review of The Analysis of Verbal Behavior (TAVB) to evaluate the trends in procedural fidelity reporting from 2007 to 2021. Of the 253 articles published in TAVB during the reporting period, 144 of the articles (168 studies) met inclusionary criteria for further analysis. Our results showed that 54% of studies reported procedural fidelity data, which is slightly higher than previous reviews. In comparison, interobserver-agreement data were reported for a high percentage of studies reviewed (i.e., 93%). Further discussion of results and applied research implications are included.

RESUMEN

This study explored predictors of community-based providers' adherence to MATCH, a modular cognitive behavioral therapy for children and adolescents. Provider-reported adherence to MATCH was measured using three increasingly strict criteria: (1) session content (whether the session covered MATCH content consistent with the client's target problem), (2) session content and sequencing (whether the session covered MATCH content in the expected sequence for the client's target problem), and (3) session content, sequencing, and participant (whether the session covered MATCH content in the expected sequence and with the expected participant(s) for the client's target problem). Session, client, provider, and organizational predictors of adherence to MATCH were assessed using multilevel modeling. Results revealed that nearly all providers delivered MATCH content that corresponded to the target problem, but only one-third of providers delivered MATCH content in the expected sequence and with the expected participant for the client's target problem. This difference underscores the need for nuanced adherence measurement to capture important implementation information that broad operationalizations of adherence miss. Regardless of the criteria used providers were most adherent to MATCH during sessions when clients presented with interfering comorbid mental health symptoms. This suggests that the design of MATCH, which offers flexibility and structured guidance to address comorbid mental health problems, may allow providers to personalize treatment to address interfering comorbidity symptoms while remaining adherent to evidence-based practices. Additional guidance for providers on managing other types of session interference (e.g., unexpected events) may improve treatment integrity in community settings.

Asunto(s)

RESUMEN

Treatment integrity is defined as the extent to which the treatment has been implemented according to a specific theoretical model. It consists of two components: adherence, that is, the degree to which the therapist implements the techniques prescribed by the theoretical model, and competence, that is, the degree to which the therapist skilfully and appropriately implements the techniques prescribed by the model. Recently, the issue of integrity is gaining increasing importance in psychotherapy research, especially in an attempt to clarify its role in influencing the effectiveness of treatments. However, most studies focus on the individual setting. Therefore, this systematic review aims at investigating the relationship between integrity and outcome in group clinical treatments. Results highlighted a positive relationship between group treatment integrity and outcome. Moreover, this review provided insights for implications for research, clinical practice, and training of therapists, identifying questions that still need to be answered and tracing possible future research directions.

RESUMEN

BACKGROUND: Treatment fidelity refers to the degree to which an intervention is implemented as intended. Promoting treatment fidelity is important to achieve a valid comparison in intervention research. However, it is often underreported: few studies detail the use and development of fidelity measures. This study aims to promote the treatment fidelity of a modified version of the Derbyshire Language Scheme (M-DLS), a manualised intervention for children with language difficulties, by exploring participants' opinions on training and intervention delivery. Results inform development of a checklist and scoring system to monitor and promote treatment fidelity in a comparison trial. METHOD: Ten student speech and language therapists (SLTs) and two research assistants (RAs) participated in the study. All received training on the M-DLS, and 10 were video-recorded completing role-plays of an M-DLS session in small groups. Feedback was gathered after training and role-plays in focus groups and interviews. Feedback was interpreted using the constructs of the Theoretical Domains Framework (TDF). A treatment fidelity checklist was then developed using the feedback. The first author and two RAs rated role-play videos using the checklist to trial it to inform amendments and to promote interrater reliability. Interrater agreement was calculated using Spearman's test of correlation. RESULTS: Participants discussed the importance of having clear materials and time to practise sessions. They suggested amendments to the materials and training to promote treatment fidelity. The checklist and scoring system accounted for participants' suggestions, with amendments detailed in a log. Spearman's correlation results suggested agreement between the raters was strong. CONCLUSIONS: Results emphasise the importance of training quality, practice and reflective opportunities and clear materials to promote treatment fidelity. The construction of the checklist and scoring system was described in detail, informing the development of future checklists. After further trialling, the checklist can be used to ensure the M-DLS is delivered with high treatment fidelity in the comparison trial. WHAT THIS PAPER ADDS: What is already known on this subject Treatment fidelity is an essential component of intervention effectiveness and efficacy studies, ensuring the intervention is delivered as intended. It is also an essential component of evidence-based clinical practice. However, few research studies report the treatment fidelity process or publish the checklists used, depriving clinicians of useful information for implementation. What this study adds This study describes in detail the iterative process of treatment fidelity checklist development, engaging those implementing the intervention in development. This ensured clarity and interrater reliability of the checklist. Furthermore, a novel scoring system was developed so that accuracy of implementation can be easily compared across users and across practice attempts. What are the clinical implications of this work? The importance of treatment fidelity when implementing effective and efficacious interventions cannot be overstated. The treatment fidelity checklist developed for research can be easily adopted to support accurate implementation in clinical practice through an audit process.

Asunto(s)

RESUMEN

BACKGROUND: Work schedules can make it challenging for professional care workers to participate in long-term training programs on managing behavioural and psychological symptoms of dementia (BPSD). Simultaneously, it is necessary to prevent caregivers' negative responses to BPSD, provide a positive environment for people with dementia, and create a system for new management plans, since the initial one is often unsuccessful. Therefore, we developed a short manual-based training system for functional analysis including positive behaviour support and strategies when management plans do not function well. This study aimed to preliminarily examine the usefulness of this system. METHODS: Thirty-five staff members from 12 care facilities participated in the training. For each facility, off-the-job training was performed in two 120-min sessions held over 2 days. Then, care plans were implemented by staff members for a month, during which on-the-job training was provided. The study included 14 people with dementia and BPSD. This was a single-arm study without a control group. Pre- and post-tests were conducted to examine the effects of the training system using the Neuropsychiatric Inventory-Nursing Home Version. RESULTS: The results of the pre- and post-tests for the total scores on severity and occupational disruptiveness significantly improved, with large effect sizes. Regarding symptom domains, delusions, agitation/aggression, and aberrant motor behaviour significantly improved in both severity and occupational disruptiveness. Depression/dysphoria and anxiety significantly improved in severity; however, there were trends of improvement in occupational disruptiveness. In addition, the effect sizes for severity and occupational disruptiveness of delusions and agitation/aggression were large. CONCLUSIONS: This preliminary study suggests that the training system is promising. A randomised controlled trial with a larger sample size is necessary to confirm the findings.

Asunto(s)

RESUMEN

Background: Literature on the association between therapist adherence and treatment success in the treatment of post-traumatic stress disorder (PTSD) is scarce, and the results are mixed.Objective: To examine the relationship between therapist adherence to dialectical behaviour therapy for PTSD (DBT-PTSD) and cognitive processing therapy (CPT) on treatment outcome in women with PTSD and emotion regulation difficulties after interpersonal childhood abuse.Method: Videotaped therapy sessions from 160 female participants of a large randomized controlled trial [Bohus, M., Kleindienst, N., Hahn, C., Müller-Engelmann, M., Ludäscher, P., Steil, R., Fydrich, T., Kuehner, C., Resick, P. A., Stiglmayr, C., Schmahl, C., & Priebe, K. (2020). Dialectical behavior therapy for posttraumatic stress disorder (DBT-PTSD) compared with cognitive processing therapy (CPT) in complex presentations of PTSD in women survivors of childhood abuse. JAMA Psychiatry, 77(12), 1235. jamapsychiatry.2020.2148] were rated. Adherence to CPT and DBT-PTSD was assessed using two specifically developed rating scales.Results: Higher therapist adherence was associated with a greater reduction of clinician-rated PTSD symptom severity. This effect was more pronounced in the CPT group than in the DBT-PTSD group. Adherence was also related to a greater reduction of self-rated PTSD symptoms, borderline symptoms, and dissociation intensity.Conclusion: Our results indicate that higher therapist adherence can lead to better treatment outcomes in PTSD treatments, especially in CPT.

Higher therapist adherence to cognitive processing therapy was associated with higher treatment gains in women with post-traumatic stress disorder (PTSD) after childhood abuse.Adherence was related to higher reductions in symptom severity of PTSD, borderline symptoms, and dissociation intensity.Adherence to dialectical behaviour therapy for PTSD did not show a strong association with treatment outcome.

Asunto(s)

RESUMEN

Background: Literature on the association between therapist competence and treatment success in posttraumatic stress disorder (PTSD) treatments is scarce and results are mixed.Aims/Objective: The relationship between different types of therapeutic competence, therapeutic alliance, and PTSD symptom reduction in patients treated with Dialectical Behaviour Therapy for PTSD (DBT-PTSD) or Cognitive Processing Therapy (CPT) was assessed. Competence types were PTSD-specific competence, treatment specific competence, and general competence in cognitive behaviour therapy (CBT).Method: Videotaped therapy sessions from N = 160 women with PTSD and emotion regulation difficulties after child abuse participating in a large randomised controlled trial (Bohus et al., 2020) were rated. Three therapeutic competence-types were assessed using specifically developed rating scales. Alliance was assessed via patient ratings with the Helping Alliance Questionnaire (HAQ). PTSD symptoms were assessed at pre- and post-treatment via clinician rating with the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) and via self-rating with the PTSD-Checklist for DSM-5 (PCL-5).Results: No significant association between competence and clinician or self-rated PTSD symptoms was found. PTSD specific competence predicted clinician rated PTSD symptom severity on a trend level. Alliance predicted both clinician and self-rated PTSD symptom reduction.Conclusion: Our results provide a starting point for future research on different competence types and their association with PTSD treatment gains. Therapists were highly trained and received weekly supervision, hence a restricted competence range is a possible explanation for non-existing associations between competence and PTSD symptom reduction in our sample. More research in naturalistic settings, such as dissemination studies, is needed.

Three different types of therapeutic competence and their association to treatment gains in women with posttraumatic stress disorder after child abuse were assessed.Therapist competence was high in all three domains of competence.No association was found between any of the competence types and reduction in posttraumatic stress disorder symptoms.

Asunto(s)

RESUMEN

The purpose of the current study was to compare the effects of omission and commission errors of reinforcement during tact instruction via telehealth with three children, 6 to 7 years of age, who were diagnosed with an autism spectrum disorder. We used an adapted alternating treatment design to evaluate skill acquisition of target stimuli across high-integrity, commission errors, and omission errors conditions. The high-integrity condition produced mastery criteria in fewer sessions compared with the integrity-error conditions in four of six comparisons, and the omission condition reached mastery criteria in fewer sessions than the commission condition in five of six comparisons.

RESUMEN

Few reviews on procedural fidelity-the degree to which procedures are implemented as designed-provide details to gauge the quality of fidelity reporting in behavior-analytic research. This review focused on experiments in the Journal of Applied Behavior Analysis (2006-2021) with "integrity" or "fidelity" in the abstract or body. When fidelity data were collected, the coders characterized measurement details (e.g., description of calculation, report of single or multiple values, frequency of fidelity checks, checklist use). The researchers found increasing trends in describing the calculation(s), reporting multiple values, and stating the frequency of measurement. Few studies described using a checklist. Most studies reported fidelity as a percentage, with high obtained values (M = 97%). When not collecting fidelity data was stated as a limitation, authors were unlikely to provide a rationale for the omission. We discuss recommendations for reporting procedural fidelity to increase the quality of and transparency in behavior-analytic research.

RESUMEN

Procedural fidelity is the extent to which an intervention is implemented as designed and is an important component of research and practice. There are multiple ways to measure procedural fidelity, and few studies have explored how procedural fidelity varies based on the method of measurement. The current study compared adherence to discrete-trial instruction protocols by behavior technicians with a child with autism when observers used different procedural-fidelity measures. We collected individual-component and individual-trial fidelity with an occurrence-nonoccurrence data sheet and compared these scores to global fidelity and all-or-nothing, 3-point Likert scale, and 5-point Likert scale measurement methods. The all-or-nothing method required all instances of a component or trial be implemented without error to be scored correct. The Likert scales used a rating system to score components and trials. At the component level, we found that the global, 3-point Likert, and 5-point Likert methods were likely to overestimate fidelity and mask component errors, and the all-or-nothing method was unlikely to mask errors. At the trial level, we found that the global and 5-point Likert methods approximated individual-trial fidelity, the 3-point Likert method overestimated fidelity, and the all-or-nothing method underestimated fidelity. The occurrence-nonoccurrence method required the most time to complete, and all-or-nothing by trial required the least. We discuss the implications of measuring procedural fidelity with different methods of measurement, including false positives and false negatives, and provide suggestions for practice and research. Supplementary Information: The online version contains supplementary material available at 10.1007/s43494-023-00094-w.

RESUMEN

Procedural integrity refers to the extent to which an independent variable is implemented as described. Measuring procedural integrity is one important factor when considering internal and external validity of experiments. Experimental articles in behavior-analytic journals have rarely reported procedural-integrity data. The purpose of this study was to update previous reviews of whether articles published in the Journal of Applied Behavior Analysis reported procedural integrity, spanning a period from 1980 to 2020, and compare reporting in JABA to recent reviews of studies published in Behavior Analysis in Practice (2008-2019) and the Journal of Organizational Behavior Management (2000-2020). Procedural integrity continues to be underreported across all three journals, but an increasing trend in reporting procedural integrity is evident in the Journal of Applied Behavior Analysis and Behavior Analysis in Practice. In addition to our recommendations and implications for research and practice, we provide examples and resources to assist researchers and practitioners with recording and reporting integrity data.

RESUMEN

Procedural fidelity is the extent to which independent variables are implemented as designed. Despite 40 years of discussion about the importance of procedural fidelity for behavioral research, reporting of fidelity data remains an uncommon practice in behavior-analytic journals. Researchers have speculated about reasons for underreporting, but the perspectives of scholars about when reporting is warranted or necessary have not yet been explored. Thus, the purpose of this study was to evaluate possible reasons for infrequent reporting of fidelity data in behavior-analytic studies. To address this purpose, we conducted focus groups with scholars in applied behavior analysis. Five themes emerged regarding why procedural fidelity data are not typically reported. We provide a discussion about how these themes are interrelated and offer suggestions and recommendations to assist with the collection and reporting of fidelity data.

Asunto(s)

RESUMEN

We examined articles with experiments published in the Journal of Applied Behavior Analysis and in Behavior Analysis in Practice from 2017 through 2021 to determine how frequently procedural fidelity was assessed. When procedural fidelity was assessed, we determined how often a measure of interobserver agreement for those fidelity data was provided. We also determined how often a measure of interobserver agreement for participants' behavior was provided. Across both journals and all years, 54.7% of relevant articles provided a measure of procedural fidelity. Of them, 17.7% provided a measure of interobserver agreement for procedural fidelity. In marked contrast, 96.4% provided interobserver agreement data for participants' behavior. It is unfortunate that applied behavior analysts frequently fail to provide procedural fidelity data and, when they do, often fail to provide interobserver agreement data for the fidelity data. Reviewers for, and editors of, behavior-analytic journals are encouraged to strongly consider the relative value of procedural fidelity and agreement on procedural fidelity measures when rendering recommendations on the suitability of a given submission.

Asunto(s)

RESUMEN

Behavioral technicians (BT) within the field of applied behavior analysis may be at greater risk for experiencing burnout and stress due to the nature of their clients, job demands, and work environments. Burnout and stress may negatively impact BT's work performances, more specifically, their treatment integrity. Acceptance and Commitment Training (ACT) may be a useful tool to address the private events as well as the covert and overt behaviors associated with burnout and stress. The purpose of this study was to investigate the effects of an ACT intervention on improving treatment integrity and reducing work-related burnout and stress amongst BTs. Four BTs participated in an ACT workshop, and their treatment integrity as well as their burnout and stress levels were measured prior to and following the ACT workshop. Treatment integrity increased for all participants, suggesting that ACT-based interventions may be an effective approach to improving work performance (i.e., treatment integrity) amongst BTs who may experience workplace burnout and stress.

Asunto(s)

RESUMEN

Objective: There is a lack of brief rating scales for the reliable assessment of psychotherapeutic skills, which do not require intensive rater training and/or a high level of expertise. Thus, the objective is to validate a 14-item version of the Clinical Communication Skills Scale (CCSS-S). Methods: Using a sample of N = 690 video-based ratings of role-plays with simulated patients, we calculated a confirmatory factor analysis and an exploratory structural equation modeling (ESEM), assessed convergent validities, determined inter-rater reliabilities and compared these with those who were either psychology students, advanced psychotherapy trainees, or experts. Results: Correlations with other competence rating scales were high (rs > 0.86-0.89). The intraclass correlations ranged between moderate and good [ICC(2,2) = 0.65-0.80], with student raters yielding the lowest scores. The one-factor model only marginally replicated the data, but the internal consistencies were excellent (α = 0.91-95). The ESEM yielded a two-factor solution (Collaboration and Structuring and Exploration Skills). Conclusion: The CCSS-S is a brief and valid rating scale that reliably assesses basic communication skills, which is particularly useful for psychotherapy training using standardized role-plays. To ensure good inter-rater reliabilities, it is still advisable to employ raters with at least some clinical experience. Future studies should further investigate the one- or two-factor structure of the instrument.

RESUMEN

Objective: To examine whether the outcomes of a chat-based suicide-prevention helpline could be improved by training counselors in motivational interviewing (MI). Methods: In a pre- and post-test design, visitors of a chat-based suicide prevention helpline received either the Five-Phase Model (treatment as usual [TAU]) or MI. They completed a pre- and post-chat questionnaire on several suicide-related risk factors. Linear mixed modeling was used to estimate the effect of the condition. Furthermore, the treatment proficiency of newly trained counselors was assessed using MI-Scope. Results: A total of 756 visitors and 55 counselors were included in this study. The visitors showed an improvement in suicidal ideation and psychological risk factors after a chat conversation. However, there were no significant differences between the MI and TAU conditions (ß = 0.03, 95% CI [-0.23-0.30], p = 0.80). The treatment integrity indices showed that the counselors mostly used MI-consistent techniques but were unable to strategically employ these techniques to evoke enough change talk. Conclusions: MI and TAU led to comparable outcomes in a chat-based suicide prevention helpline. The effectiveness of MI might improve by intensifying or improving the training of counselors, keeping the process of engaging more concise or offering visitors multiple sessions of MI.

RESUMEN

The finding that differential reinforcement of alternative behavior (DRA) is efficacious at 80% integrity when continuous reinforcement is programmed for alternative responding may have contributed to a perception that integrity at 80% or above is acceptable. However, research also suggests that other interventions (e.g., noncontingent reinforcement) may not remain effective at 80% integrity. The conditions under which 80% integrity is acceptable for common behavioral interventions remains unclear. Therefore, we conducted two human-operant studies to evaluate effects of 80% integrity for interventions with contingent or noncontingent intermittent reinforcement schedules. During Experiment 1, we compared noncontingent reinforcement (NCR) and DRA when implemented with 80% integrity. During Experiment 2, we compared 2 variations of DRA, which included either a ratio or interval schedule to reinforce alternative behavior. Results replicated previous research showing that DRA with a FR-1 schedule programmed for alternative responding resulted in consistent target response suppression, even when integrity was reduced to 80%. In contrast, neither NCR nor interval-based DRA were consistently effective when implemented at 80% integrity. These results demonstrate that 80% integrity is not a uniformly acceptable minimal level of integrity.

Asunto(s)

RESUMEN

Procedural fidelity is the extent to which components of an intervention are implemented as designed. Procedural fidelity is measured as a dependent variable and manipulated as an independent variable. In research and practice, procedural-fidelity data should be collected, monitored, and reported. Procedural fidelity as an independent variable has been investigated in humans using parametric analyses, and the current article summarizes some of the research conducted on the effects of procedural-fidelity errors in behavior-reduction and skill-acquisition interventions. Connections were drawn to applied animal researchers and the work of animal behavior practitioners to produce implications for practice with human and animal clients and suggestions for future research. Further, there are multiple ways to measure procedural fidelity, and different conclusions can be drawn based on the measure and computation method. The current article describes procedural-fidelity measures that are most applicable to animal behavior researchers and professionals.

Asunto(s)

RESUMEN

Treatment integrity is the extent to which procedures are implemented in a manner consistent with their prescribed protocols and is necessary for reaching accurate conclusions regarding functional relations between dependent (i.e., behavior) and independent (i.e., the environment) variables. Several studies assessing the frequency that studies report treatment integrity have been conducted. However, no review has included articles from Behavior Analysis in Practice. Thus, the current study reviewed Behavior Analysis in Practice between 2008 and 2019 to assess the frequency of studies reporting treatment integrity data. A total of 193 articles consisting of 205 studies met the inclusionary criteria for this review. Ninety-six studies (46.83%) reported treatment integrity data, compared to 193 (94.15%) that provided interobserver agreement data. Additionally, 98 studies (47.80%) were considered high risk for treatment implementation inaccuracies. Recommendations and implications for research and practice are discussed.