How reliable are therapeutic competence ratings?
- Assessments of psychotherapeutic competencies play a crucial role in research and training. However, research on the reliability and validity of such assessments is sparse. This study aimed to provide an overview of the current evidence and to provide an average interrater reliability (IRR) of psychotherapeutic competence ratings. A systematic review was conducted, and 20 studies reported in 32 publications were collected. These 20 studies were included in a narrative synthesis, and 20 coefficients were entered into the meta-analysis. Most primary studies referred to cognitive-behavioral therapies and the treatment of depression, used the Cognitive Therapy Scale, based ratings on videos, and trained the raters. Our meta-analysis revealed a pooled ICC of 0.82, but at the same time severe heterogeneity. The evidence map highlighted a variety of variables related to competence assessments. Further aspects influencing the reliability of competence ratings and regarding the considerable heterogeneity are discussed in detail throughout theAssessments of psychotherapeutic competencies play a crucial role in research and training. However, research on the reliability and validity of such assessments is sparse. This study aimed to provide an overview of the current evidence and to provide an average interrater reliability (IRR) of psychotherapeutic competence ratings. A systematic review was conducted, and 20 studies reported in 32 publications were collected. These 20 studies were included in a narrative synthesis, and 20 coefficients were entered into the meta-analysis. Most primary studies referred to cognitive-behavioral therapies and the treatment of depression, used the Cognitive Therapy Scale, based ratings on videos, and trained the raters. Our meta-analysis revealed a pooled ICC of 0.82, but at the same time severe heterogeneity. The evidence map highlighted a variety of variables related to competence assessments. Further aspects influencing the reliability of competence ratings and regarding the considerable heterogeneity are discussed in detail throughout the manuscript.…
| Author details: | Franziska KühneORCiDGND, Ramona Meister, Ulrike MaassORCiD, Tatjana Paunov, Florian WeckORCiDGND |
|---|---|
| DOI: | https://doi.org/10.1007/s10608-019-10056-5 |
| ISSN: | 0147-5916 |
| ISSN: | 1573-2819 |
| Title of parent work (English): | Cognitive therapy and research |
| Subtitle (English): | results of a systematic review and meta-analysis |
| Publisher: | Springer |
| Place of publishing: | New York |
| Publication type: | Article |
| Language: | English |
| Date of first publication: | 2019/11/16 |
| Publication year: | 2019 |
| Release date: | 2023/06/01 |
| Tag: | adherence; assessment; competency; psychotherapy; therapist competence |
| Volume: | 44 |
| Issue: | 2 |
| Number of pages: | 17 |
| First page: | 241 |
| Last Page: | 257 |
| Organizational units: | Humanwissenschaftliche Fakultät / Strukturbereich Kognitionswissenschaften / Department Psychologie |
| DDC classification: | 1 Philosophie und Psychologie / 15 Psychologie / 150 Psychologie |
| Peer review: | Referiert |
