The efficacy of manualized Cognitive Behavior Therapy conducted by student-therapists treating Danish youths with anxiety using a benchmark comparison

Participants in both the ST and PT group consisted of Danish youths and their parents, seeking help regarding youth anxiety. Assessment and treatment were conducted at Centre for the Psychological Treatment of Children and Adolescents (CEBU), Department of Psychology and Behavioral Science, Aarhus University, Denmark. Procedures for assessment and inclusion were identical for both groups. Inclusion criteria were that participants (7-16 years) met the criteria for an anxiety disorder according to DSM-IV as the primary diagnosis. Exclusion criteria were psychosis, untreated attention deficit hyperactivity disorder (ADHD), intellectual disability, severe behavior disorders, eating disorders, and substance abuse. Participants were encouraged not to engage in other forms of treatment or change psychopharmacological medication during the treatment period. All families signed written consent forms regarding the use of their data in scientific publications, audio-visual recording, and information exchange (e.g. physicians and teachers).

Participants in the PT group were assessed for eligibility from January 2011 to April 2012, and consisted of 56 youths aged 7 to 16 years and their parents, who were enrolled in a randomized waitlist controlled trial (24). The randomization procedure was done in three blocks of six groups per block stratified to age groups (7-9, 10-12, 13-16), resulting in three treatment groups for each age group in both the treatment and waitlist condition (24). All participants were treated by PT. In total, 56 mothers (100%) and 55 fathers (98.2%) participated in the treatment.

Participants in the ST group were assessed for eligibility from August 2014 to September 2015 and consisted of 54 youths aged 7 to 12 years and their parents. All enrolled participants received treatment in nine separate groups, with six participants in each group. Participants were allocated in groups corresponding to their age, four groups with youths aged 7 to 9 years, and five groups with youths aged 10 to 12 years. All participants were treated by ST. In total, 54 mothers (100%) and 49 fathers (90.7%) participated in the treatment.

The attrition rates were low in both groups for the clinical assessment. In the ST group, two (3.7%) and four (7.4%) participants were lost at post and follow-up (FU) clinical assessment, respectively. In the PT group, no participants were lost to post or FU clinical assessment. Missing responses on the questionnaires were generally low for the youths, mothers, and fathers at post (n: 4 youths (7.4%); 2 mothers (3.7%); 1 father (2.0%) and FU (n: 5 youths (9.2%); 3 mothers (5.5%); 8 fathers (16.3%)) in the ST group. Missing response rates were also low in the PT group among youths, mothers, and fathers at post (n: 1 youth (1.7%) and FU (n: 4 youths (7.1%); 1 mother (1.7%); 4 fathers (7.2%). There were no significant differences between the groups relating to attrition or missing responses.

The Anxiety Disorder Interview Schedule for DSM-IV, Parent and Child Versions (ADIS-C/P) (25) was utilized as the primary measure. The ADIS-C/P is a semi-structured diagnostic interview assessing youth anxiety disorders based on the criterions from the DSM-IV. Information is gathered by separate interviews with the youths (ADIS-C) and their parents (ADIS-P). A Clinician Severity Rating (CSR) is given, ranging from 0 (no interference) to 8 (extreme interference). A CSR of 4 or above indicates a disorder, while a CSR of less than 4 is considered subclinical. The most impairing ADIS diagnosis was considered the primary diagnosis. Diagnostic interviews were conducted by psychologists or graduate students trained in the use of ADIS-C/P. The training consisted of a two day workshop, watching two gold standard ADIS interviews of parents and children, observing one live interview conducted by an experienced interviewer, and conducting one ADIS interview observed by an experienced ADIS interviewer.

The ADIS-C/P has previously proven to be a reliable instrument for assessing anxiety disorder symptoms and diagnoses based on the DSM-IV, with good to excellent test-retest reliability (26). In the RCT study used as the PT group benchmark, an interrater reliability check of the Danish version of ADIS-C/P was conducted by two trained assessors watching and rating 22 (20.2%) of the video-recorded baseline interviews (24). The interrater reliability (Cohen’sκ) for the primary anxiety diagnosis was 0.77. The intraclass coefficient for the CSR of the primary anxiety diagnosis was 0.69. The education, procedure, and supervision of the assessors in the ST group were identical to the assessors in the PT study. No interrater reliability check was conducted for the ST group (24).

The Spence Children’s Anxiety Scale (SCAS) is a self-report rating scale assessing youth anxiety symptoms (27). The scale consists of 44 items (including six positive filler items), rated from 0 (never) to 3 (always). Within the scale are six subscales based on the anxiety disorders from DSM-IV: social phobia (SoP), panic disorder (PD) and agoraphobia (AP), separation anxiety disorder (SAD), generalized anxiety disorder (GAD), obsession compulsive disorder (OCD), and specific phobia (called fear of physical injury [SP]). The Danish version of the SCAS has demonstrated excellent internal consistency for the total scale in a clinical sample of youths with anxiety disorders, good test-retest reliability after two weeks and three months and good convergent and divergent validity (28). For the current study, the internal consistency for the total SCAS was good, and within the range of previous findings (α= 0.88). The SCAS parent version (SCAS-P) includes the same items as the SCAS, without the six filler items, and is administered and scored like the SCAS (29). The Danish version of SCAS-P has shown good internal consistency for both mothers (α= 0.87) and fathers (α= 0.87) in a clinical sample (Arendt et al., 2014). The current study showed good internal consistency for both mothers (α= 0.90) and fathers (α= 0.88).

The Child Anxiety Life interference Scale (CALIS) measures life interference and impairment associated with anxiety (e.g. in areas like school, at home or with friends) from the youth and parents perspective (30). The CALIS consists of 10 items where the youth reports the interference of their own life. The parent version of the CALIS (CALIS-P) consists of two subscales examining the interference of their child’s life (9 items) and of their own life (9 items). The scale is scored from 0 (not at all) to 4 (a great deal). CALIS and CALIS-P has previously demonstrated acceptable internal consistency for youths and parents, moderate-to-high test retest reliability, significant interrater reliability, and good convergent and divergent validity in a Australian population (30). The internal consistency of CALIS and CALIS-P for the current study was good for youths (α=0.85), and excellent for mothers (α= 0.91) and for fathers (α= 0.92).

The Experience of Service Questionnaire (ESQ) is a measure used to assess the youth’s and parents’ satisfaction with the treatment (31). There are separate versions for youths, with seven items, and parents, with 10 items, rated 0 (not true), 1 (partly true), or 2 (true).

The Cool Kids program is a manualized generic group CBT program for youth with anxiety disorders, involving both youths and their parents (32). The treatment aims to reduce levels of anxiety by focusing on teaching youths to recognize their emotions, restructure negative automatic thinking, and gradually confront feared situations (i.e. exposure). A systematic review and meta-analysis of 16 studies exploring the effects of the Cool Kids program, showed a moderate pre-post effect size for all the studies combined (g= 0.65). The study also included 11 studies comparing the Cool Kids program with a control group, which yielded a moderate between groups effect size (g= 0.54) (33). The efficacy and effectiveness of the Danish version of the Cool Kids program has also been found effective in treating anxiety disorders among youths (13,24).

The treatment in the current study consisted of 10 2-hr weekly group sessions, over approximately 12 weeks. The treatment sessions were typically divided so that the youths and parents were introduced to new techniques and worked with different exercises separately with assistance from the therapists. In addition to learning and assisting with the strategies taught to their children, the parents also received psycho-education in parent management strategies (34,35).

All sessions took place at CEBU, except for thein vivosession (Session 8) which took place at a local shopping center where the families and therapists practiced exposure relevant to the youths’ anxiety. Three months after the end of treatment, participants were offered a 1-hr booster group session.

The treatment was identical for both the PT and ST group. Treatment in the PT group was conducted by a psychologist, and assisted by three graduate psychology students (helped with practical tasks, and assisted families during in-session assignments). Treatments in the ST group were conducted solely by three graduate students.

Student Therapists: 24 graduate psychology students functioned as therapists in the ST group. They were in their first or second year of their master program. All students had received training in the Cool Kids program (10-hr workshop) and had assisted in one or two treatment groups using the Cool Kids program where they had been assigned to one family, and assisted in treatment assignments and exposure exercises. The ST had no other prior experience with the Cool Kids program. They received weekly (2-hr) group supervision throughout the treatment. The group supervision included a maximum of six ST, led by a psychologist with extensive experience with the Cool Kids program. During each supervision session, three ST presented their treatment case, their assigned families, their progress and possible challenges in treatment. The cases were discussed within the group, and the future work with the youth and their families were planned (e.g. planning gradual exposure or homework). Each ST presented their case, a minimum of two times during the treatment course.

Professional Therapists: two psychologists functioned as therapist in the PT group. They were both trained and supervised in the use of the Cool Kids program by an authorized psychotherapy specialist who had either developed or received training in the Cool Kids program at Macquarie University. The psychologists had five, and one year prior clinical experience, respectively. In each of the treatment groups, three graduate psychology students assisted the psychologist in treatment. Differences in therapist effects were explored for the primary treatment outcomes, and no significant differences were found between the two therapists.

Families had prior to assessment, referred themselves to CEBU, in response to recommendations from other health institutions, friends, advertisements, and the clinic’s webpage. The families where required to send a description of the youth’s anxiety symptoms, and relevant families were called in for assessment using the diagnostic ADIS-C/P interview. Families who were offered and accepted treatment completed electronically administered rating scales prior to treatment start.

Participants were assessed, at pre-treatment (pre), post-treatment (post), and at a three-month follow-up (FU), using the ADIS-C/P interview and electronically administered rating scales. Assessors were masked to youth’s prior diagnosis at post- and three-month follow-up assessments.

The analyses included all participants with pre-treatment data. Missing data were controlled for by using Little’s Missing Completely at Random test. Results indicated that the missing data were missing completely at random. Missing data at post or FU were accounted for by using the last observation carried forward approach, bringing forward the participant’s last observation (i.e. pre or post measures) replacing the missing values at post or FU. All analyses and results were conducted based on the intention to treat sample (ITT).

Participant’s baseline demographics, clinical characteristics and scores on outcome measures were compared between the two groups using independent samplest-tests and Fisher’s exact test as appropriate (i.e. gender).

The magnitude of change (effect size [ES]) within groups was estimated using Hedges’g, to control for the small sample size (36). Effect sizes were calculated for changes from pre to post and pre to FU on all measures of continuous anxiety outcomes.

Pre to post and pre to FU comparisons between the two groups were based on degree of change on continuous outcome measures following treatment. Repeated-measure, mixed-model analyses of variance (ANOVA) were conducted with group (ST vs PT) as the between-subjects variable and time (pre- vs post-treatment/and pre- vs three-month follow-up) as the within subjects variable. The magnitude of change between groups was calculated using partialη².

The proportion of youths meeting criteria for reliable change (RC) and clinical significant change (CSC) on the primary measure of youth anxiety (i.e. SCAS and SCAS-P) was calculated, in accordance to Jacobs and Truax criteria, using age and gender-specific Danish norms to calculate individual cut-off scores (28,37). The RC represents the amount of change required to make the change statistical significant. The CSC represents a change that is both statistically significant (RC) and clinically significant. Clinically significant change requires a change in means crossing a cut-off score between a clinical and non-clinical population, according to Jacobson and Truax (1991). Clinical cut-off scores were defined as the mid-point between clinical and non-clinical mean-scores from norms reported in the study of Arendt et al. (2014). Clinical significant change (CSC) was defined as the proportion of those scoring above the clinical cut-off before treatment, achieved RC, and scored below clinical cut-off after treatment (37). Reliable change and CSC calculations were compared between the two groups using Fischer’s exact test.

The proportion of participants that were free of their primary anxiety diagnosis and all anxiety diagnosis at either post-treatment or at the three-month follow-up were compared across the two groups using Fisher’s exact test.

All statistical analyses were carried out using Statistical Product and Service Solutions (SPSS) version 22.0.0.0 (38), and all ES were calculated using Comprehensive Meta-Analysis Version 3 (CMA) (39).

Tests of normality were conducted using the Shapiro–Wilk test of normality, as well as a visual inspection of the data, to assess the normal distribution for the primary and secondary measures in both groups. All scores were approximately normally distributed in both the ST and PT group, with the exception of the mothers SCAS-P score in the PT group (W= 0.96,p= 0.04) with a skewness of 0.70 (SE = 0.32) and kurtosis of 0.47 (SE = 0.63). By visual inspection, one score was detected as an outlier. By removing the outlier, the mothers SCAS-P scores in the PT were normally distributed (W= 0.96,p= 0.08) with a skewness of 0.51 (SE = 0.32) and kurtosis of 0.08 (SE = 0.63). The analysis was conducted with and without this outlier, resulting in no significant changes in outcome. The outlier was therefore dropped, and analysis conducted without the outlier.

Baseline demographic and diagnostic characteristics for the current study at pre-assessment are presented in Table1. There were no significant differences between groups regarding gender distribution, use of medication, type of primary anxiety disorder, or CSR for primary anxiety disorder. However, there was a significant difference in youth mean age, between the ST (M= 10.18, SD = 1.46) and PT group (M= 11.82, SD = 2.49;t(89) = 4.24,p< 0.005, 95% CI [0.87, 2.41]) where participants in the PT group were significantly older. There was also a significant difference in CSR for all anxiety disorders between the ST (M= 4.60, SD = 1.33) and PT group (M= 5.32, SD = 0.95;t(95) = 3.27,p< 0.005, 95% CI [1.16, 0.28] ) where the CSR was significantly higher for participants in the PT group compared to the ST group.

No significant differences in means between the groups were found for the SCAS or SCAS-P. Youth reported CALIS was significantly higher in the ST group (M= 15.10, SD = 8.09) compared to the PT group (M= 11.90, SD=7.35;t(108) = 2.17,p= 0.03, 95% CI [0.28, 6.12]). There was also a significant difference for fathers score on CALIS-P (self-reported interference of parents life) between the means of ST (M= 10.16, SD = 5.56) and PT (M= 7.85, SD = 6.09;t(102) = 2.00,p= 0.05, 95% CI [0.14, 9.39]) were the fathers ratings were higher for the ST group. No other significant differences were found for CALIS and CALIS-P (Table2).

To avoid the risk of creating biased estimates of the baseline data, no baseline adjustment or analysis controlling for these differences were conducted. The differences were taken into account in the interpretation and discussion of the results (40,41).

Changes in CSR for the primary anxiety diagnosis presented significant and large ES for both the ST (g= 1.36,p< 0.01) and PT (g= 1.51,p< 0.01) group. The within group ES were also significant and large for changes in CSR for all anxiety diagnoses for the ST (g= 1.82) and (g= 2.52) PT group. Both groups also showed a significant reduction in all the self- and parent-reported symptoms of anxiety, with most ES ranging from medium to large (ES range:g= 0.61-2.52). One exception was the CALIS-P (fathers self-reported interference of parents life) in the PT group were the ES was small (g= 0.32), but still significant.

The pre- to post-time-by-group comparison of the CSR revealed no interaction effect between the two groups for the primary anxiety diagnosis (CSR primary:F1,108 = 0.18,p= 0.72,η²= 0.01), or for all anxiety diagnosis (CSR all:F1,108 = 0.52,p= 0.47,ηp2= 0.01). Furthermore, the time-by-group comparison of youth and parent self-reported-ratings showed no interaction effect between the two groups, for any of the self-reported outcome measures (Table2).

The within group ES for changes in CSR for primary anxiety diagnosis remained significant from pre to FU in both treatment groups (ST:g= 1.88, PT:g= 1.81) group, both showing a higher ES than for pre to post. The CSR for all anxiety diagnosis also remained significant for both groups (ST:g= 2.51, PT:g= 3.27), and showed a higher ES in change at three-month follow-up than at post-treatment. All self-reported ES remained significant at three-month follow-up and there was an increase in the ES for most of the self-reported measures in both groups.

The pre to follow-up time-by-group comparison found no significant interaction effect between the two groups, regarding the CSR for primary anxiety diagnosis (CSR primary:F1,108 = 1.53,p= 0.22,η²= 0.01) or CSR for all anxiety diagnosis (CSR all:F1,108 = 0.13,p= 0.72,ηp2= 0.00) (Table3). No significant interaction effects were found for any of the self-reported anxiety measures (Table3).

Only CALIS youth for the ST group showed a significantly lower ES at three-month follow-up (g= 0.81), when compared to the ES from post-treatment (g= 0.99) (Table3).

Post-treatment comparison revealed no significant difference regarding number of youths free of primary diagnosis between the ST (27 [50.0%]) and the PT (37 [66.1%]) group (χ²= 2.92,p= 0.12). However, there was a significant difference regarding number of youths free of all anxiety disorders between the ST (14 [25.9%]) and PT (27 [48.2%]) group (χ²= 5.84,p= 0.02) (Figure1).

FIGURE 1

Note.CSR = Clinician Severity Rating, SCAS = Spence Children’s Anxiety Scale

At three-month follow-up, there was an increase in number of participants free of their primary diagnosis for both the ST (40 [74.1%]) and PT (43 [76.8%]) group, and there was no significant difference between groups in number of participants free of primary anxiety diagnosis (χ²= 0.11,p= 0.83). There was also an increase in number of participants free of all anxiety diagnosis at three-month follow-up for both the ST (25 [46.3%]) and PT (33 [58.9%]) group, and there were no difference between the groups in number of participants free off all anxiety diagnosis at three-month follow-up (χ²= 1.76,p= 0.25).

At post-treatment, a significantly larger proportion of youths in the ST group achieved RC but not CSC on the mothers ratings (SCAS-P Motherχ²= 6.79,p= 0.01). There were no other significant differences between the two groups in the proportion of participants achieving RC or CSC.

At three-month follow-up, there was a significant larger proportion of the youths in the PT group who achieved RC but not CSC (SCAS Youthχ²= 6.70,p= 0.02). Consequently there was also a significant larger proportion of the youths in the ST group who achieved CSC compared to the PT group (χ²= 6.11,p= 0.02). These differences between the ST and PT group, on CSC, was also significant for the mothers responses (SCAS-P Motherχ²= 4.66,p= 0.04). Again, there were no other significant differences between the groups achieving RC and CSC on SCAS or SCAS-P (Table4).

According to answers on the ESQ, families in both the ST and PT group were highly satisfied with the treatment. For the statement ‘The treatment helped me/my child’ 43 (87.8%) youths, 44 (83.0%) mothers, and 40 (85.1%) fathers scored 2 (true) in the ST group, while 44 (83.0%) youths, 42 (77.8%) mothers, 41 (77.4%) in the PT group scored 2 (true). For the statement ‘I had confidence in my/our therapists’ 44 (89.8%) youths, 51 (96.2%) mothers, and 44 (93.6%) fathers scored 2 (true) in the ST group, while 46 (86.8%) youths, 53 (98.1%) mothers, and 48 (90.6%) fathers in the PT scored 2 (true).

Independentt-test revealed that there was no significant difference between the total satisfaction score in the ST and PT group, for youths (t(100) = —-0.92,p=0.362, 95% CI [−0.40, 1.09]) mothers (t(105) = −0.22,p= 0.834, 95% CI [−0.58, 0.73]) or fathers score (t(98) = −1.92,p= 0.058, 95% CI [−0.03, 1.78]).