Psychometric evaluation of the Swedish Child Sheehan Disability Scale in adolescent psychiatric patients

The objective of this study was to evaluate the psychometric properties of the Swedish CSDS and CSDS-P in a sample of adolescent psychiatric patients and their parents.

This psychometric study explored the reliability and validity of the Swedish versions of the CSDS and CSDS-P in an adolescent psychiatric sample. Reliability was assessed by exploring internal consistency and factor structure. The sensitivity and specificity, and appropriate cut-offs of CSDS and CSDS-P to identify patients with at least one diagnosis versus those without a diagnosis, were tested. A functional impairment is a prerequisite for just about every diagnosis in psychiatry. Therefore, any diagnosis versus no diagnosis was chosen as the outcome. Sensitivity (to detect a diagnosis) was set to at least 80%, while specificity (to detect those without a diagnosis) was set to at least 50%.

To investigate concurrent validity, the Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (K-SADS-PL) and the Strengths and Difficulties Questionnaire (SDQ) were used for comparisons (8, 9, 26-29). Spearman’s rho correlations were used to calculate the concurrent validity of the total score of CSDS/CSDS-P and K-SADS. Moreover, the correlations between the total scores for CSDS and CSDS-P were analyzed, as well as the correlations between the items for “school” and “friends” from the two scales.

The participants were recruited from an ongoing cohort study of adolescents seeking help for psychiatric problems. Recruitment was performed between September 2011 and June 2013. All new psychiatric outpatients aged 12–17 years (N = 202) and their parents from two clinics in the County of Västmanland, Sweden, were invited to participate. Exclusion criteria were non-Swedish-speaking adolescent/parents or need of in-patient care at the time of the K-SADS-PL interview. A total of 125 (61.9%) patients participated, n = 53 (42.4%) male.

Of these, 107 adolescents provided full information on the K-SADS and CSDS, and 104 provided full information on the SDQ, while their parents completed the CSDS-P (n = 107) and SDQ (n = 104) using the Electronic Psychiatric Intake Questionnaire (EPIQ) (30) (Figure 1).

FIGURE. 1.

The remaining 18 patients were excluded from the study because of a lack of information. Internal drop-out analysis showed no significant difference in age or sex between those who completed the study (n = 107, 42% male, mean age 15.7 [SD 1.5] years) and those who did not (n = 18, 44.4% male, mean age 15.7 [SD 1.5] years).

The study was conducted in accordance with the Declaration of Helsinki (31) and approved by the Regional Ethics Committee of Uppsala (Dnr 2008/214).

At the first visit to the clinic, verbal information about the study was given, and informed consent was collected from participants. The adolescents answered the electronic psychiatric intake questionnaire (EPIQ), and parents answered the EPIQ-Parent version. Both versions include the SDQ. At the second visit, within one week, the adolescents and parents were interviewed with the K-SADS-PL. Thereafter, the adolescents and parents reported on the CSDS and the CSDS-P, respectively.

The interviewers were trained to perform the K-SADS interviews. The inter-rater reliability (IRR) was good to excellent, both before and during the period of data collection. The IRR rate was calculated for each diagnosis and each interviewer, and mean IRRs were calculated for the group and all diagnoses. Overall kappa for the group was .84 (range .54–1.00). Mean kappa was .89 for depression disorders, .79 for ADHD, and varied from .64 to 1.00 for anxiety disorders, as reported previously (32).

The CSDS and CSDS-P have been translated into Swedish. The original CSDS by Whiteside assesses general functioning in relation to “fears and worries” (9). In the Swedish version, this was changed to general functioning in relation to “troubles and feelings”. Three independent experts did the translation and back-translation for proper language adaptation and finally, the authors of this study approved the final translation.

For layout, see the Swedish CSDS and Swedish CSDS-P in Swedish and English in the Appendix.

The Swedish Child Sheehan Disability Scale (Swedish CSDS). The Swedish CSDS measures general functioning, in relation to “troubles and feelings”. It includes three items about functioning at school, with friends and at home. The questions are “How much do troubles and feelings cause problems for you when you are in school and when you do homework?”, “How much do troubles and feelings cause problems for you when you are with your friends?” and “How much do troubles and feelings cause problems for you when you are at home?”. The score of each item ranges from 0 (“not at all”) to 10 (“very, very much”), resulting in a total scale range of 0–30.

To define patients with low functioning, Leon et al. suggested a cut-off score of 5 for the SDS in adult primary care populations (25). In addition, Sheehan et al. proposed cut-off scores of ≤1 on each item of the SDS and <5 on the total score as remission criteria (14). By contrast, Whiteside studied the CSDS and the CSDS-P in two samples with a primary anxiety disorder and analysed the mean values of the most common anxiety diagnoses and the effect of the treatment without reference to a remission cut-off (9). Based on the SDS studies, a cut-off of 5 was chosen to define functional impairment on the CSDS (14, 25)

The Swedish Child Sheehan Disability Scale-Parent Report (Swedish CSDS-P). The Swedish CSDS-P (parent version) is analogous to the Swedish CSDS (child version) and has three items that assess the level of interference with the parent’s functioning at work and in social and family life: “The symptoms have disrupted your work”, “The symptoms have disrupted your social life” and “The symptoms have disrupted your family life/home responsibilities”. The CSDS-P also contains an additional two items concerning the impact on the child: “The symptoms have disrupted your child’s schooling” and “The symptoms have disrupted your child’s social life”. The score of each item ranges from 0 (“not at all”) to 10 (“very, very much”), resulting in a total scale range of 0–50. In Whiteside’s study (9), the CSDS-P total mean (M) and standard deviation (SD) were M = 22.7 (SD = 11.9) for children with anxiety diagnoses and M = 2.89 (SD = 5.5) for the community controls.

Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (K-SADS-PL). The K-SADS-PL is a semi-structured diagnostic interview that integrates information from children, parents and the clinical judgement of the interviewer (26, 27). The K-SADS-PL consists of a screening interview and eight supplements for 23 diagnostic domains (26). It has excellent IRR and test–retest reliability supporting its ability to generate valid child psychiatric diagnoses (26, 27, 33, 34). In the K-SADS-PL, comorbidity is presented as the number of specific diagnoses. The K-SADS diagnoses are grouped as follows: any depression (Major Depression, Major Depression with Psychotic Features*), any anxiety (Separation Anxiety, Specific Phobia, Social Anxiety Disorder, Panic Disorder, Agoraphobia, Generalized Anxiety, Obsessive-Compulsive Disorder, Post-traumatic Stress Disorder and Acute Stress Disorder) and any ADHD (ADHD combined, predominantly inattentive, predominantly hyperactive/impulsive and unspecified). At the time of the assessment, the only patient with bipolar diagnosis presented a manic episode. Moreover, a general K-SADS-PL symptom summation index was calculated by adding up the obtained scores during the screening interview and supplements, in a way analogous to that used by Sonnby et al. and Olofsdotter et al. in their previous studies (32, 35). The different ranges for symptom summation indexes were as follows: depression (0–114), manic episode (0–52), psychotic disorder (0–103), panic disorder (0–56), agoraphobia (0–6), separation anxiety disorder (0–27), social anxiety disorder (0–9), specific phobia (0–56), generalized anxiety disorder (0–35), obsessive-compulsive disorder (OCD; 0–98), enuresis (0–9), encopresis (0–9), anorexia nervosa (0–12), bulimia nervosa (0–30), ADHD (0–57), oppositional defiant disorder (0–24), conduct disorder (0–45), tic disorders (0–56), autism (0–43), tobacco use disorder (0–14), alcohol use disorders (0–514), drug use disorder (0–53) and post-traumatic stress disorder (0–66). Scores range from 0 to 2 or 0 to 3 for each symptom depending on the item, and the total range is 0–1488.

Furthermore, the total K-SADS function summation index was calculated by adding the obtained scores of summations of function questions of each specific diagnosis. Ranges were as follows: depression (0–6), manic episode (0–10), psychotic disorder (0–8), panic disorder (0–6), separation anxiety disorder (0–6), social anxiety disorder (0–6), specific phobia (0–6), generalized anxiety disorder (0–6), OCD (0–16), ADHD (0–6), oppositional defiant disorder (0–6), conduct disorder (0–6), tic disorders (0–6), autism (0–6) and post-traumatic stress disorder (0–8). Each function question has a score range from 0 to 2, and the total K-SADS function summation index is 0–108.

Strengths and Difficulties Questionnaire (SDQ)–self-report and parent versions. The SDQ is a brief behavioural screening questionnaire (29). There are equivalent versions for adolescents and parents/teachers (28, 29). Goodman developed the SDQ based on the brief Rutter questionnaires and the Child Behaviour Checklist (28, 29, 36, 37). The Swedish translation of the SDQ parent version has been evaluated with the parents of children aged 6–10 years and deemed to be a reliable and valid measure (29, 38). Adolescent self-reports in Nordic populations show similar distributions of SDQ scores, and in a study of adolescent self-reports, the emotional subscale of the SDQ showed usefulness in differentiating depressed from non-depressed cases (39, 40).

The SDQ consists of 25 items grouped on five subscales, each with five items, with each item using a three-point Likert scale (0 = not true, 1 = somewhat true and 2 = certainly true) (28). The subscales produce scores for emotional symptoms, conduct problems, hyperactivity-inattention, peer problems and prosocial behavior. Some items are reverse scored, and prosocial behavior is considered protective.

There was no significant age difference between the internal drop-out group and the study group (U = 947, p = .910), and no significant sex difference (χ₂ = .036, p = .850). There was no significant age difference between males and females (U = 1202, p = .223). The most frequent diagnoses in the study sample were ADHD (55.1%), MDD (42.1%) and SAD (31%) (Table 1). Diagnoses were grouped into any depression (one patient with bipolar diagnosis, who presented a manic episode at the time of the interview, was not included in depression group), any anxiety, any ADHD or any other diagnoses (Table 1). Seven patients (6.5%) did not fulfil criteria for any diagnosis.

Five (4.6%) of patients were diagnosed with bipolar or psychotic disorders; bipolar depressive episode n=1, schizoaffective disorder manic episode n=1, and unspecified psychotic disorder n=3.

Factor structure and internal consistency.

The CSDS showed one component with an eigenvalue of 2.19, explaining 72.9% of the variance. Items loaded on one component, with a maximum for “school” of .868, for “friends”.855 and a minimum for “home” of .837. For the CSDS-P, the corresponding eigenvalue was 3.09, explaining 61.8% of the variance. Items loaded on one component with a maximum “for parent’s life at home” of .874, for “parent’s work” .861, for “parent’s social life”.784, for “child’s friends”.767 and a minimum for “child’s school” of .619. Internal consistency was Cronbach’s α of .81 for the adolescent scale and .84 for the parent scale.

The CSDS at cut-off 5 and the CSDS-P at cut-off 17 are suggested for the purpose of screening, considering a sensitivity above 80% and a specificity with a minimum of 50%. The CSDS at cut-off 5 showed a sensitivity of 85% (85 patients with low function out of 100 patients with diagnosis) and specificity of 71% (five patients without low function out of seven patients without diagnosis) and the CSDS-P at cut-off 17 showed a sensitivity of 81% (81 patients with low function out of 100 patients with diagnosis) and specificity of 57% (four patients without low function out of seven patients without diagnosis).

Correlations

CSDS/CSDS-P in relation to the K-SADS symptom summation index. (Not shown in tables). The correlation of the CSDS with the K-SADS symptom summation index was low (r_s = .332, p < .001). Furthermore, among the correlations between the CSDS and the symptom summation index for each diagnosis, only the correlation between the adolescent self-report of the CSDS and the summation index for MDD was moderate (r_s = .500, p < .001). Other correlations of the CSDS in relation to the summation index for panic disorder (r_s = .322, p < .001), the summation index for separation anxiety (r_s = .317, p < .317) and the summation index for anorexia (r_s = .311, p = .001) were low, and the rest were negligible.

The correlation between the CSDS-P and the K-SADS symptom summation index and the correlations of the CSDS-P with most diagnoses were negligible (r_s ≤ .300).

Correlations of the CSDS/CSDS-P and the total score K-SADS function summation index. There were negligible correlations for both scales in relation to the total K-SADS function summation index (r_s = .268, p = .006 and r_s = .275, p =.005). With respect to correlations between CSDS and the K-SADS function summation index of each diagnosis, the correlations that stood out were manic episode (r_s = .752, p = .031), psychotic episode (r_s = .741, p = .035), panic disorder (r_s = .495 p = .043) and social anxiety (r_s = .327, p = .028). Others were negligible or insignificant.

CSDS/CSDS-P in relation to the SDQ adolescent or parent version. Table 2 shows that the correlation between the CSDS total score and the SDQ adolescent subscale score for emotional problems and the correlation of the CSDS-P and the total difficulties score of the SDQ among parents were low. Prosocial sub-scale scores were not included in the correlation because these were not considered a direct measure of the level of difficulties.

Relationship between the total score on CSDS/CSDS-P and the number of diagnoses. The correlations between the CSDS total score and the number of diagnoses were low (r = .338, p < .0001); for parents, the correlations between the CSDS-P total score and the number of diagnoses were negligible (r = .256, p < .008).

Correlations of the CSDS and CSDS-P.

The correlation for the CSDS total score and CSDS-P total score was r _s = .375, p <.001. Additionally, the correlations between the CSDS “school” item and the CSDS-P “Child’s school” item and between the CSDS “friends” item, and CSDS-P “Child’s friends” were r _s = .354, p <.01 and r _s = .375, p < .01 respectively.

The study was performed in a clinical population with multiple diagnoses and most of the participants had comorbid disorders. This limits its generalizability in diagnosing specific populations. On the other hand, our results are not limited to patients with a specific diagnosis (27). An additional limitation is the absence of a comparative community sample to evaluate the psychometric properties of the scales in different settings. Moreover, the small number of undiagnosed patients as well as the small sample size decreased the accuracy of the measurement of sensitivity and specificity.

A further limitation in the design is that no other specific scales for measuring functioning were included. However, the K-SADS, a “gold standard” among diagnostic interviews, was used to identify diagnoses and to have a diagnosis imply a clinical level of dysfunction. Therefore, this assessment was considered to be a relevant reference for the CSDS. In addition, the interview was conducted first, and then the scale was completed, which may have influenced the performance of the scale.

The study compares the CSDS and CSDS-P against the sum of symptoms and number of diagnoses obtained by using the “gold standard” K-SADS-PL interview and the SDQ, an established scale for measuring strengths and difficulties. The CSDS and CSDS-P are self-report and parent-report scales, respectively. They are easy to use and help to identify dysfunction caused by mental health problems, and thereby identify individuals who need subsequent evaluation.