The Dimensional Assessment of Personality Pathology – Short Form for Adolescents (DAPP-SF-A): normative data for Flemish adolescents aged 16 to 21 years

The Flemish Study on Parenting, Personality, and Development is a longitudinal Belgian study that started in 1999 [see Prinzie et al. (28) for a complete description of the sampling procedure]. At the start of this study, a proportional stratified sample of elementary-school-aged children attending regular schools was randomly selected. Strata were constructed according to geographical location (province), sex, and age. Participants were 680 native Flemish families (92.5% two-parent families) with an elementary-school-aged child. The present study concerns the seventh wave, collected in 2012, when 425 participants completed the Dutch versions of the DAPP-SF-A (21) and the Youth Self-Report (YSR) (29) online. The YSR was included as an instrument to identify youths that scored within the clinical range. Sample characteristics are presented in Table 1.

To compare Flemish and Dutch norms, we analyzed a subsample (n = 288, 67.8% of the total amount) that matched as closely as possible to Tromp and Koot’s (21) Dutch non-referred sample (i.e., a community sample from which clinically referred children were excluded). The clinical cut-off score of the YSR at the 80th percentile was shown to be the most accurate cut point for discriminating between referred and non-referred children (29). To this aim, we excluded adolescents potentially meeting criteria for psychopathology. That is, we excluded 89 (20.9%) adolescents scoring above the 80th percentile of either the Externalizing or Internalizing broadband scale of the YSR (29). Furthermore, we matched the non-referred sample of Tromp and Koot (21) exactly in terms of gender (49% girls) by randomly excluding an additional 48 girls from the adjusted sample. The mean age in the adjusted sample was 18.6 years (SD, 1.16 years). Note that in the current wave the DAPP-SF-A was administered for the first time. Therefore, for this instrument no data were available from this cohort at a younger age. In the non-referred sample of Tromp and Koot the mean age was 14.6 years (SD, 1.7 years).

Dimensional Assessment of Personality Disorders – Short Form for Adolescents. The DAPP-SF-A (21) consists of 144 items that are rated on a five-point Likert scale from 1 (“Very unlike me”) to 5 (“Very like me”). Eight items form a validity scale. The remaining 136 items are grouped in 18 lower-order dimensions (Table 2). These lower-order dimensions form the four higher-order factors: Emotional Dysregulation, Dissocial Behavior, Inhibitedness, and Compulsivity. Evidence for the good psychometric qualities and clinical validity of the DAPP-SF-A has been reported by Tromp and Koot (21,25).

Youth Self-Report. The YSR is a questionnaire measuring a wide variety of emotional and behavioral problems and consists of 112 items, which are scored on a three-point scale: 0 = “Not true”, 1=“Somewhat or sometimes true”, and 2 = “Very true or often true” (29). Five of the nine lower-order dimensions are grouped into two higher-order factors: Externalizing Problems and Internalizing Problems. The YSR has been widely used in both clinical and academic fields and its psychometric properties have been well established (29).

To provide norming data for a Flemish community-based sample of adolescents, means and standard deviations of the sample were computed for the lower- and higher-order dimensions of the DAPP-SF-A. Furthermore, internal consistency reliabilities for these dimensions were measured by Cronbach’s alphas. Item-rest correlations (r_ir) were examined to investigate item functioning. r_ir’s for individual items and r_ir means for lower- and higher-order dimensions were compared against a threshold value of < 0.20 as described by Kline (30) and Nunnally (31). Normality of scale distribution was investigated visually using histograms for boys and girls separately. Gender differences were examined by independent sample t-tests and, in case of non-normality, the non-parametric equivalent Mann–Whitney’s U test, to determine whether the results were influenced by non-normality.

With regard to our second aim, determining to what extend these norms differ from Dutch normative data, two-tailed independent sample t-tests were conducted to compare means on the lower- and higher-order dimensions in the current sample to those provided by Tromp and Koot (21). Effect sizes for gender and sample differences were computed as Cohen’s d and interpreted by the guidelines of Cohen (32): 0.2 = small, 0.5 = medium, and 0.8 = large. The influence of participants’ age on scores was investigated by computing correlations between age and lower- and higher-order dimensions. These correlations were interpreted by Cohen’s guidelines (0.1 = small, 0.3 = medium, 0.5 = large). All analyses were conducted with SPSS 24 (33).

Table 2 shows the internal consistency reliabilities, mean r_ir, means and standard deviations of the community sample, both for boys and girls, combined and separated. Cronbach’s alpha for both higher- and lower-order dimensions was generally high, ranging, respectively, from 0.81 for Inhibitedness to 0.97 for Emotional Dysregulation (median, 0.88), and from 0.71 for Conduct Problems to 0.87 for Affective Instability, Self-Harm, and Restricted Expression (median, 0.85). The mean r_ir’s for dimensions also showed a good level of internal consistency, ranging from 0.41 for Inhibitedness to 0.57 for Compulsivity for the higher-order dimensions, and from 0.44 for Conduct Problems to 0.67 for Self-Harm for the lower-order dimensions. Individual item functioning for the lower-order dimensions was good, all items had a r_ir larger than 0.20 (item 17, “Sex is not important in my life”, had the lowest r_ir of 0.24). For the higher-order dimensions individual item functioning was also good, 99.3% of the items had a r_ir of > 0.20. Exceptions with low-to-fair item functioning were Emotional Dysregulation (item 44, “When I was very upset, I took a drug overdose”, r_ir = 0.17; item 35, “I am not very organized when I do things”, r_ir = 0.23; and item 48, “I feel it is important that other people see me”, r_ir = 0.24), Dissocial Behavior (item 97, “As a child I started fires that destroyed things of other people”, r_ir = 0.24); and Inhibitedness (item 127, “If there was nobody in my life, I would wish that I had someone close to me”, r_ir = 0.21; and item 100, “I need a love relationship”, r_ir = 0.23).

The means in the sample showed considerable differences in overall levels of personality pathology, defined by the different lower-order dimensions, even when taking into account differences in number of items for each scale. For instance, the mean score on Self-Harm was the lowest (i.e., mean, 7.31/6 = 1.21), whereas Compulsivity showed the highest mean score (i.e., mean, 21.56/8 = 2.70). Overall, boys scored significantly higher on 10 of the 18 lower-order dimensions (d ranged from 0.21 for Rejection to 0.79 for Conduct Problems); namely, all lower-order dimensions of Dissocial Behavior and Inhibitedness and four lower-order dimensions of Emotional Dysregulation (i.e., Oppositionality, Social Avoidance, Suspiciousness, and Self-Harm). Girls scored significantly higher than boys on Affective Instability [t(423) = −3.62, p < .001, d = −0.35] and Insecure Attachment [t(423) = −3.64, p < .001, d = −0.35], which are lower-order dimensions of Emotional Dysregulation. For dimensions that showed evident non-normality (i.e., Self-Harm and Conduct Problems, which showed heavily skewed distributions), Mann–Whitney U tests were computed. These non-parametric tests resulted in p-values within the same range as the independent t-tests.

In the combined sample, correlations between age and the higher- and lower-order dimensions were small to negligible and statistically non-significant, ranging from −0.08 for Narcissism to 0.09 for Affective Instability. Intimacy Problems was an exception and showed a small significant correlation of −0.13 (p < 0.01), indicating that relatively fewer problems were reported by older adolescents.

The sample differences of the adjusted (matched) sample with the non-referred sample of Tromp and Koot (21) are presented in Table 3. The results indicate that Flemish adolescents scored significantly higher than Dutch adolescents on the higher-order dimension Compulsivity and the lower-order dimensions Submissiveness, Anxiety, Insecure Attachment, and Narcissism (d ranged from 0.13 to 0.47). Conversely, Dutch adolescents showed significantly higher scores on the higher-order dimensions Dissocial Behavior and Inhibitedness and the lower-order dimensions Cognitive Distortions, Oppositionality, Suspiciousness, Self-Harm, Stimulus Seeking, Callousness, Conduct Problems, and Intimacy Problems (d ranged from −0.22 to −1.78).

Adequate and appropriate norming data are highly relevant in the use of all assessment instruments supporting diagnosis, treatment planning, and evaluation in clinical practice. The primary goal of this study was to provide normative data for the DAPP-SF-A in a Flemish community sample, aged 16 to 21 years. The results showed that the internal consistency reliabilities of both the DAPP-SF-A lower- and higher-order dimensions were acceptable to high and were comparable in magnitude to Tromp and Koot (21). In addition, the sample means showed differences in the overall level of personality pathology defined by the different lower-order dimensions, with the lowest mean scores for Self-Harm and the highest mean scores for Compulsivity. Differences between boys and girls were small to medium. In general, boys scored significantly higher than girls on lower-order dimensions associated with externalizing problems (e.g., lower-order dimensions of Dissocial Behavior) as well as on lower-order dimensions associated with Inhibitedness. Girls only scored significantly higher than boys on Insecure Attachment and Affective Instability. This pattern of gender differences, in which boys tend to show more externalizing problem behavior and girls show more internalizing problems, resembles earlier findings, for instance, in a large-scale general population survey in the United States by Eaton et al. (34).

The second goal of the present study was to compare an adjusted, non-clinical, subsample of the Flemish data with a non-referred sample of Dutch adolescents (21). Significant differences between both countries were found with small-to-large effect sizes; Flemish adolescents scored higher on the Compulsivity, Submissiveness, Anxiety, Insecure Attachment, and Narcissism scales and Dutch adolescents scored higher on the Dissocial Behavior, Inhibitedness, Cognitive Distortions, Oppositionality, Suspiciousness, Self-Harm, Stimulus Seeking, Callousness, Conduct Problems, and Intimacy Problems scales. The results of this study are in support of earlier findings by Tromp and Koot (21) concerning adequate reliability and the use of separate norms for boys and girls. Furthermore, the results warrant the use of differential norms for Dutch and Flemish adolescents.

For the larger part, we confirmed previous findings by Tromp and Koot (21) concerning gender differences in terms of direction and magnitude. Although for some lower-order dimensions of Emotional Dysregulation, such as Submissiveness, we did not replicate Tromp and Koot’s finding that girls showed significantly higher mean scores. In general, this was only the case for differences with a small effect size in Tromp and Koot’s study. The difference between the sample size of the current study (n = 425) and the sample size in Tromp and Koot (n = 1596) explains the lack of power for tests of small effects.

In the present study differences that were found between Flemish and Dutch adolescents might be explained from the perspective of Hofstede’s cultural dimensions theory (35,36). This theoretical framework defines societal differences for many countries along six different dimensions that are Power Distance, Uncertainty Avoidance, Individualism/Collectivism, Masculinity/Femininity, Long/Short Term Orientation, and Indulgence/Restraint. Hofstede’s research shows that for the dimensions Power Distance, Masculinity, and Uncertainty Avoidance, the Belgian population scores are higher than the Dutch (35). Scoring high on Power Distance means that members of a society generally accept a hierarchy of power differences easier, whereas a high level of Uncertainty Avoidance implies that risks are avoided and rules adhered (36). This could be an explanation for the lower scores on Oppositionality, Suspiciousness, and Dissocial Behavior and the higher scores on Submissiveness, Identity Problems, and Anxiety in Flemish adolescents. Additionally, the fact that Flemish adolescents scored higher on Compulsivity and Narcissism could be explained by both the Hofstede dimension of Masculinity, a dimension that encompasses competition, achievement, and success, and the dimension of Uncertainty Avoidance that partly reflects people’s desire for fixed habits and rituals. Hofstede’s theory illustrates that the evaluation of behavior as problematic or pathological is not only relative to individual characteristics, such as gender and age, but also to what within a society is defined as deviant. Clearly, for some behavior, such as suicidality or criminal behavior, intervention is always required regardless of cultural conventions.

The current study showed a large difference between Flemish and Dutch adolescents on the lower-order dimension Intimacy Problems with relatively high scores in the Dutch non-referred population that could not be explained from Hofstede’s theory. Remarkably, the current results did match the originally reported non-referred population means in the DAPP-A manual (mean, 18.3; SD, 5.2) (25). On closer inspection, the non-referred population means reported by Tromp and Koot (21) equaled those reported in the manual, except for the means on Intimacy Problems and, consequently, its higher-order dimension Emotional Inhibitedness. Apparently, these means were adjusted after reverse-scoring errors were discovered (personal communication with Tromp). The question remains which of the different means on Intimacy Problems, and consequently Emotional Inhibitedness, are correct. It might be the case that the original report in the manual is correct because the pattern of means in that report – with higher scores on Intimacy Problems for the clinical samples compared to the non-referred sample – is more intuitive and resembles earlier findings concerning the adult version of the DAPP-SF in non-referred and clinical populations (15,37), whereas in Tromp and Koot’s (21) study this pattern is reversed. Future research must provide further insight into the magnitude of intercultural differences in personality pathology defined by the DAPP-SF-A.

A strength of this study was that we were able to derive empirical results from a broad sample of the general Flemish population and compare those directly to the original Dutch results. Given the substantial differences between the Dutch and Flemish samples, the presented Flemish norms could serve as a benchmark for the application of the DAPP-SF-A within the Flemish culture. In line with our findings, the adult version of the DAPP-BQ and DAPP-SF provides separate Dutch and Flemish norms (27). Relating to the current study that explored the DAPP’s cross-gender and cross-cultural variation in norms, the topic of measurement invariance of the DAPP model, deserves further attention. Studies that investigated the factorial stability of the PID-5 model across populations (clinical vs. non-clinical) (38), or across different cultures (39), have shown promising results; future studies on the DAPP could build on these findings.

A limitation of the current study was that the mean age of participants in the Flemish sample was higher compared to the sample of Tromp and Koot’s (21) study; as we did not have the original data from Tromp and Koot we could not control for age in both samples. However, Tromp and Koot (21) reported small-to-negligible effects of age on all Dissocial Behavior and Emotional Dysregulation scales within Dutch adolescents. Likewise, in the current sample all correlations between age and the DAPP-SF-A dimensions were small to negligible and statistically non-significant. The only exception was a significant but small correlation between age and Intimacy Problems, indicating that Intimacy Problems decreased with age. This negative correlation was mainly caused by the relatively low scores of the group aged 21 years, whereas the younger age groups scored quite constant over the years. Correspondingly, Tromp and Koot (21) reported a non-significant difference in Intimacy Problems between the age groups below 16 versus above 16 years. Summarizing, we hypothesize that in general the age difference will have a minimal effect on the comparison between both samples. Secondly, the samples were not exactly matched in terms of clinical referral status. In the current Flemish community study, clinical referral was not an exclusion criterion. Therefore, an approximate match was attempted with the YSR cut-off values for Internalizing and Externalizing Problems as external exclusion criteria. The clinical cut-off values of the YSR have shown to have good sensitivity and specificity (29). Nonetheless, the approach could have resulted in a suboptimal representation of non-referred adolescents. However, we hypothesize that potential bias was restricted to a minimum, given that the scores in the adjusted sample only slightly deviated from the original sample. In addition, the current clinical cut-off values of the YSR, as mentioned in the Method section, resulted in an exclusion of 20.9% of the participants, which is consistent with prevalence data of psychopathology in the general population (12,13). Future studies on the DAPP should be conducted in Flemish clinical adolescent population to provide clinical norms.

The main clinical implication of the current findings comprises the use of separate clinical cut-off values on the DAPP-SF-A in the Flemish-speaking part of Belgium, which were shown to be significantly different from Dutch values. If, for example, a score on the dimension Compulsivity of a Flemish adolescent is compared to Dutch norms, one could falsely interpret the self-reported scores as deviant because of the lower cut-off in the Dutch population. Conversely, if a Flemish score on the dimension Oppositionality is interpreted using Dutch norms, there is a chance on a false-negative because of the higher cut-off in the Dutch population.

To conclude, the DAPP-SF-A shows satisfactory reliability in a Flemish community sample of adolescents. Furthermore, given the differences between boys and girls the use of separate norms for gender is highly relevant. Finally, substantial differences with the Dutch general population norms warrant the use of separate norms in Flemish adolescents. In future research, cultural effects must be accounted for when norms are developed for the DAPP-SF-A in other countries.