The shortened version of the Adolescent Stress Questionnaire (ASQ-S; Sweden): a validation study in United Kingdom adolescents

The deleterious effect of stress in adolescence has been well documented (1-3), and for that reason continued scrutiny of the measurement of stress in this developmental period is essential to public health. One measurement tool with considerable promise is the Adolescent Stress Questionnaire (ASQ; [4]). Results of variable-centred studies on the structural validity of ASQ scores have been unequivocal: the ASQ yields multivariate scores structured by the domain-specificity of stressors in adolescence (e.g.4-6). Specifically, the structural validity of ASQ scores has been substantiated by confirmatory factor analysis (CFA), intercorrelations that indicated related but distinctly different constructs, and convergent validity analyses. Nevertheless, on-going inquiries into psychometric properties of ASQ scores warrant our attention.

Byrne et al. (4) developed the 56-item Adolescent Stress Questionnaire as an index of stress consisting of items forming 10 different stress components or domains. The specific domains examined are the stress of: home life; school attendance; school performance; peer pressure; emerging adult responsibility; school/leisure conflict; romantic relationships; teacher interaction; future uncertainty; and financial pressures. Psychometric studies examining the properties of ASQ scores have reported mixed results with some supportive (4,7), and others reporting problems with both structural validity and internal consistency (8), test-retest reliability (9), and problematic factor loadings (6). Although (6) reported adequate fit indices among Northern Irish adolescents (CFI = 0.95; RMSEA = 0.08), these authors also reported concerns about the reliability (a= 0.50) and validity of ‘emerging adult responsibility’ scores specifically.

One drawback of using the ASQ is its length, particularly in studies where larger batteries of questionnaires are required to be administered. A shorter version of the ASQ might also be useful for mental health professionals who engage in screening adolescents in schools and other large agencies, such as the juvenile justice system. Accordingly, a number of researchers have proposed short versions of the ASQ, and results of these studies were in keeping with studies on the full version (10,11). One such abbreviated version of the ASQ was recently piloted in Sweden using a Swedish translation (11). Results of this study (11) indicated valid and reliable ASQ-S scores. In the present study, we aimed to use the same ASQ items selected therein (11), but retain the English version in a study among adolescents in Northern Ireland. This approach allowed us to determine whether or not the psychometrics of Anniko et al. (11) proposed model were sensitive to translation in language and, or, difference in European culture. The data we used were data previously used to examine the properties of the full ASQ (6).

The specific purpose of the present study is to examine the model fit, internal consistency, and construct validity of the ASQ-S (11). Specifically, ASQ-S scores were subjected to confirmatory factor analysis (CFA), and invariance testing by gender, stage of school of participants (junior versus middle high school), and type of school attended by participants (grammar [a more academic setting] versus secondary [a more vocational setting]). Further, construct validity was assessed by examining correlations between ASQ-S scores, and scores on four self-concept measures. Finally, construct validity included an analysis of the relationship between alcohol use scores and scores on ASQ-S domains, adjusted for other variables of interest.

The 27-items of the ASQ-S (11) were used for analyses. The scale measures stress in nine of the original ten ASQ domains: home life, school attendance, school performance, romantic relationships, peer pressure, school-leisure conflict, teacher interaction, future uncertainty, and finance. For each item, participants are asked, ‘How stressful do you find’ (e.g. going to school). Participants rate the level of stress experienced on a five-point Likert-type scale (1 = not stressful at all to 5 = very stressful).

The Adolescent Alcohol Involvement Scale (AAIS): the AAIS (12) is a 14-item self-report screening measure for alcohol abuse in adolescents. It is a compilation of previously verified indicators of alcohol misuse. It functions as a research tool, which helps identify adolescents whose alcohol use impacts adversely on psychological functioning, social relations, and/or family life. Questions are answered on a scale allowing for a highest possible score of 79. The psychometric properties of the AAIS have been demonstrated to be acceptable (13).

The Rosenberg Self-Esteem Scale (RSES; 14) is a 10-item self-report measure of global self-esteem. It consists of 10 statements related to overall feelings of self-worth or self-acceptance. The scale has demonstrated good reliability and validity across a large number of different sample groups (15). The RSES has been validated for use with substance users and other clinical groups, and is regularly used in treatment outcome studies. The scale has been validated for use with both male and female adolescent, adult and elderly populations (16).

The Self-Efficacy Questionnaire for Children (SEQ-C; 17) contains 21 items assessing three domains of self-efficacy: (a) academic self-efficacy; (b) emotional self-efficacy; and (c) social self-efficacy. Each subscale consists of seven items, and respondents rate their competence in each self-efficacy domain on a five-point Likert scale (1 = not at all; 5 = very well). SEQ-C subscale scores have been found to be structurally valid and internally consistent (a< 0.80) in Dutch-speaking, Belgian-speaking, and English-speaking populations (17,18).

Each participating school was supplied with parental consent forms. A form of ‘opt out’ consent was approved so that parents received detailed information on the study and were only required to respond if they were unhappy about their child’s participation. On the day of the study, each participating pupil was also asked to give their own informed consent, indicating their willingness to take part. Data were gathered under examination-like conditions with all participants from a given school completing the questionnaires simultaneously. Participants were issued with a set of response sheets, and all questionnaires were administered verbally by the researcher, allowing pupils with literacy difficulties to take part and also to help maximise the number of fully completed response sheets. Data collection took approximately 30 min in each school.

Confirmatory factor analyses (CFA) were performed using Mplus (version 8; 19). All additional analyses were conducted using SPSS v23. A number of statistical recommendations were used to determine model fit: (a) CFI ≥ 0.90, (b) TLI ≥ 0.90, (c) RMSEA =0.05 (good) to 0.08 (acceptable), and (d) SRMR ≤ 0.08 (20-23). Measurement invariance was determined by reductions in the comparative fit indexes across increasingly restrictive models: ΔCFI ≤ 0.01 and ΔTLI ≤ 0.02 (24). Invariance was tested by factor structure (configural), factor loadings (metric), and indicator/item intercepts (scalar). To aid the interpretation of variance explained we were guided by Ferguson’s (25) recommendations for ‘practical’ effect sizes, where 0.20 is the minimal value for interpretation, and 0.50 and 0.80 demark moderate and strong effects, respectively, forβand correlation coefficients (r). Ferguson (2009) suggested that for Hedge’sg, values of 0.41, 1.15, and 2.70 demark small, moderate, and strong effects, respectively.

Of the 610 completed questionnaires, 238 (39%) were completed by males, 337 (55.2%) were completed by attendees of Grammar schools, and 361 (59.2%) were completed by pupils in Junior High school. CFA results in Tables1to3provide summaries of the fit indices for a baseline model as well as tests of gender, school stage, and school-type invariance. Results showed that ASQ-S scores fit the models well (total sample:χ^2s-b(288) = 751.60, CFI = 0.958, TLI = 0.948, SRMR = 0.040, RMSEA = 0.051 [90% CI = 0.047, 0.056]), and that scores were gender, school-type (i.e. Grammar or Secondary), and age groups (i.e. Junior versus Middle High school) invariant. Table4displays the results of independent samplest-tests for gender and stress domains for both the ASQ-S, and the corresponding nine factors from the full ASQ, as well as internal consistency (Cronbach’sa), and mean inter-item correlations for each stress factor. In keeping with the stress literature, females scored significantly higher on most stress domains than males. However, none of the Hedge’sgvalues for the ASQ-S scores reached the recommended practical effect size (g< 0.41). Regarding the full ASQ comparisons, there was only one Hedge’sgvalue that attained the recommended minimal effect size, and that was for the stress of peer pressure. Further, theavalues for the ASQ-S suggest that stress domain scores were broadly internally consistent, although four did fail to achieve a value of 0.7, albeit by a small margin. In contrast, theavalues for all but one of the full ASQ factors were all < 0.70. Mean inter-item correlations pointed to satisfactory internal consistency for both scale versions.

Table5displays the results of bivariate Pearson’s correlations (two-tailed) between scores on ASQ-S (above the diagonal), and full ASQ domains, and both self-esteem, and three domains of self-efficacy. Although the coefficients between ASQ-S scores and criterion variables were all in the expected direction, fewer than half of them (13 out of 36) met the criteria for a recommended minimum practical effect. Interesting, there were a similar number of practically significant effect sizes (15 out of 36) when examining correlations between ASQ full scores and criterion variables.

Table6displays the results of a hierarchical regression examining the relationship between AAIS score and ASQ-S stress domains, adjusted for gender and school year (a proxy for age). In order to account for clustering at school level, school was dummy coded. TheR²value for the model indicated a moderate effect size (25). Results show higher AAIS scores were significantly associated with being in higher school year (i.e. being older), and with significantly higher stress of home life, school attendance, and financial pressure, but also with significantly lower stress of peer pressure. By far, the largest observed effect was for school year, and of the significant stress domains, only the effect for stress of peer pressure (negatively) was associated with alcohol use to a practical effect size (25).

The present study was a secondary analysis of data previously gathered and was purposively conducted to examine the psychometric properties of a newly developed and shortened version of the ASQ in the Swedish context (11). Given that comparative cross cultural studies in psychological constructs are important, we deemed it both useful and important to scrutinise the ASQ-S (Sweden) scores. However, just for clarity in the stress literature it is important not to confuse this particular ASQ-S with another recently validated shortened form of the scale (10). It is an unhappy coincidence that both have been so-labelled (i.e. ASQ-S), and while both have the benefit of being relatively short, and easy-to-administer, both are compromised by the fact that they do not measure stress in the full 10 domains originally used in the ASQ (4). The two short versions of the ASQ (10,11) differ in terms of the number of original ASQ factors retained (eight and nine, respectively [10,11]), and in terms of the overall number of original ASQ items retained (24 and 27, respectively [10,11]). However, notwithstanding this, there is considerable overlap between the two versions with 20 shared items from the original ASQ.

At a variable-centred level, results of CFA suggested that the ASQ-S (11) scores were both psychometrically valid and invariant by gender, school-type, and stage of school, and therefore the ASQ-S (11) appears to be a viable measure of adolescent stress across nine domains. However, the potential utility of the scale appears limited by one concern: results suggested limited convergent validity and internal consistency for ASQ-S scores. Regarding internal consistency, most estimates exceeded or came close to the 0.7 acceptable threshold. To some extent, the below optimal internal consistency estimates could be an artefact of the low numbers of items in the ASQ-S factors. Evidence for this may be seen in the fact that internal consistency estimates for factors using all original ASQ items were substantively higher than those for the ASQ-S. However, according to Streiner (26), there are at least three other issues to be considered here. First, the Swedish sample was more homogenous in terms of age, being drawn from only two school years compared to the present five years, a fact that is also relevant to the previously discussed stage of school variance. Second, Streiner argues thatavalues that are too high can indicate item redundancy, and thatacoefficients can be negatively impacted by there being fewer numbers of items in factors. Finally, Streiner pointed out that for scales that measure narrow characteristics, mean inter-item correlations ranging between 0.40 and 0.50 are adequate. In the case of the ASQ-S all but two of the mean inter-item correlations fell within this range.

In support of the ASQ-S, mean domain scores for gender supported the widely-reported finding that females report significantly higher levels of stress than males (e.g. 3,4,6). In the present study, while females scored significantly higher on six of the nine domains, none of the Hedge’sgvalues attained what Ferguson (2009) describes as a practically significant level (g< 0.41). Further, while a broadly similar pattern of results were observed using scores from all ASQ items in each of the nine factors, only one of those results attained this practically significant level. Similarly, in the AAIS regression model, only one of theβvalues for stress domains attained a ‘practically significant’ threshold. Taken together, we suggest that while the ASQ-S is psychometrically valid, and within the context of the limitations of coefficientadiscussed above, scores are internally consistent, the utility of a shorter scale has one drawback, so that a longer and more elaborate ASQ allows researchers to assess a breadth and depth of stress not afforded by a short version. It also allows for greater variance in responses and for better discrimination between groups. Therefore, while the ASQ-S may indeed be psychometrically valid and practically useful, one of its limitations may be a lack of explanatory power in terms of the way stress differs across populations.

Examination of correlation matrix (Table5) reveals that, broadly speaking, coefficients between ASQ-S factors were generally larger when using all ASQ items (below the diagonal), than was the case with the ASQ-S (above). This is not surprising, as more variance in stress would be expected to be accounted for when using a greater overall number of items. In terms of ASQ-S viability and utility, what was interesting was the fact that coefficients using scores based on the full ASQ yielded only three more practically significant (25) coefficients than was the case when using the ASQ-S.

There are a number of limitations to the present study. First, the ASQ-S data were gathered in the context of the full ASQ (4), and this may have influenced scoring of the items. Second, all data were self-reported and were collected in a school setting, a setting directly related to three of the ASQ-S domains (school performance; school attendance; teacher interaction). Third, we did not apply person-centred analyses, and for that reason we do not know how useful a multidimensional ASQ-S Swedish version would be among Northern Irish and Scottish adolescents. That is, a recent person-centred study on ASQ-S English scores indicated that while the instrument yielded multidimensional scores, adolescents tended to experience similar levels of stress across domains (1). Said another way, the multiple dimensions of ASQ-S scores did not have much practical value with regard to individual differences.

In summary, the present study suggests broad support for the ASQ-S with the caveat of reduced explanatory power when compared to the full ASQ. For that reason, we would not recommend clinicians use the ASQ with their clients. By contrast, where researchers are interested in investigating stress as part of a large battery of questionnaires, the ASQ-S may be a practically useful addition to the literature. This is the first validation study to the best of our knowledge, and further studies are required to substantiate these findings.