How to measure the impact of the COVID-19 pandemic on quality of life: COV19-QoL – the development, reliability and validity of a new scale

The current pandemic of severe acute respiratory syndrome coronavirus 2 (SARS-COV-2) caused by the COVID-19 virus has already (as of May 27, 2020) resulted in 5,555,737 cases and 350,212 deaths worldwide (European Centre for Disease Prevention and Control, 2020). The scientific community has focused on finding an effective cure to stop the pandemic. Evidence from previous large-scale health outbreaks suggests that this type of event has a tremendous impact not only on physical health, but also on mental health and quality of life in general (Sim & Chua, 2004). This affects the whole population, both healthy people and those considered as vulnerable groups (Holmes et al, 2020).

Evidence from the current COVID-19 pandemic on mental health in the general population suggests that slightly less than 10% of interviewed participants in lockdown in China reported moderate to severe level of stress and that youger people, between 18 and 30 years old, reported to be more emotionally affected compared to the other age groups (Zhang & Ma, 2020). It was also found that medical workers involved in the treatment of COVID-19 showed levels of anxiety that were correlated with poorer sleeping patterns (Xiao et al, 2020).

People with pre-existing mental health conditions, including severe mental illness, are particularly at risk of developing or exacerbating existing symptoms (Holmes et al, 2020). Therefore, it is of high priority to monitor and measure the effect of the pandemic on this group, in order to provide prompt care and to develop interventions. Urged by the need to measure the impact of COVID-19 on mental health, to the best of our knowledge, three measures have been developed thus far: the Fear of COVID-19 Scale (FCV-19S; Ahorsu et al, 2020), the Coronavirus Anxiety Scale (CAS; Lee, 2020) and the Perception of threat from COVID-19 questionnaire (Pérez-Fuentes et al, 2020). The FCV is a 7-item scale developed on an Iranian general population (N = 717) with no formal diagnosis of mental health disorder; the scale showed good reliability and cuncurrent validity. The items cover psychological and physiological anxiety symptoms due to the COVID-19, and a 5-point Likert scale on the level of agreement is used to answer the questions (Ahorsu et al, 2020). The CAS is a brief mental health screener for dysfunctional anxiety associated with COVID-19. The scale, developed on US general population (N = 775), includes five items covering social, psychological attitudes and functional impariment and uses a 5-point Likert scale, with a mix of frenquency, likelihood and level of satisfaction types. The screener overall was found to have good reliability and validity (Lee, 2020). Finally, the questionnaire on perception of threat from COVID-19 was developed on 1014 Spanish adults and consists of five items adapted from the Brief Illness Perception Questionnaire (using a Likert scale on the level of agreement); psychometric properties of the scale, particularly internal consistency, were found to be acceptable, but not robust (Pérez-Fuentes et al, 2020). What these measures have in common is that they are short, similarly structured, focused on common mental health problems, tested on the general population and freely available. We are not aware of a measure that would cover the impact of the pandemic on the main areas of quality of life related to mental health. None of the available developed measures have been used with people with pre-existing severe mental illness who are likely to be among the most vulnerable members of society.

In this paper, we present ‘The COV-19 – Impact on Quality of Life (COV19-QoL)’, a scale aiming at capturing the effect of COVID-19 on people's quality of life. The scale is not primarily designed for people diagnosed with COVID-19.

The main goal of the study is to examine reliability and construct validity of the scale. Validation of the scale was conducted on non-clinical and clinical samples in order to examine whether its factor structure will be the same (i.e. stable) among people from the general and clinical population (to be more specific, among people with severe mental illness). Another point was its reliability check: will the scale be sufficiently internally consistent in both populations? Apart from that, we wanted to examine the perceived decrease in quality of life due to the COVID-19 pandemic in the general and clinical population.

The prerequisites for conducting PCA have been met in both the non-clinical and clinical samples (KMO = .886, χ²(15) = 4443.36, p < .001 and KMO = .862, χ²(15) = 517.90, p < .001, respectively). Based on both Kaiser–Guttman's and Cattell's scree plot criteria, one principal component emerged based on the manifest variables (item responses) in both samples. The variances explained by the extracted components in non-clinical and clinical samples are 64.13% (with eigenvalue of λ = 3.53) and 58.88% (λ = 3.85), respectively. Communalities (h²) and factor loadings (r_iF) are shown in Table 1.

As displayed in Table 1, the explained variance of each item (i.e. h²) was over .4. All factor loadings were greater than .6, with the greatest one calculated for the item “I feel more tense than before” (r_iF = .878 and .861, respectively).

Cronbach's alpha coefficient of the scale for the data from the non-clinical sample was α = .885 and for the clinical one it was equal to .856. Table 2 shows the results of item analysis (corrected item-total correlation, squared multiple correlation [SMC] and alpha coefficient if the item was deleted) conducted for both samples.

It can be noticed (Table 2) that all corrected item-total coefficients of correlation were above .5, with the highest value (item no. 4) being equal to .800 (non-clinical sample) and .762 (clinical sample). The explained variance (SMC) of the total scores ranged from .292 (item no. 1) to .676 (item no. 5) for the non-clinical sample, while it ranged from .274 (item no. 1) to .618 (item no. 4) for the clinical sample. In the non-clinical sample, if the first item was deleted, Cronbach's alpha coefficient would increase to .892. However, deletion of this item would reduce content validity of the scale because it covers quality of life in general.

All intercorrelations of the COVID19-QoL scale were of moderate to high magnitude, positive and statistically significant (p < .001; Table 3). This indicates that all items should be part of the scale. Furthermore, the mean inter-item correlation in the non-clinical and clinical samples was MIC = .561 and .500, respectively. According to Clark and Watson (1995), ideally, this value should be between .20 and .50. An equal or a slightly greater value of MIC compared to the upper bound of this interval indicates a high homogeneity of the scale in both samples.

Based on the figures shown in Table 4, participants from both samples, on average, perceived that the COVID-19 had, among the tested domains, the greatest impact on their quality of life (M = 3.34 and 3.07, respectively). The lowest perceived impact was found for the mental health domain in the non-clinical sample (M = 2.63) and for personal safety in the clinical sample (M = 2.06). However, similar mean values are found for mental health (M = 2.07) and levels of depression (M = 2.09) in the clinical sample. It seems that participants from the non-clinical sample perceived greater impact of the COVID-19 pandemic on their quality of life in general compared to those from the clinical sample (M = 2.91 vs. M = 2.40). The perceived impact is somewhat below the theoretical mean of a 5-point scale. In addition, the range of participants’ scores on each item and the whole scale was 1 to 5, therefore being the same as the theoretical range for 5-point Likert scales.

As displayed in Table 5, six and three out of seven correlation coefficients calculated between participants’ age and scores on the COV19-QoL items (along with their total scores) were, although small (from −.057 to −.158 and from .141 to .146), statistically significant in the non-clinical and clinical samples, respectively.

Only one participant in the clinical sample reported symptoms of COVID-19 (either a new continuous cough or temperature greater than 37.8°C) and none of the patients reported confirmed diagnosis of COVID-19.

In addition, the statistical significance of gender differences was not examined due to imbalance in the number of males and females in the non-clinical sample.

This is the first study reporting preliminary findings about the measurement of the quality of life impact of COVID-19 in the general population (i.e. non-clinical sample) and in people with pre-existing severe mental disorders (i.e. clinical sample). The work presents the initial phase of the development of a new scale titled COV19-QoL.

First, it was shown that the scale is, as has been expected, a unidimensional instrument. It measures perceptions of quality of life deterioration in relation to the COVID-19 pandemic. Hence, the scale has a good construct validity. Second, it was found to be internally consistent because the Cronbach‘s alpha coefficient was above .700 (as recommended by e.g. Aron et al, 2013). Next, all the corrected item-total correlations were above .30 (the criterion reported by Nunnally & Bernstein, 1994), which indicated that all the items should be part of the scale. Lastly, the values of SMCs for all the items were greater than .40 (the SMC lower bound recommended by Whitley & Kite, 2013), which confirmed that it is an internally consistent instrument. All the data presented here refer to both non-clinical and clinical samples. Hence, regardless of the population from which participants were being sampled, the latent structure of the COV19-QoL scale was the same and its reliability was estimated as very good.

Findings from the non-clinical sample indicate that the greatest impact of the pandemic was found for the quality of life area in general, whereas mental health was impacted the least. The first result could be explained by the broad meaning that quality of life has, which is comprehensive of both mental and physical health-related factors, as well as several other environmental features (e.g. accommodation, regular food supply, etc.) and economic, cultural and political influences (e.g. Priebe et al, 2015). The perceived least impact on mental health could be explained by the assumption that people are more focused on their physical health now because the COVID-19 poses an imminent and obvious risk to physical health. However, based on previous data, we can speculate that greater impact on mental health could possibly be found in the aftermath of the pandemic (e.g. Douglas et al, 2009). In the clinical sample, the lowest perceived impact was found in personal safety, then in mental health and the feeling of depression. The total value of the COV19-QoL in the clinical sample indicates a moderate level of the negative impact of the pandemic on quality of life, while greater impact was perceived by patients from the non-clinical sample. Regarding the potential decrease in quality of life in persons with severe mental illness, Fonseca et al (2020) stated that people with schizophrenia spectrum disorders are (due to COVID-19 pandemic) at greater risk for not only relapse, but also physical health deterioration (especially because some of them have difficulties to seek and receive help from medical professionals). Anxiety and depression, as well as dysfunctioning coping mechanisms (e.g. substance misuse), are the most expected mental health conditions associated with the pandemic (Holmes et al, 2020). This could be the reason why people with severe mental disorders (e.g. schizophrenia spectrum), compared to the general population, might be less prone to perceive negative impact on their mental health.

Notably, age was associated with some of the items in both non-clinical and clinical samples. However, the direction of the association was different: in the non-clinical sample, it emerged that more negatively perceived impact was associated with a younger age (all negative correlations emerged with the exception of item no. 6 about personal safety), which is in line with previous data about the association of negative feeling related to COVID-19 with age (Zhang & Ma, 2020); the opposite was found in the clinical sample, wherein the level of perceived negative impact of the pandemic correlated with older age.

When discussing these findings, it is necessary to keep in mind the socio-cultural and psychological background of populations from the participating countries. The participants were recruited from four countries in the Balkans, southeastern Europe. In recent decades, the region has been struck by war, poverty and isolation, which has inevitably imposed tough lifestyle conditions and serious life-threatening situations. Hence, the lifestyle changes due to the COVID-19 pandemic (e.g. related lockdown) might not be considered as disruptive for people in the Balkans as for people in other parts of Europe. However, it might be expected that the current situation might be a traumatic reminder of the wartime period, thus causing retraumatisation for some people and that this could be seen in upcoming months.

The scale is short, easy to use, acceptable and feasible to people with severe mental illness. These points could be regarded the as main strengths when it comes to wide application of the COV19-QoL scale.

The COV19-QoL scale has the potential to facilitate both clinical work and research on the impact of the current and future pandemics on people. Clinicians can use it for their routine assessments with patients during the pandemic and to monitor symptoms across time. There is no reason to believe that people with other diagnostic categories such as anxiety, mood or personality disorders would not find it feasible and acceptable, so the scale can potentially be used with these populations as well as with the general population.