A questionnaire for rating health-related quality of life

The study was designed as multicentric and cross-sectional.

The new questionnaire (WB-HRQoL) was developed following the guidelines set by Robert F. DeVellis (8), through eight steps: (A) HRQoL in adults was determined as object of measurement, being one of the main treatment outcomes in both clinical research and clinical practice; (B) the item pool with completely new items was generated during two brainstorming sessions, one week apart; only the authors participated in the sessions and creation of the item pool, taking care to cover the domains of quality of life encompassed in the majority of other generic instruments: physical, psychical, social and environmental (members of the group that generated the items had the following profiles: two specialists of clinical pharmacology, one specialist of general surgery, a professor of Biochemistry, a professor of Social Pharmacy, five internal medicine specialists, a specialist of infectious diseases, a specialist of epidemiology, three community pharmacists, a specialist of anaesthesiology and reanimation, a specialist of clinical pharmacy and a general practitioner); (C) the Likert’s scale was chosen as format of measurement, with the following offered answers to statements reflecting quality of life: “I do not agree completely”, “I do not agree partially”, “I neither agree, nor do not agree”, “I agree partially”, and “I agree completely” (the answers were rated from 1 [“I do not agree completely”] to 5 [“I agree completely”]); (D) the initial pool of items was revised and corrected by the three-member expert committee composed of a general surgeon, an internal diseases specialist, and a clinical pharmacology specialist (these profiles were chosen to broadly cover both surgical and non-surgical aspects of health, since the scale was intended for use primarily at the secondary and tertiary health care levels); (E) in order to determine the level of respondents’ socially desirable behaviour, one validation item was included in the questionnaire that read “I always try to be good to other people.”; (F) the initial pool of items was tested in each study centre on 3-5 PhD students for clarity and comprehension, and after the pilot and a few minor changes final versions of the questionnaire were copied in Serbian, Croatian, Bosnian, and Montenegrin (the investigators from respective language areas adapted the items’ formulation to their language preferences), and prepared for reliability testing (Supplementary file); (G) evaluating the items and (H) optimizing the questionnaire length were then made as described below. A visual analogue scale for assessing HRQoL was also offered to the study patients for validation purposes.

Cognitive status of the study subjects was assessed by the Mini-Cog screening test (9). For the purpose of convergent validation of the WB-HRQoL the study subjects were offered short form (26 items) of the World Health Organization’s generic questionnaire for measuring HRQoL (WHOQoL BREF) in Bosnian (10), as well as Visual Analogue Scale (VAS) with a ruler from 0 to 100. Divergent criteria validation was made by the 10-item Emotional Regulation Questionnaire (ERQ) in Serbian (minimally adapted for Croatian, Bosnian, and Montenegrin), which measures two strategies of emotional regulation: cognitive reappraisal and emotion suppression (11). Permissions to use these supplementary questionnaires were granted before the start of the study.

The final Serbian, Croatian, Bosnian, and Montenegrin versions of the both new (WB-HRQoL) and supplementary (Mini-Cog, WHOQoL BREF and ERQ) questionnaires were tested for reliability on convenient sample of adults who were healthy, outpatients, or hospital patients, respectively, at the five study centres: Belgrade, Kragujevac (Serbia), Zagreb (Croatia), Sarajevo (Bosnia & Herzegovina) and Podgorica (Montenegro). The surveys took place from September 2019 to July 2020. The inclusion criteria were full consciousness, literacy, and age over 18. The exclusion criteria were pregnancy, lactation, major psychiatric diseases (mood disorders, psychoses, mental retardation), psychotropic drugs abuse, chronic alcoholism, and emergency conditions. The sample of the subjects was of consecutive nature, i.e. all subjects who came into the contact with an investigator on the survey day (satisfied inclusion and did not have exclusion criteria) were offered the questionnaires. During the first encounter one copy the questionnaires was completed by the investigators who were questioning the study subjects, and another copy was given to the subjects to complete it next day by themselves, at their homes, and send it back to the investigators. The second encounter was 15-30 days later, and on that occasion both HRQoL and supplementary scales were completed by the investigators who were questioning the patients, in order to test for temporal stability. The study was approved by the Ethics Committees of Clinical Center Kragujevac, Serbia (No 01-19-3041, date 19 July 2019.), Clinical Hospital Center, Zagreb, Croatia (No 02/21 AG, date 02 July 2019.), Clinical Center of Sarajevo University, Sarajevo, Bosnia & Herzegovina (No 10-01-58803, date 18 November 2019.), Medical Faculty, University of Montenegro, Podgorica, Montenegro (No 2240/3, date 15. November 2019.), and University of Belgrade – Faculty of Pharmacy, Belgrade, Serbia (No 1122/2, date 11 July 2019.). The patients signed an informed consent before the study onset and were treated with due respect and care, according to the principles stated in Declaration of Helsinki.

The reliability of the WB-HRQoL was tested threefold. First, internal consistency was assessed by calculating Cronbach’s alpha for the whole questionnaire. Second, the questionnaire was randomly split to two halves with the same number of questions, and Cronbach’s alpha for each of the parts was calculated. Based on the alphas of both parts, the number of questions, and the average correlation between questions in the two parts, the Spearman-Brown coefficient for the questionnaire as a whole was calculated by the “prediction” formula (12). Third, for each question the mean score and its variances were calculated, in order to check for the ability to measure the whole extent of possible answers.

Exploratory factorial analysis of the questionnaire was made in order to discover principal factors (13). Principal axis factoring (14) groups the items of a scale to a smaller number of factors describing majority of the variance of the responses to the scale items. Factors covering maximal variance are kept, and others are discarded. The amount of variance covered by each factor is measured by its eigenvalue. Assumptions of the factorial analysis were tested on the sample by the Kaiser-Meyer-Olkin measure of sampling adequacy and by the Bartlett’s test of sphericity. Then, the factors were extracted at first without rotation, conditional on Eigenvalues >1.0, and with the Scree-plot. Second, the referent axes were rotated obliquely by the Promax method, and another extraction was made, using the same criteria as for the unrotated solution. The extracted factors were than given names. The calculations were performed by the SPSS statistical software, version 18.0.

Content validity of the questionnaire was evaluated by an independent panel of three experienced clinicians at Clinical Center Kragujevac, Serbia: a general surgeon, an internal diseases specialist, and a clinical pharmacology specialist.

The criterion validity was tested by three methods: (1) comparing the WB-HRQoL score with the Visual Analogue Scale (VAS), (2) comparing the WB-HRQoL score with the WHOQoL BREF domains, 1st and 2nd item scores (convergent validity testing), and (3) comparing the WB-HRQoL score with the score of the Emotional Regulation Questionnaire (ERQ) (divergent validity testing). The correlations between scores of the questionnaires and/ or VAS values were calculated. The calculations were performed by the SPSS statistical software, version 18.0. External validity was tested by comparing the WB-HRQoL scores between study subjects having at least one chronic disease and the healthy subjects. The comparison was made by Mann-Whitney U test.

Temporal stability of the WB-HRQoL results was tested by second session of completing questionnaires by the investigators who repeatedly interviewed the study subjects 15-30 days after the first encounter. The study subjects were scheduled for the second encounter at the end of the first one.

After testing original 20 items from the questionnaire, and reviewing results of correlation matrix, mean values, variance, skewness and kurtosis of response distributions for each of the items, 1 item was removed, leaving final version of the WB-HRQoL questionnaire with 19 items. The removed item had a low correlation with other items (the Spearman’s correlation coefficients were between -0.15 and 0.15) and with the total score of the remaining 19 items (the correlation coefficient was 0.092). Cronbach’s alpha of the final version with 19 items was 0.905 when the scale was rated by the investigators. The intraclass correlation coefficient (single measures) was 0.301 (95% confidence interval 0.273–0.333; F=9.627, df1=486, df2=9234, p<0.001). Mean values of responses, standard deviations, skewness, and kurtosis for each item are shown in the Table 1.

After dividing the questionnaire by the split-half method the Spearman-Brown coefficient for the questionnaire as a whole was calculated by the Spearman-Brown “prediction” formula, and its value was 0.861. Since the Spearman-Brown coefficient did not drop below 0.7 after the split-half method, the satisfactory reliability of the questionnaire is further confirmed (theoretically this coefficient may take any value between 0 and 1). When the scale was rated by the patients themselves (at the first encounter), Cronbach’s alpha was 0.900.

The principal axis factoring method was used for exploratory factor analysis. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.918 and the Bartlett’s test of sphericity was significant (p<0.001). Using oblique rotation (Promax), four factors were extracted, explaining in total 46.7% of variance. The first factor bears 4.946 eigenvalues (11.96% of variance), the second 4.818 (11.66% of variance), the third 5.644 eigenvalues (13.66% of variance), and the fourth 3.889 eigenvalues (9.41% of variance). The rotated pattern matrix is shown in the Table 2. The items 1-3, 5, and 14 belong to the first factor, which reflects physical aspects of quality of life. The items 9-11 belong to factor 2, which describes environmental aspects of quality of life, and the items 4, 6-8, and 16-19 describe the psychical aspects of quality of life. Social aspects of quality of life are described by the questions 12, 13, and 15, which belong to the fourth factor. The four-factor structure is common to other generic HRQoL instruments, due to conceptual similarity of four facets of health: physical, psychic, social life, and life in relation to environment (15).

The questionnaire’s construct validity was assessed and endorsed by the panel of experts; a few questions were slightly re-phrased by the panel.

Non-parametric correlations between scores of the WB-HRQoL scale (when it was rated by investigators and by the patients themselves) and scores of the ERQ scale (when it was rated by investigators and by the patients themselves) were calculated to test the divergent criterion validity of the WB-HRQoL. Non-parametric correlations between scores of the WB-HRQoL scale (when it was rated by investigators and by the patients themselves), scores of the domains, the 1st and 2nd item of the WHOQoL BREF scale (when it was rated by investigators and by the patients themselves), and the VAS score were used to test convergent criterion validity of the WB-HRQoL. Non-parametric correlation was chosen due to non-normal distribution of majority of the scores. Spearman’s correlation coefficients are shown in the Multi-trait, multi-method matrix (Table 3). The Spearman correlation

coefficient is a measure of the strength and direction of association between two variables, based on the rank of individual values instead of actual values; it is nonparametric correlation coefficient, and its administration does not assume normal distribution of data within the variables.

When the WB-HRQoL scores were compared between healthy study subjects and those having at least one chronic disease, they were significantly higher in the former: 79 [16] vs. 73 [19] (Mann Whitney U test, p<0.001). This confirms the instrument’s discriminative ability.

The WB-HRQoL scale showed excellent temporal stability: when the rating (by the investigators) was repeated on the same patients 15-30 days later, the correlation between the scores (Spearman’s coefficient) was 0.842 (p<0.001). Cronbach’s alpha after the repeated rating was 0.898.

The definitive version of the WB-HRQoL scale with 19 items showed very good reliability, with Cronbach’s alpha at 0.905. The scale was also temporally stable, and satisfactory results were obtained for divergent and convergent validity tests. Factor analysis showed four domains of HRQoL, physical, psychical, social, and environmental. This questionnaire can help identify patients with a low quality of life, and it can be used for better therapy monitoring and improving quality of life.

Physicians need insight into the effects of a disease on their patients’ quality of life to choose the best therapeutic method and follow the patient’s response thereto. There is no original generic instrument for measuring HRQoL developed in Serbian, Croatian, Bosnian, or Montenegrin that could cover all aspects of regional understanding of HRQoL. However, the translations of instruments developed in other, mostly more developed, parts of the world never cover all aspects of quality of life specific for western Balkan nations.

Physical domain is present in almost all generic instruments for measuring HRQoL, and is valued even in circumstances when other domains are put aside, like emergency care situations (16). Key components of the physical aspect of quality of life are absence of pain and other discomforts, and the ability to have autonomous physical activity (17). The physical domain of our questionnaire has items that cover these key components, but also has two additional items covering quality of sleep and sex life, present in other quality of life scales like the WHOQoL BREF, too (10). Because of the major influence that items measuring pain, discomfort, and ability to move or work have on quality of life, some instruments minimize the effects of sleep and sex life (18); this was not the case with our questionnaire, which showed that in this domain questions about influence of sleep, sex life, mobility, and absence of pain or discomfort are equally important (similar loadings), underlining the importance of all these aspects in regional context. The mean values of the responses on items reflecting physical aspects of quality of life in our study were close to mean of the range of possible answers, variance was of acceptable size, and skewness and kurtosis suggested minor (but significant) detachment from normal distribution (see Table 1). Such results show good discriminative ability of the items from physical domain, and suggest their capacity to capture both mild, moderate, and severe decrease in health-related quality of life (8).

Main items within the psychic domain are addressing mood, anxiety, and overall feeling of balance, well known facets of mental health. However, it is interesting that this domain encompassed also certain items that are traditionally linked to environmental aspect: adjustment to environmental temperature, air, and feeling of financial or safety in general. This is probably caused by specificity of western Balkan countries, which are extremely low-trust societies, where inhabitants either lack or have minimal trust in their governments and fellow citizens (19, 20). In such cultural contexts people do not expect that their environment will be taken care of by institutions or other individuals, therefore HRQoL will depend on how they feel about their personal ability to cope with supposedly polluted air, inappropriate ambient temperature, crime, or financial instability. In other words, individual perception of the environmental aspect of quality of life will depend more on resiliency and mood than on the real situation in the external world.

Citizens of western Balkan societies frequently have a feeling of isolation (both personal and national) and are reluctant to admit their psychological difficulties, because they are aware of their stigmatizing potential (21). Such an attitude leads to perception of the world and life as something independent of the person itself (actually environmental), i.e. they principally see the world and life in general as beautiful or not, according to the prevailing public opinion. Accordingly, the items “life is beautiful “and “the world is beautiful “in our questionnaire slipped from psychical (what was expected when the items were constructed) to the environmental domain. Traditionally, also the social item “family relations” was attached to the environmental domain, probably reflecting emerging feeling of alienation within families; however, this speculation has yet to be confirmed. Three items within the social domain are oriented towards relations with friends and colleagues at work. The participants perceive two main areas of social interaction – work and entertainment.

From demographic and data about habits within the study sample we could see that smoking, drinking alcohol and coffee were widespread and could have influenced the study participants’ perception of HRQoL. Although the process of Europeanization is taking place in countries of Western Balkans, and has already led to a decrease in alcohol consumption (22, 23), these habits are still following patterns from the past and making up one of the target population’s important specifics, further stressing the need for region-specific instruments for measuring HRQoL.

Although the convergent criteria validation of the WB-HRQoL showed very good results (see Table 3), divergent validation with the ERQ was less successful, since scores of the WB-HRQoL and ERQ showed a weak, but still significant correlation. However, significant correlation was also found between scores of the social and environmental domains of the WHOQoL BREF and that of the ERQ, while VAS, the 1st and 2nd items of the WHOQoL BREF were not correlated with the ERQ score. Probably the choice of ERQ for divergent validation purpose was not the best option; the ERQ rates how the control a person has over the expression of their emotions, which could affect social and environmental relations, and therefore quality of life. Since the WB-HRQoL does have social and environmental domains, a certain degree of correlation with the ERQ score is inevitable.

The main limitation of this study was use of convenience instead of a random sample for validating the questionnaire, which reduced potential to generalize conclusions to the whole studied populations. Second, in order to validate a scale, it is beneficial to include participants having whole spectrum of a phenomenon, which is measured, if possible, confirmed by a “gold standard”. However, the “gold standard “for HRQoL is still lacking, creating a space for uncertainty.”