Do immediate supervisors underestimate burnout in subordinates? A comparison between burnout self-assessment by nurses and assessment by immediate supervisors
Darja Maslić Seršić
Profil Orcid oraz 
Kategoria artykułu: Original article
Data publikacji: 29 gru 2024
Zakres stron: 278 - 282
Otrzymano: 01 lip 2024
Przyjęty: 01 lis 2024
DOI: https://doi.org/10.2478/aiht-2024-75-3883
Słowa kluczowe
© 2024 Darja Maslić Seršić et al., published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.
Burnout is a multidimensional syndrome. The World Health Organization’s (WHO) International Classification of Diseases (ICD-11) recognises it as an “occupational phenomenon” and defines it as a syndrome that results from chronic workplace stress that has not been successfully managed. It is characterised by three dimensions: feeling of energy depletion or exhaustion, increased mental distance from one’s job or feelings of negativism or cynicism toward one’s job, and reduced professional efficacy (1).
Although the experience of burnout can occur in any type of occupation and work environment, people-oriented human service professionals are at greater risk for developing the syndrome. According to EUROSTAT (2), around 58 % of human health and social workers report exposure to work-related risk factors that can negatively affect mental well-being.
In this context, nursing is particularly stressful as it involves direct contact with patients and their families, difficult decisions, dealing with serious illnesses and death, low autonomy, long working hours, rotating shifts, high emotional burden, and occasional aggression from patients, their families, and even colleagues (3–6).
Therefore, it is not surprising that research on burnout among nurses is common, yet it still does not provide clear insights that could help prevent or treat burnout. A review by Dall’Ora et al. (7) makes a good point in this respect as it finds that the vast majority of burnout studies used a cross-sectional design, and only a minority measured all three dimensions of the Maslach Burnout Inventory (MBI). In other words, by reducing the burnout syndrome to the experience of exhaustion or to negative effects that the nurses associate with their work, such studies fail to contribute to our knowledge of work-related mental health beyond the traditional concepts of fatigue or job dissatisfaction. In addition, many studies use very different burnout indicators (4–10).
To address this issue a new 23-item Burnout Assessment Tool (BAT-23) has been developed (11). It redefines burnout to facilitate the comparison of results collected on different samples and to enable a much-needed synthesis of knowledge in the field.
Unlike MBI, it measures four dimensions: exhaustion (severe loss of physical and mental energy and reduced ability to recover after work), emotional impairment (reduced ability to regulate emotional reactions at work), cognitive impairment (forgetfulness and lack of attention during work), and mental distancing (psychological withdrawal and experience of mental separation from one’s own job). As a self-assessment/report tool, BAT-23 (and its shorter version BAT-12) has been validated by a series of international studies (12–19).
Considering, however, the importance of supervisor support in the prevention of burnout in healthcare professionals (20–24), we wanted to see what this instrument would show, should immediate supervisors assess burnout in their nurses. In other words, the aim of our study was to examine the agreement between self-and supervisor-assessed burnout in nurses and to identify issues that may arise from disagreement to help inform supervisors in dealing with this occupational health issue.
Data were collected from a sample of 48 pairs of nurses and head nurses (N=96) from the University Hospital Centre (UHC) Zagreb, Croatia, that employed a total of 4,896 nurses at the time of data collection (between 6 February and 31 May 2023). The vast majority were women (88 %), and the age range was between 21 and 62 years. Each pair consisted of the head of a particular ward (supervisor) and a randomly selected nurse under their direct supervision. The mean age of supervisors was 47.4±8.46 years and of subordinates 34.8±10.81 years. The job tenure of the supervisors ranged between eight and 43 years (mean±SD=26.1±8.87) and of nurses between two and 43 years (mean±SD=14.2±11.32). The supervisor-subordinate relation in our pairs lasted between two and 20 years (mean±SD=7.1±4.61).
The study was approved by the Ethics Committee of the UHC Zagreb. Participation was voluntary. The participants were informed about the research by e-mail, and the research was conducted during working hours when circumstances at the ward allowed. First, the head nurse read and signed the informed consent form and then the paired nurse working a 12-hour shift read and signed the informed consent form. The shift nurse then chose a code to be placed on the questionnaires so that the questionnaires could be matched later. The codes were created by the researchers by combining letters and numbers in a sequence. The researchers gave the shift nurse the option to choose one of the offered codes to blind the researchers to their identity on the questionnaire. Each pair then received the respective version of the questionnaire in a separate envelope and were instructed verbally to complete it within a week and return to researcher’s address written on the envelope using the hospital’s internal mail.
The Croatian version of BAT-23 (25), validated by Tomas et al. (18), was applied in two versions: standard self-assessment version and supervisor-assessment version. BAT-23 consists of four dimensions: exhaustion (8 items, e.g. “At work, I feel mentally exhausted”/“At work, she/he feels mentally exhausted”), mental distance (5 items, e.g., “I struggle to find any enthusiasm for my work”/“She/He struggles to find any enthusiasm for her/his work”), cognitive impairment (5 items, e.g., “At work, I have trouble staying focused”/“At work, she/he has trouble staying focused”), and emotional impairment (5 items, e.g., “At work, I feel unable to control my emotions”/“At work, she/he feels unable to control her/his emotions”). Respondents rated each item on a 5-point scale (1=never; 2=rarely; 3=sometimes; 4=often; and 5=always).
Statistical analysis was run on the Jamovi software for Windows version 2.3.21 (26). Both BAT-23 versions show high and satisfactory internal reliability. Cronbach’s alpha coefficients for the total score were 0.89 for self-assessments and 0.94 for supervisor assessments evidence high internal consistency. As for dimensions, mental distancing had the lowest but acceptable reliability in both versions (0.72 for self-assessments and 0.73 for supervisor assessments), whereas exhaustion had the highest reliability (0.86 and 0.90, respectively).
We used bivariate Pearson’s correlation and Cohen’s kappa to test inter-rater reliability.
Pearson’s coefficients r of bivariate correlations between self-and supervisor ratings for BAT-23 dimensions ranged from nonsignificant to moderate. Non-significant correlations concern cognitive impairment (-0.16; n.s.) and mental distancing (0.14; n.s.). Moderately correlated are emotional impairment (0.45; p<0.01) and exhaustion (0.49; p<0.01). The inter-rater correlation for the total score is somewhat lower but still moderate (0.32; p<0.05). Moderate correlation is expected, at least according to the most influential meta-analysis of inter-rater reliability in the assessment of personality traits (27). Similar to burnout, self- and observer assessments share significant common variance, but the significant unique variance in self-assessments underlines the subjective nature of cognitions and emotions. We did not, however, expect that correlations for mental distancing and cognitive impairment would not be significant.
According to the BAT-23 manual (11), burnout is categorised as “low” (≤25th percentile), “average” (25–75th percentile), “high” (75–95th percentile), and “very high” (≥95th percentile). We calculated Cohen’s kappa coefficient for scores that showed significant correlations between self- and supervisor ratings, namely 0.059 (62.5 %) for the summative BAT-23 score, 0.15 (62.5 %) for the exhaustion dimension, and 0.17 (58.3 %) for the emotional impairment dimension. These coefficients show low agreement between raters.
Table 1 shows the number of nurses by burnout category according to self- and supervisor-assessment. It also shows that immediate supervisor has a tendency to underestimate the burnout experience of their subordinate. This tendency is also noticeable if we compare mean scores presented in Table 2. Paired-sample
| Burnout self-assessment and assessment by immediate supervisor | ||||
|---|---|---|---|---|
| Low | Average | High | Very high | |
| BAT-23 | 7 (12) | 25 (25) | 15 (10) | 1 (1) |
| Exhaustion | 3 (11) | 15 (22) | 26 (13) | 4 (2) |
| Emotional impairment | 4 (8) | 20 (20) | 23 (16) | 1 (4) |
| Cognitive impairment | 18 (24) | 21 (16) | 8 (7) | 1 (1) |
| Mental distance | 6 (7) | 32 (31) | 8 (10) | 2 (0) |
| Self-assessment | Assessment by immediate supervisor | |||||||
|---|---|---|---|---|---|---|---|---|
| Mean | SD | Skewness | Kurtosis | Mean | SD | Skewness | Kurtosis | |
| BAT-23* | 2.23 | 0.501 | 0.413 | 0.227 | 2.01 | 0.570 | 0.657 | -0.070 |
| Exhaustion** | 2.78 | 0.688 | 0.303 | -0.051 | 2.35 | 0.689 | 0.401 | -0.317 |
| Emotional impairment | 2.07 | 0.550 | 0.100 | -0.655 | 2.07 | 0.748 | 0.801 | 0.473 |
| Cognitive impairment | 2.06 | 0.548 | 0.105 | -0.076 | 1.93 | 0.638 | 0.652 | 0.581 |
| Mental distance | 2.05 | 0.656 | 0.463 | 0.025 | 1.92 | 0.662 | 0.595 | -0.542 |
*
**
Speaking of distribution, burnout self-assessment shows higher levels than would be expected according to statistical norms. A large number of nurses rated high and very high their exhaustion (30 out of 48) and emotional impairment at work (24 out of 48). Furthermore, they have higher mean total and exhaustion scores than reported by Schaufeli et al. (28) for healthy workers, but lower than reported for workers with the diagnosed burnout syndrome. This finding is in line with a systematic review of paramedics whose burnout was measured with other instruments than BAT-23 (3) and which shows that between 23 and 33 % of them self-rate burnout high or very high and that the prevalence of burnout is between 16 and 56 %. Similar findings are reported by other systematic reviews: 31 % of nurses experience emotional exhaustion, between 18 and 24 % high depersonalisation, and between 38 and 46 % low personal accomplishment (9–10).
The limitations of our study primarily stem from the small and homogeneous sample, and future research should include more heterogeneous samples. Also, multilevel studies based on nested groups of workers subordinate to a single supervisor will be useful to reveal inter-individual differences in supervisors’ capacities to recognise the burnout symptoms of their subordinates, but also to test intergroup differences in the level of burnout symptoms. Low and non-significant correlations between nurses’ self-assessments and those of their immediate supervisors may be the result of moderating effects of supervisors’ individual characteristics and leadership style, so this set of variables should be included in future research (29).
To our knowledge, this is the first study to investigate the agreement between workers’ burnout self-assessment and the assessment of their immediate supervisors, and it clearly demonstrates that immediate supervisors tend to underestimate burnout in their subordinate nurses. As the first such study to use a new internationally validated burnout syndrome assessment instrument (BAT-23), its results have raised new questions and directed future research on this topic. First of all, this should be research into factors that encourage faster recognition of symptoms of employee burnout. Because immediate supervisors are the first line of hierarchical communication and are responsible for the inrole behaviour and well-being of their subordinates, future research should focus on contextual (e.g., psychological climate) and personal (e.g., leadership style) variables that can enhance supervisors’ ability to recognise the burnout syndrome in their subordinates.