Standardization of quality of diagnoses, interventions, and outcomes (Q-DIO) measurement instrument for use in Slovenia
Article Category: Original scientific article
Publié en ligne: 03 janv. 2022
Pages: 14 - 23
Reçu: 29 nov. 2020
Accepté: 13 oct. 2021
DOI: https://doi.org/10.2478/sjph-2022-0004
Mots clés
© 2022 National Institute of Public Health, Slovenia, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.
A health record is the official record of all important interactions with the patient (1) in electronic or paper form (2), and an important source of data for healthcare providers, as it supports them in making clinical decisions (3, 4). Documentation also enables continuity of work and greater safety and quality of patient care (5), and is also crucial for the evaluation of patient health care (6).
Regardless of the positive effects, various authors have noted that there are often gaps in patient records, repetitive records of irrelevant information, and inaccurate data (1, 7), and further that nursing documentation does not always support nurses in making clinical decisions (3). Nursing documentation was introduced in more comprehensive form with the introduction of the Slovenian Patient Classification Tool for nursing intensity evaluation (8), while the standardized language of nursing, especially for nursing diagnosis, began to be used after the translation of the Manual of Nursing Diagnosis by Marjory Gordon (9). The introduction of the electronic health record further accelerates this process. The survey, carried out at all three levels of health care (primary, secondary and tertiary) in Slovenia, revealed that all phases of the nursing process were being documented in only one-third of cases (10). The newer version of the nursing process – called the Advanced Nursing Process (11) – has not yet been implemented in Slovenia. Based on a literature review, some questions arose: what is the actual quality of nursing records in Slovenia? Do nurses identify appropriate nursing diagnoses that are consistent with the data collected during the assessment; are nursing diagnoses based on clinical judgment? Do nurses set specific goals and effective interventions for the patient that are individualized? Studies in this field are insufficient and do not provide all the necessary answers.
Müller-Staub et al. (12) developed the Quality of Diagnoses, Interventions and Outcomes (Q-DIO) instrument to measure the accuracy of the nursing diagnosis and the coherence between nursing diagnoses, interventions, and nursing-sensitive patient outcomes in nursing documentation. The Q-DIO instrument was developed on the basis of a literature review. The concepts of nursing diagnosis, nursing interventions, nursing outcomes and the nursing process provided the theoretical framework for the development of the Q-DIO instrument. The development of a coherent nursing plan that will serve as the basis of nursing care quality requires choosing an accurate nursing diagnosis and selecting effective interventions and outcomes for each patient (6). The proper use of these concepts has a key impact on the quality of nursing documentation.
The main purpose of the Q-DIO instrument is to assess the accuracy of nursing diagnoses, the effectiveness of nursing interventions, and the quality of patient outcomes in nursing documentation. The basis for filling in the instrument are patient healthcare records, which can be in electronic or paper form. The use of a standardized nursing language in an institution is not necessary for using the Q-DIO in quality assessments (12).
The Q-DIO instrument, originally developed in English (12), has also been translated into and validated in Brazilian Portuguese (13), French (14), Italian (15, 16), German (17, 18), and Spanish (19). Psychometric testing of adapted versions of the instrument was published for the adaptation in Brazilian Portuguese and in German languages. These studies showed good psychometric properties (17, 18, 21). As in medicine and other health professions, nursing also uses rating scale questionnaires to measure characteristics for which there are no standardized physical measures. In order to use such questionnaires in the professional field, they need to be standardized and tested (validated). They also need to be tested or “calibrated” when transferred to other (cultural or linguistic) environments. The standardized transfer process is called localization and takes place in six phases: forward translation, target language version synthesis and adaptation, backward translation, source language version comparison and target adaptation, comprehensibility testing, and reliability testing (20).
The purpose of our research was to localize the Q-DIO measurement instrument (Figure 1) for use in the Slovenian setting, which includes testing its content validity and internal reliability. The Q-DIO measurement instrument was developed by Maria Müller-Staub et al. (6, 12). Its name in Slovenian is: “Kakovost negovalnih diagnoz, intervencij in izidov “.
Figure 1
Q-DIO measurement instrument – English version.
In validating the Slovenian Q-DIO instrument (Figure 2), we used psychometric testing and considered the general international recommendations (20, 22, 23). The 18 guidelines cover the following areas: pre-condition, test development, confirmation, administration, scoring and interpretation, and documentation (22).
Figure 2
Q-DIO measurement instrument – Slovene version.
The Q-DIO instrument consists of 29 items (Q), which are divided into 4 domains: nursing diagnoses as process, nursing diagnoses as product, nursing interventions and nursing-sensitive patient outcomes. The items under Nursing diagnoses as process (Q 1 to 11) measure the comprehensiveness and documentation quality of nursing history and patient demographics. The items under Nursing diagnoses as a product (Q 12 to 19) measure the accuracy and compliance of nursing diagnoses with the PES statement (problem, etiology, symptoms) (9). The items under Nursing interventions (Q 20 to 22) measure the effectiveness of intervention planning and implementation in relation to the nursing diagnosis. The items under Nursing-sensitive patient outcomes (Q 23 to 29) measure outcomes planning and achievement in relation to nursing interventions and diagnosis (6, 12, 21).
For the first domain – Nursing diagnoses as process – a 3-point Likert scale was used to distinguish between grades: 0-missing documentation, 1-partially completed documentation, and 2-comprehensive documentation (6). For the second, third, and fourth domains, a 5-point Likert scale was used to distinguish between grades: 0-no formulation, 1-partially correct only one part of diagnosis, intervention or outcome formulation (e.g. label), 2-correct one part of formulation, 3-partially correct all parts of diagnosis (label, etiology, signs and symptoms), intervention or outcome formulation (concrete, clearly named, planned, documented and affecting the etiology), 4-correct all parts of diagnosis, intervention or outcome formulation (12).
140 nursing records of children aged 1 to 9 years, who were hospitalized in 2017 and 2018 with lower respiratory tract infection within the Respiratory Diseases (J00-J99) group according to the International Classification of Diseases (ICD-10) (24), were selected in the sample for analysis from two institutions: University Medical Centre Ljubljana and University Medical Centre Maribor (70 from each institution). This group was chosen because lower respiratory tract infection is one of the most common medical diagnoses in children, and also because these children are treated in different wards, which is crucial for research. The first institution used electronic-based documentation and the standardized language for nursing diagnoses according to Marjory Gordon (9), while the second institution used paper-based documentation without a standardized language. At the end of 2017 the research setting at the first institution employed 90 registered nurses (38.8%) and 142 health technicians (61.2%) for 157 bed units (1.47 employees per bed), while the research setting at the second institution employed 24 registered nurses (32%) and 51 health technicians (68%) for 60 bed units (1.25 employees per bed).
The research sample was estimated using WINPEPI v11.32 at a statistical power of 90% and a significance level of 2.5% based on the difference of averages and standard deviation found in a comparable study (25). The sample size was calculated as 69 health records per facility; that number was rounded to 70 per facility. In one ward, there were not enough medical records from the 2017 that would meet the criteria, so medical records from 2018 were added until the sample was fleshed out.
Inclusion criteria for nursing records were: a) Records of patients with a hospital stay of at least 4 days (according to the Instructions for using the Q-DIO instrument (12)), b) which comprised an assessment, nursing problems/ nursing diagnoses, and nursing care planning documented in standardized care plan forms.
Excluded were nursing documentation for re-hospitalized patients (that were already included in the sample) and documentation that did not contain standardized forms of nursing assessments and nursing care plans (which includes nursing problems/nursing diagnoses).
The process of adaptation or so-called localization of the Q-DIO measurement instrument for the Slovenian setting is part of a broader research project conducted within the scope of a doctoral study. Data collection was conducted in both institutions from August 2019 to August 2020 by a single independent researcher, who coded and anonymized all the collected data. The principles of the Declaration of Helsinki (26) were taken into account. Data analysis was performed using the SPSS 27.0 software package.
We first obtained the consent of the author of the Q-DIO instrument for its translation into Slovenian. The instrument was then translated into Slovenian by two independent researchers – registered nurses, one also with Bachelor of Science in Nursing and the other with Bachelor of Science in Organization and Management.
Both translated versions were then compared, harmonized and unified on the basis of a discussion between the translators.
A harmonized version of the Q-DIO instrument was handed over to a third independent researcher with experience in translating professional texts on nursing from English into Slovenian. The researcher performed a backward translation – translated the instrument back into the English language without the original English version of the instrument.
We compared the new translation with the original text in English, and then harmonized and adapted the Slovenian version accordingly. The obtained and presumably final version of the translation was used for further evaluation in two steps.
The next step was comprehensibility testing, or, in other words, a content validity analysis of the entire Q-DIO instrument and of individual items, which we carried out with the help of eight experts – nurses with a bachelor’s degree, clinical mentors, and teaching assistants or lecturers on nursing. Comprehensibility was assessed using a 4-point scale (1-not relevant/not comprehensible, 2-somewhat relevant/poorly comprehensible, 3-quite relevant/partially comprehensible, 4-highly relevant/ fully comprehensible). The experts judged that the Q-DIO items constitute a representative sample of the universe of items to measure what the instrument aims to measure. Based on the scores, we calculated the item content validity index (I-CVI) and the scale validity index (S-CVI). To calculate I-CVI, we used the following formula: the number of experts who rated the item with 3 or 4 divided by the total number of all experts. S-CVI was calculated as the average of all I-CVI. According to Polit and Beck (27), the acceptable values for I-CVI are >0.78 and for S-CVI≥0.90. Based on the experts’ answers, we reworded the poorly evaluated statements (taking into account any reasonable comments and suggestions) and sent them back to the same experts for re-assessment. The subsequent comprehensibility scores were given by five experts.
In addition to the content validity index, we also checked the index of agreement between experts or modified kappa statistics (Κ*) to reduce the possibility of chance agreement. The formula Κ*=(I-CVI-PC)/(1-PC) was used. The probability of chance agreement between experts on the significance of the items (PC) was calculated with the following formula: PC=N!/A!x(N-A))x0.5N, where N represents the number of experts and A the number of experts agreeing on the relevance (scores 3 and 4). The modified Kappa is defined as excellent when the value of Κ*≥0.75, and as still acceptable when Κ*≥0.60 (27).
Divergent or discriminant construct validity analysis gather evidence that the focal measure is not a measure of a different construct (27). It was assessed with a T-test in comparison between two groups of nursing documentations: electronic-based documentation and standardized terminology for nursing diagnoses; paper-based documentation and no standardized terminology.
This was followed by reliability testing, which can be evaluated by internal consistency and stability testing. The internal consistency or internal reliability of the instrument tells us whether its components measure the same characteristic (28). We used the Cronbach coefficient ≥[0-1], whose desired value is α≥0.80 (27).
The Cronbach coefficient depends, in part, on the length of the instrument, so it is good to use additional reliability analyses (27). The analysis of the instrument’s individual items was also checked using the frequency of endorsement of items and the discrimination validity. The frequency of agreement with the statements is a function of the item’s weights, and is calculated using the weight index (<1.6 being recommended for the first 11 items, and <3.2 for the remaining 18 items of the Q-DIO instrument) (6). Item discrimination (corrected item-total correlation) is a function of internal reliability and shows us how well it is possible to replace the instrument’s score with the score of a single item. Values ≥0.3 were considered as an acceptable criterion (29).
The instrument’s stability was checked on a pilot sample of 17 patient records by two independent researchers. To be prepared for using the instrument, the researchers studied all available literature on its use that was provided from its initial author. We used two calculations: interrater reliability and interrater agreement. To calculate the reliability between researchers or the consistency of estimates between two independent researchers, we used the Pearson correlation. To calculate the agreement between two independent researchers, we used Cohen’s Kappa value.
In the first phase, i.e. the “forward translation” of the Q-DIO instrument, the translators did not encounter any major conceptual, linguistic, or terminological problems. In the second phase, they stopped to coordinate the translations of the concepts of “assessment” and “nursing-sensitive patient outcomes”. The first term, i.e. “ocenjevanje”, is not yet widely used in Slovenian nursing, so we added an explanation in parentheses that it refers to the patient’s history and condition assessment. We also do not yet have an established translation for the second term. The translators agreed to use the most semantically appropriate translation, i.e., “na zdravstveno nego odzivni izidi pacientov”. Other harmonisations were more in terms of greater comprehensibility, e.g. “informacije o situaciji” → “opis situacije”, “formulirano” → “oblikovano”, and the use of established terms, e.g., “aktivnosti za zabavo” → “prostočasne aktivnosti”.
The third phase, i.e. “backward translation”, showed almost perfect semantic correspondence between the two English texts. In the fourth phase, i.e. “source language versions comparison and target adaptation”, we made only one change (“dolgotrajne negovalne diagnoze” → “trajne negovalne diagnoze”).
In the fifth phase, i.e. “comprehensibility testing”, we calculated the content validity indexes. S-CVI of the entire instrument was 0.945, 0.877 for the first part, 1.000 for the second and third part, and 0.964 for the fourth part. The content validity indexes of individual items, with the exception of three items, were all acceptable and estimated at being between 0.857 and 1.000. The modified Kappa index also showed excellent content validity for most items (scores between 0.856 and 1.000). The exceptions were items Q7, Q11, and Q24, which were re-examined and harmonized on the basis of experts’ comments (Intimacy, being female/male: “Zasebnost/ intima, biti ženska/moški” → “Intimnost v odnosu, ženska/ moški”; The nursing diagnosis is reformulated: “Negovalna diagnoza je preoblikovana” → “Negovalna diagnoza je spremenjena”). In item Q11, a grammatical error was corrected. The revised items were sent for re-evaluation to experts (n=5). All new content validity indexes of individual items that were subsequently adjusted were estimated at 1.000.
Regarding discriminant construct validity, there was statistically significant difference between the two groups of documentations in means for the sum of all 29 items (t=39.56, p<0.001). Mean scores for group one (electronic records with standardized terminology for nursing diagnoses) were 47.8±6.2 (95% confidence interval 46.3-49.3) and for group two (paper-based records and no standardized terminology) 29.4±7.7 (95% confidence interval 27.6-31.3).
In the last, sixth, phase, i.e. “validity testing”, we performed calculations of the reliability analysis (Table 1). The value of the Cronbach coefficient for the whole Q-DIO instrument was 0.860, which corresponds to recommendations. An analysis of Cronbach’s alpha if item deleted remains above 0.80 for all items. The average values of items Q1 to Q11, measured on a 3-point Likert scale, were less than 1.6, except for items Q9 “Significant others (contact persons)” and Q10 “Activities of daily living”, which exceeded the recommended average value. The average values of most other items measured on a 5-point scale were less than 3.2, which is in accordance with the recommendations. Minor deviations were observed in three items – Q12, Q18, and Q23, and a moderate deviation in Q22 “Nursing interventions were documented”.
Item analysis of the Quality of Nursing Diagnoses, Interventions, and Outcomes Instrument.
| Subscale / Item | Item mean | Standard deviation | Corrected Item-Total Correlation | Cronbach’s Alpha if Item Deleted |
|---|---|---|---|---|
| 1. Actual situation leading to the hospitalization | 1.16 | 0.692 | 0.870 | |
| 2. Anxiety and worries related to hospitalization, expectations and desires about hospitalization | 0.27 | 0.621 | 0.377 | 0.857 |
| 3. Social situation and living environment/circumstances | 1.04 | 0.548 | 0.864 | |
| 4. Coping in the actual situation / with the illness | 0.68 | 0.833 | 0.468 | 0.854 |
| 5. Beliefs and attitudes about life (related to the hospitalization) | 1.21 | 0.560 | 0.361 | 0.857 |
| 6. Information of the patient and relatives/significant others about the situation | 0.85 | 0.795 | 0.636 | 0.850 |
| 7. Intimacy, being female/male | 1.31 | 0.507 | 0.558 | 0.854 |
| 8. Hobbies, activities for leisure | 0.20 | 0.436 | 0.349 | 0.858 |
| 9. Significant others (contact persons) | 0.487 | 0.863 | ||
| 10. Activities of daily living | 0.510 | 0.869 | ||
| 11. Relevant nursing priorities according to the assessment | 0.96 | 0.535 | 0.862 | |
| 12. Nursing problem/nursing diagnosis label is documented | 3.39 | 0.964 | 0.768 | 0.844 |
| 13. Nursing diagnosis label is formulated according to NANDA and numbered | 1.91 | 1.940 | 0.722 | 0.844 |
| 14. The etiology (E) is documented | 1.00 | 1.080 | 0.757 | 0.843 |
| 15. The etiology (E) is correct, related /corresponding to the nursing diagnosis (P) | 0.74 | 0,828 | 0.724 | 0.847 |
| 16. Signs and symptoms are formulated | 0.00 | 0.000 | 0.860 | |
| 17. Signs and symptoms (S) are correctly related to the nursing diagnosis (P) | 0.00 | 0.000 | 0.860 | |
| 18. The nursing goal relates /corresponds to the nursing diagnosis | 0.774 | 0.612 | 0.850 | |
| 19. The nursing goal is achievable through nursing interventions | 1.65 | 1.292 | 0.661 | 0.845 |
| 20. Concrete, clearly named nursing interventions according to NIC are planned (what will be done, how, how often, who does it) | 1.23 | 0.851 | 0.766 | 0.845 |
| 21. The nursing interventions affect the etiology of the nursing diagnosis | 0.35 | 0.493 | 0.594 | 0.854 |
| 22. Nursing interventions carried out, are documented (what was done, how, how often, who did it) | 0.534 | 0.867 | ||
| 23. Acute, changing diagnoses are assessed daily or form shift to shift/enduring diagnoses are assessed every fourth day | 3.37 | 0.977 | 0.769 | 0.844 |
| 24. The nursing diagnosis is reformulated | 1.73 | 1.280 | 0.876 | |
| 25. The nursing outcome is documented | 1.49 | 1.000 | 0.752 | 0.844 |
| 26. The nursing outcome is observably /measurably documented according to NOC | 0.00 | 0.000 | 0.860 | |
| 27. The nursing outcome shows |
0.84 | 0.949 | 0.501 | 0.853 |
| 28. There is a relationship between nursing-sensitive patient outcomes and nursing interventions | 1.55 | 1.889 | 0.412 | 0.860 |
| 29. Nursing outcomes and nursing diagnoses are internally related | 1.08 | 1.258 | 0.412 | 0.856 |
When reviewing the corrected item total correlations of individual items, several deviations can be observed in the first part of the instrument (items Q1 to Q11), where they are below the recommended value (r<0.3) Q1, Q3, Q9, Q10 and Q11. In other parts of the instrument (items Q12 to Q29), deviations were found only in Q22 and Q24.
Pearson’s correlation test showed excellent reliability or correlation of scores between two researchers (interrater reliability) (r=0.940, p<0.01), and the Kappa index showed good agreement of scores between two researchers (interrater agreement) (κ=0.866, p<0.01), which ensures the instrument’s good stability.
The purpose of our research was to localize the Slovenian version of the measuring instrument Quality of Nursing Diagnoses, Interventions and Outcomes (Q-DIO). Localization was performed in six phases. The first four phases (forward translation, target language version synthesis and adaptation, backward translation, source language versions comparison and target adaptation) contribute to better semantic equivalence (27) and ensure the instrument’s professional integrity (20). Through the fifth and the sixth phases of our study, the content validity and internal reliability of the Slovenian version of the instrument was confirmed, but certain research limitations need to be taken into account in the interpretation.
To assess content validity, we used the content validity index, which represents the degree of agreement of a group of experts on the relevance, integrity and balance of instrument items, and is recommended in the context of the instrument’s cultural adjustment and psychometric testing (27). The content validity index of the entire instrument was appropriate, as were the indexes of individual items. Three items that did not meet the required criteria were reviewed, harmonized, and reevaluated by experts. The new indexes were all relevant. With discriminant construct validity we assessed the ability of the Q-DIO instrument to discriminate between different types of nursing records (electronic with standardized terminology versus paper-based without standardized terminology). The analysis showed significant differences in the means of all instrument items among the two groups of documentations. Similar results were obtained in a study by da Costa Linch et al. (21), who validated a Q-DIO instrument for use in Brazil and in the United States.
The internal reliability of the entire instrument was adequate, as shown by the calculation of the Cronbach coefficient. The item mean (item difficulty criterion) exceeds the recommended values for questions Q9 and Q10, while the corrected item total correlations are lower than recommended for items Q1, Q3, Q10, and Q11 for the sub-concept “Nursing diagnoses as process”. In the section “Nursing diagnoses as product”, two items, Q12 and Q18, exceed the recommended item mean value, while the corrected item total correlations are lower than recommended for items Q16 and Q17. As part of “Nursing interventions”, only item Q22 exceeded the recommended item mean value, as well as the recommended corrected item total correlation value. In the section “Patient outcomes responsive to nursing care”, item Q23 exceeded the item mean value, while the corrected item total correlation values are lower than recommended for items Q24 and Q26. Based on these results, items Q1, Q3, Q9, Q10, Q11, Q16, Q17, Q22, Q24, and Q26 did not show favourable results and could be suggested to be left out of the instrument. When examining these items, deviations are seen as the result of poor nursing documentation quality. This also indicates that performing and documenting a comprehensive nursing assessment, accurate nursing diagnoses with related, effective interventions and measurable patient outcomes, as well as using nursing classifications, is not yet the case in the study setting. Compared to other studies, items Q3, Q9, Q11 (6), and Q22 (13) also deviated from the recommended values. However, all Q-DIO items showed good psychometric properties in other studies (17, 18, 21). The authors therefore conclude that these aspects, e.g., a more detailed nursing history involving psycho-social aspects of the individual (Q1, 3, 9, 10+11), defining the nursing diagnosis according to PES (Q16+17), observably and measurably planned nursing interventions (Q22) and observably and measurably set nursing outcomes (Q24+26), should be implemented in clinical practice and education with greater emphasis. Since all items provide an overview of the quality of nursing documentation as a whole and the key items of the instrument, i.e. Q12, Q20, and Q25 (6), showed good results, we suggest that these items remain in the instrument. The results indicate that further implementation and training of nurses in the Advanced Nursing Process and its documentation, which is based on nursing classifications, are needed (11, 30, 31).
The stability of the Q-DIO instrument was measured by interrater reliability and interrater agreement, which were both relevant and provide us with stable quality assessments even when Q-DIO is used by different assessors.
We conclude that by following the process of localization and internationally recommended phases of transferring the measuring instrument between different cultural environments, we were able to obtain a validated and professionally suitable instrument for measuring the quality of nursing documentation in Slovenia. The results of comprehensibility, internal reliability, item analysis and stability showed good psychometric properties of the Slovenian Q-DIO instrument. However, due to the poorer test results of some items, additional testing of the instrument is recommended.