Preliminary construction of evaluation indicator system for inpatients’ nursing service needs in tertiary general hospital

The research coordination group consists of six members, including three experts, all of whom have postgraduate qualification, deputy directors, and above; three coordinators, all of whom have postgraduate qualification, two intermediate titles, and one junior title.

On the basis of extensive reading of literature^6,7,8,9,10 and formerly semi-structured interviews,¹¹ referring to the “Quality Care Service Evaluation Rules (2014 Edition)” and the “Notice on Further Deepening Quality Care and Improving Care Services,” a preliminary draft of the patient care service needs evaluation indicator system was formed. In addition, intending to sample six experts in advance, according to the experts’ suggestion, after discussion and revision, the first round of consultation volume was finally formed. The consultation volume covers 5 first-level indicators such as hospitalization environment, nurse attitude, nursing skills, information notification, and emotional support and 70 second-level indicators. According to the Likert 5 scale, the importance and feasibility of the items are divided into five levels: “not important” to “very important” and “not feasible” to “very feasible,” and they are assigned 1–5 points in turn. The first-level indicator of the volume filled in the importance of assignment, revision comments, and additional entries, and the second-level indicators increased the feasibility assignment. Also, the general survey of experts and their familiarity and judgment basis were also included in it.

The principle of selection is that experts should be familiar with the subject and have authority in the industry, and they should also be familiar with well-informed consent and have ability to cooperate. According to the purpose of this study, the selection criteria of experts for consultation are as follows: (1) Nursing management experts: Associate Dean of Nursing, Director or Deputy Director of Nursing Department of tertiary general hospital, with master's degree and above, engaged in nursing management for at least 10 years. (2) Clinical nursing specialist: Head nurse of tertiary general hospital, with bachelor degree or above, engaged in clinical nursing for at least 10 years. (3) Nursing education specialist: Engaged in nursing education for at least 10 years, with master's degree or above. (4) Experts in health administration departments: At least 5 years in charge of nursing work at municipal and above health administrative departments, had working experience in tertiary general hospital, with bachelor degree or above, with deputy level and above. (5) Head of the Third Party Accreditation Agency¹²: Had previous work experience in the health industry.

Two rounds of expert consultation were conducted from July 2015 to December 2016. We used the method of destination sampling, selected experts working in nursing management, clinical nursing, and nursing education or scientific heads of third-party accreditation agencies with nursing work experience, who are familiar with the subject and with authoritative and representative, and informed well and willing to cooperate as consultant. The number of experts was based on the sample size of the consultation, depending on the size of the research question and the purpose of this study. According to epidemiology and statistical methods,¹³ the consultation experts are well represented between 15 and 50. This study was aimed at the nursing needs of inpatients in tertiary general hospitals, and the scope and scale are relatively large. According to the actual situation, 41 experts were finally consulted. According to the actual situation of the experts, it was conducted in either way by personal distribution, e-mail, or online questionnaires. After retracting the first round of questionnaires, the items that meet the following conditions are retained: importance, feasibility assignment mean ≥4.0; coefficient of variation ≤0.25; reference perfect ratio ≥0.1. We collated and returned the results of first round to experts for the second round of consultation. The research coordination group held meetings to discuss experts’ suggestions and integrate expert opinions. When the expert opinions are basically consistent, the consultation was concluded. Finally, 5 first-level indicators and 48 second-level indicators were identified.

Excel software was used to build the database, and the database was imported into SPSS 17.0 software for processing. Descriptive statistics are used to calculate the mean, standard deviation, and coefficient of variation of the importance and feasibility of each indicator (except for the primary indicator). The reliability of the expert's correspondence is tested by means of authority, enthusiasm, and coordination. The comparison of consultations of coordination of two rounds was done using chi-squared test, and P values <0.05 were considered to reflect a significant difference.

We successfully contacted 41 experts from 17 provinces and municipalities whose information are as follows: an average age of 47 ± 4.12 years and an average working age of 24 ± 3.79 years; education: 6 doctors, 29 masters, and 6 undergraduates; titles: 21 seniors (including professors), 13 vice seniors (including associate professors and deputy directors), and 7 intermediates; work units: 29 experts from hospitals, 8 experts from colleges and universities, 1 expert from health planning committee, 3 experts from third-party evaluation agencies, and some experts have work experience of two or more units.

The degree of expert opinion authority (Cr) is used to determine the reliability of the results of the consultation. It is the arithmetic average of the degree of familiarity (Cs) and the coefficient of judgment (Ca), which is obtained by the experts’ self-evaluation. Cr ≥ 0.7 is acceptable reliability. Cr ≥ 0.8 indicates that the experts have a greater grasp of the choice of content.¹⁴ The Cr values of the two rounds of consulting experts in this study were 0.76 and 0.86, so in this consultation experts’ authority is reliable.

Experts’ activeness refers to the degree of concern, participation, and cooperation of experts in the research, which is illustrated by the response rate of questionnaires and the proportion of experts who make recommendations. It is generally believed that a recovery rate of more than 70% is a good survey. The higher the response rate and the greater the number of experts’ proportion of making recommendations show the higher level of activeness of the experts. The response rates of the two rounds of this study were 82% and 94%, and the effective rates of response were 97% and 100%. The experts who made recommendations in the two rounds accounted for 54% and 21%, which indicates that the experts are highly active.

The degree of experts’ opinion coordination is mainly expressed by coefficient of variation (CV) and coordination coefficient (W). CV is the degree of coordination of all experts on a certain indicator; the smaller the CV is and the higher the degree of coordination mean more unified of the experts’ opinion. The CV value range of the second round of all indicators is 0.05–0.21, and W is the degree of coordination of all indicators for all experts. The greater the W is and the higher the degree of coordination show the more unified of the experts’ opinions.¹⁵ Under normal circumstances, W fluctuates in the range of 0.4–0.5, indicating that the experts’ opinions are well coordinated.¹⁴ The W values of the two rounds of coordination coefficient was statistically significant, P < 0.05. The specific figures are shown in Table 1.

In the first-level indicator, “information notification” was changed to “information exchange.” In the second-level indicator, in the hospital environment dimension, the “environmental cleanliness” and “the ward ventilation and no smell” were combined into “the ward is clean and odorless.” “Timely handling monitor alarm sounds,” “common patients in the same room,” and “internal entertainment resources” were deleted and “the convenience of living areas” was increased; in dimension of nursing attitudes, “speak softly,” “keep smiling,” and “timely humor” were deleted and “in special circumstances, with the ability to do things in time, and do extra things for patients” was changed to “in special circumstances for patients to solve urgent problems”; “apply patient-like title” was changed to “apply courtesy title”; in the dimension of nursing skills, “the same process for different nurses” was changed to “practicing according to law,” and the “correct and effective nursing operation” was changed to “standardized skilled nursing operation,” and “the comprehensive assessment of patients before operation” was added; in the dimension of information exchange, “information is provided with written materials” was changed to “different forms of health education, content is targeted,” and “security knowledge notification (fire channel, accident prevention, etc.)” was added; in the emotional support dimension, “quickly solve pain” was changed to “quickly and effectively treat pain,” and “encourage patients to express their emotions freely” was changed to “encourage patients to seek emotional venting methods.”

We adopted the amendments of the experts to delete, modify, and add relevant items. For the items that satisfy the importance, the feasibility assignment mean ≥4.0, the coefficient of variation ≤0.25, and the reference perfect ratio ≥0.1 were retained. Finally, the patient care service demand evaluation index system was constructed as shown in Table 2.

The choice of experts is the key to the Delphi's letter of consultation. The experts participating in this study have good regional representation and subject representativeness. In addition, the experts from third-party accreditation bodies reduced bias to a certain extent. The experts put forward revision opinions on the language expression and specific content of the questionnaire, making the indicator system more scientific and accurate. From the questionnaire recovery rate, the proportion of experts who make recommendations, the Cr value of the two rounds inquiries, the W value, and the CV value of the second round, it is known that the enthusiasm and authority of the experts were high, indicating that the opinions of the experts through the two rounds of correspondence are basically consistent and the construction of this indicator system is relatively successful.

This study was based on the principle of facilitating the work of first-line nurses in clinics. It was different from the previous theoretical framework of Maslow's needs. Based on the category of nursing services, the internal design of each dimension refers to the level of Maslow's needs, and the concept of holistic nursing and human care was combined in the design of the item.^9,16 The design of the secondary indicator considers both the importance and feasibility. The demand is determined by motivation to determine a person's mood and behavior, while the nurse has to concern about the patients’ emotions and behavior, and also to close the needs behind it. Due to the discomfort of the disease, the patients urgently need a quiet and comfortable rest environment, and fresh and interesting stimulation; comfortable and warm environment can improve the mental state of the patient.¹⁷ Therefore, the hospital environment dimension requires safe, quiet, and convenient living conditions, as well as a beautiful physical environment and a warm human environment.¹⁶ Attitude is a psychological term that refers to the individual's stable psychological tendency to a particular subject, mainly through behavior and expression. In interpersonal communication, a positive attitude can give the contact person a sense of identity and belonging.¹⁸ Therefore, the patients’ evaluation of nursing behavior was largely a response to the attitude of the nurse, so the attitude in the nursing work greatly affects the patients’ evaluation of the entire nursing service and even medical services. Therefore, this dimension had been set up with humanistic care items, such as “harmony, enthusiasm, equality, and initiative, courtesy” and professional items such as “timely and wholeheartedly.”

Nursing skills are the core of nursing services and directly affect patients’ prognosis.¹⁹ Most of the patients may pay attention to whether their familiar operations cause discomfort, which requires nurses to think about their feelings and needs from the perspective of patients. In addition, this also suggests that the design of this dimension can not only be stuck in the patients’ perspective, but also provide normative and humanistic care skills with a professional perspective. Therefore, the dimension includes “instructing daily activities, operating practices, safety, and proficiency” and items such as “predictive, adequate assessment, and comfort.”

The experts suggested changing the information notification to information exchange to reflect the twoway nature of the information. Although the information between the nurses and patients is asymmetric, however, the more information nurses know about patients, the more specific and targeted services they can provide, and the more they reflect the importance of human care and communication in nursing services, so they are adopted. At present, the use of the network in nursing is increasing, and it is more convenient for patients to obtain health education knowledge, which is conducive to patients to choose the best medical care plan according to their own situation.²⁰ Therefore, the information exchange dimension includes both communication of disease, diet, and safety, as well as the communication of daily health education and nursing operations.

A study showed that emotional support is the largest gap between patients’ perception and expectation.²¹ Emotion is the attitude experience of whether an objective thing satisfies self-demand, and it focuses more on social needs. Different hospitalization periods and special physical conditions can make the patients’ emotions change greatly in a short time. If the patients’ different emotional changes can be expected or captured in time, it is easier to understand the patients’ reaction behavior and targeted intervention. Taking into account the current situation of the patients with the same sympathy will allow the patients feel the care and understanding of the nurse from the heart.²² In addition, different patients face diametrically opposite emotions when facing the same problem. Therefore, emotional support must be personalized. Hospitalization belongs to a life event, while paying attention to its adverse effects on life. It is necessary to guide patients to think about its positive side, help patients to establish a correct outlook on life and values, and even sublimate their emotions, but there is less research and application at this level. Therefore, the emotional support dimension is based on the needs of the patients’ social level, and it fully considers the patients’ subjective initiative.

Although the research experts had certain regional and academic representations, there may still be some limitations due to the Delphi technology itself. Before the questionnaire was formed, the interviewed patients were all from the same hospital and there may be some bias.¹¹ In addition, this study is aimed at patients in general departments such as internal medicine, surgery, and obstetrics and gynecology. It is not classified and stratified according to department, hospitalization time, and patients’ demographic data. Does this evaluation system truly reflect the patients’ demand for nursing services? It needs to be verified through further exploration and practice.