Empirical evidence on the design and application of an intelligent assessment system for humanistic literacy in nursing

Principle of humanization: nursing students, as objects of training, are themselves thinking individuals with creative thinking. Therefore, in the process of cultivating medical students’ humanistic literacy, it is necessary to humanize management and promote creativity.

Principle of feasibility: the establishment of humanistic literacy training mode should be feasible, adapt to the needs of nursing students’ management, conform to the existing conditions and foundation of medical schools, fully listen to the opinions of teachers and students, and mobilize the enthusiasm of teachers and students.

Principle of scientificity: first of all, carry out investigation and research to understand the actual situation of teachers and students, as well as the feasibility of the expected goals to be achieved and the implementation of the program, the development of a scientific and reasonable cultivation system.

Principle of flexibility: The dialectical materialist view holds that “everything in the world is universally connected”. When developing the model of humanistic literacy for medical students, it is important to consider the combination of theory and practice and adjust the implementation program to fit the specific situation.

The principle of pluralism: the establishment of the humanistic literacy training model for nursing students is not only the responsibility of medical schools, but also cannot be separated from the strong cooperation of society and family, and should be an open system. With the trend of cultural pluralism, it is necessary to think about the problem from a diversified and multi-perspective, using multiple horizons.

This study collects information through a questionnaire survey to a nursing college testing professional teachers: lecturers 12, associate professors 11, professors 8 and a tertiary hospital nursing department professional and technical staff: technicians 10, supervisory technicians 20, deputy director technician 3, director technician 3 as the object of the survey, issued 60 questionnaires, recovered 58, the recovery rate of 97%.

This paper establishes a humanistic literacy evaluation index system for nursing students that contains 4 dimensions and a total of 9 indicators, as shown in Table 1.

After determining the assessment indicators of the humanistic literacy assessment model, we need to determine the relative weight of each indicator, and there are many related methods, which can be divided into three main categories:

1) Subjective empowerment method: subjective empowerment method mainly depends on the experience of experts or evaluators to the indicators of empowerment, different people in different times and environments, the empowerment given will be different, strong subjectivity, the classic subjective empowerment method has Delphi method, ring scoring method, hierarchical analysis method.

Hierarchical analysis is currently one of the most widely used decision-making techniques, it is an organized hierarchical representation of a problem methodology process, and will be decomposed at the same level of the problem of two by two comparisons, quantitatively derived weights, after layers of low-level to high-level analysis of the computation, and finally get the bottom layer of the top layer of the weights, the strengths and weaknesses of the program is the size of the weights obtained to determine the final solution to the problem [22].

The following is a brief introduction to the steps of hierarchical analysis:

First clarify the problem to be solved, analyze the structure and content of the problem, and establish a hierarchical analysis structure model, as shown in Figure 1. The top level is the problem that needs to be solved, the middle level is some guidelines to achieve the overall goal, etc., and the bottom level is the various options to solve the problem.

Figure 1.

Hierarchical analysis structure model

The G1 method starts with the experts ranking each indicator according to its importance based on their knowledge and experience, and then comparing the neighboring indicators based on this relationship of the order of importance, because the consistency test is a cumbersome process, so in order to avoid the consistency test, no longer comparing the indicators that are separated from each other, and calculating all the comparative values through mathematical relationships to determine the weights of each evaluation indicator. Therefore, it is seen that the core step of the G1 method is to rank each indicator according to its importance, and this ranking method based on the evaluation of multiple experts can be used by using the set-value iteration method [23].

The steps of the G1 method are briefly described below:

First, the ordinal relationship of the importance of indicators is determined according to the set-value iteration method. There are N experts to select s(1 ≤ s ≤ m) indicators that they consider most important among m indicators X = {x₁,x₂,…,x_m} in the same level, then the nth expert’s choice is X(n)={x1(n),x2(n),...,xs(n)}${{X}^{(n)}}=\{x_{1}^{(n)},x_{2}^{(n)},...,x_{s}^{(n)}\}$. Because of the different levels and knowledge and experience of the experts, each expert is assigned a weight (λ₁,λ₂,…,λ_N). Then the nth expert’s ith choice for the indicators in X has two cases of being in this expert’s choice X⁽ⁿ⁾ or not being in X⁽ⁿ⁾ as in Eq. (1). And the relative importance of the indicators in X, g(x_i) is obtained as in equation (2): (1) un(xt)={ 1,if xt∈X(n)0,if xt∈X(n) \[{{u}_{n}}({{x}_{t}})=\left\{ \begin{array}{*{35}{l}} 1,\text{if }{{x}_{t}}\in {{X}^{(n)}} \\ 0,\text{if }{{x}_{t}}\in {{X}^{(n)}} \\ \end{array} \right.\] (2) g(xi)=∑n=1Nλnun(xi) (l=1,2,...,m) \[g({{x}_{i}})=\sum\limits_{n=1}^{N}{{{\lambda }_{n}}}{{u}_{n}}({{x}_{i}})\;(l=1,2,...,m)\]

The ordinal relationship of indicator importance is then determined by the magnitude of g(x_i), which ultimately leads to x1*>x2*>…>xs*$x_{1}^{*}>x_{2}^{*}>\ldots >x_{s}^{*}$.

From the above overview of the two methods, it can be seen that both the G1 method and the hierarchical analysis method are based on the function-driven principle of the assignment method, while the hierarchical analysis method relies on the consistency of the judgment matrix in the calculation, but often obtains an inconsistent judgment matrix, and the computation process is cumbersome and the calculation is huge. At this time, if we use the Gl method, which has the same principle as the eigenvalue method of assignment, to obtain the local weights, and then layer by layer weighting, and finally obtain the comprehensive weights, there is no complicated calculation formula, nor do we need to judge the consistency, which overcomes the shortcomings of the hierarchical analysis method. So the hierarchical analysis method based on the G1 method is feasible.

After calculating the hierarchical analysis method and consistency test, the complete quality assessment index weights of the complete nursing students can be obtained, and the results are shown in Table 2.

Fuzzy mathematics is a collective term for the mathematical fields of fuzzy topology and fuzzy measure theory developed on the basis of fuzzy sets and fuzzy logic. It is a mathematical tool for studying many problems in the real world where the boundaries are not distinct or even very fuzzy. It has a wide range of applications in pattern recognition, artificial intelligence, and so on.

In the design of this model, the method of fuzzy mathematics will be used to model the comprehensive quality assessment system for college students at Shandong United University. Since it is difficult to accurately quantify the items of the comprehensive quality assessment of students, which can only be determined in a certain fuzzy interval, the items of the assessment are first set as fuzzy sets, then fuzzy transformations are carried out, and finally fuzzy comprehensive evaluation is carried out.

The modeling of this comprehensive quality assessment model necessitates the use of two fuzzy mathematics concepts, fuzzy transformation and fuzzy comprehensive evaluation.

Before introducing fuzzy transformations, fuzzy relations must be defined.

A fuzzy relationship is a generalization of a classical relationship. Classical relationship indicates that there is a clear relationship between two groups of things, such as greater than, less than, including, subordinate and so on, i.e., it has a clear correlation. While fuzzy relationships indicate that the relationship between two groups of things is not clear, and their relationship has some fuzzy correlations.Therefore, fuzzy relationship is a depiction of fuzzy correlation between two groups of things.

Suppose there are two finite sets U = {u₁,u₂,⋯,u_n},V = {v₁,v₂,⋯,v_m} that R satisfy if R is a fuzzy relation between U and V: (3) R=| r11r12⋯r1mr21r22⋯r2m⋯⋯⋯⋯⋯⋯⋯⋯rn1rn2⋯rnm | \[R=\left| \begin{matrix} {{r}_{11}} & {{r}_{12}} & \cdots & {{r}_{1m}} \\ {{r}_{21}} & {{r}_{22}} & \cdots & {{r}_{2m}} \\ \cdots & \cdots & \cdots & \cdots \\ \cdots & \cdots & \cdots & \cdots \\ {{r}_{n1}} & {{r}_{n2}} & \cdots & {{r}_{nm}} \\ \end{matrix} \right|\] where the fuzzy set X = {x₁,x₂,⋯,x_n} on U in (1) and the fuzzy set Y = {y₁,y₂,⋯,y_n} on V satisfy relation Y = X ⊗ R, then R is said to be a fuzzy relation from fuzzy set U to fuzzy set V.

If the above fuzzy relation is defined in another way, it can be described as follows.

Then assume that X and Y are two theses, and X ⊗ Y denotes the product of X and Y, i.e. X ⊗ Y = {(x,y):x ∈ X,y ∈ Y}. A fuzzy set R on X ⊗ Y becomes a fuzzy relation from X to Y. For a given (x,y) ∈ X ⊗ Y, R(x,y) denotes the degree to which x and y have relation R. If X = Y, then R is said to be a fuzzy relation on X.

Since fuzzy relations are arithmetic relations built on sets, which are fuzzy sets on products, fuzzy relations have the same three operations of intersection and complement as classical relations.

If matrix R is a fuzzy matrix: (4) R=| r11r12⋯r1mr21r22⋯r2m⋯⋯⋯⋯⋯⋯⋯⋯rn1rn2⋯rnm | \[R=\left| \begin{matrix} {{r}_{11}} & {{r}_{12}} & \cdots & {{r}_{1m}} \\ {{r}_{21}} & {{r}_{22}} & \cdots & {{r}_{2m}} \\ \cdots & \cdots & \cdots & \cdots \\ \cdots & \cdots & \cdots & \cdots \\ {{r}_{n1}} & {{r}_{n2}} & \cdots & {{r}_{nm}} \\ \end{matrix} \right|\]

In (4), 0 < r_ij < 1,i ≤ n,j ≤ m, and two fuzzy vectors X = {x₁,x₂,⋯,x_n}, where 0 < x_i < 1,(i = 1,2,⋯,n), Y = {y₁,y₂,⋯,y_m}, where 0 < y_j < 1,(i = 1,2,⋯,m) are said X · R = Y to be fuzzy transformations if they exist with Y = X · R.

If the set of judgments V = {v₁,v₂,⋯,v_m}, the set of factors U = {u₁,u₂,⋯,u_m}, and further assuming that the one-factor judgment matrix for the ird factor is R_i = {r₁1,r₁₂,⋯,r_ij,⋯,r_m}, then matrix R can be considered as a fuzzy subset on the set of judgments V. In matrix R_i, r_ij denotes the affiliation of the ith factor’s judgment to the jth rank, so the total judgment matrix of the n factors can be expressed as: (5) R=| R1R2⋯⋯Rn |=| r11r12⋯r1mr21r22⋯r2m⋯⋯⋯⋯⋯⋯⋯⋯rn1rn2⋯rnm | \[R=\left| \begin{matrix} {{R}_{1}} \\ {{R}_{2}} \\ \cdots \\ \cdots \\ {{R}_{n}} \\ \end{matrix} \right|=\left| \begin{matrix} {{r}_{11}} & {{r}_{12}} & \cdots & {{r}_{1m}} \\ {{r}_{21}} & {{r}_{22}} & \cdots & {{r}_{2m}} \\ \cdots & \cdots & \cdots & \cdots \\ \cdots & \cdots & \cdots & \cdots \\ {{r}_{n1}} & {{r}_{n2}} & \cdots & {{r}_{nm}} \\ \end{matrix} \right|\]

In (5), if Y = X · R, Y is the result of the combined judgment of all secondary indicators of u_i.

Generally, in practice, the result of fuzzy transformation is related to the operator used. If the fuzzy transformation operator uses the weighted average analysis operator M(*,⊕), it can take into account the weights of all the evaluation indicators, so in the case of the need to optimize the overall indicators, it is more appropriate to use the weighted average analysis.

On the other hand, for a certain content to be evaluated, when the item to be evaluated involves multiple indicators, it can be expressed as a set of multiple indicator factors U. If the evaluation involves evaluating with multiple levels of rubrics, it can be expressed as a set of rubric factors V. Using the weighted average type analysis operator, it is also necessary to establish the weighted average type analysis weight distribution fuzzy vector A. Therefore, when using the principle of fuzzy comprehensive judgment to model the comprehensive quality assessment model of college students, the following steps are followed:

1) Determine the evaluation indicator set theoretical domain U, let U = {u₁,u₂,⋯,u_n}, where n is the number of indicator items.

Through the online operation of the comprehensive assessment system for students’ quality education, the effective management of information for comprehensive quality assessment can be realized. By utilizing the technology of data collection, processing and display, it can provide rich and accurate data support for the informatization management of students, ensure the accuracy, scientificity and timeliness of the comprehensive quality assessment, and save the management resources of the school. In the design and realization of the system, the following objectives are to be met:

The PC terminal is mainly used for basic system settings or activity release settings, and only the administrator has the authority to release activity.At present, the administrators are defined as the relevant cadres of student unions, teachers of faculties and departments, relevant teachers of the Youth League Committee, and system administrators, collectively referred to as administrators.

After calculating the index weights and constructing the evaluation model, it is necessary to further test the operability and validity of the constructed assessment model in practical application, and it is necessary to carry out the reliability test and validity test on the intelligent assessment model of humanistic literacy in nursing. The evaluators consisted of five people, one senior teacher, one professor, one nursing teacher, one graduate student of nursing and one nursing trainee of a university in Beijing, and the five raters were labeled as evaluators A, B, C, D and E. The evaluation was done by disseminating questionnaires, and the evaluators employed the indicators in the evaluation index system as evaluation indexes, based on their existing experience and understanding. In the evaluation process, the evaluators, based on their existing experience and knowledge, take the indicators in the evaluation index system as evaluation points, make detailed records based on the actual situation of the evaluated samples, and score the reflections of the evaluated samples under each indicator according to the description of the indicators, and calculate the total score according to the formula of the evaluation model, and the scoring adopts the percentage system.

At the end of the evaluation, according to the scores of the five evaluators on the three teaching samples with the topic of “humanistic literacy in nursing”, the statistical analysis software was used to analyze the data and compile the final test results.The results of the independent evaluation of the three humanistic literacy samples according to the evaluating index system and the evaluation model constructed by the five raters are shown in Table 4. The results of the independent evaluation are shown in Table 4, with a score of 100. Based on the scores of the five raters on the three humanistic literacy samples of nursing on the same topic, it can be seen from the table that the teaching samples 1 and 2 are highly recognized, and sample 3 has a lower score, and the scores are decreasing from the score averages, so it can be seen that the scores of the three teaching samples are in line with the hierarchical nature of the sample selection.

The Kendall harmonic coefficient was used to test rater reliability, and the scores of each sample were entered into the SPSS software and used to perform ANOVA and Friedman tests1, and the results are shown in Table 5. The significance of the ANOVA chi-square test was 0.095,7 which is greater than 0.05, which indicates that the 3 samples of humanistic literacy in nursing are chi-square and meet the prerequisites for further analysis of variance (ANOVA), and the significance of the ANOVA was 0.757, which is much greater than 0.05, which indicates that there is no significant difference in the overall data of the evaluation. As can be seen from the table, the Kendall’s harmony coefficient of W = 0.851 (p = 0.015 < 0.05) for the scoring of the 3 teaching reflection samples by 5 experts indicates that the consistency of the raters can reach more than 95%, which is good consistency and good reliability, which represents a reasonable reliability of the model of the humanistic literacy intelligence assessment in nursing.

At the end of the evaluation, based on the completion of the questionnaire by 5 experts, the results of the evaluation were preliminarily analyzed using Excel to calculate the I-CVI of each indicator, and the results are shown in Table 6. When the number of experts is equal to 5, the value of I-CVI must be authoritatively 1.00 before the content validity of this entry can be considered good. After evaluating the content validity of each indicator, it can be found that out of 9 indicators, 7 indicators have a value of I-CVI of 1.00, and 2 indicators have a value of I-CVI of 0.90, i.e., these two indicators are recognized by only 4 experts.

Considering the reason of randomness, according to the I-CVI value estimation table under different expert numbers as shown in Table 7, the overall result of the content validity of the index model is shown in Table 8, and after the random consistency correction, the K⋆ calculated is 0.784, which is greater than 0.740, indicating that the experts believe that there is a good correlation between the measured content of these indicators and the content to be measured, indicating that the content validity is excellent. On this basis, the content validity of the whole indicator system is continuously evaluated, and S-CVI is divided into two types according to different calculation methods: one is S-CVI/UA, that is, the ratio of the number of indicators rated as “3” or “4” by all experts to the total number of indicators; The second is S-CVI/Ave, which is the average value of each indicator I-CVI. The calculated value of S-CVI/UA of this indicator system is 0.84, which meets the requirement of no less than 0.8. The value of S-CVI/Ave was 0.98, which was greater than 0.9, which met the statistical requirements, which indicated that the content validity of the index was good, that is, the intelligent assessment model of nursing humanistic literacy had good content validity both for a single item and as a whole.

The above analysis results in the conclusion that the assessment model is reliable and can be used as an assessment tool for teachers to evaluate the humanistic qualities of nursing students.

In this study, 45 subjects were randomly selected from all the subjects for the questionnaire survey, and four front-line teachers who adopted the assessment system were interviewed, in order to maximize the evaluation of the effectiveness of the use of the assessment system in terms of its application, and the specific questionnaire structure table is shown in Table 9.

In this study, the recovery rate of the questionnaire was 100%, and the valid questionnaire rate was 100%. Statistical analysis of the collected valid questionnaire data showed that since the survey mainly focused on the degree of students’ agreement with the use of the intelligent assessment system, the study only summarized the proportion of “strongly agree” and “agree” in the results table, and the specific statistical results are shown in Table 10 below.

Regarding the design of the assessment system itself, there are 5 questions, among which 93% of the students think that the interface design of the assessment system is simple and elegant, which is in line with their own cognition and preferences; 96.3% of the students thought that the operation process of the assessment system was very simple, and they only needed to enter the account number and password to achieve the login function. 93.5% of the students thought that the assessment system was convenient and could easily meet the needs of their computational thinking level test, but others thought that the limitations of network conditions might make the assessment system less convenient. 97.5% of the students thought that the font size of the questions in the assessment system was appropriate. Ninety-four percent of the students found the system to be clear and easy to navigate.

This function is important for teachers and educational administrators in analyzing and evaluating the overall humanistic development level of the participating subjects. The assessment system applies to students in the classroom, resulting in the classroom of the participating students matching the corresponding teacher user. The performance of the participating primary school students in the dimensions of humanistic literacy development level was statistically analyzed with different classification criteria of the classes, and the differences in their performance and the validity of the assessment system were further explored, mainly as follows.

Based on the fact that this study follows a combination of random and stratified sampling for the selection of samples, there are some differences in the humanistic literacy development level of students at different levels of hierarchy, as shown in Figure 3. For example, in nursing class 1 and class 3, the course performance results of students in class 1 are usually better than those of students in class 3, and the evaluation data on the assessment system show that there are indeed differences in the humanistic literacy development level of students in these two classes, which is specifically manifested in the fact that the humanistic literacy level of the students in class 1 is higher than that of the students in class 3 as a whole, and the conclusion of this assessment is in line with the actual classroom performance of the students of the two classes. This conclusion is consistent with the actual classroom performance of the students in the two classes, which also verifies the effectiveness of the assessment system.

Figure 3.

Comparison of ct levels of different class students