Development of student simulated patient training and evaluation indicators in a high-fidelity nursing simulation: a Delphi consensus study^†

Simulations were defined as the learning activities that were designed to help learners acquire insight into the interrelated knowledge structures within a specific context. Simulation-based learning consisted of the simulated scenarios, which use life-size simulators controlled by computer software and a simulated clinical setting.⁶ High-fidelity simulation (HFS) was an experiential learning through the interactions with the human body manikins, which presented actual physiological and pharmacological reactions. Knowledge was generated when people’s perspective was transformed as a result of the new experience gained through the exposure to the situation.⁷ HFS offered the students the particular learning opportunity to learn through practical experience, reflections and feedback.⁶ Students reacted to the simulated events and obtained the practical experience. Reflections encouraged them to think critically step-by-step through the experience and integrated experience with the cognitive domains.⁸ As a student-centered approach, simulation-based learning promoted the participative feedback between teachers and students, and developed the skills of communication, teamwork, and decision-making, and further helped students to better understand the role of a nurse.⁹ Simulation allowed the students not only to identify knowledge gaps and take responsibility for rectifying their shortcomings, but also to increase self-confidence, critical thinking, and psychomotor skills. The students demonstrated the increased problem-solving ability, communication competency, cooperation, and delegation skills after simulated learning. Students were trained as simulated patients (SP) by using a HFS, which allowed for realistic practical experience.¹⁰

Portrayed patient referred to a standardized or SP and was planted in a scenario by health professionals, who acted as a proxy for the client with a specific disease. SP was defined as the person who was trained to precisely and reliably play the role of a client with health concerns.¹¹ SPs realistically imitated the situation of real patients in order to help students prepare and improve professional knowledge and communication skills by working with the SPs.¹² SSPs have been used for training students’ clinical skills and communication skills¹³ as well as skills in the objective structured clinical examination for comprehensive clinical competency⁵. Previous studies reported positive impacts on nursing education especially in the development of nursing skills and critical nursing care.¹⁴ SP simulation was necessary to motivate nursing students’ abilities to establish therapeutic relationships with the real patient.¹³ SP simulation provided an active learning environment for students to gain the confidence in clinical practicums. The interaction with the SP improved the psychological and emotional fidelity to a greater extent than the interaction with manikins.¹³ Based on best practice for SP training, the training principles included were safety, quality professionalism, accountability, and collaboration. SPs should be trained for portrayal role, feedback, and complete assessment by a safe way that minimizes or reduces the stakeholder risk.¹⁵ Quality assurance and continuous improvement were necessary for the development of SP training or practice. The content and feedback technology of SP training were determined, which is the key to ensure the training quality.¹⁶

The purposes of this study were to construct SSP training content and evaluation indicators, and to explore their validity and reliabilities.

A descriptive design and Delphi expert consultations were conducted.

The recruitment of an expert panel was conducted through emails sent from the researcher as well as leaflet explaining the objectives of consultations. The expert panel consisted of 20 experts who were from Beijing, Shanghai, Guangzhou, Sichuan, Yunnan, Zhejiang, Inner Mongolia and Macao, China (Table 1).

The Delphi consultation was a technique for achieving consensus by the expert panel that expressed their opinions and comments on a particular issue.¹⁷ The Delphi method was a well-developed, completely anonymous group process, in which opinions were expressed to experts in the form of a consultation questionnaire.¹⁷ The preliminary dimensions and indicators were developed through the literature retrieval and collation analysis. The questionnaire included questions related to the research purpose, general information of the experts, self-assessment survey of expert familiarity and judgment basis, and importance consultations on the first-level and secondary-level indicators using a five-point scale (from 1 = not important to 5 = very important). In addition, a supplementary opinion column was set for experts to give their comments.

The questionnaires were sent via e-mail in order to allow experts to fully express their opinions. Responses to each item were collected and analyzed along with experts’ comments. The adding, revising, or dropping were done in the subsequent round consultations until panel consensus was achieved. After the collection of the questionnaires in the first round, the mean score and coefficient of variation (CV) of each item were calculated. The mean score of importance over 3.00 points and the CV <0.25 were inclusion criteria for item screening.¹⁸ The questionnaire was revised based on experts’ comments, and then was sent back to expert panel. Any modification of questionnaire was informed to the panel. The interval between two rounds was 2–3 weeks.

All quantitative analyses were performed using the Statistical Package for the Social Sciences (SPSS, 26.0 IBM Corp). Mean and standard deviation were calculated for the importance of each item. The experts’ enthusiasm was determined by response rate. The authority (Cr) was calculated as follows: Cr = (Ca + Cs)/2 (judgment coefficient Ca, familiarity coefficient Cs).¹⁷ The weights of judgment basis were assigned and presented in Table 2. The sum was reported as the judgment coefficient. The familiarity of the questionnaire was reported by experts with the help of the scoring values—1.00 = very familiar, 0.80 = familiar, 0.60 = general, 0.40 = unfamiliar, and 0.20 = very unfamiliar. The average was calculated as the familiarity coefficient. The degree of expert coordination represented the credibility of consultation and reflected the consensus of different experts. The coordination was analyzed by the coefficient of variation (CV) and the Kendall coordination coefficient (kendall w). The CV is the difference in experts’ understanding of the importance of a specific indicator. It was calculated by the formula as “coefficient of variation = standard deviation/mean score.” The CV greater or equal to 0.25 indicated a large difference between experts.¹⁷ The coordination coefficient (Kendall w) was analyzed by the non-parametric K-related-sample test. The larger W value indicated the better coordination. Analytic hierarchy process (AHP) was used to calculate the weight of each item, which showed the relative importance of indicator in the overall evaluation.¹⁷

The positivity coefficients were 0.952 in the first round and 1.00 in the second round. The judgment coefficient was 0.89 and the familiarity coefficient was 0.85, while the authority coefficient was 0.87. The authority coefficient over 0.80 indicated that experts had great power and confidence on consultation content.¹⁷ Table 3 shows that the coordination coefficient in the second round consultation was higher than that in the first round. It meant that the consensus of experts’ judgment on indicators was enhanced.

After the first round of expert consultation, the average score of each item was found to be over 3.00, so all items are retained. SSP training included role and responsibility of SP, script interpretation, plot performance and training for a rater, with a total of 17 indicators (see Table 4). The SSP evaluation included disease knowledge, role portrayal, and performance fidelity, and being a rater, with 20 indicators in total (see Table 5). Each indicator was scored as “2—passed, 1—needs to be improved, and 0—failed.” The accuracy of assessment was calculated by the ratio of the trainee’s scoring to trainer’s scoring (2 points ≧85%, 1 point 60 ~ 84%, 0 points <60%). The trainees with a total score rate of 85% or over were considered as the qualified SSP.

Based on the strategic key elements of SSP training (shown in Figure 1), a pilot study was conducted. The best practices for SP training were applied to ensure a safe learning environment, to train the roles of portrayal and raters, and to give the feedback to trainees during debriefing.¹⁵

Figure 1

The high-fidelity simulators were used to design the scenarios which were controlled by computer software. For example, the simulated scenario was about abdominal pain. The situation was performed in a simulated surgery department. Mrs. Lu, 25 year-old female, was admitted to surgery department. The training process was shown in Figure 2.

Figure 2

Trainers were the senior teachers qualified for SP training. Eleven voluntary trainees were recruited from senior Baccalaureate nursing students through advertisements in News-board. The mean age was 22.82 ±1.25 years old. They had experienced clinical practicums for 24.82 ± 4.85 weeks, and had participated in simulation learning for 12.64 ± 4.18 hours. The 20-hour SSP training included role and responsibilities of SP, context and format of scenarios, script read through, education regarding the disease, role demonstration and training for raters. The SSP performance of role portrayal was expected to be consistent and accurate, and also to understand and maintain confidentiality principles related to the simulated events. Students rehearsed using high-fidelity simulated scripts under the trainers’ guidance. Each session was videotaped in order to provide detailed feedback to the trainees. The feedback and debriefing were conducted in each session in order to help the trainees to improve their performance and reduce their anxiety during the simulated experience.

The open-ended questions were used to investigate trainees’ views on SSP training. Students considered that scenario design was realistic (100%), and they understood the patient’s feelings by playing the role of a patient (90.9%), had deeper understanding of disease knowledge (81.8%), learned the interrogation skills (72.7%) and plot performance (63.6%), and improved communication skills (54.5%), but they encountered the difficulties in memorizing script lines by heart (63.6%), appropriate and timely response to the plot changing (54.5%), and attention to student’s performance (36.4%).

The content validity of SSP evaluation indicators was granted by three experts in nursing education, SP training and instrument development, and the content validity indicator (CVI) was reported as 0.95. The two qualified tutors observed trainee’s performances at the same time and marked the score based on the SSP evaluation indicators. The inter-rater reliability was calculated by the correlation of the scorings between two raters, and reported that the Kendall r value was 0.866 (P = 0.000). The reliability of internal consistency (Cronbach’s α) was reported as 0.727.