Effect of simulation-based teaching on nursing skill performance: a systematic review and meta-analysis

: Objective: To summarize and produce aggregated evidence on the effect of simulation-based teaching on skill performance in the nursing profession. Simulation is an active learning strategy involving the use of various resources to assimilate the real situation. It enables learners to improve their skills and knowledge in a coordinated environment. Methods: Systematic literature search of original research articles was carried out through Google Scholar, Medline, and Cochrane Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases. Studies conducted on simulation-based teaching and skill performance among nursing students or clinical nursing staff from 2010 to 2019, and published in the English language, were included in this study. Methodological quality was assessed by Joanna Briggs Institute, and the risk of bias was also assessed by Cochrane risk of bias and the risk of bias assessment tool for non-randomized studies (ROBINS-I) checklists. Results: Initially, 638 titles were obtained from 3 sources, and 24 original studies with 2209 study participants were taken for the final analysis. Of the total studies, 14 (58.3%) used single group prep post design, 7 (29.1%) used high fidelity simulator (HFS), and 7 (29.1%) used a virtual simulator (VS). Twenty (83.3%) studies reported improved skill performance following simulation-based teaching. Simulation-based teaching improves skill performance among types of groups (single or double), study regions, high fidelity (HF), low fidelity (LF), and standard patient (SP) users. But the effect over virtual and medium fidelity simulators was not statistically significant. Overall, simulation-based teaching improves the skill performance score among the experimental group (d = 1.01, 95% confidence interval [CI] [0.69–1.33], Z = 6.18, P < 0.01, 93.9%). Significant heterogeneity and publication bias were observed during the pooled analysis. Conclusions: Simulation did improve skill performance among the intervention groups, but the conclusion is uncertain due to the significant heterogeneity. The large extent of difference among original research has necessitated the development of well-defined assessment methods for skills and standardized simulation set-up for proper assessment of their effects


Introduction
Simulation is an active learning strategy involving the use of various resources to assimilate the real situation. 1reover, it allows students to practice skills, exercise clinical reasoning, and make patient care decisions in a safe environment. 2It is also ideal for teaching reflective skills and management of patients in a crisis situation.performance in the nursing profession, this review followed the guidelines proposed by Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA).

Eligibility criteria
Literature published in the English language, original, which deal with nursing students or nursing professionals, and which compare any type of simulation with no simulation or traditional lecture-based teaching, were included.Moreover, studies available in full text that measure the effect of simulation on skill performance, and published between 2009 and 2019 (10-year review), were also included.But, qualitative study, interprofessional study, non-nursing study, review study, study population patient, observational study, and combination training (simulation-based + other, and then simulation alone) were excluded from the review and analysis.

Participants
Participants were undergraduate nursing students and clinical nursing staff.

Intervention
Intervention was based on simulation-based teaching (using low fidelity [LF], HF, medium fidelity, standard patient (SP), and virtual based teaching).

Control
No treatment or other conventional training such as interactive lecture alone or in combination with conventional manikin-based teaching.

Outcomes
The primary outcome was skill performance score after intervention.The term score was used because an inconsistency was observed in separate reporting of acquisition and retention of skill performance.For this review, skill score was used as a general term representing a change in skill performance score following simulation-based teaching.The skill performance score was taken as it was reported by original researchers.

Information sources
Study data were obtained from the databases of Google Scholar, PubMed, Cochrane database (CINAHL), and other references.Bland et al (2011) summarized features of simulation as a learning strategy, as it encompasses creating a hypothetical opportunity, authentic representation, active participation, integration, repetition, evaluation, and reflection.As a result, it promotes active learning, creative thinking, and high-level problem solving that can produce the capability of independent work among students. 3n contrast with this, the use of simulation also has disadvantages such as high cost, the need for staff development to manipulate the performance, limited time for training of faculty, and some chance of false transfer due to wrong adjustment of simulators. 4 Again, higher psychological preparation of students is needed since most of the simulation activities cause students to be anxious and frustrated. 5ome of the driving forces for current attention for simulation-based teaching are the patient bill of write, a greater need for high competency, and the changing trend of teaching approach from passive to experiential learning.Besides, a professional obligation to keep patient safety, difficulties to find clinical sites, and the greater need to provide high-quality clinical practice also influenced the current trends of teaching. 2n nursing, there was a lack of high-stake research that can provide strong evidence on the effect of simulation with a well-organized procedure. 6This indicates the need to conduct more investigations and arrive at a consensus on the issue among nurse experts.
The individual studies reported both negative and positive effects of simulation-based teaching.For example, in medicine, the use of high fidelity (HF) simulation is criticized for causing overconfidence in students that was even hampering their real practice. 7On the other hand, nursing literature also reported no effect of simulation on knowledge, skill, and confidence. 8As a result, this analysis aimed to narrow this gap by producing pooled evidence about the effect of simulation-based teaching over skill performance in the nursing profession.Moreover, this study considers the students and clinical nursing staff as a comparison group to ascertain differences, if any, in skill performance.
Simulation has many advantages and effects for learners and as well as the health care industry as a whole.[11]

Protocol and registration
To summarize and produce aggregated evidence on the effect of simulation-based teaching on nursing skill

Studies
Both non-randomized (quasi) and randomized original trail studies were included in the review and analysis.

Study selection
At first instance, literature were retrieved from original sources and merged using the software package EndNote X8 (reference management software) and an Excel sheet.Thereafter, the duplicate records were removed.Titles and abstracts were used for primary screening; then, the full text was used if needed.The two authors independently screened each study according to the inclusion criteria.Studies were included if they: (1) include undergraduate nursing students and/or clinical nursing staffs, (2) measure the effect of simulation-based teaching using various types of simulators, (3) use skill performance score as the primary outcome, (4) are randomized controlled trials (RCTs) or non-RCTs (quasi), and ( 5) produce sufficient data for calculation of sizes of effect.At the same time, the following criteria were used to exclude specific studies from the review process, including nonnursing, not assess simulation, interprofessional study, not original study, qualitative study, result that was not readily used as the report of median and different study populations.

Data collection process
The two review authors (AA and NA) independently extracted the data using an Excel sheet for a one-page summary.Accordingly, the information about the general overview of the article, the study design, country, population, sample size, intervention, the comparison, duration of the simulation, the outcome, and the methodological quality by JBI score checklist was filled over the pre-defined Excel sheet.

Risk of bias across studies
The risk of bias was assessed using the Cochrane Collaboration's Risk of Bias Tool for RCTs. 12This tool has 6 areas to assess experimental study and the authors decide to use the tool without modifications.Each study was scored (1) for a high risk of bias, (2) for the unclear statements about specific areas of bias, and (3) for low risk of bias.The non-randomized trials were evaluated against the Risk of Bias Assessment tool for Non-randomized Studies (Robins-I).Robins-I have 5 domains to be scored for individual studies.They are (1) bias arising from the randomization process, (2) bias due to deviation from intended interventions, (3) bias due to missing outcome data, (4) bias in the measurement of the outcomes, and (5) bias in the selection of reported result.Each domain is expected to report scores of low, high, or concern. 13he quality of the included studies was also done using JBI critical appraisal checklist. 14The tool was used to judge a study over 9 areas and researchers used 4 phrases with justification: Yes, No, Unclear, and Not applicable. 15Additionally, publication bias was tested by Trim and Fill methods to assess the effect of publication bias on effect size.

Summary measures
The composite score of skill performance reflects an overall aggregate score derived from various tools designed by the original researcher or adopted that were used to assess skill ability or performance before and after the experiment.The tools were varied in terms of their type, content, and number of points included in rubrics or checklists.

Synthesis of results
The analysis was performed by comprehensive metaanalysis version 2 (CMA) software.The quantitative description of pooled analysis was planned.The final discussion of pooled results is dictated by the level of heterogeneity obtained.Then subsequent subgroup analysis was done for the type of study groups, level of fidelity, study regions, types of participants, and types of outcome variables.The heterogeneity was assessed using the Cochran χ 2 test (Q-test) with the alpha level of significance set at 0.10. 16The degree of heterogeneity was also estimated and interpreted using the I 2 statistic Cochrane Handbook for Systematic Reviews of Interventions recommendations with the alpha level of significance set at 0.10, 12 which describes the percentage of total variation across studies that result from heterogeneity rather than chance.Finally, based on the final level of heterogeneity, pooled estimate was reported, discussed, and generalized to the group based on the significance level.The rest of the individual studies were included in a systematic review to avoid misleading readers.
The final size of effect was estimated and reported using a computed random standard deviation (SD) of mean difference (d) with a respective confidence interval (CI).This estimate is appropriate for effect size computed from a different study with different measurement context of outcome variables. 17

Risk of bias across studies
Assessment of quality of studies and risk of bias at study level was done by JBI and Cochrane checklist.Overall publication bias was tested by using Trim and Fill methods, which have a higher level of sensitivity to assess the effect of publication bias on effect size. 18

Patient and public involvement
This review had no contact with patients.All information was obtained from published studies and electronic databases.

Study characteristics
The included studies varied in terms of their design, the population used, and duration of simulation, type of test used to evaluate outcome variable, type of interventions, learning theory used, and level of fidelity in the simulator.Totally 2209 study subjects participated in 24 original studies with a maximum of 367 20 to a minimum of 30 21 sample size.The proportion of studies that involved clinical nursing staff amounted to 13.4%, while the rest comprised undergraduate nursing students (86.6%).A large proportion of individual studies came from Turkey (33.3%) followed by the USA (29%), in which both constituted more than half of all studies.Moreover, more than three-fourth of the studies were quasi-experimental (n = 20; 83.3%), (29%) used HF, (29%) also used virtual simulators (VSs), and (58.3%) used both control and experimental group (double group).The total duration spent for simulation intervention ranged from a maximum of 24 h 22 to a minimum of 20 min. 23The simulation duration was not clearly mentioned in 3 studies [24][25][26] (Table 1).
The control group was mostly taking the conventional or lecture method of teaching as a comparator or no intervention.The dominant scenario used by individual researchers was acute cases: mainly cardiopulmonary cases (41.6%).The second most common cases were drug dose calculation (8.3%), proper drug administration (8.3%), and securing peripheral intravenous line catheter and phlebotomy (8.3%) (Table 1).
To measure the effectiveness of the intervention, 12 (50%) used direct observation of skill performance using a checklist, 6 (25%) reported the use of OSCE, 4 studies (16.6%) used self-assessment of skill performance improvement, and 1 (4.2%) reported a rating of documents.In 3 studies the skill performance evaluation was assisted by VSs.Of this, virtual computerguided performance was used in 1 (4.2%), 4 (16.7%)used self-assessment, and another one (4.2%)used direct actual patient-based performance evaluations (Table 1).

Type of scenario
Different type of scenarios were used for simulation activity in all studies.Almost half of the scenarios were having the nature of acute cases, such as CPR, resuscitation, arrhythmia, deteriorating patient, pre-post case, and shock.The remaining scenarios were nonacute or cold cases such as medication administration, phlebotomy, diabetes mellitus (DM), and communication skills.

Quality of individual studies
The risk of bias in included studies ranged from unclear to high due to issues with 6 areas of risk of bias assessments for RCTs.These are random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting.From 7 studies, it could be ascertained that 5 of them scored moderate risk of bias while the rest were high risk of bias.Using Robins-I tool for non RCT, we discovered that 6 studies scored with no risk of bias, 7 with low risk of bias, 3 with moderate risk bias, and 1 with serious risk of bias.Moreover, from the total of 24 included studies, only 4 (6.7%)studies were categorized as high-quality research, 2 (8.3%) as low-quality research, and the remaining 18 (75%) studies ranked as medium-level quality studies.In most of the studies, quality issues were related to lack of control group, unclear outcome measurements, and failure to clearly state what treatment was given for study groups.

Result of individual studies
Even though individual studies reported additional outcomes as primary and/or secondary objective to their studies, this review considers and takes only the outcome related to skill performance.From a total of 24 studies, 20 reported positive effects of simulation-based teaching, while the rest reported a lack of evidence to support the positive effects of simulation-based teaching.
Simulation-based teaching improves skill performance among the experimental group with an overall random effect size of d = 1.01, 95% CI [0.69-1.33],Z = 6.18,P < 0.01.From this, it is understood that >79% of control group skill performance is below experimental group skill performance.But it is uncertain to conclude this finding because significant heterogeneity (I 2 = 93.9%)was observed during analysis.
Initially, 4 individual studies 26,33,35,37  over a participant's skill performance.At the same time, the meta-analysis also confirmed this by reporting a statistically insignificant effect size for those studies.(Figure 3; forest plot).

Subgroup analysis
Because of overall significant heterogeneity (I with types of study design, type of participants, study regions, and simulation fidelity.The heterogeneity level was maintained high despite variation in the effect size across the moderator variable analysis.

Effect of simulator type
Five types of the simulation were considered for this analysis.Except for medium fidelity simulator (MFS), all of the simulation types scored large effect size favoring the skill performance score among the experimental group.But only the low fidelity simulator (LFS) obtained a larger and statistically significant effect size with an acceptable level of heterogeneity d = 0.89 (CI [CI 0.24, 2.29], P = 0.02, I 2 0%).This group of studies involved study participants.We are confident that using LFS improved the skill performance of the experimental group (Table 2).

Types of group
The effect of group type used for the individual study was tested for all studies as subgroup analysis, which was tested as to whether individual studies used single pre-post or double group pre-post design.The single group pre-post users score large effect size d = 1.02 (CI [0.52, 1.50], P < 0.01).Again, the double group also score almost similar effect size d = 1.00 (CI [0.56, 1.44], P < 0.01).In both cases, significant heterogeneity was observed.So, it is understood there is no effect on size, whether we have used a single group or double group for the experiments (Table 2).Here also, we have no confidence to discuss the pooled analysis due to significant heterogeneity observed during analysis.But it is visible that the effect size was almost similar and statistically significant (Table 2).

Study sesign
There is no difference in whether RCT or quasiexperimental design was used to evaluate the effect of simulation on skill performance.The skill performance score was increased among study experimental group participants.The effect size for 7 RCTs was d = 1.14 (CI [0.54, 1.75], P < 0.01) and for the rest of quasiexperimental was 0.96 (CI [0.57, 1.34], P < 0.01).In both cases, considerable heterogeneity precludes us from drawing a conclusion and recommending the result (Table 2).

Types of scenario
Another comparison was done to ascertain whether nursing skill performance was different due to the use of categories of scenarios.The scenarios were categorized as acute and cold cases.The effect size for both groups of scenarios was similar and considerable heterogeneity was observed in both cases.Thus, we can conclude that in the current study, types of scenarios used for simulation have no effect on nursing skill performance (Table 2).

Sensitivity analysis
The pooled effect size was tested for a possible change by one study remove method.Accordingly, there is no large change over the overall effect size due to the removal of individual studies one by one.The maximum change was observed (d = 1.11) when Stayt et al. 41 was removed from the analysis.Further, the minimum effect size (d = 0.97) was also obtained when Jaberi and Momennasab 34 was removed from the analysis.The overall variation was d = 0.13.Thus, it is understood that the removal of 1 study has no significant effect on overall effect size (Figure 2).

Risk of bias
The risk of publication bias was tested using 4 common methods.Except for Egger's regression (intercept = 2.61, P = 0.08), the Trim and Fill methods (d = 0.62, [0.28, 0.96]), classic Fail-safe N, and the Begg and Mazumdar (b = 0.35, P = 0.01), all confirm the presence of publication bias under the random-effects model.The point estimate and 95% CI for the combined studies is 1.01035 (0.69, 1.33).Using Trim and Fill, the imputed point estimate is 0.62 (0.28, 0.95) (Figure 4).

Discussions
This review and meta-analysis were intended to present the result of the review, and produce a pooled estimate regarding the effect of simulation-based teaching on nursing skill performance in nursing.Most of the studies were from developed and middle-level countries, and original researches were varied in terms of the study context such as the types of the scenario used, the number of study participants, the duration of the simulation, and tools to measure outcomes.Moreover, the pooled estimate of included studies did prove the positive effect of simulation-based teaching in improving nursing skill performance.Since significant heterogeneity was observed during analysis, the reader needs to use the pooled analysis result with caution.The agreement among specific studies on the simulation was not complete.Some studies 26,[33][34][35]37 still reporting inconsequentiality of simulation-based teaching got improving skill performance in nursing. Thisgives an assignment for researchers to answer why, and users to continuously assess their success after the implementation of the simulation.The simulation-based teaching helps learners or users to assume the complexity of health service delivery and allow repeated exercise.10 Moreover, participation in simulation decrease mistakes in actual practice and increases flexibility during practice.45 In the current review, regardless of simulation types, the effect of simulation over skill performance showed a larger effect size that favors the users, which is consistent with a systematic review done by others.9,[46][47][48] In contrast with the result obtained in overall effect size, some individual studies reported and scored result that shows lack of evidence to prefer the use of simulation from traditional teaching method.26,[33][34][35]37 This indicates a need for further evidence and searching for potential factors significantly affecting the success and failure of this teaching strategy.Another factor may be the level of information contamination among controls and experimental groups.A significant number of specific studies were not strict on blinding participants and evaluators of performance.
This review and meta-analysis obtained significant heterogeneity in the overall and moderator analysis.Even though the sizes of effect were statistically significant, we lack the confidence to recommend this effect size due to large heterogeneity.Moreover, this might be due to a combination of studies with different scenarios, designs, and assessment tools.As a result, further work is expected from the nurse researchers to justify its effect confidently in a well-organized and standardized manner.
The larger proportion of studies was drawn from the developed and middle-level countries.Similar results were also reported consistently in various reviews and meta-analyses.This might be associated with a lack of financial support, simulation facility, and motivation on the part of the researchers to handle experimental studies that are accompanied by strict procedures.
We may think that high fidelity simulators (HFS) are better than LFS, 49 but the current review shows the opposite.The estimated effect size for LF was higher for LF with an acceptable range of heterogeneity.Even in medicine, the students prefer LF, focused, and shorter duration of the simulation. 50Again, Massoth et al., reported in 2019 that LFS helped to improve skill performance as compared to HFS, and the HF was criticized for letting the students have overconfidence. 7Again, another RCT reported HF that had no effect on students' retention of neonatal resuscitations. 51he students' preference as well as larger effect size in LFS-based teaching may be associated with the extent of time spent in simulation and mental adjustment of students for the simulation environment.It tends to happen that that students spent more time in LF.Moreover, the level of anxiety at the time of teaching in LF may also favor learning.Another justification may be the distracting nature of HFS from basic concept learning by increasing extraneous cognitive load; this was also given as the reason for impaired learning in HF simulation room. 52n contrast with the current study, many reviews of original studies showed a higher advantage of HFS than LFS in neonatal resuscitation, 9 identification, and management of deteriorating patient, 46 and performance of basic life support. 53As a controversial finding, having different types of fidelity levels has not shown a significant difference in student skill performance in all types of simulation.This result indicated not to depend on the level of fidelity and has rather resulted in the revelation that use of the mixed method may be more advisable. 10Again, it helps us to conclude that focused training, student handling, and duration of simulation matters more than types of fidelity used.Thus, the upcoming research needs to identify and address the factors that determine success in using simulator other than changing fidelity.
The use of standardized patients is preferred for the noninvasive procedure and skills, such as physical examination, history taking, communication exercise, and improvement of confidence for clinical skill management.This review also identified the use of standardized patients as a simulator improves the skill performance of participants with large effect sizes.Oh et al. (2015), show that the use of standardized patients improves communication skills with large effect size. 54

Conclusions
Assisting teaching with simulation did improve nursing skill performance.Again, the use of simulation-based teaching showed a positive effect both for student and clinical nursing staff training.The level of fidelity showed little difference and even LFS produced a greater effect size than others.Along with investing in equipment and teaching aid, equal attention should be given to faculty development to improve the style of teaching, student handling, and facilitation of teaching sessions.Since most studies were done in simulated environments, their application and significance for actual patient care need to be proved with further research.

Strength and limitations
Analysis of single outcome of simulation-based teaching aid is understood to cause focused result and implication.Moreover, focusing on the most important aspect of nursing education (skill) also helps to inform the most important aspect of nursing.
The confidence in generalizability and overall recommendation is limited by significant heterogeneity in the pooled analysis.Variety and difference in the type of scenario and outcome measuring tool were the major challenges of these combined studies.
The scope of the literature search was narrow due to the subscription challenge, which might reduce the depth of the literature search.Bias may also be introduced during searching, screening, and selecting literature, which directly affect the pool of literature for the final analysis.The number and quality of included and excluded literature were dependent on the critical appraisal ability of researchers.Again, this review was not specific and it considers every study that assessed a skill performance while they were using a different scenario, and research context that ends up with significant heterogeneity.The true effects of simulation-based teaching may be obscured due to the inclusion of freely available literature.

Figure 1 .
Figure 1.Flow diagram showing the process of study identification and selection.

Figure 4 .
Figure 4. Funnel plot showing publication bias among included studies.

Table 1 .
already reported that simulation has no statistically significant change Characteristics of included studies.

name Statistics with study removed Std diff in means (95% CI) with study removed
2= 93.9%), subgroup analysis with moderator variables were done Forest plot showing the effect size of individual studies.

Sensativity Analysis with one study remove Random Effect analysis(Isquare, 93%) Figure
3.Forest plot showing sensitivity analysis by one study remove method.

Table 2 .
Summary of effect size for subgroup analysis.