Teaching efficacy of U.S. Air Force enlisted professional military educators during the COVID-19 pandemic

U.S. Air Force EPME prepares Airmen for leadership roles as they increase in both rank and responsibility (O’Neil 2020; Rivera & Shufelt 2016). Students are noncommissioned officers serving in roles which range from immediate supervisors to leaders responsible for managing, inspiring and motivating thousands of Airmen worldwide (O’Neil 2020; Bangari 2014; Rivera & Shufelt 2016). Air Force EPME consists of four levels of training, with Airmen attending each course as they advance in rank (AF/A1D 2018). There are over 500 instructors teaching the various levels of EPME across 80 schools globally (Thomas N. Barnes Center for Enlisted Education 2020a, 2020b, 2020c).

Tens of thousands of Airmen who are annually selected for promotion require EPME completion before assuming the next rank (AF/A1D 2018). Therefore, there was a need to continue courses during the CoViD-19 pandemic. Due to the necessity of continued EPME, resistance to change was not an option, and EPME instructors and schoolhouses were forced to adopt online delivery methods for courses that had traditionally been taught in face-to-face environments (Culbert 2020). Air Force guidance stated that “…distance learning instructors must complete the same qualification process as a traditional class room instructor” (CCAF 2017, p. 25) and did not mandate pre-service or in-service training specific to teaching online. Therefore, this study aimed to glean insight into the perceived self-efficacy of U.S. Air Force EPME instructors to better inform future decisions regarding training and development for online faculty.

Rapid technological developments have enabled the rise of effective and efficient online learning systems (Dhawan 2020). Online learning uses computer networks to allow for the possibility for students and teachers to use computers and internet-connections to interact anywhere, anytime and in a rhythm and manner which is most beneficial to learning (Cojocariu et al. 2014). Synchronous online learning allows the learning environment to mimic in-person instruction with live lectures, discussions and interactions between students and teachers, whereas in asynchronous environments, learning can occur in different times and spaces that are particular to each learner (Guo 2020; Dhawan 2020). During the COVID-19 pandemic, U.S. Air Force EPME leveraged both delivery methods (Culbert 2020).

Bernhardt (2018) noted that there are different varieties of data which one can analyse to examine the efficacy of learning environments such as student learning data (e.g. success in reaching learning outcomes), process data (e.g. cost, organisational processes) and demographic data (e.g. attendance, graduation rates). This study examined teachers’ self-efficacy by obtaining perceptions data directly from faculty members, since perceptions data is ideal for analysing thoughts, feelings and opinions of participants (Bernhardt 2018).

While Bandura (1977) defined the concept of self-efficacy as an individual's belief that they are capable of executing specific performance attainments, teachers’ self-efficacy refers to the self-efficacy of professional educators, and can be defined as the belief in one's abilities to “plan, organise, and carry out activities required to attain given educational goals” (Skaalvik & Skaalvik 2007, p. 612). Tschannen-Moran et al. (1998), in their comprehensive literature review, noted that higher senses of teachers’ self-efficacy bring forth feelings of perseverance, curiosity and drive to continuously improve pedagogic practices. Bandura (1977) noted that teachers with high levels of self-efficacy could maintain a positive outlook and succeed in situations where they feel unprepared or uncomfortable.

Teachers’ self-efficacy impacts teachers, students and schools (Stephanou et al. 2013). Skaalvik and Skaalvik (2019) noted a correlation between higher senses of self-efficacy and increased levels of student engagement. Zhang and Liu (2019) explained that teachers’ sense of self-efficacy plays a moderating role when examining motivational regulation, perceived task value and learner engagement, with the moderating effect increasing as the teachers’ sense of self-efficacy increases. Teachers’ self-efficacy has been examined from a myriad of angles, such as the difference in efficacy between male and female college professors (Chang et al. 2011), the difference in efficacy based on years of experience (Mehdinezhad 2012) and the difference in perceived self-efficacy of online teachers based on the age, education level, gender and number of classes taught online (Robinia and Anderson 2010).

Built upon foundational practices from eight previous methods used to measure teachers’ efficacy, Tschannen-Moran and Hoy (2001) crafted and validated an instrument known as the Ohio State teacher efficacy scale (OSTES), measuring the teachers’ sense of efficacy of K-12 educators. A factor analysis using pre-service teachers’ responses to the 24 items on the instrument identified three main factors: efficacy in instructional strategies (8 items; alpha = 0.91), efficacy in student engagement (8 items; alpha = 0.87), efficacy for classroom management (8 items; alpha = 0.90) and an overall combined OSTSES score (24 items; alpha=0.94).

The OSTES was further refined and validated when Robinia and Anderson (2010) modified the instrument (with permission) for use by nursing educators teaching online at the collegiate level. In addition to examining the three dimensions of teachers’ sense of efficacy within the OSTES, this revised instrument, known as the Michigan Nurse Educators Sense of Efficacy for Online Teaching (MNESEOT; see Appendix), brought the OSTES into the digital age by incorporating and examining an additional fourth dimension of teachers’ self-efficacy: computer use. The researchers validated the 32-item instrument by conducting a factor analysis, resulting in four factors: efficacy in online student engagement (0.93), efficacy in online instructional strategies (0.94), efficacy for online classroom management (0.93) and efficacy in the use of computers (0.86). A total score for the entire instrument (0.93) was also derived.

Robinia and Anderson (2010) administered the MNESEOT to 327 participants, receiving a 43% usable response rate, or 140 participants, from college-level nursing educators across the state of Michigan. Results indicated a positive and significant correlation between teachers who had met with an instructional support expert (such as an expert in educational technology or online delivery methods) and sense of teaching self-efficacy, as well as a positive and significant correlation between educators who had taken a course in online teaching and sense of teaching self-efficacy. Robinia and Anderson (2010) also noted a positive and significant correlation between the number of online teaching experiences and higher senses of teachers’ self-efficacy. Participants felt higher senses of teachers’ self-efficacy in the subscales of classroom management and instructional strategies, whereas student engagement was noted as the category where educators felt least confident when teaching online.

Horvitz et al. (2014) used a modified web-based MNESEOT instrument to ascertain the sense of efficacy of 91 college-level educators within a variety of organisations specialising in liberal arts and humanities, STEM and professional skills development. Results mirrored those of Robinia and Anderson (2010): the highest efficacy scores were observed in the instructional strategies and classroom management categories, whereas the lowest efficacy scores were observed in the student engagement category. Results indicated that there was a significant and positive correlation between the number of semesters taught online and the teachers’ sense of self-efficacy, indicating that as an educator has more experience teaching online, their sense of efficacy rises. Unique to the study was data indicating that educators affiliated with professional schools, as opposed to traditional colleges, displayed a significantly higher sense of efficacy in the subscale of computer use.

Aside from the MNESEOT, other instruments have been used to measure teachers’ sense of efficacy. Chang et al. (2011) examined the teachers’ self-efficacy of 513 university-level educators in Taiwan using the Faculty Teaching Efficacy Questionnaire, an instrument comprised of 28 four-point-Likert items. The items were affiliated with six factors: efficacy for course design, technology usage, instructional strategy, class management, interpersonal relation and learning assessment. Factor analysis results for each factor were consistently large, between 0.58 and 0.88. 73.59% of the total variance was accounted for by all six factors. The categories of internal consistency reliability for course design, instructional strategy, technology usage, class management, interpersonal relation, learning assessment and the total scale were 0.91, 0.88, 0.93, 0.90, 0.86, 0.87 and 0.95, respectively (Chang et al. 2011). Results indicated that educators’ sense of teaching efficacy was lower for teachers with five or fewer years of experience than their counterparts who had taught for 6 years or longer. Results also indicated that institutional investment in educators’ professional development, such as assigning coaches, mentors or specialists to hone the skill-sets of educators, positively correlated with teachers’ sense of self-efficacy.

Although teachers’ self-efficacy has been thoroughly studied within K-12 and postsecondary education (Norton 2013; Walsh et al. 2020; Avalos 2011; Loughland and Nguyen 2020), it is true that less is known about the self-efficacy of U.S. Air Force EPME instructors teaching leadership interventions at the collegiate level. This study aimed to continue the work of Robinia and Anderson (2010) by utilising their self-efficacy measurement instrument (with permission) to examine the variables that affect the self-efficacy of EPME instructors who were forced to teach online during the COVID-19 pandemic without specialised training to instruct in online learning environments.

Online EPME instructors were not required to be trained to teach in ways specific to online learning environments (CCAF 2017). With hundreds of instructors teaching online with no formalised training specific to online instruction, this study began with the following research questions:

How confident do EPME instructors feel in preparing, conducting and evaluating online courses? Is there a difference in online teaching efficacy in relation to the variables: (a) military rank, (b) years of teaching EPME, (c) number of online classes taught, (d) level of EPME at which the instructor is teaching, (e) instructor education level and (f) the qualification level of the instructor?

This study examined instructors who were not formally trained to utilise online instructional technologies. As Robinia and Anderson's (2010) results indicated a positive relationship between the number of online teaching experiences and increased self-efficacy levels for online teaching, as well as a positive relationship between working with instructional experts and increased self-efficacy, the hypotheses for this study were as follows:

Of the four dimensions of self-efficacy which were examined, the dimension of technology use will be the lowest amongst EPME instructors.

The instrument for this study replicated Robinia and Anderson's (2010) sense of efficacy survey. Since they were not tailored to any specific institution or profession, all 32 questions pertaining to the four dimensions of teachers’ self-efficacy (classroom management, student engagement, technology use and instructional strategies) used the exact wording from Robinia and Anderson's (2010) instrument. Modifications were only made to demographic questions at the end of the survey, as this study examined military professionals as opposed to nurse educators.

This study was undertaken to provide actionable data to U.S. Air Force decision-makers. Therefore, the question of gender was intentionally omitted from the background questionnaire, because the U.S. Air Force does not consider gender in recruitment, hiring, training, evaluation or retention of EPME faculty (Military Assignments Programs Branch 2020). This study viewed the EPME instructor as a gender-neutral position.

Cronbach's alpha for the entire instrument was established as 0.97. Based on a factor analysis of the 32 items in the instrument, four items were derived coinciding with Robinia and Anderson's (2010) four dimensions of teachers’ self-efficacy. Subscale Cronbach's alphas were computed for student engagement (0.92), instructional strategies (0.90), classroom management (0.84) and computer skills (0.85).

These data were collected using an online survey, and all questionnaires were stored and administered by utilising commercial survey software. After obtaining official e-mail addresses for all instructors from EPME headquarters, instructors were contacted through official U.S. Air Force e-mail, which requested their participation. Within the e-mail, a link provided directions and access to the survey. The survey was closed 2 weeks after the initial solicitation for participants.

This study employed descriptive statistics to describe the sample through frequencies and cross tab comparisons to population statistics. Means and standard deviations were calculated to answer questions posed within this research. When determining correlations between interval-level variables, Pearson's product–moment correlation coefficient was used. Analysis of variance was used to test differences between the means of the online teaching efficacy scores. This study used alpha of 0.05 throughout all tests.

Factor analysis was conducted to ensure items within the questionnaire measured the appropriate category of teachers’ self-efficacy. With a sample size of 129 participants, principal axis factor analysis with varimax rotation was conducted to assess the underlying structure of the 32 items of the instrument using SPSS version 27. Four factors were requested, based on the presumption that questions were designed to index four constructs: classroom management, technology use, learner engagement and instructional strategies. After rotation, four components emerged with eigenvalues greater than one, explaining 19%, 17%, 15% and 8% of the variance respectively. As recommended by Morgan et al. (2019), factors less than 0.3 were omitted to improve clarity, since the correlation matrix revealed a majority of coefficients of 0.3 and greater. Used to measure sampling adequacy, results from the Kaiser–Meyer–Olkin test were > 0.70, exceeding the recommended value of 0.6 (Pallant 2020). Bartlett's test of sphericity produced a value < 0.05, substantiating factorability of the correlation matrix (Morgan et al. 2019).

Varimax rotation indicated that 11 items loaded strongly on one component consisting of questions related to classroom management. Twelve items loaded strongly on two components consisting mainly of questions related to technology use. Nine items loaded strongly on a third component consisting of questions related to instructional strategies. Questions related to student engagement were represented across all four components.

The 32-item instrument directed participants to respond to items using a Likert-scale format ranging from nothing (1) to a great deal (9) in relation to questions pertaining to online teaching.

Mean subscale scores within the domains of classroom management, instructional strategies, student engagement and technology use were calculated (Table 3). These scores were then combined to create a total sense of efficacy score, ranging from 4 to 36. Generally, participants indicated they felt they could do more than “some” to “quite a bit” when tasked with preparing, conducting and evaluating online courses. No participant felt they could do “nothing” in any category. The mean score for the instrument was 26.92 with a standard deviation of 5.25.

One-way between-groups analysis of variance showed no significant difference in sense of efficacy scores for rank [F(4,123) =.957), p = 0.434], level of EPME [F(6,122) = 1.85), p = 0.95], instructor's education level [F(2,126) = 1.09), p = 0.34] or number of online courses taught [F(5,121) = 1.73) p = 0.13].

An independent samples t-test found no significant difference between instructor trainees (n = 9, M = 7, SD = 0.76) and fully qualified instructors (n = 116, M = 6.69, SD = 1.26), t(123) =.713, p = 0.48, or respondents who had a degree in education (n = 19, M = 6.57, SD = 1.24) and respondents without a degree in education (n = 107, M = 6.71, SD = 1.22), t(124) = −0.465, p = 0.64.

One-way between-groups analysis of variance showed a significant difference in sense of efficacy scores based on number of years having taught EPME [F(3,124) = 4.148), p = 0.008]. Responses regarding years of teaching EPME were further categorised as (a) less than one year, (b) between 1 and 2 years, (c) between 2 and 3 years and (d) three or more years, as there was noted a rise or fall of sense of teachers’ self-efficacy at each of those points (see Figure 1). The effect size was medium to large at 0.09 according to Cohen (2013). Post hoc comparisons using the Tukey's honestly significant difference test indicated that a significant difference was found between instructors who had taught between 1 and 2 years and instructors who had taught between 2 and 3 years (p = < 0.05).

Fig. 1

Participants were asked to identify whether they had any of five possible preparatory experiences for online teaching including: (a) having a teaching degree, (b) taking a seminar on online teaching, (c) taking a course on online teaching, (d) meeting with a peer mentor on a regular basis during an online teaching experience and (e) meeting with an instructional support expert during an online teaching experience. Affirmative responses were directed to additional questions which asked participants to rate their agreement that the experience prepared them in the skills necessary to teach online. A Likert-type scale was used with ratings of 1 = strongly disagree, 2 = slightly disagree, 3 = neutral, 4 = agree and 5 = strongly agree.

An independent sample t-test found significant differences in total sense of efficacy scores for instructors who had met with an instructional support expert during an online teaching experience (M = 7.09, SD = 1.07) compared to instructors who did not (M = 6.56, SD = 1.21) t(122) = −2.02, p = 0.04. Pearson's product–moment coefficient revealed positive and significant correlations between both meeting with an instructional support expert during an online teaching experience and overall sense of efficacy, with a medium or typical effect size (0.47) according to Cohen (2013).

Independent samples t-tests found no significant differences in the efficacy levels between instructors who had a teaching degree (M = 6.57, SD = 1.24) and those who did not have a teaching degree (M = 6.71, SD = 1.21), t(124) = −0.465, p = 0.66, those who had taken a course on online teaching (M = 6.57, SD = 1.30) and those who had not (M = 6.74, SD = 1.19), t(124) = −0.672, p = 0.5, those who had attended a seminar on online teaching (M = 6.7, SD = 1.30) and those who had not (M = 6.7, SD = 1.19), t(124) = 0.032, p = 0.97 or those who had met with peer mentors on a regular basis during an online teaching experience (M = 6.83, SD = 1.11) and those who had not (M = 6.57, SD =1.29), t(124) = 1.18, p = 0.24.

Hypothesis 1 was not supported, as the self-efficacy dimension of technology use was not the lowest reported among EPME instructors. Hypothesis 2 was also not supported, as there was no positive relationship between the number of online teaching experiences and increased self-efficacy levels for online teaching. Hypothesis 3 was supported, as EPME instructors who reported working with instructional support specialists also reported significantly higher levels of online teaching efficacy.