The Value of Postgraduate Students Opinions in the Quality Management of Academic E-Learning

It is quite universal that humans usually value the opinions of other users of a service or product before they decide to buy it themselves. Based on the experience of others, they often determine whether a given product or service is suitable for them and can meet their needs and expectations.

In e-learning, the opinions of other students also have a significant influence on the choice of the educational institution. Moreover, as emphasized by numerous authors (Almusharraf and Khahro, 2020; Segovía-García and Said-Hung, 2021; Watts, 2019), students’ perspectives and thoughts on those aspects of processes in which they are active participants are inevitable and essential criteria for creating highly qualitative educational context in a virtual environment. The disadvantages of these processes identified by students are a signpost for the university to introduce improvements. Moreover, as Żuraw (2015) claims: “determining the level of satisfaction in the case of student surveys allows assessing the quality of education in a given field of study and university, as well as to formulate reports and summaries that are the subject of rankings prepared, for example, by government institutions” (p.172).

In Poland, many universities and public opinion research institutions surveyed students’ opinions on the quality of e-learning during the Covid-19 pandemic. These studies usually present unsatisfactory results, and they showed that some universities “somehow” managed to go from offline to online and conduct classes in a virtual learning environment. However, this “somehow” did not meet the expectations of many students.

This was confirmed by the results of a nationwide survey conducted by Flow Centrum Research (2020) among 1,232 students from 76 Polish universities on the students’ opinions about distance learning in the pandemic era. As many as 41% of the surveyed students believed that the quality of distance learning is worse than in traditional education. The reduction of this quality is influenced by various factors, including the low quality of teaching materials, few practical exercises, low level of communication with lecturers, and relatively low competencies of lecturers in the field of conducting remote classes. As we can read in one of the reviews: “Some lecturers based their cooperation on sharing presentations, which without their comments make little sense […]. Practical exercises were often challenging to perform without consulting colleagues due to the lack of knowledge of the lecturers supplementing their presentations and training instructions” (UAM, 2020, p.2).

According to the report of the research team from the Institute of Philosophy and Sociology of the Pedagogical University in Krakow on the opinions of students of the University of Life Sciences on e-learning (Długosz, 2020), the main disadvantages of this form of education, mentioned by the respondents, included: overload with tasks to be performed, too much self-study, lack of contacts with teachers and peers, and decreased motivation to work independently. The main advantages of e-learning, though, were reported as: saving time by not commuting to the university and the possibility of studying in comfortable conditions with time flexibility.

In the opinion of the Medical University of Warsaw students (CDEM, 2020), the strengths of e-learning were the possibility of studying anywhere, the possibility of returning to the didactic materials placed on the platform, and the repeatable way of delivering content. In their view, learning was smooth, thanks to the use of new technologies. As for the disadvantages, the students mentioned: the feeling of isolation; the lack of the possibility of substantive discussion in real time, reducing the motivation to study; the lack of direct contact with the lecturer; and complex communication with the lecturer.

As claimed by the students of the University of Economics in Katowice, e-learning is a convenient form of studying. Still, it will not replace face-to-face meetings with lecturers. Among the positives, the students indicated the possibility of using intuitive Information and Communications Technology (ICT) tools and software that facilitate studying. On the other hand, they negatively assessed online exams, which exacerbated stress (Warchala, 2020).

A leading Polish Business School located in Central Poland has also conducted a pandemic-induced online learning quality survey of students to identify the strengths and weaknesses in this form of learning and to take action to improve those aspects of e-learning that have shown a good level of quality. The School is aware that e-learning will continue to develop regardless of some of its disadvantages and many students will prefer this form of education. Therefore, it wants to prepare for the future and offer online education at the highest level. This paper aims to present the results of the research in quantitative and qualitative terms.

Quality can be understood in many ways. According to the Organization for Economic Co-operation and Development (OECD, 2015), the quality of education is the one that ensures that all students acquire the knowledge, abilities, skills, and attitudes necessary in adult life. On the other hand, the United Nations Educational, Scientific and Cultural Organization (UNESCO, 2021), considering that high-quality education is the right of all those in education, has identified five main foundations that guarantee the quality of education: 1)

Equality: equal access to education for all students, equality in access to educational resources and infrastructure, and equality in students’ assessment;

Arias (2015) claimed that the quality of education is a complicated process, especially when it comes to e-learning, due to constant changes in it and the number of factors that determine this quality. For example, for García Peñalvo, et al. (2020), the quality of e-learning depends on the following factors: the technology used to implement the online course, access to various types of additional services, strategies for assessing student progress, course content, and the human factor (teachers’ competences). For Fainholc (2012), the determinants of e-learning quality are student satisfaction, the effectiveness of the learning process, positive learning outcomes, and positive social impact.

According to the UNE 66181 standard (AENOR, 2012), the quality of e-learning is related to student satisfaction. The factors that determine this satisfaction are the possibility of employment after completing the e-learning course (to what extent the completed study will increase the likelihood of getting a job or promotion), availability of the course and the teacher, and teaching methodology.

Moore and Kelly (2011) mention five pillars of e-learning quality: teaching effectiveness, student satisfaction, satisfaction of e-learning course leaders, cost-effectiveness ratio, and access to supporting materials and services. For the Online Learning Consortium (ONL, 2015), the quality of academic e-learning is influenced by institutional support provided to students, technological support, development and design of online courses, the structure of online classes, didactic strategies used by teachers during online courses, the support provided by teachers to students, and the methods for assessing student competences.

One can refer to the definition of e-learning by Harvey and Green (1993), who define it as a unique phenomenon (good idea for a course, compliance with specific standards), perfection or consistency (equivalent to zero defects and an appropriate quality culture), adaptation to the goal (adequacy of processes to achieve the set didactic goals), value for money (economic efficiency), and as a transformation (qualitative change understood as a continuous process of transformation of the student).

For Prodanović and Gavranović (2021), the quality of e-learning and student satisfaction with this form of education are related to the organization of online classes, teaching methods and dynamics, the variety and availability of teaching materials and necessary information, communication with lecturers, and other students, and students’ satisfaction with progress in science.

Marciniak and Gairín (2017), by analyzing 25 models of academic e-learning quality proposed by authors around the world, classify the determinants of this quality into six key groups: 1)

Institutional context (it is assessed, among other things, whether the management of the institution is focused on the development of e-learning, whether the e-learning courses meet the needs of the society, whether the technological infrastructure is appropriate for the implementation of education in a virtual environment, whether the institution has competencies necessary for the performance of e-learning courses, and whether the economic situation of the institution is sufficient to conduct these courses successfully)

As can be seen, the quality of e-learning is a complicated concept and has not yet received one universal definition. Some authors (UNESCO, 2021; ONL, 2015; Prodanović and Gavranović, 2021) associate it with the learning process, product, or services supporting online learning. Other authors refer to it as the perfect education system, with its effectiveness and positive learning outcomes (Moore and Kelly, 2011), and others define it based on the satisfaction of the leading actors of e-learning, that is, students (Harvey and Green, 1993; AENOR, 2012). We also believe that one of the main determinants of academic e-learning quality is the satisfaction of students, as they are the main clients and recipients of this form of education. Their opinions should be considered in planning, organizing, and implementing this form of education to ensure quality at the highest level.

The study was carried on MBA and Executive MBA (EMBA) students at one of the leading Polish business schools located in Central Poland. The study aimed to answer the following questions:

How did students of MBA and EMBA programs perceive e-learning during the Covid-19 pandemic, and what was their experience with it?

To find answers to the above questions, online surveys were conducted with students of the Business School. The main research tool was a survey on the Surveymonkey (in Polish and English) containing 17 questions, including 14 closed questions with two possible answers (yes, no), one question with ratings on a Likert scale from 1 to 5 (1- very bad, 2- bad, 3- hard to say, 4- good, 5- very good), one question asking them to specify which particular program they studied, and one open-ended question, which was not mandatory. The respondents also could comment on each of these questions, but it was up to them whether they wished to use it. For the analysis, the questions were grouped in terms of themes. Before sending the questionnaire, a mock survey was carried out on a group of 10 students using the telephone interview method to check the unambiguity of the questions. The research was conducted after the completion of 2020/2021 academic year.

The survey was anonymous and voluntary, which all the students were informed about. The questionnaire was sent by e-mail to all 176 MBA and EMBA students/or alumni affected by the restrictions due to Covid-19 and experienced online delivery as a forced mode of study. Seventy of them returned a completed valid questionnaire, which equals a 40% response rate.

Before sending the questionnaire, a pilot survey was conducted among 10 students using the telephone interview method. The purpose of this study was to make sure that the questions contained in the questionnaire were understandable and unambiguously defined and to find out how the respondents responded to some of the survey questions. After this pilot research, the questionnaire was revised based on the feedback received.

Having collected correctly filled-in questionnaires, their quantitative and qualitative analysis was carried out. A statistical program was used for the quantitative analysis, which allowed for the collection and analysis of numerical data obtained from the questionnaires. The qualitative research was carried out based on the students’ comments that justified their assessment and suggestions for improving the quality of e-learning.

The research results indicated that the surveyed students were satisfied with their e-learning classes (Table 1). Seventy-six percent of the respondents generally evaluated the quality of online courses as very good or good, and 69% of them valued the organization of such classes. In terms of the content of online courses, 75% of the respondents graded this aspect as good or very good and 63% appreciated the quality of teaching materials. Only a small number of students gave low grades to these aspects.

In terms of the didactic materials, they were rated as very high by the respondents. This meant access to various didactic materials and their attractiveness, incentivizing students to study (Table 2).

Some students commented that lecturers shared information on inspirational books or papers, which they found very helpful. They were also pleased to have all the didactic materials stored on the Moodle platform. Their suggestion for improvement would be to make sure that all these recommended readings are available in the School's library. Some students pointed out that although most of the didactic materials were of the highest quality, modern, and visually attractive, in rare cases, they were old fashioned and should be more engaging.

Contrary to the previous positive answers obtained about the online classes, in the case of issues related to networking with other program participants, a large group of respondents (Table 3) indicated that it was not sufficient.

In their comments, some students stated that networking was enough in the scope of performing joint projects. Still, involving the participants’ integration in a course organized by the School should be improved. Other respondents also pointed out that due to the Covid-19, some international business trips had to be cancelled, and this was disappointing as trips are great opportunities for networking and integration. There were also some comments that online delivery limits possibilities for building networking and long-term relationships. This is one of the most crucial parts of the MBA/EMBA program value.

The contact during classes or breaks between the students is not limited solely to them, but also includes the relationship between the student/s and the lecturer. The respondents were asked to assess whether this contact was sufficient and whether they received adequate support from the lecturers. In the case of lecturers, the connection and the support were rated high (Table 4). Students also appreciated the feedback they received from the lecturers for each didactic activity performed by them.

In their comments, respondents appreciated the possibility of having contact with lecturers after the classes. However, they still felt that a discussion over a coffee, where you can have face-to-face contact, is more valuable and they missed it. In terms of feedback, they felt it was of very high quality; however, there were some exceptions, which should be improved, as in some cases, the feedback was unclear and needed to be more precise.

Later in the study, students were asked to provide their opinion on their perception of the online classes’ delivery, regarding whether they were interesting, engaging, and dynamic. As shown in Table 5, most respondents assessed that the online courses they participated in were interesting for them. The results were not that spectacular in the other two aspects, but many respondents still perceived the activities as engaging and dynamic.

The students’ comments indicated that the engagement was there, but students were not participating as much as they probably would have on face-to-face deliveries. It was a challenge for the lecturers to gain much dialog, besides frequently inviting students to speak up.

When it comes to the students’ evaluation of the online classes’ assessment methods, the majority of the respondents considered them satisfactory. It confirmed that they were familiar with the evaluation criteria of the subject (Table 6). There were, however, some comments indicating that the assessment criteria, in exceptional cases, should have been more precise.

In the case of online delivery, a well-functioning e-learning platform is critical for its efficient implementation. As the survey results showed (Table 7), both the e-learning platform and the support from the School were rated as very high by the respondents.

In their comments, students praised the engagement and professionalism of the program coordinators, who were very supportive.

The last question asked students to specify which aspects of e-learning in their view need improvement and why. This question was not mandatory, and we received answers from 40 students. The results are presented in Table 8.

The majority of the students who responded to the qualitative question regarding their views on these aspects of e-learning indicated the networking needs improvement. Some of them suggested more group work and more interactive presentations. They recommended using online tools such as, for example, Mentimeter, Mural, or Kahoot, which would make the classes more vivid and dynamic. Gamification would engage students more in the classes.

Having cameras on mandatory during the classes would also improve students’ engagement. The respondents appreciated lectures and consultations delivered online, but suggested that all the classes are recorded and the recordings are made available to students. It would also be helpful to have more availability of lecturers after the lessons for extra consultations. The online delivery allows synchronous classes and asynchronous methods to be used. Some respondents appreciated introductory videos before the course, prepared some modules, and recommended applying such an approach to all the classes. Some students also pointed a necessity to make sure that no external noise is disturbing the classes, especially if the lecturer is delivering them from home.

Generally, the students’ approach to online delivery was very positive. They felt that after the pandemic, it could be beneficial to offer some classes online as this would save their time and would not require investing in travel and accommodation, especially if the students do not live in proximity to the university.

Measuring students’ opinions about the quality of e-learning is a complex process, mainly due to the number of factors contributing to this satisfaction (Wei and Chou, 2020). However, this measurement is necessary to improve the quality of this form of education and students’ satisfaction (Ramírez, 2020).

This paper analyzes the assessments and reflections of MBA and EMBA students on the quality of e-learning elements, such as didactic materials, online classes delivery, students’ communication with lecturers, strategies for measuring students’ achievement, networking, institutional support, and the quality of the e-learning platform.

The surveyed students assessed the level of e-learning offered by the School well. They graded the majority of aspects connected with their e-learning experienced as good or very good. Looking at some of their additional comments and especially their suggestions, it is possible to identify which e-learning aspects are crucial for them and which could be improved in the future. One of these aspects is the didactic materials. In the respondents’ view, a few lecturers’ materials seemed not very diverse and not attractively designed, and when improved, could attract the students more to study. There is no doubt that teaching materials are crucial in online education. The quality of these materials largely determines the quality of the entire learning process and participants’ satisfaction (Marciniak and Cáliz, 2021). The high quality of teaching materials also translates into a significant increase in the level of knowledge acquisition by students and, as the Spanish Association for Standardization and Certification (AENOR, 2012) points out, is effective in terms of teaching, technology, and the availability of teaching materials. Hence, we recommend universally that lecturers, when selecting didactic materials for their classes, focus on the most appropriate type, quality, and quantity of them, considering the didactic goals, content, and learning outcomes and the profile of students to whom these materials are addressed.

Another essential element influencing the quality of e-learning is the lecturers’ attitude, in particular, as stated by the students, the lecturers’ contact, support, and the feedback they provide to students about their performance. In the case of the surveyed students, these aspects were rated very high. The comments from some of the respondents stressed that it is beneficial to have an opportunity to have personal contact with a lecturer not only during the class, but also during coffee breaks, which unfortunately is not possible online.

Communication is the key to every teaching process. In e-learning, it becomes even more important, as this form of education requires constant interaction between the lecturer and students to counteract the possible isolation of some students. Systematic communication determines the success of a didactic undertaking, and above all, it is one of the factors motivating one to undertake an educational effort (Gawlik-Kobylińska, 2014).

When it comes to the feedback, it was generally assessed well by the respondents. However, some of them felt that in some exceptional cases, it was vague. Feedback, no matter if delivered orally or in writing, should always be clear and precise. The proper feedback for the student, rather than being long and imprecise, should be short and include mainly the comments on what was good at work, what needs to be improved, and how it should be improved.

Another essential factor in online delivery is whether the classes are perceived as interesting, engaging, and dynamic. That is a challenge, especially in the case of online delivery, as students can get distracted, bored, and tired much easier. Monotonous lessons quickly bore students, and inactivity or the lack of ambitious and inspiring tasks during these lessons does not encourage students to participate in them actively. The dynamization of classes is not an easy task, which is noted, among others, by Sorkosz (2021), arguing that for students, “participation in an online lesson is the training of mindfulness and the ability to register messages, which requires attention to strictly defined issues, tasks or situations. In remote learning, this task is challenging, the more that concentration of attention is subject to certain disturbances and is characterized by a variable intensity over time” (p.1).

Although increasing the dynamics of online lessons is difficult, lecturers should accept the challenge because if we want students not to be just passive subjects of the didactic process, they should be involved in this process. As Bonwell and Eison (1991) and Grace College (2021) postulate, they should do something other than passive listening; students should be involved and should actively participate in the entire learning process. Therefore, we suggest lecturers use various activating methods during online lessons, that is, those in which the activity of students will be higher than that of the lecturer conducting the online class. There is a wide variety of these methods, and their selection should be based on the learning objectives. Narrowing it only to the suggestions presented by the surveyed students, lecturers could activate students more by using even more often tools such as Mentimeter, Mural, and Kahoot, which, by their nature, enforce active participation. The request of having cameras on during the classes would also encourage students to be engaged in the co-creation of the classes.

Another significant qualitative aspect in academic e-learning is the criteria for assessing the activity performed by students, which, as the study showed, are generally very highly rated by the respondents. It is always crucial to inform students precisely about the assessment methods and criteria and select the appropriate ones. Imprecise and concealed would generate dissatisfaction among students. On the other hand, well-chosen, clearly defined assessment criteria allow for an objective, transparent, and fair assessment, consistent with the learning objectives and appropriate to the activity. They are a way of explaining what the lecturers assess and how.

The research results show that e-learning causes great dissatisfaction among students due to the lack or slight possibility of building relationships with other students. In the case of postgraduate studies, especially MBA and EMBA programs, networking is a significant factor. The lack of personal contact with other students throughout classes or breaks, during which students often exchange their opinions among themselves, is perceived by them as a significant weak point in online learning. This was visible in the case of the surveyed students, as the aspect of networking experience was rated insufficient by a lot of them. Moreover, according to World Bank experts (World Bank, 2020), “staying at home is also affecting students’ physical, mental and emotional health, as well as their vulnerability to engage in risk behaviors” (p.43). We agree with both the World Bank experts and Weber and Rothe (2013), who consider networking as an “indirect support of educational settings, in which they can serve as ‘social glue’ and help the students persist and be more successful by making them feel well connected and part of a community” (p.1957). We suggest more efforts to networking, such as “virtual coffee meetings” or chats before classes to have casual conversations.

It is also essential to appreciate the role of the School's support during the online classes, which is not only limited to lecturers, but also involves program coordinators and all the IT support as well as the e-learning platform. In the case of the Business School investigated, these aspects were rated exceptionally high. Students praised not only the support and engagement of the administrative members of staff, but also the highest level of their professionalism. Online delivery providers should never neglect these aspects, as they play a crucial role in the overall satisfaction of students’ experience.

Even though the study was conducted on quite a small sample of students, the obtained opinions provide helpful information for universities on the quality of postgraduate e-learning from the point of view of its main actors, that is, students. They show which aspects of e-learning should be paid particular attention to, so that its quality is at the highest level and meets the expectations of students in this regard.

The main limitation in the conducted study was the research sample. Although the questionnaires were sent to all the MBA and EMBA students who were studying during the pandemic, the feedback rate was at 40%. The study was limited to measuring the opinions of postgraduate students, and it is advisable to expand the profile of the research group and compare the results obtained to determine the similarities and differences in the perception of the quality of e-learning by students of different levels of study (both postgraduate and undergraduate studies).