An analysis of team projects outcomes from student and instructor perspectives in online computing degrees

Team working is an essential skill required in any university graduate and is particularly important in programmes aiming to impart computing knowledge (Shneiderman, 2016; Vivian et al., 2013). The stereotype of the solitary coder, wearing a hoodie and hunched over a machine with three (sometimes up to five) screens is becoming a thing of the past (Cheryan et al., 2015; Vera, 2021). Employers are keen to ensure that their computing employees can function in a team (Hiter, 2021; Riebe et al., 2010; Vogler et al., 2018), and some even use this as an interview differentiator. Group work is highlighted as an assessment example by the Quality Assurance Agency (QAA) benchmark statement for master’s programmes in computing (Quality Assurance Agency for Higher Education, 2019). In fact, peer assessment is seen as offering an opportunity for “innovative and flexible means of assessment” (Quality Assurance Agency for Higher Education, 2019). In line with industry expectations, Professional Accreditation, Professional Statutory and Regulatory Bodies (PSRBs) such as The Chartered Institute for IT (BCS) in the UK, also require the inclusion of team-working skills in the development and delivery of applicable Computing Programmes (The Chartered Institute for IT [BCS], 2020). Furthermore, the social constructivist approach to learning is based on the concept that knowledge develops because of learners’ interactions with others, and that learning is therefore a shared experience, not an individual one; this extends ideas around the importance of synchronous learning in distance education (Peterson et al., 2018). Considering these requirements, many higher education (HE) institutions ensure team projects are included in their curriculum design. Both the research literature and experiences of the research team reinforce the fact that team work is challenging for students and can be resisted (Tucker & Abbasi, 2016). These effects are particularly amplified in the online learning environment (Smith et al., 2011), as utilised by the University of Essex Online (UoEO), the institution which is the focus of this case study. With an increasing move to blended or fully online delivery, considering the forced migration during the coronavirus (COVID-19) pandemic, there is a need to ensure the utilisation of an effective means of the delivery and assessment of team projects. A selection of steps taken to facilitate effective team work practices is discussed in the literature, for both the face-to-face (Gardner & Korth, 1998; Hosam & Noria, 2018) and online learning environments (Falls et al., 2014). Research published as recently as 2021 indicates that this is an ongoing research challenge (Wildman et al., 2021) and one that commands further research today. Hence, this research aims to provide an in-depth understanding of a viable team project delivery process in an online environment for computing students.

In line with the benefits of team working as highlighted above, and the widespread industry demand for this skill, all postgraduate computing programmes offered by the University include team projects, which contribute 60%–70% of the overall mark for an applicable module. This responds to the programme learning outcome, ‘Working with others’, which is included in the Key Skills outcome of each programme: “Develop and apply key team-working skills by working with peers to develop and implement plans, as well as explore and develop viable computing solutions, as applied in, for example, software development.” Similar learning outcomes are specified for all applicable modules across the computing programmes; assessments that provide students with the opportunity to demonstrate the ways in which they meet this learning outcome are provided.

At this point, it is best to address the elephant in the room: are we focussing on team or group assessments? This needs clarification as there is a significant difference in how each of these operates. In a group project, there is usually very little interaction between members. Each member does a specific task without learning much from the other members of the team. On the other hand, in a team project, consensus must be reached as the members work collaboratively to achieve a common goal (Pursel, n.d). This is sometimes referred to as students working “as a group” (collaboratively) compared to working “in a group” (cooperatively) (Chiriac, 2014). The operational ramifications of this distinction constitute the most common reason why employers rate team-working skills so highly.. The question then arises as to how such skills can be developed in an academic setting.

Getting students to work as a group/team has been a regular feature in many HE programmes, cutting across disciplines. This is due to the noticeable increase in student achievement, collaboration, motivation, and self-efficacy (Chiriac, 2014; Wilson et al., 2018; Chang & Brickman, 2018). However, such tasks have not been implemented without equally noticeable challenges, which include negative student experience (non-participating members/free-riders, overbearing members, interpersonal conflicts) and implementation issues (composition, structures, and norms, assessing individual contributions) (Wilson et al., 2018; Thorn, 2020). These implementation issues can be exacerbated when dealing with virtual teams, as we have in the Computing Master’s programmes delivered by the UoEO (Morrison-Smith & Ruiz, 2020). The pragmatic issues observed concerning the virtual teamwork activities forming part of the online curricula of several university programmes have given rise the requirement for improving the online delivery of learning, since the elimination of online assessments from the curriculum is not an option for many institutions, as explained in the Introduction. (Thorn, 2020).

With the move to fully online/blended educational delivery, we recognise that a research gap continues to persist around team work processes that work, and that many in academia are keen to hear about how such processes can be utilised in their discipline. Though this study cannot guarantee outcomes in other disciplines, it does provide a starting point for review. Hence, to achieve the aim of providing an in-depth understanding of a viable team project delivery process in an online environment for computing students, answers to the following questions, based on student and instructor experiences (Chiriac, 2014), are required:

In what ways does team work contribute to (your) learning?

The interview/focus group questions utilised were based on research undertaken by Chang and Brickman (2018) and focuses on the core questions above (see Appendix A). The study by Chang and Brickman (2018) does encompass some of the processes used in fostering collaborations in teams. Their results enhance the outcomes of this case study, as well as bringing to the fore issues that still need addressing, which we do attempt to address in this paper.

The computing programmes delivered by the UoEO in the timeframe considered in this paper (2020–2021) are conversion master’s in computer science and cyber security. These programmes are aimed at noncomputing graduates, providing them with an entry route into the computing industry. It should be noted that we typically have more students in the Cyber Security Programme than in the Computer Science programme.

Delivery of the team assessments in these programmes is based on extended research and brings together best practices from various institutions (Chapman Learning Commons, 2016; Race, 2001; University of Waterloo, n.d. - a). The process involved in the delivery of the team work opportunity and good team-working practices in general is as follows:

An Introductory session in Week 1 of the module where the instructor allocates teams based on time zones and experience (based on the information available on their learning platform profile), while still maintaining diversity among team members (Freeman et al., 2017). The teams are then asked to draw up a team contract, assigning roles and tasks to each team member to ensure they can achieve the given goal(s) for the project/assignment in the given timescale. The signed contract is then shared with the module instructor. While it is not formally graded, it is used as an artefact that students are required to reflect on later in the module (see below).

This process has been utilised in four modules since January 2021, with generally positive feedback from both students and staff. The External Examiner for the MSc Cyber Security degree programme highlighted (2020–21 academic year report) this assessment format as an example of good practice and innovation. The four modules are the focus of the study and are titled as follows: Information Risk Management, Secure Software Development (Computer Science), Secure Software Development (Cyber Security), and Network and Information Systems Management.

A case study approach was used, relying on qualitative data to provide insightful information on the team project delivery mechanism. The qualitative data from the students were mapped against the quantitative assessment grades. The protocol used for this study is detailed as follows (Pervan & Maimbo, 2005; Yin, 2017):

Phase 1 – Case study design (defining the research questions and selecting the population for the study/setting boundaries for the study). The real-world case for our abstraction is the small group of students in the UoEO Computing programmes who take on a module with team-based assessments. The context of delivery is described above.

Students across the four aforementioned modules were invited to participate in the research project via online focus groups and/or individual interviews. The two instructors of these modules are part of the research team and provide their perspectives on the delivery in this paper. The modules include those taught in the MSc Computer Science and MSc Cyber Security Conversion programmes. Ethical approval was sought from and granted by the University before students were contacted. Information on the four modules used for the study is as follows (Table 1):

IRM, information risk management; NISM, network and information security management; SSD, secure software development; SSDCS, Secure Software Development (Computer Science).

In total, six students agreed to take part in the study, where one was only available for one-to-one interviews while the others took part in focus group discussions. The same set of questions was asked of all participants regardless of mode of participation. The output of these interviews (the next section) was collated according to discipline, to determine if this would affect the responses to the research questions posed. The viewpoints of the instructors are also captured in the next section, to provide further insight to the responses.

Core aspects of the student team work experiences will be discussed in the following section, correlated with the overall module performance of the students. It also includes instructor reflections on the implementation of this assessment design. Based on a qualitative evaluation of the performance of groups, we state our observations on the strengths of the team work experience.

Firstly, the results can be discussed from the perspective of the teams from the May-to-July-2021 run of the SSD module, as well as the instructor’s perspectives on the delivery and outcomes. The profiles of the students from this module who took part in the focus group are presented in Table 2. The outcomes of the interviews are reviewed in this section based on the research questions posed in the Introduction.

In what ways does team work contribute to (your) learning?

There is evidence that the team-working experience becomes a fundamental part of the student learning process, with Student B joking that he talked more with his team than he did with his wife. During the focus group, he admitted that ‘We met almost every night without any issues.’ Student B repeated throughout the module and the focus group that he believed that belonging to a ‘good group’ is a matter of luck. Trying to support this student’s learning in relation to the teamworking experience, he was given feedback to try to help him to explore this thought process more deeply, in the sense of how he might react if assigned into a team that was naturally less effective, to get the most out of the team. There was limited evidence provided, however, that these questions were examined by the student.

Considering in more depth beyond the focus groups the organisation of students into teams, for one cohort, the instructor assigned students into teams and one student contacted the instructor to ask if they could instead be in a team with at least one of two other named students. This was due to his familiarity with the students through past experience, and his knowledge of their approaches and compatibilities. He was subsequently allocated into a group of his choice, and overall, the team performed highly. Taking this experience into account, in a later cohort, the instructor offered students the opportunity to self-organise into groups. Interestingly, the students did not wish to make use of this opportunity and preferred to be allocated into teams by the instructor. Further of interest in relation to team organisation, it has been observed that teams that were allocated in SSD have self-organised into the same groups in a later module – the formation of bonds during one module could then be exploited further. However, the unpredictable nature of a design to self-organise or not makes it more difficult to offer a single approach that can be guaranteed to respond to the needs of all students, and it may be the case that a degree of openness is required by the instructor in the allocation and potential reallocation of teams at the beginning of a module.

Despite the discussion around the contract possibly playing a superficial role during the focus group, there was a scenario where the contract did have a role to play, in the resolution of a dispute between two students. One student was dissatisfied with the contribution being made by another team member, and instead of speaking directly to the team member, as had been agreed in the team contract, she first spoke to the instructor. When the instructor then spoke to the other student, he was confused about the reasons why the student had spoken to the instructor, as that line of action had not been agreed in their team contract. In this instance, the contract therefore could have played a valuable role, if all students had followed the decisions initially agreed upon.

While the team organisation approach is primarily based on location, and there were some challenges associated with this discussed during the focus group, in one situation, the instructor paired students firstly according to gender, and then according to time zone. The notion of gender was used as an influential factor, in recognition of the fact that females can become marginalised in engineering groups (Russo & Stol, 2020). In this cohort, there were three females, who were assigned to a team with one another. There were no complaints from the team about this and they appeared to have a beneficial module experience.

While not discussed during the focus group, there was some evidence from the teaching experience that students may not feel completely comfortable with peers being aware of their own skillset. One student, for example, arranged a code development session with the instructor for his component of the team project. It later became obvious, however, that other members in the team had the skillset that was needed. This student had therefore avoided asking his peers, and instead wished to build his own skillset in the background, aided by the support of his instructor.

The peer review scores play an interesting role in supporting an instructor’s evaluation of the team work experience, and particularly, if any marginalisation has taken place owing to the actions of the team members. In the situation of the student making a complaint to the instructor about the participation of another team member, a third team member then scored the student who had complained lower than a student who had admitted to not completing much of the work. There therefore appeared to be some unfair scoring in this participants situation, which had not been resolved. This therefore provided some evidence that the peer scoring may not be entirely fair in all circumstances. This is the rationale underlying our recommendation for instructor moderation of peer review scores.

Here, the results are discussed from the perspective of the teams that were part of the Cyber Security modules, as well as the instructor’s perspectives on the delivery and outcomes. The profiles of the students from these modules who took part in the focus group and single interview are presented in Table 3.

As has been discussed previously, there are various methods available to assign students into groups. As well as the aforementioned self-organisation and geographicparameter (i.e. timezone) based organisation, there is also skill-focussed organisation. The overriding intention has always been to create groups that knit and work together well, with a good balance of skills and knowledge. The method of group creation has often had a strong bearing on how well a group works together and meets the above expectations.

Skill-based organisation relies on some knowledge of the cohort. It tries to organise students into teams based on demonstrated or self-indicated skill sets. Generally, each team should have students with an aptitude for, or competence in, programming, project management, English writing skills, and general administration/organisation. Unfortunately, it can be quite difficult to find an adequate number of students with these skills to populate multiple groups equally. Instructor experience thus far demonstrates that groups tend to fit into three general categories: excellent, intermediate, and dysfunctional. The outcomes of the interviews for each group in this section have been mapped to the relevant research questions.

In what ways does team work contribute to (your) learning?

The excellent groups self-organise, allocate work evenly within the group and assign it to fit with each student’s strengths, and demonstrate comprehensive communication, project management, and administration skills. They exhibit maturity in their interactions, respecting each individual’s opinions, skills, strengths, and contributions. As Student D commented, ‘In general or in the context of what we have applied it to, essentially team work is carrying out an exercise together. So, you just divide the exercise in parts and then everybody just has something to do, a part to do. What happened in our case, our group, is that we naturally gravitated towards our specialisation, so like strong suits, while also, learning from each other and I’m saying this because it’s an integral part of teamwork, I think, for me. it’s like you, strengthen each other and it may sound cheesy but you kind of do a better job because everybody can just chip in with their strongest suit.’

Student A (from Computer Science) also commented, ‘Teamwork is essentially leveraging the skillsets within the collective to achieve a specific task…. I guess it’s almost like a bonding or an experience, where you know you complete a task together, so you build friendship and connection between you and your fellow peers. And so, to me that’s a sort of component of teamwork, which was unexpected actually I didn’t expect to sort of form a bond with this, the team that I was sort of tasked to work with, and that was quite meaningful, and we continue to connect to this day, which is really cool.’

As both comments illustrate, excellent teams manage disagreements and disputes within the group and rarely need to use the peer review process, preferring to rate each individual equally – demonstrating respect for each other’s skills and contributions. Excellent groups may not always be the top graded team in a module, but they are consistently within the top 10% for every module.

What negative experiences have you had while working in/supporting team(s) in your module(s)?

However, even membership of excellent groups has some drawbacks. Comments and feedback suggest that many group members stay in their comfort zone and do not try to stretch themselves – or learn new skills. For example, some students complain in various cyber security modules that they have not had a chance to learn more programming, even though large sections of the course are designed to allow them to do just that. It would seem that an extra few percent on top of the overall grade is more valuable than learning a new skill for some.

Intermediate groups are less easy to categorise. They are certainly less consistent than the excellent groups and may have a less-than-optimum mixture of skills and/or geographic distribution of members. Although intermediate groups may get the best grades in a single module (where their skills mix is a good fit, for example), they are less consistent across multiple modules and activities and often have different members between modules as well.

In what ways does team work contribute to (your) learning?

Some intermediate groups can be characterised by a ‘hero complex’ where one or two individuals own most of the tasks and it is typically their skills that ultimately achieve the intermediate grades. Even though ‘heroes’ ensure that work gets done, it is not always the best outcome for the team, or even for other individuals in the team. Often some members of intermediate teams know what a team should do, but the team itself does not live up to that standard. As Student F commented, ‘My takeaway… is that there’s an expectation that a larger task will be broken down into smaller chunks and individuals will perform their portion and then it will come back together. It’s usually not like that but that’s what the word means to me anyway.’ In intermediate groups, there is also an expectation that some of the members will not contribute, or that their contribution will not be up to the required standard. As Student F mentioned, ‘I knew what I was getting into… in most cases I’ve been the one that pulled an all-nighter and finished a deliverable to get it across the line.’

‘Some people, step up to (it), and some people don’t but you get a lot of junior work from what I’ve seen even in in this group.’

The biggest difference between intermediate and excellent groups is evidenced in their peer evaluations. Unlike the excellent groups who respect individual opinions and skills, and apportion equal peer grades based on that, intermediate groups tend to utilise much more polarising peer assessment scores. Thus, while the excellent groups will almost always apportion equal marks to all group members, the intermediate groups will often demonstrate wildly differing peer grades up to the extreme where some members can score over 90% while others score less than 40% for the same group work.

In what ways does team work contribute to (your) learning?

The final category of group is the dysfunctional group. This type of group exhibits the opposite of many of the characteristics of the excellent group: for example, the group may exhibit poor project management and organisational skills; individuals may not attend meetings or respond to messages; the group may not have established a common communication channel; it may also lack one or more key skills such as programming or report writing. As Student F commented, ‘I think one problem with a group project is that if you get somebody who chooses not to participate, and they are assigned a group, they still get a mark. Even though you go in and you take marks off at the end, they’re still going to pass if your group does well.’ Student E commented, ‘The team contract just stated tasks would be allocated but didn’t mention who would do what.’ Student E also mentioned that the team had no regular meetings, but just carried on with tasks and communicated when necessary. One member of the team was referred to as the ‘coding team member’ who did most of the coding for the assessment.

The above comments illustrate the lack of structure, organisation and, in some cases, skills that typify the excellent category of group. One of the main reasons for this, as explained above, is that members of excellent groups tend to self-organise into the same groups for each module, whereas other high achieving individuals will use unofficial or student community networks to form intermediate groups with friends and/or individuals with a history of good grades. This means that people who have not been so successful in previous modules, or do not have core skills such as project management or programming, tend to find themselves outside the excellent and intermediate group ‘collectives’ and naturally gravitate towards the dysfunctional groups. This ultimately leads to a self-fulfilling failure mechanism.

The focus group conversation also provided some feedback on the use of a peer review grading system. It was felt that the zero-sum scoring did not provide the flexibility the students needed to give meaningful scores to their teammates, with Student D describing it as ‘unfair’ and Student C saying there should be an opportunity to ‘pick one person in the team you’d like to reward with more marks, like an honourable mention’. Based on this feedback, we have decided to utilise a new format in the upcoming runs of these modules that is based on a simplified 1-to-5 scoring system (see Appendix 3)

Group or team-oriented assessments will often seem like gifts to students who are in ‘excellent groups’: an opportunity to exercise key strengths, to work in an environment where you are respected and valued and be in a supportive team where you are all learning and working together towards the same goal. It should be the goal of all instructors to encourage such a supportive and valuable learning environment. Unfortunately, it is not always possible, as an excellent computing group requires certain properties: a good mix of key skills including programming, report writing, project management, communication, and organisation. It also requires a certain maturity from the members. These skills are in limited availability within any given cohort. Intermediate groups can still be effective, and provide a supportive learning environment, but only if monitored and supported by the instructor. Dysfunctional groups can lead to destructive and negative behaviours and these kinds of groups should be discouraged and avoided wherever possible.

How can the instructor help in organising and managing online groups? Our recommendation is that the first step is to try and establish a skill-based approach to group setup. The instructor should try and allocate someone with each of the key skills to each group (bearing in mind potential restrictions around location and time zones). In addition, students have asked that instructors attend kick off meetings and assist with initial setup and organisation. Instructors can also help by signposting students to training and resources around key skills such as project management.

A key objective must be to avoid dysfunctional groups. Some of the abovementioned techniques may help with that, but ultimately there may be a need to offer remedial training or coaching, or even to participate more directly. However, if these approaches fail, instructors may need to consider alternative routes and mechanisms to support students who cannot (or will not) work within team-oriented assessments, particularly as this form of assessment is sometimes critical to a course’s learning outcomes. Such alternatives could be a mixture of group and solo work, offering the option of alternative paths through the course, or even a greater emphasis on the e-portfolio assessment, where the student can select which assessments s/he wishes to participate in. In every case, the instructor would still need to emphasise the need for a balance of individual and group-based assessments.

The research is limited by the relatively small sample size of students participating in the team work experience. However, we justify this as being a preliminary piece of work, the argument of which will be supplemented in future work. Furthermore, we appreciate that qualitative evaluation of the individual performance of groups on a per module basis limits the overall value of team working throughout the entire degree programme, which is a further observation we have come to recognise. In summary, we see evidence across multiple modules that the capable groups continue to work together beyond a single module, with subsequent strong performance across their degree programme. We plan to investigate this finding further in our future work.

There is a need to encourage the development of team-working skills in HE delivery. This becomes particularly problematic when the HE programme is an online master’s in computing that is attended by students of varying abilities and in various locations across the globe. The fact that there is limited research covering this area has motivated this case study-based research on the outcomes of team-based projects on such programmes. This study provides information on a pedagogical design for team-based assessments for online programmes, and then presents quantitative and qualitative outcomes of the implementation. We justify this as being significant today given the continual move to online education.

The quantitative outcomes (module scores) are positive overall, and the qualitative outcomes (both from students and instructors) highlight requirements for creating these teams and the need for instructor oversight (as described in the reflections of the instructors involved). It should be noted that one key shortcoming of this research is the number of students involved in the focus group (6 participants out of a total number of 81 students taking the modules in question). However, the instructor perspective in this study has been included to bring into play comments and actions from other students who did not take part in the study, as well as for providing a practitioner’s view on the delivery.

One challenge with online learning is the fact that students do not always wish to participate in a synchronous manner. They join the online learning experience as a result of the flexibility it can bring, allowing study to take place around a full-time job and a family. There are benefits, nonetheless, of being engaged in a more synchronous manner. This fact is particularly true for students who have not necessarily opted into the online and distance education model but have been forced as a consequence of circumstances beyond their control (such as COVID-19). The groupwork mode, to some degree, ‘forces’ students to function in a more synchronous way, allowing the subsequent benefits to be achieved. This can be seen in the quantitative and qualitative outcomes provided in this study.

Finally, the insights from the instructors in this study highlight key ‘Do’s and Don’ts’ (recommendations for organising and managing online groups, as well as moderation of their scores). These should be considered when using any pedagogical design for team assessments, particularly where online delivery is concerned. Team composition, instructor participation, and peer review scoring formats play key roles and will need to be reviewed with each cohort and the mode of delivery.