Subject Area Risk Assessment of Four Hungarian Universities with a View to the QS University Rankings by Subject

After the turn of the millennium, the emergence of global media rankings transformed university operations and priorities. Universities are at the intersection of global and national ranking benchmarks, which is mostly a communication challenge for them and an interpretative challenge for their users (Johnes, 2018). One of the most well-known of these rankings is the QS ranking (QS World University Rankings), which publishes not only general but also field-specific rankings annually (URL1). The latter is also attracting increasing attention because, unlike the general rankings, the rankings by subject provide an opportunity to compare institutions with similar structures and a similar disciplinary focus (García et al., 2012; López-Illescas et al., 2011). Primarily, the requirement to be ranked and to move up in the rankings has also emerged in Hungary among institutions that have moved to a foundation structure. The drastic restructuring of the Hungarian higher education system began in 2019–2020, however the “shift of university governance models” is just one element of the restructuring, it plays a core role in achieving a higher competitiveness and a more flexible operating model according to the Hungarian government. Restructuring the university system means that they are transformed from a state-funded institution into one that is maintained by a nonprofit foundation. Through this change, the foundation controls the funding of the university, its operations, and maintains the rights of the university. The government managed, on the one hand, to step away from directly funding the university, while on the other hand, it has endowed the university with formerly publicly-held assets. With all these, the key goals are to empower the universities to play a central role in research, development, and innovation, as well as to strengthen their inter-institutional, corporate, and international relations in order to make Hungary a regional knowledge center (URL2). Hungarian universities with funding have made commitments (for extra resources) to a steady increase in Scopus (and Web of Science) indexed publications over the next 5 years.

The development of world-class universities is high on the policy agenda of the stakeholders in higher education around the world (Altbach & Balán, 2007). Several reforms can be studied both at the national and institutional levels in the quest for academic excellence, while these policy concerns let reinforce the role of university rankings (Salmi, 2009). In parallel, in the last decades, a growing number of nations introduced national policies in higher education dedicated to higher competitiveness. The same process can be observed in Hungary through the shift of governance models of universities. Usually, these policies are based on the peer-reviewed method, while rankings are centered around the research performance as well. Performances of universities are compared at the discipline level and not at the individual level, allowing a more rational comparison of universities having similar research profiles (Abramo et al., 2012). In this highly competitive environment, universities are exposed to their top scientists in producing high-quality publications and maintaining competitive research performance. This led universities to introduce risk assessment techniques, based on the following criteria (Syreyshchikova et al., 2020): -

systematic approach;

The risk concerning the educational field lies in the lack of meeting stakeholder expectations and goals, which depends on the quality of education received and the research capacities of the staff (Girdzijauskaite et al. 2019; Shatunova et al., 2019). In educational activities, the objects of risk management are the following processes: resource management, management of innovative and scientific and methodological activities, personnel management, the provision of educational services, procurement management, maintenance, disposal, and repair management (Suray et al., 2019). Risk assessment also deals with keeping up the position of universities in international university rankings.

As the question of competitiveness emerges, it is essential to study further the tools and strategies universities can use to maintain and increase their position in the international university rankings being important measuring tools in today's highly globalized and competition-centered higher education sphere (Altbach, 2012). Although risk assessment is widely used by universities, including academic risks, there has been only a handful of studies focusing on the research contribution of the mentors. In their remarkable study, Piro, Rørstad, and Aksnes (2016) investigated the mentors' contribution to their institution or department through their citation impact and found that the overall effect of the most prolific professors or mentors on their institution's citation impact was quite moderate and greatly varied by scientific fields as well as research topics.

In alignment with these, the main research questions of the study are the following: To what extent are the institutions exposed to the attrition of their mentors, and are there already signs that the institutions will be able to replace them with other mentors?

The paper is divided into six sections. After the introduction, the second chapter maps the current literature on the significance of building an institutional-level research strategy. It includes argumentations about the role of mentors in successfully implementing the research strategy. The methodology is described in the third section introducing the general overview of the analyzed universities, the methodology of QS ranking by subjects, and the methodology of calculation of the risk assessment giving the core of this empirical analysis. The fifth section presents the research results, while the conclusions are drawn in the final section.

Aboagye and co-authors (2021) have shown that institutional governance promotes the improvement of academic performance. They identified the following instruments: -

a research plan defined at the institutional level,

It is also the responsibility of the university management to support the work of the research teams and the mentors leading them to implement the research strategy (Felisberti & Sear, 2014). Research teams sustain the scientific output of the institution, producing publications with greater impact and more references (Wuchty et al., 2007).

It is worth noting here that the literature divides a research career into several stages. Super (1980) distinguished five stages in the life cycle:

growth (interest and curiosity; 0–14 years),

Hall and Chandler (2007) identified the following four career stages:

discovery (discovering the identity of the researcher),

Within a given career stage, there may be a number of smaller “mini-cycles”, which provide opportunities for the redesign. Due to the different roles and responsibilities that characterize the stages of a research career, institutions should consider the strategic tools they can use to offer their employees a satisfying and motivating career. The process of academic career progression determines not only research positions but also teaching and administrative positions (Baldwin & Blackburn, 1981).

Within each research team, work is organized along different roles. While the younger researchers play a follow-up role in the project, the older, senior researchers play a project designer and leadership role. The study of the role of leaders is a popular topic in management and organization studies. Research on this topic demonstrates that leaders play a significant role in achieving organizational performance (Hermalin, 2012; Van der Heijden et al., 2006). However, leaders are highly mobile within an organization, often joining other organizations or being lost to retirement, illness, or death, after which their replacement poses a challenge for the organization.

The key tasks of mentors in these research teams are: -

Researching strategy and the appropriate allocation of tasks among the members of the research team,

The mentor is described in the literature as the lead researcher who takes up a central position in the overall network—he or she has a direct relationship with the members of the research team and is also the most productive researcher in the network (Hayat et al., 2020).

From the institution's point of view, the organizational work of mentors is essential in the implementation of research strategies. These senior, leading researchers also ensure the functioning of the research teams and sustaining of scientific output. It is precisely for this reason their absence places a considerable burden on the organization and makes it a challenge to replace them. The work of Győrffy et al. (2020) highlights that the peak of a research career comes decades after the start of a research career, at the age of 48–49. This shows that “training” new mentors and preparing them for leadership roles are quite time-consuming for the institution. It is also important for institutional strategies that active researchers do not change their research field during their career, which they try to limit by means of grants and by following up research proposals (Zeng et al., 2019).

As already indicated above, it has become a priority for foundation universities to be ranked and to move up the rankings, not only to get into the rankings but to maintain the performance continuously and move up in them. As academic performance at the institutional level does not depend on a single author, it is worth looking at well-functioning, productive research teams that contribute to publication performance if universities seek to avoid taking the unnecessary risk of a measurable decrease in scientific output. By the working definition of this paper, the risk is the potential loss of a number of publications written by a lead researcher or by the members of a research team led by a mentor. Age risk refers to lead researchers and mentors heading towards the end of their academic careers; in other words, the older a mentor gets, the more likely their scientific performance suffers as a result of either retirement, a chronic illness, or a lack of ambition. If a lead researcher contributes to the majority of publications in a given topic or topic cluster, their withdrawal from the university as an author, regardless of its reason, poses a double risk to the university in the form of an almost disproportionately high loss in the number of publications. Therefore, it is in the best interest of universities to map these productive teams and identify the authors who connect and coordinate them, often in a mentoring role, and then to assess the number and age risk of publications by lead researchers and mentors in each topic cluster. It should be a priority for universities to develop an appropriate strategy because researchers standing at different stages of their careers can only be retained by universities through different motivational tools and different tasks (Brazeu & Woodward, 2012). It is also important to note however that, mentors have a core role in maintaining the publication performance of the university—not just being productive authors but serving as a bridge to connect other authors—a more inclusive and comprehensive research strategy is required to maintain the prominent ranking of universities.

The framework for the present analysis is the QS rankings by subject so it is worth briefly reviewing its methodology. The QS is organized along six pillars. The two most important of these are the reputation and the volume of references of the researchers at the university under review, and the others are -

the esteem of the employer,

For the subject-specific QS ranking, lists are published in 5 disciplines divided into 51 science categories. Four components are used to rank universities by specialization for QS: -

Academic reputation,

Four universities in Hungary were included in the study: -

Semmelweis University

Their position in the QS rankings by subject is summarised in Table 1.

The four institutions studied are among the best-positioned universities in higher education in Hungary. The University of Szeged (SZTE) and the University of Debrecen (DE) have a multidisciplinary profile, while Semmelweis University (SE) is specialized in Life Sciences. The University of Pécs (PTE) is ranked exclusively in the field of Medicine. A common feature of the universities is that they are all ranked in the field of Medicine.

The empirical study draws on data from SciVal's research support online platform. SciVal is a tool that draws on data from the Scopus database, allowing the overview, tracking, trending, and comparison of different units (researchers, institutions, publications, countries, and subject areas).

The Scopus and Scimago classification distinguishes 27 scientific disciplines and 330 science categories. The science categories can be further broken down into 1,500 topic clusters (TC). The topic clusters consist of 97,000 topics (T) altogether. Three keywords are used to describe the topic clusters and topics in SciVal. Topics contain more than 70 million publications in Scopus.

Topics (T) and topic clusters (TC) can be characterized by -

the number of publications,

In SciVal, the Field-Weighted Citation Impact (FWCI) indicates how the number of citations received for a publication or a topic or topic cluster (entities for short) or author's publications compares to the average number of citations received for all other similar publications in the same discipline, or, in simpler terms, how the number of citations received for the entity's publications compares to the world average. An FWCI of 1.00 indicates that the number of citations received for the entity's publications is exactly what would be expected based on the global average of similar publications; the FWCI for the “World”, i.e., the entire Scopus database. An FWCI greater than 1.00 means that the citations of the publications are above the global average, for example, an FWCI value of 2.11 means 111% more than the world average. An FWCI value below 1.00 means that the citation value of publications in similar disciplines is below the global average, e.g., an FWCI value of 0.87 means 13% less than the world average.

For a given topic or topic cluster, the name of the institutional author with the most publications and the number of publications can be determined. In other words, the author who co-authors the most publications on a given topic stands out. They are the mentoring authors (mentors) who take care of a given topic and a given cluster.

As it was discussed in the second chapter, a mentor may pose a significant risk to the university if they take part in a high proportion or they are frequent co-authors of publications in a given topic or topic cluster. The increased level of risk may lead to the vulnerability of the university, which means that the potential loss of a mentor followed by the loss of numerous publications might lead to a drastically reduced publication output, negatively affecting the institution's overall position in the university rankings. Taking this into consideration, the vulnerability of any given university can be calculated using the following formula: sTCmax=xTCmaxz {s_{TC\max }} = {{{x_{TC\max }}} \over z} where x_TCmax = the number of publications by the author with the most publications for a given topic or topic cluster at a given institution and in a given period,

z = total number of publications for a given topic or topic cluster, for a given institution and period.

s_TCmax = percentage risk value (0–100%] for authors with the most publications, for a given topic cluster, for a given institution and period.

A value higher than 50% is considered high risk for s_TCmax.

Another risk factor may be the age of the mentor. In the case of -

old age, retirement or superannuation, possible illness or death, and lack of individual motivation for promotion, moral and financial motivation,

Although we relied on the work of Super (1980) and Chandler and Hall (2007) when defining each stage, the specific structure of education and academic progression in Hungary had to be taken into consideration as well (Sasvári et al., 2021). Its significance can mostly be described by the different levels of frequency of publications in each identified stage: in the early stage researchers are more likely to publish a number of papers as it is needed for their academic promotion, in the middle stage, leading authors still publish regularly while organizing and operating a well-functioning research team or network, and finally, the late stage can be characterized by a declining number of occasional publications or complete withdrawal.

The exposure of the four universities to the QS rankings by subject was assessed along the following criteria: -

Position in the QS rankings by subject,

The scientometrics of the topic clusters of each university surveyed that can be considered in the QS rankings by subject are summarised in Table 2. It shows that SE, SZTE, and DE have similar numbers of publications, while PTE has a lower number of publications in these rankings. For each of these institutions, most of the publications considered were in the field of Medicine. In terms of the number of authors, SE stands out with more than 5,000 authors. SE also leads in terms of the number of references, followed by SZTE, DE, and PTE. Accordingly, SE stands out in terms of the number of references per publication. The main indicator in the table is the tendency of the FWCI values, for which the ranking between institutions is similar. However, it is worth highlighting that the FWCI values are around average in all cases, with only certain specializations standing out. Among these, the field of Physics and Astronomy at SZTE (4.88) stands out, while DE has a FWCI of 2.28 in the same specialization. Relatively lower FWCI values are found in the fields of Computer Science and Information Systems and Mathematics at SZTE and Mathematics at DE.

Table 3 shows the concentration of each topic and topic cluster in the universities surveyed. In the case of the two larger universities, SZTE and DE, both the number of topic clusters and the number of topics are higher than for the other two institutions. This is also due to the fact that they can be found in more than one rankings by subject at the same time, as they have more topics simultaneously than the discipline-specific SE or PTE that is prominent in the Medical Sciences. However, if we look at the clusters of Medical Sciences as a point of comparison, we can see that within these, SE has the most topic clusters (458), while SZTE, DE, and PTE are present with fewer topic clusters. The same order can be observed but with a significantly larger variation among institutions in terms of the number of topics. In the field of Medicine, SE is active in 3,138 topics, while SZTE is active in 1,857. Concentration can be measured by the number of topic clusters accounting for 50% of the publications published by the university. This shows that SZTE (106) and PTE (104) are more fragmented, while the strongest concentration is observed in SE. When compared to the total number of topic clusters, we can see that in general, 10% of the topic clusters account for 50% of the scientific output of the university. In comparison, DE is slightly more concentrated (9.53%), while PTE (12.7%) is more fragmented.

Figure 1 shows the age distribution of mentors in the four universities. Overall, there are no major differences in age among the researchers surveyed. The average age is the lowest at SZTE with 52.9 years and the highest at PTE with 55.5 years. The median age is also the lowest at SZTE (51 years), while the median age at the other three universities is 54 years.

Figure 1

Figure 2 shows the levels of risk at the four surveyed universities in each topic cluster. At the bottom left corner, those topic clusters are shown that pose the lowest level of risk to the universities, and topic clusters with the highest risk are shown at the top right corner. The varying size of circles express the level of risk in percentage, that is, the bigger the circle is, the higher the rate of risk is measured. A more detailed summary of the results of the risk analysis can be found in Table 4 in the Appendix. The first column in Figure 2 shows low-risk topic clusters (below 50%), while the second one reveals high-risk topic clusters (above 50%). The latter column can be considered to be a double risk, i.e., it is considered to be risky according to both the age of the mentor and the extent of his or her contribution to the cluster. Overall, we can see that SE has the lowest risk cluster (30%), followed by SZTE (28%), DE (28%), and PTE (19%). These are topic clusters that can be sustained in the longer term. The highest proportion of topic clusters with double risk is found in PTE (13%) and the lowest in SE (1%). Due to the extent of mentor input, the institutions surveyed have a similar proportion of high-risk topic clusters, for example, 40% in PTE (=13%+14%+13%; including double-risk clusters as well). The pattern is different in SE (14%=6%+7%+1%). The critical points by specialization are as follows: -

in the case of SE, in Anatomy and Physiology 50% (column V in the table, 1 topic cluster) due to the age of the mentor,

Figure 2

SE is the best overall performer. DE and SZTE are exposed in the field of Mathematics, while SZTE is more at risk in Computer Science and Information Systems and DE in Physics and Astronomy.

In the politics of higher education in Hungary, especially among the universities maintained by foundations, getting on, staying present, and achieving more favorable positions in international university rankings over time has become a primary issue. Universities have a better chance of achieving this in certain specializations so for them, it is worth considering the rankings by subject. Hungarian universities are generally small in number and can only focus on specific areas. Research teams in these fields are key to maintaining the performance of the institutions, as only they can ensure sufficient productivity at the international level as well. One of the messages of the study is therefore that universities need to support their research teams and also have a key role to play in the selection of the lead researcher (mentor). Furthermore, as we have seen that the age distribution of mentors is similar both in terms of mean ages and medians of ages, this highlights that it is in fact the distribution of researchers by topic clusters that influences the stability or even the exposure of the institution. This is in line with the primary priority that the composition of the institution's researchers, the motivational factors, and the tasks associated with each career stage should be kept in mind by the university management. By considering these, universities can develop a strategy tailored to their researchers to ensure their competitiveness. Where the contribution of mentors to the leading topic clusters is high, it is worth motivating them towards mentoring and involving young researchers, while at those universities where the contribution of younger researchers is higher, it is worth motivating them with appropriate working conditions. However, identifying the exact elements of this strategy is beyond the scope of this analysis and will be addressed in a future paper.

The analysis looks at four Hungarian universities, two of which have been ranked in several disciplines, while the other two are ranked in the field of Medicine. The selected universities have been ranked in the QS rankings by subject for several years, of which SZTE is ranked in eight, DE in five, SE in three and PTE in one place, respectively. Maintaining these rankings is a priority for the universities, which they can ensure by examining their discipline-specific publication performance and the specificities of their researchers. This is what the analysis has attempted to do, using SciVal data. The results of the measurement can be summarised in a few key findings: -

10% of the topic clusters managed by researchers at the universities alone account for 50% of the scientific output of the given university. This rule of thumb helps to identify those topic clusters that are represented by a sufficient number of publications to give the university international visibility. At the same time, within each specialization, a number of smaller topics ensure a diversity of research, thus giving researchers scope to add new topics to their portfolio. These minor topics contribute to the better ranking of the institution in the rankings by subjects.

We believe that the research strategy of Hungarian universities should be designed by the university management, considering the exposure, because it is not enough to be ranked, they have to stay ranked. It is clearly deriving from our results that the studied universities have a rather elder group of mentors, representing a high risk in maintaining the rank achieved by these universities. In order to keep the prominent position of these universities, it is essential to elaborate an inclusive and comprehensive research strategy with a strong emphasis on talent management and the promotion of young researchers becoming mentors of their disciplines. Building a more inclusive strategy requires the “inclusivity” of publication practice as well, regardless of age, nationality, or status of the researchers. This should be reinforced by new incentives to develop a more comprehensive model of research and publication mentoring, including the development of the proper reward system, facilitating new and already existing international and national cooperation, and a better understanding and inclusive communication about the latest tendencies observed in the international academic community and publishing.

As a final remark, we would like to put an emphasis on the fact that although the empirical analysis presented a case study of four Hungarian universities—being mainly relevant for the Central—Eastern European higher education policy –, the methodology and the concept of risk assessment based on the mentors can be used for every university around the globe.