Implications of Publication Requirements for the Research Output of Ukrainian Academics in Scopus in 1999–2019

This article aims to explore the implications of publication requirements for the research output of Ukrainian academics in Scopus in 1999–2019. As such, the study contributes to the knowledge about the qualitative and quantitative aspects of research evaluation policies (Abramo, D’Angelo, & Felici, 2019; Abramo, D’Angelo, & Grilli, 2021; Csomós, 2020; Butler, 2003; Schneider, Kaare, & Bloch, 2016; Korytkowski & Kulczycki, 2019; Kyvik & Aksnes, 2015; Vanecek, 2014; Vanecek & Pecha, 2020). Initially, Scopus and Web of Science (WoS) included journals from western states, most of which have been top-ranked journals. Thus academics from these states have a long tradition of publishing in Scopus- and WoS-indexed journals (Nesvetailov, 1995). Consequently, publications in Scopus- and WoS-indexed journals have been considered the proxy of research quality, and non-western states have adopted policies requiring academics to publish in these journals (Csomos, 2020; Good et al., 2015; Grancay, Vveinhardt, & Sumilo, 2017; Kulczycki, 2017; Kuzhabekova & Ruby, 2018; Nazarovets, 2020; Olechnicka & Płoszaj, 2019). However, the earlier studies highlighted that the quantitative criteria in research evaluation policies can be achieved through publications in journals of low impact (Butler, 2003; Demir, 2018; Korytkowski & Kulczycki, 2019; Vanecek & Pecha, 2020). Thus the national research policies should prevent this unintended consequence. This issue is crucial for the countries that do not have a long-lasting tradition of publishing extensively internationally because their publication requirements were nationally-oriented for a long time (Hladchenko & Moed, 2021).

Ukraine is one of these countries. Though, in the last years, Ukraine claimed to have aligned its higher education system with the Bologna Process, the remnants of the Soviet model still persist. Ukraine has preserved the two-level systems of doctoral degrees and scientific titles. In some EU countries, habilitation (Switzerland, Germany, Poland) can be viewed as the second doctoral degree. However, while in Ukraine, the second doctoral degree is about writing a second doctoral thesis, in the EU countries, it is about summarising research done after PhD research that indicates the scholarly level of academic required for promotion to professor. Moreover, Germany has developed a career path alternative to habilitation. There are also countries that abolished the second-level doctoral degree e.g. Latvia, Estonia, Georgia and Croatia. Ukraianian academic, having been awarded a doctoral degree and a scientific title, also receives financial bonuses that ensure a life-long extra payment to the salary. After 1991 publication requirements for doctoral degrees and scientific titles included publications only in Ukrainian journals but in 2013 articles in foreign journals or Ukrainian journals indexed in international databases were introduced into doctoral publication requirements (Ministry of Education and Science, Youth and Sport of Ukraine, 2012). In 2016 publications in the Scopus-and WoS-indexed journals became compulsory for the scientific titles of associate professor and professor (Cabinet of Ministers of Ukraine, 2015). In the following years publications in Scopus- and WoS-indexed journals were introduced into the criteria for the attestation of higher education institutions (Cabinet of Ministers of Ukraine, 2018) and the allocation of the state funding for research (Ministry of Education and Science of Ukraine, 2019).

This study explores the implications of publication requirements for the research output of Ukrainian authors in Scopus in 1999–2019. Three metrics were elaborated to analyse the quality of research output: publications in predatory and local journals and publications per SNIP quartile from the disciplinary perspective.

After 1991, the higher education system in Ukraine underwent marketisation and massification (Shevchenko, 2019). Meanwhile, a significant part of the Soviet model of higher education was preserved (Oleksiyenko, 2016). For example, the division between primarily teaching-oriented higher education institutions and research institutes of the National Academy of Sciences. Moreover, next to the National Academy of Sciences, sectoral academies in pedagogy, agriculture, medicine, education, law and art were established. Despite Ukraine joined the Bologna process in 2005, which mentions PhD as the third cycle, the two-level system of scientific degrees (PhD and Doctor of Sciences) was preserved.

In addition to a two-level system of doctoral degrees, a two-level system of scientific titles—docent (associate professor) and professor inherited from the Soviet model also was preserved. A PhD degree allows promotion to associate professor (docent), while the second-level doctoral degree allows promotion to professor. Doctoral degrees and scientific titles provide extra payment to the salary of academics: 15% and 35% of a salary increase for the PhD and Doctor of Sciences and 25% and 33% for the scientific titles of associate professor and professor. Academic hierarchy with the two-level systems of doctoral degrees and scientific titles while the most “distinguished” academics are awarded the status of the corresponding member and “academik” (academician) was created under the Soviets to ensure the loyalty of the academic community to the communist system. Individual academics strived to move up the academic hierarchy by accepting and promoting communist ideology, thereby reinforcing and increasing the legitimacy of communist academic institutions. As neither Soviets nor communists do not take rule in Ukraine anymore, academic hierarchy needs to be updated to correspond to the nowadays societal and economic needs of Ukraine. The life-long incomes for the titles of academician and member-correspondent cost Ukraine €5 million per year (Kovalchuk, 2010). This funding could be allocated on compative basis among Ukrainian academics or spent on high-tech equipment, which could really contribute to the development of Ukrainian science.

After 1991 Ukrainian academics, as a rule, published in Ukrainian journals as these publications, were prioritised at the state level and were required for the doctoral degrees and scientific titles, attestation of higher education institutions and for the application for the state funding for research.

In November 2012, the education ministry introduced the articles in “journals of foreign countries or Ukraine indexed in international databases” into the requirements for doctoral degrees. However, already in December 2012, these publications were changed for publications in international journals which can be substituted with publications in Ukrainian journals indexed in international databases. Publication requirements came into force in September 2013 (Table 1). One article in an international journal and four articles in Ukrainian professional journals became obligatory for the PhD degree, and four international articles and sixteen articles in Ukrainian professional journals for the second-level doctoral degree (Ministry of Higher Education and Science, Youth and Sport 2012). As most Ukrainian journals charge a publication fee, many Ukrainian academics erroneously think that all foreign journals also charge a publication fee (Hladchenko & Westerheijden, 2018).

In 2014 the Candidate of Sciences degree was renamed PhD. A PhD awarded by a Western university equals to Ukrainian PhD and is viewed as a first-level doctoral degree. Till 2015 among the requirements for the promotion to associate professor and professor were textbooks approved by the education ministry and articles in Ukrainian professional journals. In 2015 the new requirements were approved. They came into force in 2016. According to these requirements for the scientific title of docent (associate professor), academics with a PhD are obliged to have a five-year work experience, an article in a journal indexed in Scopus or WoS, an international certificate that proves level B2 in English and study visit or participation in a conference or symposium in OECD countries and/or European Union countries (Cabinet of Ministers of Ukraine, 2015).

Academics with a second-level doctoral degree are required to have a ten-year work experience, two articles in journals indexed in Scopus or WoS to be promoted to professor. An international certificate that proves level B2 in English and a study visit to OECD countries and/or European Union countries are also required. Meanwhile, in both cases, the international certificate which proves level B2 in English can be substituted with 10 articles in journals indexed in Scopus or WoS. It is necessary to denote that all PhD students pass an exam in English.

In 2018 articles in journals indexed in Scopus and WoS became one of the requirements of the state attestation of Ukrainian higher education institutions (Cabinet of Ministers of Ukraine, 2018). Publications in the journals indexed in Scopus and WoS are among the criteria for the assessment of research projects applying for state funding (Cabinet Ministers of Ukraine, 2019).

In September 2019, changes were made in publication requirements for the assessment of the individuals when applying for a PhD degree. Articles in so-called Q1, Q2, and Q3 sources allow decreasing the total number of publications (Ministry of Education and Science of Ukraine, 2019). However, these requirements still preserve the opportunity for Ukrainian academics to be awarded a doctorate for low-quality publications. Q1, Q2, and Q3 sources are journals that are ranked in the first three quartiles of the distribution of citation impact measures (either Scimago Journal & Country Rank or Journal Citation Reports) among journals in a particular subject category.

Having explored the implications of publication requirements for the research output of Ukrainian academics, Nazarovets (2020) revealed that in the period 2015–2019, the most popular journals among Ukrainian authors, according to the number of publications, were journals of Ukrainian publishers and English-language translations of Ukrainian journals. However, CiteScore indicated that these were generally not the most influential journals. In 2016, Scopus indexed 47 Ukrainian journals (Hladchenko & Moed, 2021b), classified in the following disciplinary areas: natural sciences (23 journals), engineering (12 journals), clinical medicine & biomedical sciences (7 journals) and social sciences (5 journals). Nine of them have been published since the Soviet period. Twelve of them are in English. In 2014–2016 publications by Ukrainian authors constituted 68.2% of overall publications in Ukrainian journals, with peaks in clinical medicine & biomedical sciences (78,0%), and engineering (77.7%). Concerning the citations, 51.8% of overall citations of publications in Ukrainian journals came from Ukrainian authors, with maximum in engineering (68.9%) and minimum in the social sciences (27.8%). Findings of Hladchenko and Moed (2021a) highlight that in 2019, 29% of publications of Ukrainian authors were in national journals.

Having analysed the publications of Ukrainian authors in titles excluded from Scopus in 2011–2020, Nazarovets (2022) showed that in 2011–2020, in the field of Economics, Econometrics, and Finance, 46.92% of papers of Ukrainian authors were in these journals.

This article aims to provide a scientometric analysis of the effect of publication requirements on the research output of Ukrainian authors in Scopus in 1999–2019. The study analyses articles, reviews, proceeding papers and surveys. It explores the research output of Ukrainian academics affiliated with higher education institutions as well as with the research institutes of the academies of sciences.

Three metrics were chosen to investigate the quality of the research output of Ukrainian authors in Scopus: publications in predatory journals, publications in local journals and publications per SNIP quartile from the disciplinary perspective. A comparison among disciplines is taken as one of the metrics because apart from the national context scholarly publication patterns depend on the discipline (Engels, Ossenblok, & Spruyt, 2012; Korytkowski & Kulczycki, 2019) which refers to epistemic cultures (Cetina 1999).

Firstly, journals with publications of Ukrainian authors were checked against Beall's list. This revealed 40 predatory journals among journals in which Ukrainian authors published in 1999–2019. Secondly, among the whole list of journals in which Ukrainian authors published in 1999–2019 journals were found 101 local journals. Journals were categorised as Ukrainian ones because Ukraine is stated as a publishing country or Ukrainian organisation is stated as a publishing organisation or the journals are English translations of Ukrainian journals. Thirdly, a field-normalised metric of SNIP which allows a comparison between subject fields was used to measure the citation impact of the sources. SNIP indicates the relative standing of a serial title within subject fields. All subject fields are divided into 100 equal-sized percentiles based on the number of serial titles, and a serial title is assigned to a percentile based on its SNIP. Quartiles are bands of serial titles that have been grouped because they occupy a similar position within their subject categories. The quartiles are quartile 1 (Q1): serial titles in 99-75^th percentiles, quartile 2 (Q2): serial titles in 74-50^th percentiles, quartile 3 (Q3): serial trials in 49-25^th percentiles, quartile 4 (Q4): serial titles in 24-0th percentiles. The dynamic of publications of Ukrainian authors in Scopus sources within the SNIP quartiles was explored from the disciplinary perspective: biomedical sciences, clinical medicine, engineering, natural sciences, social sciences & humanities.

During the analysis it was found that due to an inappropriate presentation of the author affiliation, there are Ukrainian journals that contain publications with undefined author affiliation and territory, these publications were checked manually and allocated to the corresponding category.

This article explored the effects of publication requirements on the research output of Ukrainian academics in Scopus in 1999–2019. Three metrics were elaborated to investigate the implications of publication requirements for the quality of publications: publications in predatory journals, publications in local journals and publications per SNIP quartile from the disciplinary perspective. Data were extracted from Scopus for the period from 1999 to 2019.

Firstly, an analysis of publications of Ukrainian authors in predatory journals revealed that the number of publications of Ukrainian authors in predatory journals started to rise after 2013 and reached 1% in 2019. As such, the study findings resonate with Grancay, Vveinhardt, and Sumilo (2017), who revealed that research evaluation policies without defined qualitative requirements for the journals resulted in academics from Central and Eastern European countries stimulated to increase the number of publications in predatory journals. Compared globally, Ukraine has a low percentage of publications in predatory journals in general but a high percentage in the social sciences (Macháček & Srholec, 2021).

Secondly, the exploration of publications of Ukrainian authors in local journals indicated an increase in the number and percentage of these publications after 2012 with the peak in 2015, which can be attributed to the changes in publication requirements for doctoral degrees and scientific titles. In the following years the percentage of publications in the local journals fell. The number of these publications also fell insignificantly after 2015. However, it was much higher than before papers in Scopus-indexed journals were introduced into publication requirements. As such, the study findings support Nazarovets (2020), who revealed that publication requirements based on quantitative criteria do not “encourage Ukrainian scientists to seek the optimal channel for the presentation of their research output”. This also resonates with Pajic (2015), who indicated that national journals from Eastern European countries indexed by WoS are used for “local promotions and formal fulfilment of policy rules, rather than the true promotion of national science”. Local high-quality journals are crucial for the development of the science. They require publication requirements to be focused not only on quantity but also on quality, as it is stated in earlier studies (Krawczyk & Kulczycki, 2021).

Thirdly, the total number of publications has risen dramatically since 2011. The analysis of publications per SNIP quartile highlighted an increase in the number of both Q1+Q2 and Q3+Q4 but from 1999 to 2019 except for 2004 the share of Q3+Q4 exceeded the share of Q1+Q2. In 2019 there was a rise in the share of publications in journals with missing SNIP. These journals are either local journals or foreign open access journals.

As regards the disciplinary perspective, after 2012 in the biomedical sciences the gap between Q1+Q2 and Q3+Q4 & publications in journals with missing SNIP was almost the same as in prior years, while in the natural science it narrowed insignificantly and the share of Q1+Q2 increased a little bit. In clinical medicine in 2012 and in 2015–2019 the share of Q3+Q4 exceeded the share of Q1+Q2, though in previous years it was the opposite way. In engineering and social sciences, the share of Q3+Q4 has exceeded Q1+Q2 since 2012 even more than in previous years. However, in 2019 in the social sciences and humanities, the number and the percentage of Q1+Q2 publications exceeded Q3+Q4 and publications in journals with missing SNIP. However, the increase in the number of Q1+Q2 occurred due to the rise in publications in local journals and open access journals, the peer-review standards of which are under question. As academics publish more and more in these journals they are supposed to cite more and more publications from these journals and as a consequence, the journals shift to higher SNIP percentiles. So not always the high SNIP percentile can be seen as evidence of quality. In the social sciences and humanities, publication requirements led to the creation of income for particular journals indexed in Scopus which for a publication fee can publish an article without maintaining the required standards of peer review. Distinctions among the disciplines support the earlier studies that indicated that publication practices of academics are related not only to financial incentives but also to reputational and intrinsic reasons (Andersen & Pallesen, 2008; Lam 2011) as well as the publication culture of discipline (Engels, Ossenblok, & Spruyt, 2012; Korytkowski & Kulczycki, 2019).

To summarise, the study findings highlight that the introduction of Scopus publication requirements resulted in the increase in the research output of Ukrainian authors in Scopus. However, the study findings also indicate that research evaluation policies should define qualitative criteria for journals to prevent quantitative criteria to be achieved through publications in journals of lower impact. This resonates with the earlier studies on quantitative and qualitative aspects of research evaluation policies (Butler, 2003; Demir, 2018; Korytkowski & Kulczycki, 2019; Kyvik & Aksnes, 2015; Rochmyaningsih, 2019; Schneider, Kaare, & Bloch, 2016; Vanecek, 2014; Vanecek & Pecha, 2020). Qualitative criteria in publication requirements prevent the waste of time and money of academics and orient them to high standards in research.