Scientometric Implosion that Leads to Explosion: Case Study of Armenian Journals

While attempting to evaluate the academic impact of scientific journals, the journal impact factor (JIF) has become the most commonly used measure for journals. Over the last several decades, the interest in this metric and its role in research evaluation has progressively increased within the academic community and beyond.

The aim of the first studies on the quality of scientific journals was about how to filter the most important and influential periodicals in the given scientific field to purchase for the libraries of particularly small American colleges to stimulate their development (Gross & Gross, 1927). The idea that was not initially aimed at research evaluation was then developed to one of the most widely used tools—journal impact factor—with its apparent strength and shortcomings. What we nowadays understand as impact factor was presented by Eugene Garfield in 1955 which led to a publication of Science Citation Index (SCI) in 1961 (Garfield, 2006). In order to accelerate the selection of journals for SCI, Garfield and Irving Sher have introduced journal impact factor by re-sorting the author citation index into the journal citation index, thus creating Journal Impact Factor (Garfield, 2006). The first sample ranking of journals by impact factor appeared in 1969, which was followed by a later annual publication of Journal Citation Reports (JCR). Among the criticisms of this system were geographic (mainly US-based) and language (mainly English - language) bias of journals as a source for research (Aksnes & Siversten, 2019). Although the Web of Science is trying to address this shortcoming by including regional and national indexes (e.g. China Knowledge Resource Integrated Database, Korean Journal Database, SciELO Citation Index, Russian Science Citation Index), there is still a huge misrepresentation of national journals and journals in national languages in the system.

Meanwhile, there was and still is a criticism concerning the journal impact factor, however, as Hoeffel (1998) rightly notes “IF is not a perfect tool to measure the quality of articles, but there is nothing better. . .” This continues to be true until now and despite its shortcomings the Journal Citation Report and its Journal Impact Factor are greatly effecting research, journals, and scientists’ evaluations around the world (Archambault & Larivière, 2009).

The issue of evaluation of scientific journals has become important also in the Republic of Armenia but quite late after the country has regained its independence from the USSR in 1991. Integration into the international scientific community as an independent unit became one of the first challenges for the young republic. Development and further preservation of high-quality Armenian academic journals was conceived as one of the steps towards this goal. However, until 2010 there were only two tools that were used to evaluate Armenian academic journals: a) international indexing platforms (which were then not so popular and were not understood as prestigious) and b) the list of the recommended journals developed by the Supreme Certifying Commission

For more on the Supreme Certifying Commission, see www.bok.am.

The situation has changed in the first decade of the 2000s. One of the aims of the establishment of the Center for Scientific Information Analysis and Monitoring

The Center for Scientific Information Analysis and Monitoring was established in January 2010 by the initiative of the State Committee of Science and National Academy of Sciences of the Republic of Armenia (NAS RA). The Center is functioning within the Institute for Informatics and Automation Problems of NAS RA. Among the main activities of the Center are: studying and monitoring science in Armenia; calculating Impact Factors for the journals published in Armenia; importing and developing Scientometrics as a separate science sub-field in Armenia; developing the Armenian Science Citation Index.

The definition of implosion is used in the natural sciences, social sciences implying compression of a territory (Dvoryadkina & Kaibicheva, 2017). As such, when an object is subjected to endogenous and exogenous pressure, it “explodes.” Projecting this phenomenon in scientometrics, we argue that the existence of a compressed space in the country narrows the interest in the journals published in the local languages. Because of the limited audience, the journals are not developed being unable to reach international levels. After fulfilling the technical requirements, citations in ISD are the second prerequisite for the national journals to be included there. Thus, as soon as one of the local journals breaks through the compressed space, an explosion happens. This affects also the dynamic of citations of these journals. When the explosion happens, the other journals acknowledge the prospects of development and follow the same path. This results in an internationalization of science. Such processes are being observed in the countries of Asia and former Soviet states. The aim of this article is to introduce the new term scientometric implosion into the scientific discourse and test the idea on the example of the Armenian journals.

This article is an extended version of the thesis by the authors “Sargsyan Sh., Mirzoyan A., Blaginin V. Scientometric implosion of Armenian journals, 17th International Conference on Scientometrics and Informetrics, ISSI 2019 Proceedings, pp. 2642–2643.” Here a more detailed analysis related to the Armenian journal indexed in national and international scientific databases and their citation was made. The article also gives some forecasts related to the possible behavior of Armenian journals and their inclusion in ISD and proves the possibility to use the term scientific implosion in scientometric discourse.

The present work is based on the data obtained from four databases: the Clarivate Analytics’ Web of Science (WoS Core Collection and Emerging Source Citation Index), Elsevier's Scopus, the Russian Index of Scientific Citation's Core Collection (RISC CC), developed by the Russian Scientific Electronic Library (e-Library), and the Armenian Science Citation Index (ASCI) which is in the process of development by the CSIAM. From the first three databases, we have retrieved journals that are affiliated with Armenia. From the ASCI all journals that were indexed for a given year were retrieved. The used time window of citations received by the Armenian journals from the WoS and RISC is 2013–2017.

ArmJIF is calculated using the same methodology as Journal Impact Factor. ArmJIF is being calculated for the Armenian journals indexed in the Armenian Science Citation Index. Nowadays ArmJIF is being calculated for all Armenian journals indexed in the ASCI. In the future, however, the Core Collection of journals will be identified.

The subject fields of journals are classified into 15 fields according to Glänzel and Schubert (2003).

Inclusion of the Armenian journals in the national scientific database (ASCI) was the first step towards the standardization and then internationalization of the Armenian journals. Currently, there are about 120 scholarly journals in Armenia, but only about 100 of them are included in ASCI database due to different reasons connected with predominantly technical requirements—the absence of archive, irregular periodicity, a large number of non-scientific articles, etc. For all the journals indexed in the ASCI that have availability of issues (for three consecutive years) ArmJIF is being calculated. The number of journals for which ArmJIF has been and is calculated, varied from 50 to 99. According to Figure 1, the number of journals in ArmJIF database has started to increase for the first six consecutive years, then began to fall. This was connected with the policy of CSIAM; at first the approach was to include as many journals as possible (depending on the availability of journal issues necessary for ArmJIF calculation). Later, only those journals that met minimum technical requirements, among which online archives for the three necessary years, were taken. Although the ASCI hasn’t a so-called Core Collection yet, it is planned, as making this difference will necessarily further increase the quality of the Armenian journals.

Figure 1

Half of the Armenian journals indexed in the ASCI are in the fields of Social Sciences and Humanities, while the second half is being distributed among the other 12 subject categories (Figure 2).

Figure 2

Meanwhile, the journals from Armenia indexed in ISD are predominantly from the natural sciences and only one of them, Wisdom, is in the field of Arts and Humanities. At this moment there are six journals from Armenia indexed in WoS, Scopus, and RISC CC. It should also be noted that the same six journals appear in nearly three databases (Table 1) with the RISC CC being almost the first, followed by Scopus (or simultaneously) and then WoS, which implies that the inclusion in the RISC CC is a sort of the first step for those journals.

The present study has revealed that after the investment of journal metrics in Armenia, the great majority of Armenian journals have considerably improved their technical characteristics (Table 2). Additionally, the new requirements of the science management authorities advanced also a peer review process making it mandatory for the Armenian journals. Other improvements are related to language—a lot of journals start to publish also in English (or at least having their abstracts in English); editorial boards of the journals—foreign scholars are included in the board; international requirements related to publishing ethics are introduced, etc. This allows the national journals to make an implosion that will later transform into an explosion, bringing some of them to ISD, thus to a wider scientific community.

As a final step of our research, we have studied citations to the Armenian journals not indexed in any ISD to see the reaction of the international scientific community to the changes of national journals. For that purpose, we have taken the six Armenian journals already indexed in the studied ISD and another six Armenian journals not indexed in that ISD (Table 3). The six journals not yet indexed in several cases have received even more citations than those already indexed. According to our prognosis and in support of our thesis of the scientometric explosion, these journals have all the chances to break the national territory and be internationalized.

When after the USSR dissolution Armenia regained its independence in 1991, there was a scarcity of publications from Armenia in ISD. Moreover, there was no knowledge and consequently interest in them. Before independence, national journals of the Soviet states indexed in the ISI database were affiliated with Soviet Union, so, although there were very few (2–3 depending on the period) Armenian journals in the ISI, they were attributed to the Soviet Union. Plus, the great majority of national publications were not visible to the international scientific community thus making the Armenian science isolated and out of the scope of any bibliometric analysis. There was a serious scientific enclosure in the area and the journals did not tend for attaining the international level or even pursue that goal. The situation hadn’t changed for the next two decades. In order to revive the national science and bring the Armenian journals to the world standards, CSIAM had initiated the establishment of a national scientific database to collect bibliometric data on national journals, citations received by the national journals and also started calculating impact factors for the local journals. The new requirements considerably affected the quality of the Armenian journals.

Nowadays only 5% of Armenian academic journals (published in English) are presented in ISD, in particular WoS, Scopus, and RISC CC. According to the obtained data, the first journal from Armenia that appeared in ISD is Astrophysics, which was well before the introduction of journal technical requirements and ArmJIF into the scientific community of Armenia. This is also true for the other two journals, Journal of Contemporary Physics and Journal of Contemporary Mathematical Analysis. Their inclusion can be explained also by the fact that traditionally these scientific directions are quite strong in Armenia (Gzoyan et al., 2015) and the management of the journals is internationally active.

Interestingly, in nearly all studied cases, the first ISD where the Armenian journals appear is RISC CC, which is understandable taking into consideration historical relations between Armenia and Russia. Today nearly all Armenian journals have full or partial representation in the RISC which is a result of activities of CSAIM and its collaboration with the Russian part. This process continues and more and more Armenian journals put their full content in RISC, enlarging their attainability to the Russian readership and pave a path to other ISD.

The number of local journals in ASCI is also gradually and firmly increasing and at some point Core Collection of the Armenian journals will be formed, which will further tighten the requirements for the academic journals and will create more competition. The application of ArmJIF in the assessment of Armenian scholarly journals has contributed to the rise of competition, maintenance of publishing ethics, online access of journals, trilingual usage of bibliographic data, etc. If in 2009 only 33% of the local journals had a webpage, now nearly all journals meet this requirement. The number of journals with full information and archive (PDFs of all articles) has reached nearly 100%. Another important development is connected with the shift of languages: more articles started to appear in foreign languages—Russian as the second spoken language in Armenia and English as lingua franca of nowadays science. Generally, however, there is a clear shift towards a trilingual model.

These local developments in a national science market and a compressed territory resulted in the explosion and integration of some Armenian journals to ISD, as they start seeking international recognition and readership. We have further studied international citations to the Armenian journals from the WoS database to assess the chances of other local journals to be included in the ISD. For that purpose, we have identified the first six journals which had received the most citations in WoS. The conclusion is that those journals have all the prerequisites and also intend to break the national boundaries and appear in international ISD. The steady inclusion of Armenian journals in RINC is one of the sure indicators of scientific explosion. This, as well as the latest inclusion of two Armenian journals in the WoS and Scopus, will definitely lead to the inclusion of more Armenian journals in ISD.