The Three-Step Workflow: A Pragmatic Approach to Allocating Academic Hospitals’ Affiliations for Bibliometric Purposes
Article Category: Research Paper
Publicado en línea: 03 feb 2022
Páginas: 20 - 36
Recibido: 16 feb 2021
Aceptado: 17 ene 2022
DOI: https://doi.org/10.2478/jdis-2022-0006
Palabras clave
© 2022 Andrea Reyes Elizondo et al., published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Determining a universal definition for what constitutes a university is a challenging task (Birtwistle, 2003; Rochford, 2006) which is complicated by internationalization (Altbach, 2004), rankings (Dill, 2006), strategic mergers (Labi, 2011), and the evolving roles and social expectations placed on universities (Denman, 2005). Likewise, characterizing the relationships between universities and their affiliated organizations can be problematic given the different degrees of interdependency that exist between them. This is particularly evident in the case of academic hospitals (Barrett, 2008).
Not only do these institutions tend to follow particular national systems, but they also evolve constantly in response to economic factors and sociopolitical expectations (Wartman, 2008). Building on Praal et al.'s (2013) research into the challenges of including affiliated institutes in a university's publication count, this paper describes the various academic hospital systems we have encountered and proposes a mode of classifying the university-hospital relationship in the context of the Leiden Ranking.
Hospital nomenclatures vary worldwide to denote some form of collaboration with a university: academic hospitals, teaching hospitals, university hospitals, and academic medical centres do not correspond to universally standard definitions. Rather than seeking a normative definition of “academic hospital”, we are proposing a method of allocating publications that only mention affiliations with such hospitals. For the purpose of the Leiden Ranking, we distinguish between three general models of hospital-university relationships in order to measure the research output of universities through publications. This exercise is necessary because the organizational boundaries of universities are not always clear. In addition, the information provided under author affiliations may not always be accurate, for example when researchers mention their working address at the affiliated hospital but omit their university affiliation. In this respect, our methodology aligns with Hardeman's (2013) claim that university output cannot be objectively identified because there is no universal and objective definition of what constitutes a university. This is exacerbated by the fact that affiliations serve two general purposes: providing contact information for the authors and attributing credit to the employer(s).
A wide variety of definitions exist for what constitutes a university, a hospital, and in particular an academic hospital. Below we outline the concepts that will appear in this article.
As per the Leiden Ranking framework, we consider universities as higher education institutions that award degrees of the third cycle (Leiden Ranking, 2021) in accordance with the
In general terms, a hospital is perceived as an institution for healthcare provision. In most countries, a hospital will be considered as academic when it is connected to a higher education institution in some form. Although a precise definition and clear organizational structure for academic hospitals do not currently exist, the attribute of “academic” indicates such hospitals have a tripartite mission of patient care, education, and research. For the purposes of our classification system, we are most concerned with the hospital's relationship to the university in the areas of education and research.
Before we move on, it is worth noting that academic medical centres can differ substantially in size and organizational complexity. In addition, these organizations are constantly changing. What was once a single academic hospital may now be an entire campus or an academic health science centre comprising several hospitals, clinics, laboratories, and even research centres; we do not differentiate between single academic hospitals and these larger structures. As aforementioned, what we seek is to employ the university-hospital relationship in the areas of education and research as an instrument to measure publication output. More specifically, we consider whether the publications mentioning only an affiliation with an academic hospital should be included or disregarded in the count of a university's output.
The majority of the literature on academic hospitals either examines the organizational aspect of such institutions or focuses on their various outcomes (Zinner & Campbell, 2009). Below we provide a non-exhaustive overview of the most common research topics.
Many studies discuss the various organizational structures of academic centres (Barrett, 2008; Daniels & Carson, 2011; Davies, Tawfik-Shukor, & de Jonge, 2010), including their governance (Griner & Blumenthal, 1998; Gaynor, Laudicella, & Propper, 2012), and the effects of ongoing restructuring and mergers. Researchers also often highlight the complexity of processes that arises from specific policies (French, Ferlie, & Fulop, 2014) and budget constraints (Fuchs, 2013).
Other scientific literature focuses on the different expectations placed on academic hospitals due to their tripartite mission (Lozon & Fox, 2002) as well as on the changing roles and challenges that academic medical education faces (Gallin & Smits, 1998; Nonnemaker & Griner, 2001; Wartman et al., 2009). Related to this are studies analyzing the conflicting interests between universities, public health organizations, and industry (Brennan et al., 2006; Cohen & Siegel, 2005; Rothman & Chimonas, 2010; Washington et al., 2013). This is especially the case when higher education institutions must find befitting partnerships to fulfil the demands for innovation (Fish, 2013).
Several articles treat specifically the subject of academic science centres and health networks (Fish, 2013; Nicholson et al., 2015; Ovseiko et al., 2014; Wijgert, 2010). Such articles reflect upon the expanding and conflicting expectations that public institutions face in the wake of budget constraints and cost-reduction mergers, and the accountability issues that arise from these events.
Aside from some articles on resource utilization in relation to teaching activities (Sato & Fushimi, 2012), the area of education, a fundamental part of the tripartite mission of academic hospitals, has received little scholarly attention. When it does receive attention, rarely a distinction is made between the different types of education such as patient, professional, postgraduate, or undergraduate education. This lack of distinction in the literature is an indication of the organizational complexity whereby the different types of education provided at hospitals follow the priorities of different governmental departments, most often those of education and health. Therefore, the term “teaching” is generally used in the literature in its broadest sense.
Although most articles focus on regional or national cases, many authors devote attention to developing a general definition of what constitutes an academic hospital. However, the diversity of educational, research, and public health models, coupled with the ever-changing relationships between universities and hospitals, make organizational definitions challenging even at the national level. As several scholars note, if you have seen one [academic medical hospital or medical school], you have seen only one (Lozon & Fox, 2002).
This section describes the three types of affiliation that we distinguish between an academic hospital and university. It then outlines how to identify the type of relationship by applying the three-step workflow and how to interpret the findings.
Hospitals affiliated with universities are relevant elements of academic systems where education and research take place (Washington et al., 2013). Nevertheless, there are significant differences in the types of relationships between hospitals and universities both on an international and national level: on one end of the spectrum there are fully integrated hospitals and medical faculties, while on the other end there are non-integrated hospitals offering clinical placements for students. In between lies an array of structures at varying degrees of interdependency, all determined by national and regional systems.
Examining the organizational structures of university-affiliated hospitals, Wartman describes two prototypical models of academic health centres, while Levine considers five possible structures with varying degrees of interdependency (Levine, 2002; Wartman, 2015). For the purpose of the Leiden Ranking, we identify three general models for relationships between universities and hospitals:
full integration of the hospital and the medical faculty into a single organization; health science centres in which hospitals and medical faculty are still separate entities, albeit within the same governance structure; structures in which universities and hospitals are separate entities and collaborate with one another.
The case of fully integrated academic hospitals shows a strong degree of interdependency with a particular university, for example, when the medical faculty is located at the hospital or vice versa (see Figure 1). This model is found in most academic hospitals of Belgium, the Netherlands, Germany, and Switzerland.
Figure 1
Full integration model.
The system of Health Science Centers also presents a strong degree of interdependency between academic hospitals and universities but the structure differs from the full integration model. In the full-integration model, the medical faculty and hospital are often the same institution. Health science centres, on the other hand, are organizations that coordinate the medical faculty, the various patient-oriented medical facilities, and the research centres (see Figure 2). They can fall directly under the university or stand as a separate legal entity. Health science centres exist predominantly in the United States of America (e.g. the
Figure 2
Health science centres.
It is important to note the difference between health science centres (as modelled in Figure 2) and academic health science networks such as the
The third model refers to a relationship of collaboration and support between hospital and university in which the collaboration is structural rather than incidental: there are some shared activities between the two, but no full integration of the medical faculty into the hospital (see Figure 3). These types of relationships are the most common worldwide.
Figure 3
Collaboration and support.
Around the world, hospitals are referred to as “academic hospital”, “teaching hospital”, “university hospital”, “third-tier hospital”, and “academic medical centre” to denote some form of collaboration with a university; however, these nomenclatures do not constitute universally accepted definitions of specific types of hospital-university relationship, and thus can denote different things in different countries or even regions of the same country (Shi et al., 2021; Sato & Fushimi, 2012). Additionally, the level of collaboration between a university and a hospital may vary substantially between countries and regions, further rendering such labels unsuitable for global comparisons.
Sometimes the relationship between a hospital and a university may seem clear according to local perceptions. However, such local understandings of the interconnectedness between hospitals and universities cannot be used for international comparisons as they are not sufficiently consistent and demand specific local knowledge. On a similar note, some bibliometric databases assign affiliation between teaching/academic hospitals and universities based on local perceptions as communicated by each university to the database owner. This approach presents also shortcomings when doing international comparisons as it lacks consistency and depends on the client base of the various databases.
Following our previous work on academic hospitals (Praal et al., 2013), we at the Leiden Ranking have developed a three-step workflow to classify the relationship between universities and their affiliated hospitals. This workflow effectively transposes the three above-noted relationship models into the two types of instrumental relationships that the Leiden Ranking employs for the allocation of the publication output of academic hospitals. We distinguish between academic hospitals that are associated with universities (“associate”) and academic hospitals that can be considered as part of the university (“component”).
When a hospital and a medical faculty are fully integrated or when a hospital is part of a health science centre (see Figures 1 and 2) the relationship is classified as component. When a hospital follows the model of collaboration and support (see Figure 3) the relationship is classified as associate. Although distinct in their organizational structures, the full-integration and health science centre models suggest a strong integrated relationship concerning the tasks of education and research. In both cases the medical faculty is difficult to separate from the hospital; therefore we treat them in the same way. In the collaboration and support model this is not the case.
Although the proposed scheme cannot do justice to the myriad forms of integration that might exist, it seeks to address the complexity of the relationship between academic hospitals and universities. The scheme is pragmatic and focuses on three aspects that can aid in evaluating the degree of interdependence: legal status, shared educational mandate, and level of physical integration (see Figure 4).
Figure 4
Three-step workflow.
First, a hospital's legal status is examined. If the ownership lies with the university or a foundation owned by or related to the university, the hospital is considered a component. In many cases, ownership can be easily determined for a whole country or region given specific health services regimes. For example, in the United Kingdom, hospitals and trusts that collaborate intensively with universities are owned by the
For the first step in the workflow, not only the direct legal ownership is considered but also whether the hospital is part of a university health system. We find many cases like this in the United States of America. For example, the
Some cases present a seemingly opposite ownership situation that nevertheless results in a classification as component. For example, the
The second step in the workflow examines a hospital's mandate. Given the Leiden Ranking's focus on universities, the tasks considered here are core-curriculum education and research. In general, every hospital affiliated with a university will conduct research, except for smaller clinics which solely provide clinical placements. A more difficult function to determine is the type of education provided by a hospital. Over all, we have found that hospitals provide different levels of education ranging from patient education (for example on nutrition), specialist training or continuous education, to core-curriculum education for undergraduate students. On some occasions, a hospital has a clear mandate to aid a university in the provision of undergraduate education, in which case the hospital is considered to act as a component of the university.
The mandate of a hospital is not always publicly available, although for some countries it is part of the national or regional legislation. In a few cases, the mandate is part of a hospital's mission statement. For example, the
In some countries, we have observed a clear pattern in which public academic hospitals provide core education. However, it is possible that the wave of public-services privatization in those countries may influence the extent of educational tasks provided by hospitals, as privatized hospitals tend to become less involved with education. In Germany, for example, the majority of large academic hospitals are traditionally public and involved with core education, but as some of these hospitals have been recently privatized, not all German hospitals can be automatically considered as components of universities.
Often, information about hospitals’ mandates is not available at all. For example, the
The third and final step of the workflow examines to what extent a hospital and medical faculty are integrated. This step covers two major elements: the physical location of the faculty and the publication behaviour of authors mentioning the hospital affiliation. For the former we consider whether the hospital and the medical faculty have shared locations. For the latter we examine whether the authors publishing for the hospital are members of the medical faculty.
When a hospital shares locations and staff with a medical faculty, it is considered a component of the university. As in the previous steps, the outcome of this review is rarely a binary yes-no. In some cases, the medical school or most of its departments will be located on the hospital grounds, indicating a tight physical integration. For example, the medical school of
With this complexity in mind, the third step also considers the publication behaviour of authors using the hospital affiliation: we examine a sample of publications comprising authors using the hospital affiliation without mentioning the university to which the hospital is supposedly affiliated. We then investigate whether those authors are formally employed by the university. For example, the majority of the authors from our sample who publish using the
When the workflow shows a low level of interdependency, we classify the relationship of the hospitals as that of associate to the university. As an example, a hospital owned by the regional health authority, without a clear mandate for core-curriculum education, with one or no university department located on the hospital premises, and few authors publishing with the hospital's address who hold a faculty position, would be considered as associate. Such is the case of the
It is worth emphasizing that this is not a straightforward process: often it is difficult or impossible to retrieve all the information necessary to complete the workflow, and even with all of the information, the results of the assessment may be ambiguous. Thus the decision of classifying the relationship as associate or component typically depends on a combination of the available information. Lastly, because of its resource-intensive nature, the assessment is only carried out for academic hospitals whose publication output surpasses a specific threshold.
Determining the relationship model can have a substantial impact on the publication counts for universities (as shown in Table 1). For example, the
Overview of the separate output per university and hospital as well as their combined output. The percentage shows the increase of publication count of the university when the hospital is included as a component.
| Organization | Separate output | Combined output | Percentage |
|---|---|---|---|
| Université de Tours | 3,463 | 4,525 | 31% |
| Centre Hospitalier Régional Universitaire de Tours | 1,062 | ||
| University of Zurich | 18,573 | 22,297 | 20% |
| University Hospital Zurich | 3,724 | ||
| University of Parma | 5,222 | 5,924 | 13% |
| Azienda Ospedaliero-Universitaria di Parma | 702 | ||
| Complutense University of Madrid | 12,908 | 14,300 | 11% |
| Hospital Clínico San Carlos | 1,392 | ||
| University of Ottawa | 14,959 | 16,474 | 10% |
| The Ottawa Hospital | 949 | ||
| Children's Hospital of Eastern Ontario | 566 | ||
| University of Ioannina | 3,479 | 3,689 | 6% |
| University Hospital of Ioannina | 210 | ||
| University of Turku | 7,849 | 8,195 | 4% |
| Turku University Hospital | 346 | ||
| Palacký University Olomouc | 4497 | 4,620 | 3% |
| University Hospital Olomouc | 123 | ||
| Australian National University | 13,932 | 14,158 | 2% |
| Canberra Hospital | 226 | ||
| University of Hong Kong | 17,790 | 18,015 | 1% |
| Queen Mary Hospital, Hong Kong | 225 | ||
| University of Buenos Aires | 8,434 | 8,492 | 1% |
| Hospital de Clínicas José de San Martín | 58 | ||
| University of New Mexico | 8,138 | 8,179 | 0% |
| UNM Health Sciences Center | 41 |
Caption: Output figures are based on publications indexed for the SCIE, SSCI, and AHCI citation indexes as part of the Web of Science produced by Clarivate Analytics. Only core publications from 2017–2020 that were classified as articles or reviews have been taken into account. Please see the Leiden Ranking website for more information on the selection of publications (Leiden Ranking, 2021) On the Leiden Ranking website this way of counting is denoted as ‘full counting method’ as opposed to the fractional counting method in which each affiliation in a paper receives only partial credit for a paper based on the proportion of the authors using that affiliation. Although the fractional counting method is the preferred counting method in the Leiden Ranking, for this paper we chose to use the full counting method as this method is more straightforward to understand and most university rankings only use this counting method. In the fractional counting method, differences between the separate output of a university and the output combined with the hospital tend to be larger as affiliations with academic hospitals often appear in papers alongside affiliations with the university.
These differences in counting publications output are also evident in other cases. For example in Italy, the
Notably, the discrepancy on publication assignations is not present for all universities. For example, the
As mentioned above, this paper does not seek to provide a normative definition of an academic hospital rather than a pragmatic process for assigning publications with only a hospital address in university rankings. The three-step model allows for a standard approach that can facilitate comparison on the level of integration between hospitals and universities on a global scale in a fair and transparent manner.
Further, by managing the affiliation relationships through linkages instead of automatically cleaning hospital publications as those of the university, data can be disaggregated and analyzed at different levels and combinations. Likewise, the university-hospital linkage is not an obstacle to recognize other types of collaboration between those same institutions. For example, an academic hospital may be an associate of a university yet collaborate tightly with it on a research lab. If the research lab has a significant publication output it can be created as unit of analysis which is component to both the university and the academic hospital. Thus the different collaboration paths can be recognized without focusing solely on the output of one organization (Hicks & Katz, 1996).
The approach is not without issues, especially as the information necessary to assign the type of affiliation is not always available. Owing to the different information-seeking paths and a non-binary decision process, the model cannot be applied automatically. Being labour and time intensive may limit its application by other research organizations. Nevertheless this model is a step in an ongoing research on publication patterns of hospitals and universities.
As the model is not normative, we do not foresee any sort of policy implications for academic hospitals and universities. The risk that some organizations or regions may game the rankings by specifically rewriting mandates is a possibility. However, the implications for the organizations themselves would be considerable as such change would involve a whole administrative and budgetary adjustment. At the same time, being transparent on the processes to categorize organizations and their affiliations may help universities and academic hospitals to communicate their relationship more clearly to the outside world.
Building upon Praal et al.'s (2013) discussions on how to delimit universities with regard to academic hospitals for publication count, the Leiden Ranking proposes a characterization and methodology that allows us to evaluate the degree of integration between these two types of organizations. Although the methodology presented herein is somewhat arbitrary, employing these standards to compile data allows for a more balanced and fair comparison between worldwide educational and research systems, without losing disaggregated data at different levels.
The Leiden Ranking's current approach has some limitations. Firstly, it requires a resource-intensive assessment process. We have sought to streamline this process by setting a threshold of publications before submitting a hospital to the three-step review. Secondly, examining the relationships between universities and hospitals in countries with non-Latin-based alphabets poses a challenge and can be a disadvantage for those organizations that do not have comprehensive faculty websites in English or do not update their faculty pages often. Nevertheless, the model provides a standard approach for evaluating integration between these organizations.
To conclude, we would like to add that organizational structures will continue evolving in order to better respond to public and government expectations of accountability and efficiency, requiring our methodologies to evolve as well. The experience and knowledge acquired through the in-depth evaluation of academic hospitals following the three-step model, will allow us to follow these changes better.