Academic Collaborator Recommendation Based on Attributed Network Embedding

At present, the popular academic mining task is academic recommendation, which aims to help scholars deal with a large amount of scholarly data and find potential collaborators. Usually, the collaborator recommendation can be regarded as a problem of measuring the similarity among scholars. Approaches for recommending collaborator using similarity mainly include network structure-based, random walk-based, and deep learning methods. CN (Lü & Zhou, 2011) is a network structure-based approach, holding that the more scholars who have common neighbors, the more likely they will collaborate in the future. The method based on Random Walk with Restart (RWR) mainly uses random walk to capture the relationship among nodes. For example, Xia et al. (2014) explore three academic factors, i.e., coauthor order, latest collaboration time, and times of collaboration, to define link importance in academic social networks. Zhou et al. (2021) define the transition probability among different types of nodes via quantifying three academic relationships, i.e., researcher-researcher, researcher-article, and article-article. For recommending the most Beneficial Collaborator, Kong et al. (2017) studied researchers’ topic distribution of research interest, interest variation with time, and researchers’ impact in collaborators network.

Recently, some methods based on deep learning have been proposed. Liu, Xie, and Chen (2018) proposed a context-aware academic collector recommendation model CACR, which consists of the collaborative entity embedding network (CEE) and the hierarchical factorization model (HFM). CEE learns joint embeddings for researchers and topics with their mutual dependency. HFM exploits researchers’ activeness and conservativeness to make high-quality new collaborator recommendations. Zhang et al. (2020) perform an improved random walk, generate specific node sequences capturing network structures, and then employ a novel graph recurrent neural framework to embed scholar attributes. ACNE (Wang et al., 2021) proposed by Wang et al., which extracts four types of scholar attributes based on the proposed scholar profiling model, can learn a low-dimensional representation of scholars considering both scholar attributes and network topology simultaneously. However, while many of the recommended models by academic collaborators take into account the fusion of network topology and scholarly attribute information, exploration of the highly nonlinear relationship between them is rarely studied.

In recent years, network embedding has flourished. The main idea of network embedding is to map the network into a low-dimensional latent space to maximize the preservation of network topological and node attributes information. In general, network embedding technology can be divided into two categories, topology network embedding, and attributed network embedding. Perozzi, Al-Rfou, and Skiena (2014) introduce word embedding into graph embedding. The proposed model DeepWalk adopts a truncated random walk to obtain the structural information of the network and uses the skip-gram model to learn the low-dimensional node embedding. The Node2vec (Grover & Leskovec, 2016) designed by Grover et al. proposes a biased random walk that can flexibly explore diverse neighborhoods. Wang, Cui, and Zhu (2016) utilize an auto-encoder to preserve the first order and second order proximity simultaneously. Most recently, some embedding methods enhance node representation by joining node attributes. To preserve the unique properties of each relationship while integrating information of different types of relationships, MNE (Zhang et al., 2018) represents it with high-dimensional public embedding and low-dimensional additional embedding. Cen et al. (2019) apply the multi-view network embedding approach to heterogeneous networks, which proposed a transductive and Inductive model to learn node embedding. Shi et al. (2019) adopt a meta-path based random walk strategy on heterogeneous information network embedding for the recommendation.

Usually, working with others is inevitable for scholars engaged in scientific research. Collaboration among scholars is now becoming more and more common. Studies have shown that collaboration will also promote the improvement of scientific output. Academic collaboration networks can help us discover academic social relationships among scholars. For example, the first-order neighbors of scholars in the network represent his co-authors. Describing relationships in networks with first-order neighbors proved efficient, but there are certain limitations in using it directly to mine academic relationships. In fact, for a specific scholar, scholars who have worked with him are not necessarily more likely to cooperate in the future than those who have never worked with him. That is, the relationships among scholars do not only depend on the near neighbors in the network. It also inspires us to reconsider the capture of academic network structure when predicting potential collaborators.

To more comprehensively capture non-local academic collaboration network structure, we define a new neighbor for each scholar, called non-local neighbors. Specifically, for each node in the academic collaboration network, the non-local neighbor comes from the nodes that start from the start node and passes through after T times of walk with a length of l. Note that our walk strategy builds on the Node2vec (Grover & Leskovec, 2016). After walking T times, we did not select the nodes of all the walk sequences. Here we set a filtering threshold Freq, to take partial nodes with their higher occurrence frequency as non-local neighbors of the start node. The reason for that is when we are building an academic collaboration network, the weight of links in the network is the number of collaborations among scholars. After a certain number of random walks and frequency filtering, the nonlocal neighbor nodes have a strong relationship with the start node.

Take an example, as shown in Fig. 2. First, we select scholar a as the start node and set the number of walks as 5, and the walk length as 5. Under the guidance of link weight, we start the biased random walk from scholar a, after that, a series of scholars related to a will be sampled. Finally, based on the sampled nodes, after the operation of frequency filtering, we get scholar a's non-local neighbors b, c, e, f, h. It should be noted that among all the non-local neighbors of scholar a, even if some first-order neighbors are filtered out in the frequency filtering, their first-order neighbors should be involved.

Figure 2

Recent studies have shown that it is necessary to consider the scholar's comprehensive attributes. However, although some recent works combine the scholar's attributes, they almost only consider one aspect, which is relatively not comprehensive for the grasp of the overall information of scholars. Therefore, for the recommendation of academic collaborators, it is natural for us to consider various attributes information of scholars. The following attributes for scholars are explored:

Academic Age (AA): Academic age can be represented by the interval between the first paper and the last paper published by scholars within the period of our study. AA=tcur−tfirst AA = {t_{cur}} - {t_{first}} where t_cur is the year of the last paper published by the scholar up to the research point, and t_first is the year of the first paper published by the scholar in the research period.

For the six different attributes mentioned above, we integrate them through the concatenate operation. Scholar attributes matrix A can be obtained by concatenation operation concat() as shown in Eq. (1). (1) A=concat(AA, RI, PUB, AC, CO, H) A = concat\left( {AA,\,RI,\,PUB,\,AC,\,CO,\,H} \right)

Based on the attribute matrix A, we can calculate the attribute similarity among scholars with a cosine function. For example, assuming that A_i and A_j are the attributes embedding of scholar i and scholar j, respectively. The calculation of attributes similarity between them is shown as follows: (2) Sim(Ai,Aj)=Ai⋅Aj|Ai||Aj|=∑mda(Ai,m*Aj,m)∑m=1daAi,m2*∑m=1daAj,m2 Sim\left( {{A_i},{A_j}} \right) = {{{A_i} \cdot {A_j}} \over {\left| {{A_i}} \right|\left| {{A_j}} \right|}} = {{\sum\nolimits_m^{{d_a}} {\left( {{A_{i,m}}*{A_{j,m}}} \right)} } \over {\sqrt {\sum\nolimits_{m = 1}^{{d_a}} {A_{i,m}^2} } *\sqrt {\sum\nolimits_{m = 1}^{{d_a}} {A_{j,m}^2} } }}

After obtaining the attributes similarity among scholars, we establish relationships for all scholars with their Top-K attributes similar neighbors. Here, we take nodes with high attribute similarity as attr_sim neighbors. As shown in Fig. 3, suppose we aim to find scholar a's attr_sim neighbors. First, we use Eq. (2) to calculate the similarity of the attributes between a and other scholars. Then, we find scholars k, j and a have higher attributes proximity when we set the number of attr_sim neighbors as 2. Therefore, scholars k, j serve as the attr_sim neighbors of a, which could preserve the intrinsic similarity among scholars.

Figure 3

The non-local neighbors and attr_sim neighbors describe the proximity of scholars on network topology and academic attributes levels, respectively. To integrate these two proximities, we construct a new multi-type relationship network for all scholars, combining their non-local neighbors and attr_sim neighbors, as shown in Fig. 4. By integrating the two proximity, our multi-type relational network can also capture the similarity of attributes among scholars while preserving non-local network topology.

Figure 4

To encode the academic collaboration network topology and scholar attributes into a low-dimensional representation space, we extend the traditional deep auto-encoders according to the literature (Salakhutdinov & Hinton, 2009). Deep auto-encoder is a deep neural network model for feature learning, which can learn highly non-linear topological and attribute features. It is an unsupervised model, consists of an encoder and a decoder. Formally, given the i-th row x_i of the adjacency matrix X of multi-relational network, the hidden representations for each layer are shown as follows: (3) yi(1)=σ(W(1)xi+b(1)) y_i^{(1)} = \sigma \left( {{W^{(1)}}{x_i} + {b^{(1)}}} \right) (4) yi(k)=σ(W(k)yi(k−1)+b(k)), k=2,…,K y_i^{(k)} = \sigma \left( {{W^{(k)}}y_i^{(k - 1)} + {b^{(k)}}} \right),\,k = 2, \ldots ,K where k is the number of layers for the encoder and decoder. σ() represents the nonlinear activation function. W^(k) and b^(k) are the trainable parameters. Here, yi(k) y_i^{(k)} is the final low-dimensional representation of the i-th scholar. After obtaining yi(k) y_i^{(k)} , we can get the output x^i {{\hat x}_i} by reversing the calculation process of encoder. The learning process can be simply described as minimizing the difference between the input x_i and the reconstructed data x^i {{\hat x}_i} . Therefore, we minimize the reconstruction loss L_Non-local as follows: (5) LNon−local=∑i=1|V|‖x^i−xi‖22 {L_{Non - local}} = \sum\nolimits_{i = 1}^{\left| V \right|} {\left\| {{{\hat x}_i} - {x_i}} \right\|} _2^2

In addition, we should also consider the importance of local network structure. That is to say, for the scholars who have collaborated, we hope that their representation will be closer, simultaneously. Here, the supervised loss L_Local for this goal is designed as follows: (6) Llocal=∑i, j=1|V|Si,j‖yi−yj‖22 {L_{local}} = \sum\nolimits_{i,j = 1}^{\left| V \right|} {{S_{i,j}}\left\| {{y_i} - {y_j}} \right\|} _2^2 where S_i,j ∈ {0,1}, which indicates whether there is a link between the node i and j. Finally, to preserve the global and local structure of the network simultaneously, we propose a semi-supervised model ACR-ANE based on deep auto-encoder, which combines Eq. (5) and Eq. (6) and joint minimizes the following objective function: (7) L=LNon−local+λLLocal+LReg=∑i=1|V|‖x^i−xi‖22+λ∑i,j=1|V|Si,j‖yi−yj‖22+12∑i=1|K|(‖W(k)‖22++‖W^(k)‖22) \matrix{ L \hfill & = \hfill & {{L_{Non - local}} + \lambda {L_{Local}} + {L_{{Reg}}}} \hfill \cr {} \hfill & = \hfill & {\sum\nolimits_{i = 1}^{\left| V \right|} {\left\| {{{\hat x}_i} - {x_i}} \right\|} _2^2 + \lambda \sum\nolimits_{i,j = 1}^{\left| V \right|} {{S_{i,j}}\left\| {{y_i} - {y_j}} \right\|} _2^2 + {1 \over 2}\sum\nolimits_{i = 1}^{\left| K \right|} {\left( {\left\| {{W^{\left( k \right)}}} \right\|_2^2 + + \left\| {{{\hat W}^{\left( k \right)}}} \right\|_2^2} \right)} } \hfill \cr }

Specifically, λ is a linear trade-off parameter that adjusts the penalty between L_Non-local and L_Local. L_Reg is an L2-norm regularizer term to prevent overfitting.

To evaluate the performance of ACR-ANE, we conduct experiments using two real-world scholarly datasets, including Aminer

https://www.aminer.cn/

We adopt three widely used metrics to evaluate the proposed recommendation model ACR-ANE, including Precision@k, Recall@k, and F1@k. Precision@k is a metric for indicating the proportion of new scholars in the recommendation list who have collaborated with the target scholars, defined as: Precision@k=Number of correct recommended collaboratorsNumber of recommended scholars {Precision}@k = {{Number\,of\,correct\,recommended\,collaborators} \over {Number\,of\,recommended\,scholars}}

And for Recall@k, which denotes the proportion of new collaborators recommended by the model who have actually collaborated with target scholars, defined as: Recall@k=Number of correct recommended collaboratorsNumber of true collaborators {Recall}@k = {{Number\,of\,correct\,recommended\,collaborators} \over {Number\,of\,true\,collaborators}}

F1@k is an integrated metric of Precision@k and Recall@k, defined as: F1@k=2*(Precision @k*Recall@k)Precision @k+Recall@k F1@k = {{2*\left( {{Precision}\,@k*{Recall}@k} \right)} \over {{Precision}\,@k + {Recall}@k}}

To evaluate the performance of the proposed recommendation model ACR-ANE, here we use the following baseline methods. For the rest baseline methods, their parameters are set following their original papers. A brief description of these methods is as follows:

DeepWalk (Perozzi, Al-Rfou, & Skiena, 2014): DeepWalk is a classic deep learning model for network embedding. It uses a truncated random walk to obtain node sequence and then adopts Skip-gram model to train and generate scholars embedding. The gained scholar embedding can be used for similarity calculation for the recommendation.

In this part, we mainly study the impact of different experimental parameter settings, including scholar embedding dimension, scholar non-local neighbor filtering parameters. Finally, to demonstrate the effectiveness of the proposed method, we compare our model to the baseline models from three evaluation metrics.

5.4.1

Influence of non-local filtering parameter Freq

Freq indicates the filtering threshold when selecting the non-local neighbors. We analyze how the adjustment parameter Freq influences the performance of the proposed model on Precision, Recall, and F1. To obtain a better Freq, we conducted experiments on two datasets with 5 possible values for adjustment parameter, i.e. {10, 20, 30, 40, 50}.

The experimental results are shown in Fig. 5 and Fig. 6, respectively. From Fig. 5, we can observe that the model performance is gradually better when we increase the frequency from 10 to 30, while the Freq is greater than 30, the performance of the model shows a downward trend. Unlike Fig. 5, Fig. 6 shows the performance of ACR-ANE on the APS dataset. From Fig. 6(a)(b)(c), we can see that the model performs best when Freq is set to 10. Therefore, we set Freq as 30 and 10 for these two datasets, respectively. In addition, as shown in Fig. 5(a) and Fig. 6(a), we can see that the Precision shows an overall downward trend with the increasing value of recommendation list k. It is not hard to understand that as the recommendation list k increases, the probability that new collaborators of the target scholar will be recommended is greater. However, the increase in the number of new collaborators recommended is less than the increase in the recommendation list length k. Therefore, according to the definition of precision, we can see that precision decreases.

Figure 5

Figure 6

5.4.2

Influence of scholar embedding dimension

To obtain the ideal dimension of the embedding, we set the length k of the recommended list as 20, and the scholar embedding dimension is set as five different values, i.e. {40, 60, 80, 100, 120, 140}. We have conducted extensive experiments on AMiner and APS, and Precision@k, Recall@k, F1@k are used to evaluate the recommendation performance of the model under the different scholar embedding dimensions. What can be seen from Fig. 7 and Fig. 8 is that with the increase of scholar embedding dimension Dim, the overall performance of ACR-ANE is expected to increase accordingly, and then remain steady after a certain value. Specifically, the best performance of the model is achieved with a dimension of 120. Therefore, considering the above model performance, we set the scholar embedding dimension as 120 in the following experiments.

Figure 7

Figure 8

5.4.3

Comparison with baselines

Our ultimate task is to recommend potential academic collaborators for scholars. In this section, we compare ACR-ANE with several baseline models. At the same time, to demonstrate the importance of scholar attributes information, we propose a variant of the ACR-ANE model, which is a weakened version that only considers the structure of the academic collaboration network and does not incorporate attributes in learning scholar embedding. Fig. 9 and Fig. 10 present the experimental comparison results on the Aminer dataset and APS datasets in terms of Precision, Recall, and F1. According to the experimental results, we can get the following observations: 1)

Our proposed ACR-ANE has better performance in recommendation precision, recall, and f1 than all the state-of-the-art methods. This result shows the effectiveness of our proposed model in the task of academic collaborator recommendation.

2)

While comparing with other network structure-based embedding models, ACR-NE performs best in the final recommendation task. It indicates the effectiveness of our proposed method to consider scholars’ non-local neighbors in terms of capturing academic collaboration network structure. In contrast with ACNE, our proposed method performs better on two more datasets. Specifically, ACNE is an embedding model dominated by attribute information, while our proposed model is structure-dominated. The results demonstrate the importance of focusing on preserving the network structure for the final node representation learning.

3)

Through the comparative analysis of ACR-ANE and ACR-NE, we can see that ACR-ANE performs better, which shows that it is significant to consider the structure of academic collaborate network and scholar attributes when recommending collaborators.

Figure 9

Figure 10