<abstract xmlns="http://www.w3.org/1999/xhtml"><sec id="j_jdis-2019-0019_s_007_w2aab3b7b3b1b6b1aab1c17b1Aa"><h3>Purpose</h3><p>To uncover the evaluation information on the academic contribution of research papers cited by peers based on the content cited by citing papers, and to provide an evidence-based tool for evaluating the academic value of cited papers.</p></sec><sec id="j_jdis-2019-0019_s_008_w2aab3b7b3b1b6b1aab1c17b2Aa"><h3>Design/methodology/approach</h3><p>CiteOpinion uses a deep learning model to automatically extract citing sentences from representative citing papers; it starts with an analysis on the citing sentences, then it identifies major academic contribution points of the cited paper, positive/negative evaluations from citing authors and the changes in the subjects of subsequent citing authors by means of Recognizing Categories of Moves (problems, methods, conclusions, etc.), and sentiment analysis and topic clustering.</p></sec><sec id="j_jdis-2019-0019_s_009_w2aab3b7b3b1b6b1aab1c17b3Aa"><h3>Findings</h3><p>Citing sentences in a citing paper contain substantial evidences useful for academic evaluation. They can also be used to objectively and authentically reveal the nature and degree of contribution of the cited paper reflected by citation, beyond simple citation statistics.</p></sec><sec id="j_jdis-2019-0019_s_010_w2aab3b7b3b1b6b1aab1c17b4Aa"><h3>Practical implications</h3><p>The evidence-based evaluation tool CiteOpinion can provide an objective and in-depth academic value evaluation basis for the representative papers of scientific researchers, research teams, and institutions.</p></sec><sec id="j_jdis-2019-0019_s_011_w2aab3b7b3b1b6b1aab1c17b5Aa"><h3>Originality/value</h3><p>No other similar practical tool is found in papers retrieved.</p></sec><sec id="j_jdis-2019-0019_s_012_w2aab3b7b3b1b6b1aab1c17b6Aa"><h3>Research limitations</h3><p>There are difficulties in acquiring full text of citing papers. There is a need to refine the calculation based on the sentiment scores of citing sentences. Currently, the tool is only used for academic contribution evaluation, while its value in policy studies, technical application, and promotion of science is not yet tested.</p></sec></abstract>

PurposeTo uncover the evaluation information on the academic contribution of research papers cited by peers based on the content cited by citing papers, and to provide an evidence-based tool for evaluating the academic value of cited papers.Design/methodology/approachCiteOpinion uses a deep learning model to automatically extract citing sentences from representative citing papers; it starts with an analysis on the citing sentences, then it identifies major academic contribution points of the cited paper, positive/negative evaluations from citing authors and the changes in the subjects of subsequent citing authors by means of Recognizing Categories of Moves (problems, methods, conclusions, etc.), and sentiment analysis and topic clustering.FindingsCiting sentences in a citing paper contain substantial evidences useful for academic evaluation. They can also be used to objectively and authentically reveal the nature and degree of contribution of the cited paper reflected by citation, beyond simple citation statistics.Practical implicationsThe evidence-based evaluation tool CiteOpinion can provide an objective and in-depth academic value evaluation basis for the representative papers of scientific researchers, research teams, and institutions.Originality/valueNo other similar practical tool is found in papers retrieved.Research limitationsThere are difficulties in acquiring full text of citing papers. There is a need to refine the calculation based on the sentiment scores of citing sentences. Currently, the tool is only used for academic contribution evaluation, while its value in policy studies, technical application, and promotion of science is not yet tested.

CiteOpinion: Evidence-based Evaluation Tool for Academic Contributions of Research Papers Based on Citing Sentences

Journal of Data and Information Science

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

PurposeTo uncover the evaluation information on the academic contribution of research papers cited by peers based on the content cited by citing papers, and to...

CiteOpinion: Evidence-based Evaluation Tool for Academic Contributions of Research Papers Based on Citing Sentences

Article Category: Research Paper

Published Online: Dec 27, 2019

Page range: 26 - 41

Received: Nov 08, 2019

Accepted: Nov 28, 2019

DOI: https://doi.org/10.2478/jdis-2019-0019

Keywords
Cited paper, Citing paper, Citing sentence, Citation motive, Citation sentiment, Academic contribution, Evaluation

© 2019 Xiaoqiu Le, Jingdan Chu, Siyi Deng, Qihang Jiao, Jingjing Pei, Liya Zhu, Junliang Yao, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

CiteOpinion: Evidence-based Evaluation Tool for Academic Contributions of Research Papers Based on Citing Sentences

Article Category: Research Paper

Published Online: Dec 27, 2019

Page range: 26 - 41

Received: Nov 08, 2019

Accepted: Nov 28, 2019

DOI: https://doi.org/10.2478/jdis-2019-0019

KeywordsCited paper, Citing paper, Citing sentence, Citation motive, Citation sentiment, Academic contribution, Evaluation

© 2019 Xiaoqiu Le, Jingdan Chu, Siyi Deng, Qihang Jiao, Jingjing Pei, Liya Zhu, Junliang Yao, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Keywords
Cited paper, Citing paper, Citing sentence, Citation motive, Citation sentiment, Academic contribution, Evaluation