<abstract xmlns="http://www.w3.org/1999/xhtml">Detection of local text reuse is central to a variety of applications, including plagiarism detection, origin detection, and information flow analysis. This paper evaluates and compares effectiveness of fingerprint selection algorithms for the source retrieval stage of local text reuse detection. In total, six algorithms are compared – <italic>Every p-th</italic>, <italic>0 mod p</italic>, <italic>Winnowing</italic>, <italic>Hailstorm</italic>, <italic>Frequency-biased Winnowing</italic> (<italic>FBW</italic>), as well as the proposed modified version of <italic>FBW</italic> (<italic>MFBW</italic>).Most of the previously published studies in local text reuse detection are based on datasets having either artificially generated, long-sized, or unobfuscated text reuse. In this study, to evaluate performance of the algorithms, a new dataset has been built containing real text reuse cases from Bachelor and Master Theses (written in English in the field of computer science) where about half of the cases involve less than 1 % of document text while about two-thirds of the cases involve paraphrasing.In the performed experiments, the overall best detection quality is reached by <italic>Winnowing</italic>, <italic>0 mod p</italic>, and <italic>MFBW</italic>. The proposed <italic>MFBW</italic> algorithm is a considerable improvement over <italic>FBW</italic> and becomes one of the best performing algorithms.The software developed for this study is freely available at the author’s website <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.cs.rtu.lv/jekabsons/">http://www.cs.rtu.lv/jekabsons/</ext-link>.</abstract>

Detection of local text reuse is central to a variety of applications, including plagiarism detection, origin detection, and information flow analysis. This paper evaluates and compares effectiveness of fingerprint selection algorithms for the source retrieval stage of local text reuse detection. In total, six algorithms are compared – Every p-th, 0 mod p, Winnowing, Hailstorm, Frequency-biased Winnowing (FBW), as well as the proposed modified version of FBW (MFBW).Most of the previously published studies in local text reuse detection are based on datasets having either artificially generated, long-sized, or unobfuscated text reuse. In this study, to evaluate performance of the algorithms, a new dataset has been built containing real text reuse cases from Bachelor and Master Theses (written in English in the field of computer science) where about half of the cases involve less than 1 % of document text while about two-thirds of the cases involve paraphrasing.In the performed experiments, the overall best detection quality is reached by Winnowing, 0 mod p, and MFBW. The proposed MFBW algorithm is a considerable improvement over FBW and becomes one of the best performing algorithms.The software developed for this study is freely available at the author’s website http://www.cs.rtu.lv/jekabsons/.

Evaluation of Fingerprint Selection Algorithms for Local Text Reuse Detection

Applied Computer Systems

This work is licensed under the Creative Commons Attribution 4.0 Public License.

{"article-title":"Evaluation of Fingerprint Selection Algorithms for Local Text Reuse Detection"}

Detection of local text reuse is central to a variety of applications, including plagiarism detection, origin detection, and information flow analysis. This...

Evaluation of Fingerprint Selection Algorithms for Local Text Reuse Detection

Published Online: Jun 05, 2020

Page range: 11 - 18

DOI: https://doi.org/10.2478/acss-2020-0002

Keywords
Document fingerprinting, fingerprint selection, local text reuse detection, plagiarism detection

© 2020 Gints Jēkabsons, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 Public License.

Evaluation of Fingerprint Selection Algorithms for Local Text Reuse Detection

Published Online: Jun 05, 2020

Page range: 11 - 18

DOI: https://doi.org/10.2478/acss-2020-0002

KeywordsDocument fingerprinting, fingerprint selection, local text reuse detection, plagiarism detection

© 2020 Gints Jēkabsons, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 Public License.

Keywords
Document fingerprinting, fingerprint selection, local text reuse detection, plagiarism detection