1. bookVolume 9 (2019): Issue 3 (July 2019)
Journal Details
License
Format
Journal
eISSN
2449-6499
First Published
30 Dec 2014
Publication timeframe
4 times per year
Languages
English
access type Open Access

Swarm Algorithms for NLP - The Case of Limited Training Data

Published Online: 09 May 2019
Volume & Issue: Volume 9 (2019) - Issue 3 (July 2019)
Page range: 219 - 234
Received: 07 Dec 2018
Accepted: 30 Jan 2019
Journal Details
License
Format
Journal
eISSN
2449-6499
First Published
30 Dec 2014
Publication timeframe
4 times per year
Languages
English
Abstract

The present article describes a novel phrasing model which can be used for segmenting sentences of unconstrained text into syntactically-defined phrases. This model is based on the notion of attraction and repulsion forces between adjacent words. Each of these forces is weighed appropriately by system parameters, the values of which are optimised via particle swarm optimisation. This approach is designed to be language-independent and is tested here for different languages.

The phrasing model’s performance is assessed per se, by calculating the segmentation accuracy against a golden segmentation. Operational testing also involves integrating the model to a phrase-based Machine Translation (MT) system and measuring the translation quality when the phrasing model is used to segment input text into phrases. Experiments show that the performance of this approach is comparable to other leading segmentation methods and that it exceeds that of baseline systems.

Keywords

[1] D. Klein and C. D. Manning, A generative constituent-context model for improved grammar induction, Proceedings of 40th ACL Meeting, Philadelphia, USA, pages 128–135, July 2002.10.3115/1073083.1073106Search in Google Scholar

[2] D. Klein and C. D. Manning, Corpus-based induction of syntactic structure: Models of dependency and constituency, Proceedings of 42nd ACL Meeting, Barcelona, Spain, pages 478–485, July 21-26, 2004.10.3115/1218955.1219016Search in Google Scholar

[3] Y. Seginer, Fast unsupervised incremental parsing, Proceedings of 45th ACL Meeting, Prague, Czech Republic, pages 384–391, June 2007.Search in Google Scholar

[4] E. Ponvert, J. Baldridge, and K. Erk, Simple unsupervised grammar induction from raw text with cascaded finite state models, Proceedings of 49th ACL Meeting, Portland, Oregon, USA, pages 1077–1086, 2011.Search in Google Scholar

[5] D. Yarowsky and G. Ngai, Inducing multilingual PoS taggers and np bracketers via robust projection across aligned corpora, Proceedings of NAACL-2001 Conference, Pittsburgh, PA, USA, pages 200-207, 2-7 June 2001.10.3115/1073336.1073362Search in Google Scholar

[6] L. Zhu, D.F. Wong, and L.S. Chao, Unsupervised chunking based on graph propagation from bilingual corpus, The Scientific World Journal, Article ID 401943, 2014.10.1155/2014/401943397742424772017Search in Google Scholar

[7] S. Goldwater, T.L. Griffiths, and M. Johnson, Contextual dependencies in unsupervised word segmentation, Proceedings of 21st International Conference on Computational Linguistics and 44th ACL Meeting, Sydney, Australia, pages 673–680, 2006.10.3115/1220175.1220260Search in Google Scholar

[8] D. Mochihashi, T. Yamada, and N. Ueda, Bayesian unsupervised word segmentation with nested Pitman-Yor language modeling, Proceedings of 47th ACL Meeting, Suntec, Singapore, pages 100–108, August 2009.10.3115/1687878.1687894Search in Google Scholar

[9] T. Nguyen, S. Vogel, and N.A. Smith, Nonparametric word segmentation for machine translation, Proceedings of COLING-2010, Beijing, China, pages 815–823, August 2010.Search in Google Scholar

[10] G. Tambouratzis, M. Vassiliou, & S. Sofianopoulos, Machine Translation with Minimal Reliance on Parallel Resources, Springer Briefs in Statistics series, Springer-Verlag, Berlin, 2017.10.1007/978-3-319-63107-3Search in Google Scholar

[11] E.F. Tjong, K. Sang, and H. Dejean, Introduction to the CONLL-2001 shared task: Clause identification, Proceedings of CoNLL-2001, Toulouse, France, pages 53–57, 6-7 July 2001.Search in Google Scholar

[12] J. Lafferty, A. McCallum and F.C.N. Pereira, Conditional random fields: Probabilistic models for segmenting and labelling sequence data. ICML Conference Proceedings, Williamstown, USA, pages 282–289, 28 June-1 July 2001.Search in Google Scholar

[13] F. Sha and F.C.N. Pereira, Shallow parsing with conditional random fields, Proceedings of HLTNAACL Conference, Alberta, Canada, pages 213–220, 27 May – 1 Jun 2003.Search in Google Scholar

[14] Y. Tsuruoka, J. Tsujii, and S. Ananiadou, Fast full parsing by linear-chain conditional random fields, 12th EACL Conference Proceedings, Athens, Greece, pages 790–798, 30 March-3 April 2009.10.3115/1609067.1609155Search in Google Scholar

[15] R.O. Duda, P. E. Hart, and D.G. Stork, Pattern Classification (2nd edition), Wiley Interscience, New York, U.S.A., 2001.Search in Google Scholar

[16] G. Tambouratzis, Conditional Random Fields versus template-matching in MT phrasing tasks involving sparse training data, Pattern Recognition Letters, 53:44-52, 2015.10.1016/j.patrec.2014.11.002Search in Google Scholar

[17] A.J. Booker, J.E. Dennis Jr., P.D. Frank, D.B. Serafini, V. Torczon, and M.W. Trosset, A Rigorous Framework for Optimization of Expensive Functions by Surrogates, Structural Optimisation, 17:1-13, 1999.10.1007/BF01197708Search in Google Scholar

[18] Y. Jin, Surrogate-assisted evolutionary computation: Recent advances and future challenges, Swarm and Evolutionary Computation, 1(2):61-70, 2011.10.1016/j.swevo.2011.05.001Search in Google Scholar

[19] K. Stoeber, P. Wagner, J. Helbig, S. Koester, D. Stall, M. Thomae, J. Blauert, W. Hess, and H. Mangold, Speech synthesis by multilevel selection and concatenation of units from large speech corpora. Verbmobil: Foundations of Speech-to-Speech Translation, Symbolic Computation, Springer, Berlin, pages 519–537, 2000.10.1007/978-3-662-04230-4_38Search in Google Scholar

[20] J. Kennedy and R. Eberhart, Particle swarm optimization, IEEE International Conference on Neural Networks, Perth, WA, USA, pages 1942–1948, 27 Nov-1 Dec 1995.Search in Google Scholar

[21] G. Tambouratzis, Applying PSO to Natural Language Processing Tasks: Optimizing the Identifi-cation of Syntactic Phrases, Proceedings of CEC-2016 Conference, Vancouver, Canada, pp. 1831-1838, July 2016.Search in Google Scholar

[22] R. Mendes, J. Kennedy, and J. Neves, The fully informed particle swarm: Simpler maybe better, IEEE Transactions on Evolutionary Computation, 8(3):204–210, 2004.10.1109/TEVC.2004.826074Search in Google Scholar

[23] Z.-H. Zhan, J. Zhang, Y. Li, and H.S.-H. Chung, Adaptive particle swarm optimization, IEEE Transactions on Systems, Man Cybernetics Part B: Cybernetics, 39(6):1362–1381, 2009.10.1109/TSMCB.2009.201595619362911Search in Google Scholar

[24] G. Tambouratzis (2017) The effectiveness of surrogate functions in improving the accuracy of PSO-type algorithms in an NLP task, Proceedings of 10th SSCI Conference, Honolulu, USA, pp. 3214-3221, IEEE Press, 27 November - 1 December 2017.Search in Google Scholar

[25] K. Papineni, S. Roukos, T. Ward, and W.J. Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. 40th ACL Meeting Proceedings, Philadelphia, USA, pages 311–318.Search in Google Scholar

[26] NIST. Automatic evaluation of machine translation quality using n-gram cooccurrences statistics. Report available at http://www.itl.nist.gov/iad/mig/tests/mt/doc/ngramstudy.pdf, 2002.Search in Google Scholar

[27] K.R. Harrison., A.P. Engelbrecht, B.M. Ombuki-Berman, An adaptive particle swarm optimization algorithm based on optimal parameter regions, Proceedings of 10th SSCI Conference, Honolulu, USA, pp. 1606-1613, IEEE Press, 27 November - 1 December 2017.Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo