<abstract xmlns="http://www.w3.org/1999/xhtml"><p>Using software applications or services, which provide word or even word pattern recommendation service has become part of our lives. Those services appear in many form in our daily basis, just think of our smartphones keyboard, or Google search suggestions and this list can be continued. With the help of these tools, we can not only find the suitable word that fits into our sentence, but we can also express ourselves in a much more nuanced, diverse way. To achieve this kind of recommendation service, we use an algorithm which is capable to recommend word by word pattern queries. Word pattern queries, can be expressed as a combination of words, part-of-speech (POS) tags and wild card words. Since there are a lot of possible patterns and sentences, we use Big Data frameworks to handle this large amount of data. In this paper, we compared two popular framework Hadoop and Spark with the proposed algorithm and recommend some enhancement to gain faster word pattern generation.</p></abstract>

Using software applications or services, which provide word or even word pattern recommendation service has become part of our lives. Those services appear in many form in our daily basis, just think of our smartphones keyboard, or Google search suggestions and this list can be continued. With the help of these tools, we can not only find the suitable word that fits into our sentence, but we can also express ourselves in a much more nuanced, diverse way. To achieve this kind of recommendation service, we use an algorithm which is capable to recommend word by word pattern queries. Word pattern queries, can be expressed as a combination of words, part-of-speech (POS) tags and wild card words. Since there are a lot of possible patterns and sentences, we use Big Data frameworks to handle this large amount of data. In this paper, we compared two popular framework Hadoop and Spark with the proposed algorithm and recommend some enhancement to gain faster word pattern generation.

Word pattern prediction using Big Data frameworks

Acta Universitatis Sapientiae, Informatica

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

{"article-title":"Word pattern prediction using Big Data frameworks"}

Using software applications or services, which provide word or even word pattern recommendation service has become part of our lives. Those services appear in...

Word pattern prediction using Big Data frameworks

Published Online: Jul 16, 2020

Page range: 51 - 69

Received: Jan 31, 2020

Accepted: Feb 28, 2020

DOI: https://doi.org/10.2478/ausi-2020-0004

Keywords
word-pattern, word-prediction, big data, hadoop, spark, nlp, map reduce, snappy, lz4, data compression

© 2020 Bence Szabari et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Word pattern prediction using Big Data frameworks

Published Online: Jul 16, 2020

Page range: 51 - 69

Received: Jan 31, 2020

Accepted: Feb 28, 2020

DOI: https://doi.org/10.2478/ausi-2020-0004

Keywordsword-pattern, word-prediction, big data, hadoop, spark, nlp, map reduce, snappy, lz4, data compression

© 2020 Bence Szabari et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Keywords
word-pattern, word-prediction, big data, hadoop, spark, nlp, map reduce, snappy, lz4, data compression