<abstract xmlns="http://www.w3.org/1999/xhtml"><p> In both Chinese and Dzongkha languages, the greatest challenge is to identify the word boundaries because there are no word delimiters as it is in English and other Western languages. Therefore, preprocessing and word segmentation is the first step in Dzongkha language processing, such as translation, spell-checking, and information retrieval. Research on Chinese word segmentation was conducted long time ago. Therefore, it is relatively mature, but the Dzongkha word segmentation has been less studied by researchers. In the paper, we have investigated this major problem in Dzongkha language processing using a probabilistic approach for selecting valid segments with probability being computed on the basis of the corpus.</p></abstract>

In both Chinese and Dzongkha languages, the greatest challenge is to identify the word boundaries because there are no word delimiters as it is in English and other Western languages. Therefore, preprocessing and word segmentation is the first step in Dzongkha language processing, such as translation, spell-checking, and information retrieval. Research on Chinese word segmentation was conducted long time ago. Therefore, it is relatively mature, but the Dzongkha word segmentation has been less studied by researchers. In the paper, we have investigated this major problem in Dzongkha language processing using a probabilistic approach for selecting valid segments with probability being computed on the basis of the corpus.

Analysing the Methods of Dzongkha Word Segmentation

Applied Computer Systems

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

{"article-title":"Analysing the Methods of Dzongkha Word Segmentation"}

In both Chinese and Dzongkha languages, the greatest challenge is to identify the word boundaries because there are no word delimiters as it is in English and...

Analysing the Methods of Dzongkha Word Segmentation

Publié en ligne: 13 juin 2017

Pages: 61 - 65

DOI: https://doi.org/10.1515/acss-2017-0008

Mots clés
Dzongkha word segmentation, maximal matching, n-gram, natural language processing

© Riga Technical University

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Analysing the Methods of Dzongkha Word Segmentation

Publié en ligne: 13 juin 2017

Pages: 61 - 65

DOI: https://doi.org/10.1515/acss-2017-0008

Mots clésDzongkha word segmentation, maximal matching, n-gram, natural language processing

© Riga Technical University

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Mots clés
Dzongkha word segmentation, maximal matching, n-gram, natural language processing