<abstract xmlns="http://www.w3.org/1999/xhtml"><p>In the era of big data, people have to face information filtration problem. For those cases when users do not or cannot express their demands clearly, recommender system can analyse user’s information more proactive and intelligent to filter out something users want. This property makes recommender system play a very important role in the field of e-commerce, social network and so on. The collaborative filtering recommendation algorithm based on Alternating Least Squares (ALS) is one of common algorithms using matrix factorization technique of recommendation system. In this paper, we design the parallel implementation process of the recommendation algorithm based on Spark platform and the related technology research of recommendation systems. Because of the shortcomings of the recommendation algorithm based on ALS model, a new loss function is designed. Before the model is trained, the similarity information of users and items is fused. The experimental results show that the performance of the proposed algorithm is better than that of algorithm based on ALS.</p></abstract>

In the era of big data, people have to face information filtration problem. For those cases when users do not or cannot express their demands clearly, recommender system can analyse user’s information more proactive and intelligent to filter out something users want. This property makes recommender system play a very important role in the field of e-commerce, social network and so on. The collaborative filtering recommendation algorithm based on Alternating Least Squares (ALS) is one of common algorithms using matrix factorization technique of recommendation system. In this paper, we design the parallel implementation process of the recommendation algorithm based on Spark platform and the related technology research of recommendation systems. Because of the shortcomings of the recommendation algorithm based on ALS model, a new loss function is designed. Before the model is trained, the similarity information of users and items is fused. The experimental results show that the performance of the proposed algorithm is better than that of algorithm based on ALS.

Application of Improved Recommendation System Based on Spark Platform in Big Data Analysis

Institute of Disaster Prevention, Yanjiao Town 065201,

Cybernetics and Information Technologies

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

In the era of big data, people have to face information filtration problem. For those cases when users do not or cannot express their demands clearly,...

Application of Improved Recommendation System Based on Spark Platform in Big Data Analysis

Published Online: Jan 25, 2017

Page range: 245 - 255

DOI: https://doi.org/10.1515/cait-2016-0092

Keywords
Spark, recommendation system, collaborative filtering, alternating least squares

© 2016 Li Xie et al., published by De Gruyter Open

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Application of Improved Recommendation System Based on Spark Platform in Big Data Analysis

Published Online: Jan 25, 2017

Page range: 245 - 255

DOI: https://doi.org/10.1515/cait-2016-0092

KeywordsSpark, recommendation system, collaborative filtering, alternating least squares

© 2016 Li Xie et al., published by De Gruyter Open

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Keywords
Spark, recommendation system, collaborative filtering, alternating least squares