Crawler Technology Based on Scrapy Framework

[1]Jing Wang, Yuchun Guo. Scrapy-Based Crawling and User-Behavior Characteristics Analysison Taobao[P]. Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), 2012 International Conference on, 2012.WangJingGuoYuchunScrapy-Based Crawling and User-Behavior Characteristics Analysison Taobao[P]Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), 2012 International Conference on201210.1109/CyberC.2012.17Search in Google Scholar

[2]McGuffeeJames W.. Non-profit geographically constrained locator[J]. ACM SIGCASComputers and Society,2015,45(2).McGuffeeJames W.Non-profit geographically constrained locator[J]ACM SIGCASComputers and Society201545210.1145/2809957.2809971Search in Google Scholar

[3]Yuhao Fan. Design and Implementation of Distributed Crawler System Based on Scrapy[J].IOP Conference Series: Earth and Environmental Science,2018,108(4).FanYuhaoDesign and Implementation of Distributed Crawler System Based on Scrapy[J]IOP Conference Series: Earth and Environmental Science2018108410.1088/1755-1315/108/4/042086Search in Google Scholar

[4]Shen Jie, Li Yifan. Application of Web crawler system in cloud media [J]. China Cable Television, 2018 (05): 595-597.JieShenYifanLiApplication of Web crawler system in cloud media [J]China Cable Television201805595597Search in Google Scholar

[5]Zhang Jin. Research on Web crawler technology based on Hadoop platform [D].Nanjing University of Posts and Telecommunications, 2017.ZhangJinResearch on Web crawler technology based on Hadoop platform [D]Nanjing University of Posts and Telecommunications2017Search in Google Scholar

[6]Zhao Fen, Lei Zhenzhen, Yang Xiaoyun, Su Pengju and Wang Shunye. Based on Baidu Tieba College Students’Network Public Opinion Analysis [J]. Computer Knowledge and Technology, 2018, 14 (28): 227-229.ZhaoFenLeiZhenzhenYangXiaoyunSuPengju and WangShunyeBased on Baidu Tieba College Students’Network Public Opinion Analysis [J]Computer Knowledge and Technology20181428227229Search in Google Scholar

[7]Ding Zhongxiang, Yang Yanhong, Du Yanming. Design and implementation of video information crawling based on Scrappy framework [J]. Journal of Beijing Printing Institute, 2018, 26 (09): 92-97.DingZhongxiangYangYanhongDuYanmingDesign and implementation of video information crawling based on Scrappy framework [J]Journal of Beijing Printing Institute201826099297Search in Google Scholar

[8]Xie Zhu. Emotional Tendency Analysis for Chinese Short Texts [D]. Hunan University, 2018.XieZhuEmotional Tendency Analysis for Chinese Short Texts [D]Hunan University2018Search in Google Scholar

[9]Wei Chengcheng. Data Information Crawler Technology Based on Python [J]. Electronic World, 2018 (11): 208-209.WeiChengchengData Information Crawler Technology Based on Python [J]Electronic World201811208209Search in Google Scholar

[10]Ye Xiqiezhong. Research and Implementation of Tibetan Text Automatic Classification Based on Web [D]. Qinghai University for Nationalities, 2014.YeXiqiezhongResearch and Implementation of Tibetan Text Automatic Classification Based on Web [D]Qinghai University for Nationalities2014Search in Google Scholar

[11]Zhong Jiajun. Research on Copyright Infringement Recognition of News Aggregation Platform [D]. Lanzhou University, 2018.ZhongJiajunResearch on Copyright Infringement Recognition of News Aggregation Platform [D]Lanzhou University2018Search in Google Scholar