A Use Case of Patent Classification Using Deep Learning with Transfer Learning

Abdelgawad, L., Kluegl, P., Genc, E., Falkner, S., & Hutter, F. (2020). Optimizing Neural Networks for Patent Classification. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). volume 11908 LNAI. doi:10.1007/978-3-030-46133-1{\_}41. AbdelgawadL. KlueglP. GencE. FalknerS. HutterF. 2020 Optimizing Neural Networks for Patent Classification In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) volume 11908 LNAI 10.1007/978-3-030-46133-1{\_}41 Open DOISearch in Google Scholar

Aristodemou, L., & Tietze, F. (2018). The state-of-the-art on Intellectual Property Analytics (IPA): A literature review on artificial intelligence, machine learning and deep learning methods for analysing intellectual property (IP) data. World Patent Information, 55, 37–51. doi:10.1016/J.WPI.2018.07.002. AristodemouL. TietzeF. 2018 The state-of-the-art on Intellectual Property Analytics (IPA): A literature review on artificial intelligence, machine learning and deep learning methods for analysing intellectual property (IP) data World Patent Information 55 37 51 10.1016/J.WPI.2018.07.002 Open DOISearch in Google Scholar

Bispo, T.D., Macedo, H.T., Santos, F.D.O., Da Silva, R.P., Matos, L.N., Prado, B.O., Da Silva, G.J., & Guimarães, A. (2019). Long short-term memory model for classification of english-PtBR cross-lingual hate speech. Journal of Computer Science, 15. doi:10.3844/jcssp.2019.1546.1571. BispoT.D. MacedoH.T. SantosF.D.O. Da SilvaR.P. MatosL.N. PradoB.O. Da SilvaG.J. GuimarãesA. 2019 Long short-term memory model for classification of english-PtBR cross-lingual hate speech Journal of Computer Science 15 10.3844/jcssp.2019.1546.1571 Open DOISearch in Google Scholar

Quinta de Castro, P.V., Félix Felipe da Silva, N., & da Silva Soares, A. (2018). Portuguese Named Entity Recognition Using LSTM-CRF. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). volume 11122 LNAI. doi:10.1007/978-3-319-99722-3{\_}9. Quinta de CastroP.V. Félix Felipe da SilvaN. da Silva SoaresA. 2018 Portuguese Named Entity Recognition Using LSTM-CRF In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) volume 11122 LNAI 10.1007/978-3-319-99722-3{\_}9 Open DOISearch in Google Scholar

De Castro, P.V.Q., Da Silva, N.F.F., & Da Silva Soares, A. (2019). Contextual representations and semi-supervised named entity recognition for Portuguese language. In CEUR Workshop Proceedings. volume 2421. De CastroP.V.Q. Da SilvaN.F.F. Da Silva SoaresA. 2019 Contextual representations and semi-supervised named entity recognition for Portuguese language In CEUR Workshop Proceedings volume 2421. Search in Google Scholar

Derieux, F., Bobeica, M., Pois, D., & Raysz, J.P. (2010). Combining semantics and statistics for patent classification. In CEUR Workshop Proceedings. volume 1176. DerieuxF. BobeicaM. PoisD. RayszJ.P. 2010 Combining semantics and statistics for patent classification In CEUR Workshop Proceedings volume 1176. Search in Google Scholar

Devlin, J., Chang, M.W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL HLT 2019–2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies—Proceedings of the Conference. volume 1. DevlinJ. ChangM.W. LeeK. ToutanovaK. 2019 BERT: Pre-training of deep bidirectional transformers for language understanding In NAACL HLT 2019–2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies—Proceedings of the Conference volume 1. Search in Google Scholar

Espacenet (2021). Espacenet Patent search. URL: https://lp.espacenet.com/?locale=pt_LP. Espacenet 2021 Espacenet Patent search URL: https://lp.espacenet.com/?locale=pt_LP. Search in Google Scholar

Feldman, R., & Sanger, J. (2006). The Text Mining Handbook. doi:10.1017/cbo9780511546914. FeldmanR. SangerJ. 2006 The Text Mining Handbook 10.1017/cbo9780511546914 Open DOISearch in Google Scholar

Gomez, J.C., & Moens, M.F. (2014). A survey of automated hierarchical classification of patents. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8830. doi:10.1007/978-3-319-12511-4. GomezJ.C. MoensM.F. 2014 A survey of automated hierarchical classification of patents Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8830 10.1007/978-3-319-12511-4 Open DOISearch in Google Scholar

Gonçalves, T., Silva, C., Quaresma, P., & Vieira, R. (2006). Analysing part-of speech for Portuguese text classification. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). volume 3878 LNCS. GonçalvesT. SilvaC. QuaresmaP. VieiraR. 2006 Analysing part-of speech for Portuguese text classification In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) volume 3878 LNCS 10.1007/11671299_57 Search in Google Scholar

Hu, J., Li, S.B., Hu, J.J., & Yang, G.C. (2018). A hierarchical feature extraction model for multi-label mechanical patent classification. Sustainability (Switzerland), 10. doi:10.3390/su10010219. HuJ. LiS.B. HuJ.J. YangG.C. 2018 A hierarchical feature extraction model for multi-label mechanical patent classification Sustainability (Switzerland) 10 10.3390/su10010219 Open DOISearch in Google Scholar

Instituto Nacional da Propriedade Intelectual (2018). Código da Propriedade Industrial. URL: https://inpi.justica.gov.pt/Portals/6/PDF%20INPI/Legisla%C3%A7%C3%A3o%20e%20outros%20documentos/CPI%20-%202018.pdf?ver=2019-06-28-153157-733. Instituto Nacional da Propriedade Intelectual 2018 Código da Propriedade Industrial URL: https://inpi.justica.gov.pt/Portals/6/PDF%20INPI/Legisla%C3%A7%C3%A3o%20e%20outros%20documentos/CPI%20-%202018.pdf?ver=2019-06-28-153157-733. Search in Google Scholar

IP5 (2019). IP5 Statistics Report 2018 Edition. URL: https://www.fiveipoffices.org/statistics/statisticsreports/2019edition IP5 2019 IP5 Statistics Report 2018 Edition URL: https://www.fiveipoffices.org/statistics/statisticsreports/2019edition Search in Google Scholar

Kowsari, K., Meimandi, K.J., Heidarysafa, M., Mendu, S., Barnes, L., & Brown, D. (2019). Text classification algorithms: A survey. doi:10.3390/info10040150. KowsariK. MeimandiK.J. HeidarysafaM. MenduS. BarnesL. BrownD. 2019 Text classification algorithms: A survey 10.3390/info10040150 Open DOISearch in Google Scholar

Krestel, R., Chikkamath, R., Hewel, C., & Risch, J. (2021). A survey on deep learning for patent analysis. World Patent Information, 65, 102035. KrestelR. ChikkamathR. HewelC. RischJ. 2021 A survey on deep learning for patent analysis World Patent Information 65 102035 10.1016/j.wpi.2021.102035 Search in Google Scholar

Lai, K., & Wu, S.J. (2005). Using the patent co–citation approach to establish a new patent classification system. Information Processing and Management, 41(2), 313–330 LaiK. WuS.J. 2005 Using the patent co–citation approach to establish a new patent classification system Information Processing and Management 41 2 313 330 10.1016/j.ipm.2003.11.004 Search in Google Scholar

Lee, J.S., & Hsiang, J. (2020). Patent classification by fine-tuning BERT language model. World Patent Information, 61. doi:10.1016/j.wpi.2020.101965. LeeJ.S. HsiangJ. 2020 Patent classification by fine-tuning BERT language model World Patent Information 61 10.1016/j.wpi.2020.101965 Open DOISearch in Google Scholar

Li, S.B., Hu, J., Cui, Y.X., & Hu, J.J. (2018). DeepPatent: patent classification with convolutional neural networks and word embedding. Scientometrics, 117. doi:10.1007/s11192-018-2905-5. LiS.B. HuJ. CuiY.X. HuJ.J. 2018 DeepPatent: patent classification with convolutional neural networks and word embedding Scientometrics 117 10.1007/s11192-018-2905-5 Open DOISearch in Google Scholar

Liddy, E.D. (2001). Natural Language Processing. In Encyclopedia of Library and Information Science. Encyclopedia of Library and Information Science. LiddyE.D. 2001 Natural Language Processing In Encyclopedia of Library and Information Science Encyclopedia of Library and Information Science. Search in Google Scholar

Manning, C.D., Raghavan, P., & Schutze, H. (2008). Introduction to Information Retrieval. doi:10.1017/cbo9780511809071. ManningC.D. RaghavanP. SchutzeH. 2008 Introduction to Information Retrieval 10.1017/cbo9780511809071 Open DOISearch in Google Scholar

Pan, S.J., & Yang, Q. (2010). A survey on transfer learning. doi:10.1109/TKDE.2009.191. PanS.J. YangQ. 2010 A survey on transfer learning 10.1109/TKDE.2009.191 Open DOISearch in Google Scholar

Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. In NAACL HLT 2018—2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies—Proceedings of the Conference. volume 1. doi:10.18653/v1/n18-1202. PetersM.E. NeumannM. IyyerM. GardnerM. ClarkC. LeeK. ZettlemoyerL. 2018 Deep contextualized word representations In NAACL HLT 2018—2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies—Proceedings of the Conference volume 1. 10.18653/v1/n18-1202 Open DOISearch in Google Scholar

Risch, J., & Krestel, R. (2019). Domain-specific word embeddings for patent classification. Data Technologies and Applications, 53. doi:10.1108/DTA-01-2019-0002. RischJ. KrestelR. 2019 Domain-specific word embeddings for patent classification Data Technologies and Applications 53 10.1108/DTA-01-2019-0002 Open DOISearch in Google Scholar

Rodrigues, R.C., Rodrigues, J., de Castro, P.V.Q., da Silva, N.F.F., & Soares, A. (2020). Portuguese language models and word embeddings: Evaluating on semantic similarity tasks. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). volume 12037 LNAI. doi:10.1007/978-3-030-41505-1{\_}23. RodriguesR.C. RodriguesJ. de CastroP.V.Q. da SilvaN.F.F. SoaresA. 2020 Portuguese language models and word embeddings: Evaluating on semantic similarity tasks In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) volume 12037 LNAI 10.1007/978-3-030-41505-1{\_}23 Open DOISearch in Google Scholar

dos Santos, C., & Guimarães, V. (2015). Boosting Named Entity Recognition with Neural Character Embeddings. doi:10.18653/v1/w15-3904. dos SantosC. GuimarãesV. 2015 Boosting Named Entity Recognition with Neural Character Embeddings 10.18653/v1/w15-3904 Open DOISearch in Google Scholar

Silva, C., & Ribeiro, B. (2010). Inductive Inference for Large Scale Text Classification: Kernel Approaches and Techniques. volume 255. doi:10.1007/978-3-642-04533-2. SilvaC. RibeiroB. 2010 Inductive Inference for Large Scale Text Classification: Kernel Approaches and Techniques volume 255. 10.1007/978-3-642-04533-2 Open DOISearch in Google Scholar

Souza, F., Nogueira, R., & Lotufo, R. (2019). Portuguese Named Entity Recognition using BERT-CRF. arXiv. URL: https://arxiv.org/abs/1909.10649v2. SouzaF. NogueiraR. LotufoR. 2019 Portuguese Named Entity Recognition using BERT-CRF arXiv URL: https://arxiv.org/abs/1909.10649v2. Search in Google Scholar

Trappey, A.J., Hsu, F C., Trappey, C.V., & Lin, C.I. (2006). Development of a patent document classification and search platform using a back-propagation network. Expert Systems with Applications, 31. doi:10.1016/j.eswa.2006.01.013. TrappeyA.J. HsuF C. TrappeyC.V. LinC.I. 2006 Development of a patent document classification and search platform using a back-propagation network Expert Systems with Applications 31 10.1016/j.eswa.2006.01.013 Open DOISearch in Google Scholar

Trappey, A.J., Trappey, C.V., Chiang, T.A., & Huang, Y.H. (2013). Ontology-based neural network for patent knowledge management in design collaboration. International Journal of Production Research, 51. doi:10.1080/00207543.2012.701775. TrappeyA.J. TrappeyC.V. ChiangT.A. HuangY.H. 2013 Ontology-based neural network for patent knowledge management in design collaboration International Journal of Production Research 51 10.1080/00207543.2012.701775 Open DOISearch in Google Scholar

Trappey, A.J.C., Trappey, C.V., Wu, C.-Y., & Lin, C.-W. (2012). A patent quality analysis for innovative technology and product development. Advanced Engineering Informatics, 26, 26–34. doi:10.1016/j.aei.2011.06.005. TrappeyA.J.C. TrappeyC.V. WuC.-Y. LinC.-W. 2012 A patent quality analysis for innovative technology and product development Advanced Engineering Informatics 26 26 34 10.1016/j.aei.2011.06.005 Open DOISearch in Google Scholar

Wagner Filho, J.A., Wilkens, R., Idiart, M., & Villavicencio, A. (2019). The BRWAC corpus: A new open resource for Brazilian Portuguese. In LREC 2018—11th International Conference on Language Resources and Evaluation. Wagner FilhoJ.A. WilkensR. IdiartM. VillavicencioA. 2019 The BRWAC corpus: A new open resource for Brazilian Portuguese In LREC 2018—11th International Conference on Language Resources and Evaluation Search in Google Scholar

World Intellectual Property Organization (2008). WIPO Intellectual Property Handbook: Policy, Law and Use. doi:1. World Intellectual Property Organization 2008 WIPO Intellectual Property Handbook: Policy, Law and Use doi:1. Search in Google Scholar

Wu, J.L., Chang, P.C., Tsao, C.C., & Fan, C.Y. (2016). A patent quality analysis and classification system using self-organizing maps with support vector machine. Applied Soft Computing Journal, 41. doi:10.1016/j.asoc.2016.01.020. WuJ.L. ChangP.C. TsaoC.C. FanC.Y. 2016 A patent quality analysis and classification system using self-organizing maps with support vector machine Applied Soft Computing Journal 41 10.1016/j.asoc.2016.01.020 Open DOISearch in Google Scholar

Zhang, X.Y. (2014). Interactive patent classification based on multi-classifier fusion and active learning. Neurocomputing, 127. doi:10.1016/j.neucom.2013.08.013. ZhangX.Y. 2014 Interactive patent classification based on multi-classifier fusion and active learning Neurocomputing 127 10.1016/j.neucom.2013.08.013 Open DOISearch in Google Scholar

Zhuang, F.Z., Qi, Z.Y., Duan, K.Y., Xi, D.B., Zhu, Y.C., Zhu, H.S., Xiong, H., & He, Q. (2021). A comprehensive survey on transfer learning, in Proceedings of the IEEE, 109(1), Jan. 2021. doi:10.1109/JPROC.2020.3004555. ZhuangF.Z. QiZ.Y. DuanK.Y. XiD.B. ZhuY.C. ZhuH.S. XiongH. HeQ. 2021 A comprehensive survey on transfer learning in Proceedings of the IEEE 109 1 Jan. 2021 10.1109/JPROC.2020.3004555 Open DOISearch in Google Scholar