SARL: A reinforcement learning based QoS-aware IoT service discovery model

[1] L. Atzori, A. Iera and G. Morabito, “The internet of Things: a survey”, Computer Networks, 54(15), pp. 2787-2805, 2010.Search in Google Scholar

[2] S. Li, L. D. Xu and S. Zhao, “The internet of things: a survey”, Information Systems Frontiers, 17(2), pp. 243-259, 2015.10.1007/s10796-014-9492-7Search in Google Scholar

[3] P. Kamalinejad, C. Mahapatra, Z. Sheng, S. Mirabbasi, V. C. Victor and Y. L. Guan, “Wireless energy harvesting for the Internet of Things”, IEEE Communications Magazine, 53(6), pp. 102-108, 2015.10.1109/MCOM.2015.7120024Search in Google Scholar

[4] G. Kecskemeti, G. Casale, D. N. Jha, J. Lyon and R. Ranjan, “Modelling and simulation challenges in internet of things”, IEEE Cloud Computing, 4(1), pp. 62-69, 2017.10.1109/MCC.2017.18Search in Google Scholar

[5] A. H. Ngu, M. Gutierrez, V. Metsis, S. Nepal and Q. Z. sheng, “IoT middleware: a survey on issues and enabling technologies”, IEEE Internet of Things, 4(1), pp. 1-20, 2017.10.1109/JIOT.2016.2615180Search in Google Scholar

[6] A. Lazaro, R. Villarino and D. Girbau, “A survey of NFC sensors based on energy harvesting for IoT applications”, Sensors, 18, 2018.10.3390/s18113746626377630400233Search in Google Scholar

[7] O. Bello and S. Zeadally, “Toward efficient smartification of the internet of things (IoT) services”, Future Generation Computer Systems, 92, pp. 663-673, 2019.10.1016/j.future.2017.09.083Search in Google Scholar

[8] M. P. Papazoglou, “Service-oriented computing: concepts, characteristics and directions”, 4^th International Conference on Web Information Systems Engineering, pp. 3-12, 2003.Search in Google Scholar

[9] ME Khanouche, Y. Amirat, A. Chibani, M. Kerkar and A. Yachir, “Energy-centered and QoS-aware services selection for internet of things”, IEEE Transactions on Automation Science, 13(3), pp. 1256-1269, 2016.Search in Google Scholar

[10] L. Li, S. Li and S. Zhao, “QoS-aware scheduling of services-oriented internet of things”, IEEE Transactions on Industrial Informatics, 10(2), pp. 1497-1507, 2014.Search in Google Scholar

[11] N. B. Mabrouk, S. Beauche, E. Kuznetsova, N. Georgantas and V. Issarny, “QoS-aware service composition in dynamic service oriented environments”, ACM/IFIP/USENIX International Conference on Distributed Systems Platforms and Open Distributed Processing, pp. 123-142, 2009.10.1007/978-3-642-10445-9_7Search in Google Scholar

[12] A. Kouicem, A. Chibani, A. Tari, Y. Amirat and Z. Tari, “Dynamic services selection approach for the composition of web complex services in the web of objects”, IEEE World Forum on Internet of Things (WF-IoT), pp. 298-303, 2014.10.1109/WF-IoT.2014.6803176Search in Google Scholar

[13] G. White, A. Palade, and S. Clarke, “QoS prediction for reliable service composition in IoT”, International Conference on Service-Oriented Computing. pp. 149-160, 2017.10.1007/978-3-319-91764-1_12Search in Google Scholar

[14] B. Sapkota, D. Roman, S. R. Kruk, and D. Fensel, “Distributed Web service discovery architecture”, Proceedings of the Advanced International Conference on Telecommunications and International Conference on Internet and Web Applications and Service, pp. 136-136, 2006.10.1109/AICT-ICIW.2006.85Search in Google Scholar

[15] Q. He, J. Yan, Y. Yang, R. Kowalczyk and H. Jin, “A decentralized service discovery approach on peer-to-peer networks”, IEEE Transactions on Services Computing, 6, pp. 64-75, 2013.10.1109/TSC.2011.31Search in Google Scholar

[16] J. Li, Y. Bai, N. Zaman and V. C. Leung, “A decentralized trust-worthy context and QoS-aware service discovery framework for the internet of things”, IEEE Access, 5, pp. 19154-19166, 2017.Search in Google Scholar

[17] Q. He, J. Yan, Y. Yang, R. Kowalczyk and H. Jin, “Chord4S: A P2P-based decentralised service discovery approach”, IEEE International Conference on Services Computing, pp. 221-228, 2008.10.1109/SCC.2008.74Search in Google Scholar

[18] E. Rapti, C. Houstis, E. Houstis and A. Karageorgos, “A Bio-inspired service discovery and selection approach for IoT applications”, IEEE International Conference on Services Computing, pp. 868-871, 2016.10.1109/SCC.2016.126Search in Google Scholar

[19] F. Wilhelmi, B. Bellalta, C. Cano and A. Jonsson, “Implications of decentralized Q-learning resource allocation in wireless networks”, IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, pp. 1-5, 2017.10.1109/PIMRC.2017.8292321Search in Google Scholar

[20] C. Sommer, I. Dietrich and F. Dressler, “Simulation of Ad hoc routing protocols using OMNeT++: a case study for the DYMO protocol”, Mobile Networks and Applications, 15, pp. 786-930, 2010.10.1007/s11036-009-0174-5Search in Google Scholar

[21] P. Asghari, A. M. Rahmani and H. H. S. Javadi, “Service composition approaches in IoT: A systematic review”, Journal of Network and Computer Applications, 120, pp. 61-77, 2018.10.1016/j.jnca.2018.07.013Search in Google Scholar

[22] M. Hamzei and N. J. Navimipour, “Toward efficient service composition techniques in the internet of things”, IEEE Internet of Things Journal, 5(5), pp. 3774-3787, 2018.Search in Google Scholar

[23] X. Jin, S. Chun, J. Jung and K. H. Lee, “A fast and scalable approach for IoT service selection based on a physical service model”, Information Systems Frontiers, 19(6), pp. 1357-1372, 2017.Search in Google Scholar

[24] O. Alsaryrah, I. Mashal and T. Y. Chung, “Bi-Objective optimization for energy aware internet of things service composition”, IEEE Access, 6, pp. 26809-26819, 2018.Search in Google Scholar

[25] D. Duque, L. Lozano and A. L. Medaglia, “An exact method for the biobjective shortest path problem for large-scale road networks”, European Journal of Operational Research, 242(3), pp. 788-797, 2015.10.1016/j.ejor.2014.11.003Search in Google Scholar

[26] Y. Chu, S. Kosunalp, P D. Mitchell, D. Grace and T. Clarke, “Application of reinforcement learning to medium access control for wireless sensor networks”, Engineering Applications of Artificial Intelligence, 46, pp. 23-32, 2015.10.1016/j.engappai.2015.08.004Search in Google Scholar

[27] S. H. Park, P. D. Mitchell and D. Grace, “Reinforcement learning based MAC protocol (UW-ALOHA-Q) for underwater acoustic sensor networks”, IEEE Access, 7, pp. 165531-165542, 2019.Search in Google Scholar

[28] N. Morozs, T. Clarke and D. Grace, “Cognitive spectrum management in dynamic cellular environments: a case-based Q-learning approach”, Engineering Applications of Artificial Intelligence, 55, pp. 239-246, 2016.10.1016/j.engappai.2016.07.002Search in Google Scholar

[29] T. Jiang, D. Grace and P. D. Mitchell, “Efficient exploration in reinforcement learning-based cognitive radio spectrum sharing”, IET Communications, 5(10), pp. 1309-1317, 2011.Search in Google Scholar

[30] Y. Chawathe, S. Ratnasamy, L. Breslau, N. Lanham and S. Shenker, “Making gnutella-like P2P systems scalable”, SIGCOMM ’03: Proceedings of the 2003 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 407-418, 2003.10.1145/863955.864000Search in Google Scholar

[31] W. Qu, W. Zhou and M. Kitsuregawa, “Sharable file searching in unstructured peer-to-peer systems”, Journal of Supercomputing, 51(2), pp. 149-166, 2010.10.1007/s11227-009-0282-9Search in Google Scholar

[32] Z. Jia, J. You, R. Rao and M. Li, “Random walk search in unstructured P2P”, Journal of Systems Engineering and Electronics, 17(3), pp. 648-653, 2006.10.1016/S1004-4132(06)60111-4Search in Google Scholar

[33] F. Alanazi and T. Yeferny, “Reinforcement learning based query routing approach for P2P systems”, Future Internet, 11(12), pp. 253, 2019.10.3390/fi11120253Search in Google Scholar

[34] Y. Deng, F. Wang and A. Ciura, “Ant colony optimization inspired resource discovery in P2P Grid systems”, Journal of Supercomputing, 49(1), pp. 4-21, 2009.10.1007/s11227-008-0214-0Search in Google Scholar

[35] S. Asghari and N. J. Navimipour, “Resource discovery in the peer to peer networks using an inverted ant colony optimization algorithm”, Peer-to-Peer Networking and Applications, 12(1), pp. 129-142, 2019.10.1007/s12083-018-0644-2Search in Google Scholar

[36] J. A. Torkestani, “A distributed resource discovery algorithm for P2P grids”, Journal of Network and Computer Applications, 35(6), pp. 2028-2036, 2012.Search in Google Scholar

[37] G. Mei, N. Xu, J. Qin, B. Wang and P. Qi, “A survey of internet of things (IoT) for geo-hazards prevention: applications, technologies, and challenges”, IEEE Internet of Things Journal, 7(5), pp. 4371-4386, 2019.Search in Google Scholar

[38] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari and M. Ayyash, “Internet of things: a survey on enabling technologies, protocols, and applications”, IEEE Communications Surveys and Tutorials, 17(4), pp. 2347-2376, 2015.Search in Google Scholar

[39] L. P. Kaelbling, M. L. Littman and A. W. Moore, “Reinforcement learning: a survey”, Journal of Artificial Intelligence Research, 4, pp. 237-285, 1996.10.1613/jair.301Search in Google Scholar

[40] C. J. C. H. Watkins, Learning from delayed rewards, PhD Thesis, 1989.Search in Google Scholar

[41] M. Kusy and R. Zajdel, “Stateless Q-learning algorithm for training of radial basis function based neural networks in medical data classification”, Intelligent Systems in Technical and Medical Diagnostics, pp. 267-278, 2014.10.1007/978-3-642-39881-0_22Search in Google Scholar

[42] C. Claus and C. Boutilier, “The dynamics of reinforcement learning in cooperative multiagent systems”, 15^th National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence/Innovative Applications of Artificial Intelligence, pp. 746-752, 1998.Search in Google Scholar

[43] M. McGlohon and S. Sen, “Learning to cooperate in multi-agent systems by combining Q-learning and evolutionary strategy”, International Journal on Lateral Computing, 1(2), pp. 58-64, 2005.Search in Google Scholar

[44] A. Varga, “OMNeT++, in: modeling and tools for network Simulation,” Springer Berlin Heidelberg, pp. 35-59, 2010.10.1007/978-3-642-12331-3_3Search in Google Scholar

[45] S. X. Sun and J. Zhao, “A decomposition-based approach for service composition with global QoS guarantees”, Information Sciences, 199, pp. 138-153, 2012.10.1016/j.ins.2012.02.061Search in Google Scholar

[46] I. Yaqoob, I. A. T. Hashem, Y. Mehmood, A. Gani, S. Mokhtar and S. Guizani, “Enabling communication technologies for smart cities”, IEEE Communications Magazine, 55(1), pp. 112-120, 2017.10.1109/MCOM.2017.1600232CMSearch in Google Scholar

[47] R. Duan, X. Chen and T. Xing, “A QoS architecture for IoT”, IEEE International Conference on Cyber, Physical and Social Computing, pp. 717-720, 2011.10.1109/iThings/CPSCom.2011.125Search in Google Scholar

[48] M. Ripeanu, “Peer-to-peer architecture case study: Gnutella network, 1^st International Conference on Peer-to-Peer Computing, pp. 99-100, 2001.Search in Google Scholar

[49] S. Vuong and J. Li, “Efa: An efficient content routing algorithm in large peer-to-peer overlay networks, 3^rd International Conference on Peer-to-Peer Computing, pp. 216-217, 2003.Search in Google Scholar