1. bookVolume 24 (2017): Issue s3 (November 2017)
Journal Details
License
Format
Journal
eISSN
2083-7429
First Published
20 Jul 2007
Publication timeframe
4 times per year
Languages
English
access type Open Access

A Deep Q-Learning Network for Ship Stowage Planning Problem

Published Online: 22 Nov 2017
Volume & Issue: Volume 24 (2017) - Issue s3 (November 2017)
Page range: 102 - 109
Journal Details
License
Format
Journal
eISSN
2083-7429
First Published
20 Jul 2007
Publication timeframe
4 times per year
Languages
English
Abstract

Ship stowage plan is the management connection of quae crane scheduling and yard crane scheduling. The quality of ship stowage plan affects the productivity greatly. Previous studies mainly focuses on solving stowage planning problem with online searching algorithm, efficiency of which is significantly affected by case size. In this study, a Deep Q-Learning Network (DQN) is proposed to solve ship stowage planning problem. With DQN, massive calculation and training is done in pre-training stage, while in application stage stowage plan can be made in seconds. To formulate network input, decision factors are analyzed to compose feature vector of stowage plan. States subject to constraints, available action and reward function of Q-value are designed. With these information and design, an 8-layer DQN is formulated with an evaluation function of mean square error is composed to learn stowage planning. At the end of this study, several production cases are solved with proposed DQN to validate the effectiveness and generalization ability. Result shows a good availability of DQN to solve ship stowage planning problem.

Keywords

1. M. Omar, S. S. Supadi. 2012. Integrated models for shipping a vendor’s final production batch to a single buyer under linearly decreasing demand for consignment policy. Sains Malaysiana 41.3: 367-370.Search in Google Scholar

2. C. Mi, Z. W. Zhang, Y. F. Huang, Y. Shen, 2013. A fast automated vision system for container corner casting recognition. Journal of Marine Science and Technology- Taiwan, 24(1): 54-60. DOI: 10.6119/JMST-016-0125-810.6119/JMST-016-0125-8Open DOISearch in Google Scholar

3. X. P. Rui, X. T. Yu, J. Lu, et al. 2016. An algorithm for generation of DEMs from contour lines considering geomorphic features. Earth Sciences Research Journal, 20(2): G1-G9, 20(2):G1-G9.10.15446/esrj.v20n2.55348Open DOISearch in Google Scholar

4. Y. Shen, 2016. An Anti-Collision Method of Slip Barrel for Automatic Ship Loading in Bulk Terminal. Polish Maritime Research, 23(s1).10.1515/pomr-2016-0059Open DOISearch in Google Scholar

5. C. Mi, Y. Shen, W. J. Mi, Y. F. Huang, 2015. Ship Identification Algorithm Based on 3D Point Cloud for Automated Ship Loaders. Journal of Coastal Research, 2015(SI.73): 28-34. DOI: 10.2112/SI73-006.10.2112/SI73-006Open DOISearch in Google Scholar

6. C. Mi, Z. W. Zhang, X. He, Y. F. Huang, W. J. Mi, 2015. Two-stage classification approach for human detection in camera video in bulk ports, Polish Maritime Research, 22(SI.1):163-170. DOI: 10.1515/pomr-2015-004910.1515/pomr-2015-0049Open DOISearch in Google Scholar

7. C. Mi, H. W. Liu, Y. F. Huang, W. J. Mi, Y. Shen, 2016. Fatigue alarm systems for port machine operators. Asia Life Sciences, 25(1): 31-41.Search in Google Scholar

8. Yifan S, Ning Z, Weijian M. 2016. Group-Bay Stowage Planning Problem for Container Ship. Polish Maritime Research, 23(s1).10.1515/pomr-2016-0060Search in Google Scholar

9. Mengjue X., Ning Z, Weijian M. 2016. Storage Allocation in Automated Container Terminals: the Upper Level. Polish Maritime Research, 23(s1).10.1515/pomr-2016-0061Open DOISearch in Google Scholar

10. C. Mi, X. He, H. W. Liu, Y. F. Huang, W. J. Mi, 2014. Research on a Fast Human-Detection Algorithm for Unmanned Surveillance Area in Bulk Ports. Mathematical Problems in Engineering. DOI: 10.1155/2014/38676410.1155/2014/386764Search in Google Scholar

11. D. S. Todd, P. Sen, 1997. A Multiple Criteria Genetic Algorithm for Containership Loading International Conference on Genetic Algorithms, East Lansing, Mi, Usa, July. DBLP, 674-681.Search in Google Scholar

12. N. Zhao, W. J. Mi, 2008. Robust approach in stowage planning at contianer terminals. IEEE proceeding of the 4th International Conference on Intelligent Logistic System, 191-204.Search in Google Scholar

13. A. Moura, J. Oliveira, C. Pimentel, 2013. A Mathematical Model for the Container Stowage and Ship Routing Problem. Journal of Mathematical Modelling and Algorithms in Operations Research, 12(3): 217-231.10.1007/s10852-012-9207-3Search in Google Scholar

14. M. Avriel, M. Penn, 1993. Exact and approximate solutions of the container ship stowage problem. Computers & Industrial Engineering, 25(1-4):271-274.10.1016/0360-8352(93)90273-ZSearch in Google Scholar

15. J. J. Shields, 1984. Containership Stowage: A Computer- Aided Preplanning System. Marine Technology, 21(4): 370-383. 10.5957/mt1.1984.21.4.370Search in Google Scholar

16. A. Imai, T. Miki, 1989. A heuristic algorithm with expected utility for an optimal sequence of loading containers into a containerized ship. Journal of Japan Institute of Navigation, 80: 117-124 (in Japanese).10.9749/jin.80.117Search in Google Scholar

17. A. Imai, E. Nishimura, K. Sasaki, S. Papadimitriou, 2001. Solution comparisons of algorithms for the containership loading problem. Proceedings of the International Conference on Shipping: Technology and Environment, available on CD-ROM.Search in Google Scholar

18. A. Imai, E. Nishimura, K. Sasaki, S. Papadimitriou, 2001. Solution comparisons of algorithms for the containership loading problem. Proceedings of the International Conference on Shipping: Technology and Environment, available on CD-ROM.Search in Google Scholar

19. A. Haghani, E. I. Kaisar, 2001. A model for designing container loading plans for containerships. In: 80th Transportation Research Board Annual Meeting, Washington, DC, USA.Search in Google Scholar

20. J. F. Álvarez, 2006. A heuristic for Vessel planning in a reach stacker terminal. Journal of Maritime Research Jmr, 3(1): págs. 3-16.Search in Google Scholar

21. K. H. Kim, 1994. Analysis of rehandles of transfer crane in a container yard. APORS-Conference, 3: 357-365.Search in Google Scholar

22. K. H. Kim. 1997. Evaluation of the number of rehandles in container yards. Computers & Industrial Engineering, 32: 701-711.Search in Google Scholar

23. K. H. Kim, Y. M. Park, K. R. Ryu, 2000. Deriving decision rules to locate export containers in container yards. European Journal of Operational Research, 124: 89-101.10.1016/S0377-2217(99)00116-2Open DOISearch in Google Scholar

24. K. H. Kim, J. S. Kang, K. R. Ryu, 200. A beam search algorithm for the load sequencing of outbound containers in port container terminals. OR Spectrum, 26: 93-116.10.1007/s00291-003-0148-0Search in Google Scholar

25. Y. Lee, J. Kang, K. R. Ryu, K. H. Kim, 2005. Optimization of Container Load Sequencing by a Hybrid of Ant Colony Optimization and Tabu Search, Natural Computation Lecture Notes in Computer Science, 3611, 1259-1268.10.1007/11539117_167Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo