<abstract xmlns="http://www.w3.org/1999/xhtml"><p> Ship stowage plan is the management connection of quae crane scheduling and yard crane scheduling. The quality of ship stowage plan affects the productivity greatly. Previous studies mainly focuses on solving stowage planning problem with online searching algorithm, efficiency of which is significantly affected by case size. In this study, a Deep Q-Learning Network (DQN) is proposed to solve ship stowage planning problem. With DQN, massive calculation and training is done in pre-training stage, while in application stage stowage plan can be made in seconds. To formulate network input, decision factors are analyzed to compose feature vector of stowage plan. States subject to constraints, available action and reward function of Q-value are designed. With these information and design, an 8-layer DQN is formulated with an evaluation function of mean square error is composed to learn stowage planning. At the end of this study, several production cases are solved with proposed DQN to validate the effectiveness and generalization ability. Result shows a good availability of DQN to solve ship stowage planning problem.</p></abstract>

Ship stowage plan is the management connection of quae crane scheduling and yard crane scheduling. The quality of ship stowage plan affects the productivity greatly. Previous studies mainly focuses on solving stowage planning problem with online searching algorithm, efficiency of which is significantly affected by case size. In this study, a Deep Q-Learning Network (DQN) is proposed to solve ship stowage planning problem. With DQN, massive calculation and training is done in pre-training stage, while in application stage stowage plan can be made in seconds. To formulate network input, decision factors are analyzed to compose feature vector of stowage plan. States subject to constraints, available action and reward function of Q-value are designed. With these information and design, an 8-layer DQN is formulated with an evaluation function of mean square error is composed to learn stowage planning. At the end of this study, several production cases are solved with proposed DQN to validate the effectiveness and generalization ability. Result shows a good availability of DQN to solve ship stowage planning problem.

A Deep Q-Learning Network for Ship Stowage Planning Problem

Polish Maritime Research

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

{"article-title":"A Deep Q-Learning Network for Ship Stowage Planning Problem"}

Ship stowage plan is the management connection of quae crane scheduling and yard crane scheduling. The quality of ship stowage plan affects the productivity...

A Deep Q-Learning Network for Ship Stowage Planning Problem

Article Category: Special Issue Title: Marine Processes Studies And Marine Engineering

Published Online: Nov 22, 2017

Page range: 102 - 109

DOI: https://doi.org/10.1515/pomr-2017-0111

Keywords
Deep Q-Leaning Network (DQN), Container terminal, Ship stowage plan, Markov decision process, Value function approximation, Generalization

© 2017

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

A Deep Q-Learning Network for Ship Stowage Planning Problem

Article Category: Special Issue Title: Marine Processes Studies And Marine Engineering

Published Online: Nov 22, 2017

Page range: 102 - 109

DOI: https://doi.org/10.1515/pomr-2017-0111

KeywordsDeep Q-Leaning Network (DQN), Container terminal, Ship stowage plan, Markov decision process, Value function approximation, Generalization

© 2017

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Keywords
Deep Q-Leaning Network (DQN), Container terminal, Ship stowage plan, Markov decision process, Value function approximation, Generalization