[Åström, K.J. (1965). Optimal control of Markov decision processes with incomplete state estimation, Journal of Mathematical Analysis and Applications 10(1): 174-205.10.1016/0022-247X(65)90154-X]Search in Google Scholar
[Barto, A.G. and Mahadevan, S. (2003). Recent advances in hierarchical reinforcement learning, Discrete Event Dynamic Systems 13(4): 341-379.10.1023/A:1025696116075]Search in Google Scholar
[Bonet, B. (2009). Deterministic POMDPs revisited, Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI), Montreal, Canada, pp. 59-66.]Search in Google Scholar
[Bui, H.H., Phung, D.Q. and Venkatesh, S. (2004). Hierarchical hidden Markov models with general state hierarchy, Proceedings of the 19th National Conference on Artificial Intelligence (AAAI), San Jose, CA, USA, pp. 324-329.]Search in Google Scholar
[Chang, H.S., Fard, P.J., Marcus, S.I. and Shayman, M. (2003). Multi-time scale Markov decision processes, IEEE Transactions on Automatic Control 48(6): 976-987.10.1109/TAC.2003.812782]Search in Google Scholar
[Charlin, L., Poupart, P. and Shioda, R. (2007). Automated hierarchy discovery for planning in partially observable environments, in B. Schölkopf, J.C. Platt and T. Hofmann (Eds.), Advances in Neural Information Processing Systems 19 (NIPS 2006), The MIT Press, Cambridge, MA, pp. 225-232.]Search in Google Scholar
[Chatzis, S.P. and Kosmopoulos, D. (2014). A partially-observable Markov decision process for dealing with dynamically changing environments, in L. Iliadis, I. Maglogiannis and H. Papadopoulos (Eds.), Artificial Intelligence Applications and Innovations, Springer-Verlag, Berlin, pp. 111-120.10.1007/978-3-662-44654-6_11]Search in Google Scholar
[Dean, T., Angluin, D., Basye, K., Engelson, S., Kaelbling, L., Kokkevis, E. and Maron, O. (1995). Inferring finite automata with stochastic output functions and an application to map learning, Machine Learning 18(1): 81-108.10.1007/BF00993822]Search in Google Scholar
[Dietterich, T.G. (2000). Hierarchical reinforcement learning with the MAXQ value function decomposition, Journal of Artificial Intelligence Research 13: 227-303.10.1613/jair.639]Search in Google Scholar
[Doshi-Velez, F. (2009). The infinite partially observable Markov decision process, in Y. Bengio et al. (Eds.), Advances in Neural Information Processing Systems 22 (NIPS 2009), Curran Associates Inc., Red Hook, NY, pp. 477-485.]Search in Google Scholar
[Doshi-Velez, F., Pfau, D., Wood, F. and Roy, N. (2015). Bayesian nonparametric methods for partially-observable reinforcement learning, IEEE Transactions on Pattern Analysis and Machine Intelligence 37(2): 394-407.10.1109/TPAMI.2013.19126353250]Search in Google Scholar
[Drake, A. (1962). Observation of a Markov Process Through a Noisy Channel, Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA.]Search in Google Scholar
[Fine, S., Singer, Y. and Tishby, N. (1998). The hierarchical hidden Markov model: Analysis and applications, Machine Learning 32(1): 41-62.10.1023/A:1007469218079]Search in Google Scholar
[Foka, A. and Trahanias, P. (2007). Real-time hierarchical POMDPs for autonomous robot navigation, Robotics and Autonomous Systems 55(7): 561-571.10.1016/j.robot.2007.01.004]Search in Google Scholar
[Gavaldà, R., Keller, P.W., Pineau, J. and Precup, D. (2006). PAC-learning of Markov models with hidden state, Proceedings of the 17th European Conference on Machine Learning (ECML), Berlin, Germany, pp. 150-161.]Search in Google Scholar
[Heller, K.A., Teh, Y.W. and Görür, D. (2009). Infinite hierarchical hidden Markov models, Proceedings of the 12th International Conference on Artificial Intelligence and Statistics (AISTATS), Clearwater Beach, FL, USA, pp. 224-231.]Search in Google Scholar
[Hengst, B. (2011). Hierarchical approaches, in M. Wiering and M. van Otterlo (Eds.), Reinforcement Learning: State of the Art, Springer-Verlag, Berlin, pp. 293-323.]Search in Google Scholar
[Hinton, G.E., Osindero, S. and Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets, Neural Computation 18(7): 1527-1554.10.1162/neco.2006.18.7.152716764513]Search in Google Scholar
[Hoey, J., Poupart, P., Bertoldi, A., Craig, T., Boutilier, C. and Mihailidis, A. (2010). Automated handwashing assistance for persons with dementia using video and a partially observable Markov decision process, Computer Vision and Image Understanding 114(5): 503-519.10.1016/j.cviu.2009.06.008]Search in Google Scholar
[Holmes, M.P. and Isbell Jr., C.L. (2006). Looping suffix tree-based inference of partially observable hidden state, Proceedings of the 23rd International Conference on Machine Learning (ICML), Pittsburgh, PA, USA, pp. 409-416.]Search in Google Scholar
[Kaelbling, L.P., Littman, M.L. and Cassandra, A.R. (1999). Planning and acting in partially observable stochastic domains, Artificial Intelligence 101(1-2): 99-134.10.1016/S0004-3702(98)00023-X]Search in Google Scholar
[Kolobov, A. (2012). Planning with Markov decision processes: An AI perspective, Synthesis Lectures on Artificial Intelligence and Machine Learning 6(1): 1-210.10.2200/S00426ED1V01Y201206AIM017]Search in Google Scholar
[Kołodziej, J., Khan, S.U., Wang, L., Min-Allah, N., Madani, S.A., Ghani, N. and Li, H. (2011). An application of Markov jump process model for activity-based indoor mobility prediction in wireless networks, 9th IEEE International Conference on Frontiers of Information Technology (FIT), Islamabad, Pakistan, pp. 51-56.]Search in Google Scholar
[Li, H., Zhao, Q. and Yang, Z. (2007). Reliability modeling of fault tolerant control systems, International Journal of Applied Mathematics and Computer Science 17(4): 491-504, DOI: 10.2478/v10006-007-0041-0.10.2478/v10006-007-0041-0]Search in Google Scholar
[Lim, Z., Sun, L. and Hsu, D.J. (2011). Monte Carlo value iteration with macro-actions, in J. Shawe-Taylor et al. (Eds.), Advances in Neural Information Processing Systems 24 (NIPS 2011), Curran Associates Inc., Red Hook, NY, pp. 1287-1295.]Search in Google Scholar
[Littman, M.L. (1996). Algorithms for Sequential Decision Making, Ph.D. thesis, Brown University, Providence, RI.]Search in Google Scholar
[Mahadevan, S. (1998). Partially observable semi-Markov decision processes: Theory and applications in engineering and cognitive science, AAAI Fall Symposium on Planning with Partially Observable Markov Decision Processes, Orlando, FL, USA, pp. 113-120.]Search in Google Scholar
[Mihalkova, L. and Mooney, R.J. (2007). Bottom-up learning of Markov logic network structure, Proceedings of the 24th International Conference on Machine Learning (ICML), Corvallis, OR, USA, pp. 625-632.]Search in Google Scholar
[Murphy, K.P. (2002). Representing hierarchical POMDPs as DBNs, with applications to mobile robot navigation, www.cs.ubc.ca/murphyk/mypapers.html.]Search in Google Scholar
[Oliver, N., Garg, A. and Horvitz, E. (2004). Layered representations for learning and inferring office activity from multiple sensory channels, Computer Vision and Image Understanding 96(2): 163-180.10.1016/j.cviu.2004.02.004]Search in Google Scholar
[Oniszczuk, W. (2009). Semi-Markov-based approach for the analysis of open tandem networks with blocking and truncation, International Journal of Applied Mathematics and Computer Science 19(1): 151-163, DOI: 10.2478/v10006-009-0014-6.10.2478/v10006-009-0014-6]Search in Google Scholar
[Pineau, J., Montemerlo, M., Pollack, M., Roy, N. and Thrun, S. (2003). Towards robotic assistants in nursing homes: Challenges and results, Robotics and Autonomous Systems 42(3): 271-281.10.1016/S0921-8890(02)00381-0]Search in Google Scholar
[Poupart, P. and Vlassis, N. (2008). Model-based Bayesian reinforcement learning in partially observable domains, Proceedings of the 10th International Symposium on Artificial Intelligence and Mathematics (ISAIM), Fort Lauderdale, FL, USA, p. 8.]Search in Google Scholar
[Rao, V. and Teh, Y. W. (2013). Fast MCMC sampling for Markov jump processes and extensions, The Journal of Machine Learning Research 14(1): 3295-3320.]Search in Google Scholar
[Ross, S., Pineau, J., Chaib-draa, B. and Kreitmann, P. (2011). A Bayesian approach for learning and planning in partially observableMarkov decision processes, Journal of Machine Learning Research 12: 1729-1770.]Search in Google Scholar
[Roy, N., Pineau, J. and Thrun, S. (2000). Spoken dialogue management using probabilistic reasoning, Proceedings of the 38th Annual Meeting on Association for Computational Linguistics (ACL), Hong Kong, China, pp. 93-100.]Search in Google Scholar
[Rusek, K., Janowski, L. and Papir, Z. (2014). Transient and stationary characteristics of a packet buffer modelled as an MAP/SM/1/b system, International Journal of Applied Mathematics and Computer Science 24(2): 429-442, DOI: 10.2478/amcs-2014-0033.10.2478/amcs-2014-0033]Search in Google Scholar
[Sallans, B. (2000). Learning factored representations for partially observable Markov decision processes, in S.A.]Search in Google Scholar
[Solla, T.K. Leen and K. Müller (Eds.), Advances in Neural Information Processing Systems 12 (NIPS 1999), TheMIT Press, Cambridge, MA, pp. 1050-1056.]Search in Google Scholar
[Shani, G., Brafman, R.I. and Shimony, S.E. (2005). Model-based online learning of POMDPs, Proceedings of the 16th European Conference on Machine Learning (ECML), Porto, Portugal, pp. 353-364.]Search in Google Scholar
[Spaan, M.T.J. and Vlassis, N. (2005). Perseus: Randomized point-based value iteration for POMDPs, Journal of Artificial Intelligence Research 24: 195-220.10.1613/jair.1659]Search in Google Scholar
[Theocharous, G. (2002). Hierarchical Learning and Planning in Partially Observable Markov Decision Processes, Ph.D. thesis, Michigan State University, East Lansing, MI.]Search in Google Scholar
[Theocharous, G. and Mahadevan, S. (2002). Approximate planning with hierarchical partially observable Markov decision process models for robot navigation, Proceedings of the 2002 IEEE International Conference on Robotics and Automation (ICRA), Washington, DC, USA, pp. 1347-1352.]Search in Google Scholar
[Theocharous, G., Murphy, K. and Kaelbling, L.P. (2004). Representing hierarchical POMDPs as DBNs for multi-scale robot localization, Proceedings of the 2004 IEEE International Conference on Robotics and Automation (ICRA), New Orleans, LA, USA, Vol. 1, pp. 1045-1051.]Search in Google Scholar
[Toussaint, M., Charlin, L. and Poupart, P. (2008). Hierarchical POMDP controller optimization by likelihood maximization, Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence (UAI), Helsinki, Finland, pp. 562-570.]Search in Google Scholar
[Wakabayashi, K. and Miura, T. (2012). Forward-backward activation algorithm for hierarchical hidden Markov models, in F. Pereira et al. (Eds.), Advances in Neural Information Processing Systems 25 (NIPS 2012), Curran Associates Inc., Red Hook, NY, pp. 1502-1510.]Search in Google Scholar
[White, C.C. (1976). Procedures for the solution of a finite-horizon, partially observed, semi-Markov optimization problem, Operations Research 24(2): 348-358. Young, S., Gašić, M., Thomson, B. and Williams, J.D. (2013). POMDP-based statistical spoken dialog systems: A review, Proceedings of the IEEE 101(5): 1160-1179.]Search in Google Scholar
[Youngblood, G.M. and Cook, D.J. (2007). Data mining for hierarchical model creation, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 37(4): 561-572.10.1109/TSMCC.2007.897341]Search in Google Scholar
[Youngblood, G.M., Heierman, E.O., Cook, D.J. and Holder, L.B. (2005). Automated HPOMDP construction through data-mining techniques in the intelligent environment domain, Proceedings of the 18th International Florida Artificial Intelligence Research Society Conference (FLAIRS), Clearwater Beach, FL, USA, pp. 194-199.]Search in Google Scholar
[Zamani, Z., Sanner, S., Poupart, P. and Kersting, K. (2012). Symbolic dynamic programming for continuous state and observation POMDPs, in F. Pereira et al. (Eds.), Advances in Neural Information Processing Systems 25 (NIPS 2012), Curran Associates Inc., Red Hook, NY, pp. 1403-1411. ]Search in Google Scholar
