A Flexible Approach to Modeling Unpredictable Events in MDPs. [Go Back]


Publication Info

Stefan J. Witwicki, Francisco S. Melo, Jesús Capitán Fernández, and Matthijs T.J. Spaan. In Proceedings of the 23rd International Conference on Automated Planning and Scheduling, (ICAPS-2013). Rome, Italy. June 2013. To appear.


Abstract

In planning with a Markov decision process (MDP) framework, there is the implicit assumption that the world is predictable. Practitioners must simply take it on good faith the MDP they have constructed is comprehensive and accurate enough to model the exact probabilities with which all events may occur under all circumstances. Here, we challenge the conventional assumption of complete predictability, arguing that some events are inherently unpredictable. Towards more effectively modeling problems with unpredictable events, we develop a hybrid framework that explicitly distinguishes decision factors whose probabilities are not assigned precisely while still representing known probability components using conventional principled MDP transitions. Our approach is also flexible, resulting in a factored model of variable abstraction whose usage for planning results in different levels of approximation. We illustrate the usage of our modeling framework in a robot surveillance domain.


Bibtex
@inproceedings{Witwicki:ICAPS2013,
    author = {Stefan J. Witwicki, Francisco S. Melo, Jes\'{u}s Capit\'{a}n Fern\'{a}ndez, 
	      and Matthijs T.J. Spaan},
    title = {A Flexible Approach to Modeling Unpredictable Events in {MDP}s},
    booktitle = {Proceedings of the 23rd International Conference on 
		Automated Planning and Scheduling (ICAPS-2013)},
    month = {June},
    year = {2013},
    pages = {},
    address = {Rome, Italy}
    note = {To appear.}
}
[Go Back]