gaips_bea image1 image2 image3 image4 image5 gaips_ecute_beach_bar_banner gaips_ecute_train_incorrect_ticket_banner
An Associative State-Space Metric for Learning in Factored MDPs


Abstract In this paper we propose a novel associative metric based on the classical conditioning paradigm that, much like what happens in nature, identifies associations between stimuli perceived by a learning agent while interacting with the environment. We use an associative tree structure to identify associations between the perceived stimuli and use this structure to measure the degree of similarity between states in factored Markov decision problems. Our approach provides a state-space metric that requires no prior knowledge on the structure of the underlying decision problem and is designed to be learned online, i.e., as the agent interacts with its environment. Our metric is thus amenable to application in reinforcement learning (RL) settings, allowing the learning agent to generalize its experience to unvisited states and improving the overall learning performance. We illustrate the application of our method in several problems of varying complexity and show that our metric leads to a performance comparable to that obtained with other well-studied metrics that require full knowledge of the decision problem.
Year 2013
Keywords Reinforcement Learning;Neural Computation;
Authors Pedro Sequeira, Francisco S. Melo, Ana Paiva
Booktitle Proceedings of the 16th Portuguese Conference on Artificial Intelligence
Pages 163-174
Series EPIA 2013
Publisher Springer-Verlag
Address Berlin Heidelberg
Month September
Pdf File \"pdf
BibTex bib icon or see it here down icon

@inproceedings { sequeira13, abstract = {In this paper we propose a novel associative metric based on the classical conditioning paradigm that, much like what happens in nature, identifies associations between stimuli perceived by a learning agent while interacting with the environment. We use an associative tree structure to identify associations between the perceived stimuli and use this structure to measure the degree of similarity between states in factored Markov decision problems. Our approach provides a state-space metric that requires no prior knowledge on the structure of the underlying decision problem and is designed to be learned online, i.e., as the agent interacts with its environment. Our metric is thus amenable to application in reinforcement learning (RL) settings, allowing the learning agent to generalize its experience to unvisited states and improving the overall learning performance. We illustrate the application of our method in several problems of varying complexity and show that our metric leads to a performance comparable to that obtained with other well-studied metrics that require full knowledge of the decision problem.}, address = {Berlin Heidelberg }, booktitle = {Proceedings of the 16th Portuguese Conference on Artificial Intelligence}, keywords = {Reinforcement Learning;Neural Computation;}, month = {September}, pages = {163-174}, publisher = {Springer-Verlag}, series = {EPIA 2013}, title = {An Associative State-Space Metric for Learning in Factored MDPs}, year = {2013}, author = {Pedro Sequeira and Francisco S. Melo and Ana Paiva} }

up icon hide this content