Emergence of Emotional Appraisal Signals in Reinforcement Learning Agents (JAAMAS Extended Abstract)


Pedro Sequeira, Francisco S. Melo and Ana Paiva

In Proceedings of the 15th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2016, pp. 466–467.  Emergence of Emotional Appraisal Signals in Reinforcement Learning Agents (JAAMAS Extended Abstract)Emergence of Emotional Appraisal Signals in Reinforcement Learning Agents (JAAMAS Extended Abstract)


Abstract
The positive impact of emotions in decision-making has long been established in both natural and artificial agents. In the perspective of appraisal theories, emotions complement perceptual information, coloring our sensations and guiding our decision-making. However, when designing autonomous agents, is emotional appraisal the best complement to their perceptions? Mechanisms investigated in the affective sciences provide support for this hypothesis in biological agents. In this paper, we look for similar support in artificial systems. We adopt the intrinsically motivated reinforcement learning framework to investigate different sources of information that can guide decision-making in learning agents, and an evolutionary approach based on genetic programming to identify a small set of such sources that have the largest impact on the performance of the agent in different tasks, as measured by an external evaluation signal. We then show that these sources of information: (i) are applicable in a wider range of environments than those where the agents evolved; (ii) exhibit interesting correspondences to emotional appraisal-like signals previously proposed in the literature. The results of the study thus point towards our departing hypothesis that the appraisal process might indeed provide essential information to complement perceptual capabilities and thus guide decision-making.

Paper

Presentation

Full JAAMAS paper

Posted on