About the project

The AMIGOS (Affect Modeling for robots In GrOup Social interactions) project investigates the role of emotions and adaptation in interactions between a robot and a group of users, contrasting to the typical one-robot one­user paradigm in Human­Robot Interaction (HRI). Despite the complex social challenges that long­term HRI will soon bring, so far little is known about how perception and action selection systems, typically designed for one­to­one interactions, will perform in multiparty settings. Recent studies in this area indicate that data­driven perception mechanisms trained with information from individual interactions do not generalize well in group settings, raising the need to investigate new adaptive mechanisms for robots interacting with groups of users. Previous research by members of this team studied social robots in multiparty interactions, yet these robots had limited capabilities and were evaluated in single interactions with users.

We address the issue of social adaptation for robots in group settings focusing on computational modeling of emotions. Emotions play a critical role in HRI. Several authors have reported the relevance of emotions in the establishment of social interactions between one robot and one user, in particular the role of empathy. Despite these efforts, further research is necessary to verify whether similar results hold (1) when aiming for longer term social interactions, and (2) when the robot is in the presence of a group of people.

To endow a robot with the ability to cope with changes in the number of users around it, we propose the use of interactive machine learning techniques that allow the robot to quickly adjust its behavior, depending on both the situational context (i.e., the number of users in the environment) and the preferences of a particular user, and generate adaptive social responses. Such adaptive responses contrast with existing approaches built on rule­based stereotypes from cognitive/psychological theories, or on the result of a process of optimization that computes the "best" intervention depending on the situation. These approaches become insufficient as the social environment around the robot becomes more complex, for example, as the number of users around the robot increases. Our main hypothesis is that if a robot can capture the dynamics of the affective interactions of a small group and is able to adapt its emotional behavior accordingly, users will be more willing to sustain the interaction with the robot for longer periods of time, bringing us closer to the establishment of sustainable and engaging long­term interactions. With this purpose, we employ two related classes of machine learning techniques: (1) online Learning by regret minimization, where the robot has a set of pre­determined high­level emotional behaviors (nonverbal and verbal) that, depending on the situation, indicate how the robot should respond, and (2) reinforcement Learning, where the robot is endowed with a repertoire of lower­level response primitives and slowly adapts its responses in a context­specific manner, using feedback from the user(s).

To evaluate the effectiveness of the robot in dealing with a variety of group sizes, we will carefully design an experimental collaborative scenario using a social robot that will be acquired for the purpose of this project. To create believable experimental situations with the robot while limiting the perception challenges that multiparty human­robot interactions will bring, the scenario will feature simple game tasks where one user or a small group of users engage in the interaction with the robot, trying to solve the tasks in the game. The INESC­-ID team has previous experience in designing interactive HRI scenarios with multiple users, and has available perception systems that will be the basis of the perception work. Our experiments will be conducted outside laboratory conditions with young adults, to assess the real impact that the emotional and adaptive behaviors will have in relational measures with users.

The project involves the collaborative effort of INESC­-ID (principal contractor) and ISCTE­-IUL. The senior members have an excellent track record of research outputs in the areas of Human­Robot Interaction, Machine Learning, Affective Computing, and Social and Health Psychology, making this a unique multidisciplinary team. Previous successful collaborations among the partners in the team evidence the potential for a well functioning and productive team.