Degree : |
PhD |
Institution : |
INESC-ID |
Address : |
IST Tagusparque, Av. Prof. Dr. Cavaco Silva, 2744-016 Porto Salvo, Portugal |
Office : |
2N7.15 |
Telephone : |
+351 210 407 002 |
Interests
I am an Associate Professor at the Department of Computer Science and Engineering of Instituto Superior Técnico, in Lisbon, Portugal.
In my research, I am interested in the application of machine learning in the development of general intelligent autonomous agents capable of interacting with complex, possibly unknown environments. Concretely, my primary research interests are in general reinforcement learning both in single and multiagent systems and in the applications of this framework to real world problems, including robotic and sensor network applications. I am also particularly interested in developing computational models that may contribute to a better understanding of learning in biological systems.
Current Projects
Publications
2022
Cooperation and learning dynamics under risk diversity and financial incentives
Ramona Merhej and Fernando P. Santos and Francisco S. Melo and Francisco C. Santos, in Proc. of the 21st International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2022), Full paper, IFAAMAS, 2022
2021
A Methodology for the Development of RL-Based Adaptive Traffic Signal Controllers
Guilherme Varela and Pedro Pinto Santos and Alberto Sardinha and Francisco S. Melo, AAAI Workshop on AI for Urban Mobility (AI4UM), February, 2021
Ad Hoc Teamwork in the Presence of Non-Stationary Teammates

Pedro M. Santos and João Ribeiro and Alberto Sardinha and Francisco S. Melo, 20th EPIA Conference on Artificial Intelligence, September, 2021
Cooperation between independent reinforcement learners under wealth inequality


Ramona Merhej and Fernando P. Santos and Francisco S. Melo and Francisco C. Santos, in Proc. of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021), U. Endriss, A. Nowé, F. Dignum, A. Lomuscio (eds.), May 3-7, 2021, Full Paper, IFAAMAS (2021) [Nominated for the AAMAS best paper award]]., 2021
Helping People On The Fly: Ad Hoc Teamwork for Human-Robot Teams

João Ribeiro and Miguel Faria and Alberto Sardinha and Francisco S. Melo, 20th EPIA Conference on Artificial Intelligence, September, 2021
Interactive Teaching with Groups of Unknown Bayesian Learners


Carla Guerra and Francisco S. Melo and Manuel Lopes, International Conference on Artificial Intelligence in Education, pg. 178--182, Springer, June, 2021
Learning Collective Action under Risk Diversity

Ramona Merhej and Fernando P. Santos and Francisco S. Melo and Francisco C. Santos, Cooperative AI Workshop (CoopAI 2021), NeurIPS 2021 - Conference on Neural Information Processing Systems , 2021, 2021
Teaching Unknown Learners to Classify via Feature Importance


Carla Guerra and Francisco S. Melo and Manuel Lopes, Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, pg. 1524--1526, 2021
Understanding Robots: Making Robots More Legible in Multi-Party Interactions


Miguel Faria and Francisco S. Melo and Ana Paiva, 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), pg. 1031-1036, IEE, Vancouver, BC, Canada, August, 2021
2020
A new convergent variant of Q-learning with linear function approximation
Diogo Carvalho and Francisco S. Melo and Pedro Santos, NeurIPS 2020: Thirty-fourth Conference on Neural Information Processing Systems, International Conference on Machine Learning, pg. 21-55, December, 2020
Emergence of Cooperation in N-Person Dilemmas through Actor-Critic Reinforcement Learning

João Vítor Barbosa and Francisco S. Melo and Anna H. Reali Costa and Jaime S. Sichman and Francisco C. Santos, Adaptive Learning Workshop (ALA), AAMAS 2020 - Autonomous Agents and Multi-Agent Systems, 2020
Interactive Teaching with Unknown Classes of Bayesian Learners


Carla Guerra and Francisco S. Melo and Manuel Lopes, 2nd AIMA4EDU Workshop, IJCAI, 2020
2019
An approach for personalized social interactions between a therapeutic robot and children with autism spectrum disorder

Kim Baraka and Marta Couto and Francisco S. Melo and Ana Paiva and Manuela Veloso
Effect of Agents’ Transparency on Teamwork


Silvia Tulli and Filipa Correia and Samuel Mascarenhas and Samuel Gomes and Francisco S. Melo and Ana Paiva, Springer post-proceedings Lecture Notes in Artificial Intelligence (LNAI)
Empathic robot for group learning: A field study


Patrícia Alves-Oliveira and Pedro Sequeira and Francisco S. Melo and Ginevra Castellano and Ana Paiva, ACM Transactions on Human-Robot Interaction (THRI), Vol. 8, No. 1, pg. 3, ACM, 2019
Exploring prosociality in human-robot teams


Filipa Correia and Samuel Mascarenhas and Samuel Gomes and Patrícia Arriaga and Iolanda Leite and Rui Prada and Francisco S. Melo and Ana Paiva, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pg. 143--151, 2019
For The Record-A Public Goods Game For Exploring Human-Robot Collaboration


Filipa Correia and Samuel Mascarenhas and Samuel Gomes and Silvia Tulli and Fernando P. Santos and Francisco C. Santos and Rui Prada and Francisco S. Melo and Ana Paiva, Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, pg. 2351--2353, 2019
Group intelligence on social robots


Filipa Correia and Francisco S. Melo and Ana Paiva, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pg. 703--705, 2019
Multi-task Learning and Catastrophic Forgetting in Continual Reinforcement Learning


João G. Ribeiro and Francisco S. Melo and João Dias, 5th Global Conference on Artificial Intelligence, pg. 163-175, December, 2019
Project INSIDE: towards autonomous semi-unstructured human–robot social interaction in autism therapy


Francisco S. Melo and Alberto Sardinha and David Belo and Marta Couto and Miguel Faria and Anabela Farias and Hugo Gambôa and Cátia Jesus and Mithun Kinarullathil and Pedro Lima and Luís Luz and André Mateus and Isabel Melo and Plinio Moreno and Daniel Osório and Ana Paiva and Jhielson Pimentel and João Rodrigues and Pedro Sequeira and Rubén Solera-Ureña and Miguel Vasco and Manuela Veloso and Rodrigo Ventura, Artificial Intelligence in Medicine, Vol. 96, pg. 198 - 216, 2019
Walk the Talk! Exploring (Mis) Alignment of Words and Deeds by Robotic Teammates in a Public Goods Game


Filipa Correia and Shruti Chandra and Samuel Mascarenhas and Julien Charles-Nicolas and Justin Gally and Diana Lopes and Fernando P. Santos and Francisco C. Santos and Francisco S. Melo and Ana Paiva, 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pg. 1--7, 2019
2018
"I Choose... YOU!" Membership preferences in human--robot teams

Filipa Correia and Sofia Petisca and Tiago Ribeiro and Francisco S. Melo and Ana Paiva, Autonomous Robots, Jun, 2018
A Platform for the Geometry Friends Cooperative Game AI Competition


Ricardo Costa and Rui Prada and Francisco S. Melo, Proceedings of Videojogos 2018 - the 10th Conference on Videogame Sciences and Arts, pg. 153-167, i2ADS and SPCV, Porto, Portugal, September, 2018
An Ensemble Inverse Optimal Control Approach for Robotic Task Learning and Adaptation

Hang Yin and Francisco S. Melo and Ana Paiva and Aude Billard, Autonomous Robots, Springer, 2018
Boosting Robot Learning and Control with Domain Constraints


Hang Yin and Francisco S. Melo and Aude Billard and Ana Paiva, In Proceedings of Robotics: Science and Systems (RSS), RSS Pioneer Workshop, Pittsburgh, USA, June, 2018
Exploring the Impact of Fault Justification in Human-Robot Trust


Filipa Correia and Carla Guerra and Samuel Mascarenhas and Francisco S. Melo and Ana Paiva, Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, pg. 507--513, International Foundation for Autonomous Agents and Multiagent Systems, Stockholm, Sweden, 2018
Group-based Emotions in Teams of Humans and Robots


Filipa Correia and Samuel Mascarenhas and Rui Prada and Francisco S. Melo and Ana Paiva, Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, pg. 261-269, ACM, New York, NY, USA, March, 2018
Interactive Consoles for Adjustable Autonomy in Assistive Robots: a Case Study with ASD Children


Jhielson M Pimentel and Carla Guerra and Pedro Sequeira and Andre Mateus and Alberto Sardinha and Francisco S. Melo and Pedro U Lima, Workshop: Robots for Assisted Living (IROS 2018), 2018
Interactive Optimal Teaching with Unknown Learners


Francisco S. Melo and Carla Guerra and Manuel Lopes, Proceedings of the 27th International Joint Conference on Artificial Intelligence, pg. 2567--2573, Stockholm, Sweden, 2018
Learning Variational Latent Dynamics: Towards Model-based Imitation and Control


Hang Yin and Francisco S. Melo and Aude Billard and Ana Paiva
Project INSIDE: Towards Autonomous Semi-Unstructured Human-Robot Social Interaction in Autism Therapy


Francisco S. Melo and Alberto Sardinha and Miguel Faria and Ana Paiva and Ruben Solera and Miguel Vasco and Mithun Kinarullathil and Pedro Lima and Luís Luz and André Mateus and Jhielson Pimentel and Rodrigo Ventura and Marta Couto and Anabela Farias and Isabel Melo and Hugo Gambôa and David Belo and Daniel Osório and João Rodrigues and Pedro Sequeira and Manuela Veloso
2017
A Conversational Agent Powered by Online Learning

Vânia Mendonça and Francisco S. Melo and Luísa Coheur and Alberto Sardinha, Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, pg. 1637--1639, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 2017
A Social Robot as a Card Game Player


Filipa Correia and Patrícia Alves-Oliveira and Tiago Ribeiro and Francisco S. Melo and Ana Paiva, Proceedings of the Thirteenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-17), AAAI Press, 2017
Associate Latent Encodings in Learning from Demonstrations


Hang Yin and Francisco S. Melo and Aude Billard and Ana Paiva, Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI). San Francisco, California, USA., 2017
Groups of Humans and Robots: the AMIGOS Project


Filipa Correia and Patrícia Arriaga and Sofia Petisca and Patrícia Alves-Oliveira and Raquel Oliveira and Samuel Mascarenhas and Iolanda Leite and Francisco S. Melo and Ana Paiva, IEEE International Symposium on Robot and Human Interactive Communication Workshop on Groups in Human-Robot Interaction, August, 2017
Groups of humans and robots: Understanding membership preferences and team formation


Filipa Correia and Sofia Petisca and Patrícia Alves-Oliveira and Tiago Ribeiro and Francisco S. Melo and Ana Paiva, Robotics: Science and Systems 2017 (RSS), July, 2017
Me and You Together: Movement Impact in Multi-user Collaboration Tasks


Miguel Faria and Rui Silva and Patricia Alves-Oliveira and Francisco S. Melo and Ana Paiva, , 2017
Online Learning for Conversational Agents


Vânia Mendonça and Francisco S. Melo and Luísa Coheur and Alberto Sardinha, Progress in Artificial Intelligence, pg. 739--750, Springer International Publishing, 2017
“ Me and You Together ” Movement Impact in Multi-user Collaboration Tasks


Miguel Faria and Rui Silva and Patricia Alves-Oliveira and Francisco S. Melo and Ana Paiva, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pg. 2793--2798, IEEE, Vancouver, BC, Canada, September, 2017
2016
Ad hoc teamwork by learning teammates' task

Francisco S. Melo and Alberto Sardinha, Autonomous Agents and Multi-Agent Systems, Vol. 30, No. 2, pg. 175--219, Mar, 2016
Ad Hoc Teamwork by Learning Teammates' Task


Francisco S. Melo and Alberto Sardinha, Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, pg. 577--578, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 2016
Dynamics of Fairness in Groups of Autonomous Learning Agents


Fernando P. Santos and Francisco C. Santos and Francisco S. Melo and Ana Paiva and Jorge M. Pacheco, Autonomous Agents and Multiagent Systems, AAMAS 2016 Workshops Best Papers, pg. 107-126, Vol. 10002, Springer International Publishing, 2016
Building a social robot as a game companion in a card game


Filipa Correia and Tiago Ribeiro and Patrícia Alves-Oliveira and Francisco S. Melo and Ana Paiva, The Eleventh ACM/IEEE International Conference on Human Robot Interation Extended Abstracts - Videos, pg. 563--563, 2016
Discovering Social Interaction Strategies for Robots from Restricted-Perception Wizard-of-Oz Studies


Pedro Sequeira and Patrícia Alves-Oliveira and Tiago Ribeiro and Eugenio Di Tullio and Sofia Petisca and Francisco S. Melo and Ginevra Castellano and Ana Paiva, , pg. 197--204, 2016
Just follow the suit! Trust in Human-Robot Interactions during Card Game Playing


Filipa Correia and Patrícia Alves-Oliveira and Nuno Maia and Tiago Ribeiro and Sofia Petisca and Francisco S. Melo and Ana Paiva, International Symposium on Robot and Human Interactive Communication (RO-MAN), pg. 507-512, August, 2016
Learning to be Fair in Multiplayer Ultimatum Games (Extended Abstract)


Fernando P. Santos and Francisco C. Santos and Francisco S. Melo and Ana Paiva and Jorge M. Pacheco, Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems , pg. 1381-1382 , IFAAMAS, 2016
Multiplayer ultimatum game in populations of autonomous agents


Fernando P. Santos and Francisco C. Santos and Francisco S. Melo and Ana Paiva and Jorge M. Pacheco, Proceedings of Adaptive and Learning Agents Workshop (ALA 2016), part of 2016 Int. Conf. Autonomous Agents and Multiagent Systems (AAMAS 2016), (Best paper award), 2016
Rapidly-Exploring Random Tree approach for Geometry Friends


Rui Miguel Barata Castanho Soares and Francisco Leal and Rui Prada and Francisco S. Melo, Proceedings of the First International Joint Conference of DIGRA and FDG, 2016
Synthesizing Robotic Handwriting Motion by Learning from Human Demonstrations


Hang Yin and Patrícia Alves-Oliveira and Francisco S. Melo and Aude Billard and Ana Paiva, Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), July 09-15, 2016
2015
"It's amazing, We Are All Feeling it!" Emotional Climate as a Group-Level Emotional Expression in HRI
Patrícia Alves-Oliveira and Pedro Sequeira and Eugenio Di Tullio and Sofia Petisca and Carla Guerra and Francisco S. Melo and Ana Paiva, AAAI Fall Symposium, 2015
A Reinforcement Learning Approach for the Circle Agent of Geometry Friends


João Quitério and Rui Prada and Francisco S. Melo, CIG’2015 - IEEE Conference on Computational Intelligence and Games, pg. 423-430, IEEE Computer Society, Tainan, Taiwan, August, 2015
An Empathic Robotic Tutor for School Classrooms: Considering Expectation and Satisfaction of Children as End-Users


Patrícia Alves-Oliveira and Tiago Ribeiro and Sofia Petisca and Eugenio Di Tullio and Francisco S. Melo and Ana Paiva, In Social Robotics, pg. 21-30, Springer International Publishing, 2015
The Development of Cooperation in Evolving Populations through Social Importance


Pedro Sequeira and Samuel Mascarenhas and Francisco S. Melo and Ana Paiva, Proceedings of the 5th International Conference on Development and Learning and on Epigenetic Robotics, pg. 308--313, IEEE Computer Society, Providence, Rhode Island, Aug, 2015
The Geometry Friends Game AI Competition


Rui Prada and Phil Lopes and João Catarino and João Quitério and Francisco S. Melo, CIG’2015 - IEEE Conference on Computational Intelligence and Games, pg. 431-438, IEEE Computer Society, Tainan, Taiwan, August, 2015
The “Favors Game”: A Framework to Study the Emergence of Cooperation through Social Importance ( Extended Abstract )


Pedro Sequeira and Samuel Mascarenhas and Francisco S. Melo and Ana Paiva, Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, pg. 1899--1900, Istanbul, Turkey, May, 2015
“Let's Save Resources!”: A Dynamic, Collaborative AI for a Multiplayer Environmental Awareness Game


Pedro Sequeira and Francisco S. Melo and Ana Paiva, Proceedings of the 2015 IEEE Conference on Computational Intelligence and Games, pg. 399--406, IEEE Computer Society, Tainan, Taiwan, Aug-Sept, 2015
2014
Emergence of emotional appraisal signals in reinforcement learning agents

Pedro Sequeira and Francisco S. Melo and Ana Paiva, Autonomous Agents and Multi-Agent Systems, April, 2014
Learning by Appraising: An Emotion-based Approach for Intrinsic Reward Design


Pedro Sequeira and Francisco S. Melo and Ana Paiva, Adaptive Behavior, Vol. 22, No. 5, pg. 330-349, October, 2014
2013
An Associative State-Space Metric for Learning in Factored MDPs

Pedro Sequeira and Francisco S. Melo and Ana Paiva, Proceedings of the 16th Portuguese Conference on Artificial Intelligence, pg. 163-174, Springer-Verlag, Berlin Heidelberg , September, 2013
Emergence of Emotion-Like Signals in Learning Agents


Pedro Sequeira and Francisco S. Melo and Ana Paiva
Towards agents with human-like decisions under uncertainty


Nuno Marques and Francisco S. Melo and Samuel Mascarenhas and João Dias and Rui Prada and Ana Paiva, Cogsci 2013- The annual Meeting of the Cognitive Science Society, 2013
2012
Associative metric for learning in factored MDPs based on classical conditioning

Pedro Sequeira and Francisco S. Melo and Ana Paiva
Decentralized Multiagent Planning for Balance Control in Smart Grids


Francisco S. Melo and Alberto Sardinha and Stefan Witwicki and Laura M. Ramirez-Elizondo and Matthijs T. J. Spaan, International Workshop on Information Technology for Energy Applications (IT4ENERGY 2012), September, 2012
Exploiting Sparse Interactions for Optimizing Communication in Dec-MDPs


Francisco S. Melo and Matthijs Spaan and Stefan Witwicki, Proceedings of the Workshop on Multiagent Sequential Decision Making Under Uncertainty (MSDM-2012), pg. 40-47, June, 2012
Learning by Appraising: An emotion-based approach for intrinsic reward design


Pedro Sequeira and Francisco S. Melo and Ana Paiva
2011
Emerging social awareness: Exploring intrinsic motivation in multiagent learning

Pedro Sequeira and Francisco S. Melo and Rui Prada and Ana Paiva, Proceedings of the First Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, pg. 1--6, IEEE, 2011
Emotion-Based Intrinsic Motivation for Reinforcement Learning Agents (Best Paper Award)


Pedro Sequeira and Francisco S. Melo and Ana Paiva, Affective Computing and Intelligent Interaction (ACII 2011), pg. 326-336, Springer Berlin / Heidelberg, 2011
2010
Abstraction levels for robotic imitation: Overview and computational approaches
Manuel Lopes and Francisco S. Melo and Luis Montesano and José Santos-Victor, Abstraction Levels for Robotic Imitation: Overview and Computational Approaches, pg. 313-355, Springer, Olivier Sigaud and Jan Peters (Eds.), 2010
A computational model of object affordances

Luis Montesano and Manuel Lopes and Francisco S. Melo and Alexandre Bernardino and José Santos-Victor, Advances in Cognitive Systems, IET Publishers, Samia Nefti-Meziani and John Gray (Eds.), 2010
Analysis of inverse reinforcement learning with perturbed demonstrations


Francisco S. Melo and Manuel Lopes and Ricardo Ferreira, Proceedings of the 19th European Conference on Artificial Intelligence, pg. 349-354, 2010
Approximate planning for decentralized MDPs with sparse interactions (Extended Abstract)


Francisco S. Melo and Manuela Veloso, Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, pg. 1389-1390, 2010
Coordinated learning in multiagent MDPs with infinite state spaces


Francisco S. Melo and M. Isabel Ribeiro, Journal of Autonomous Agents and Multiagent Systems , Vol. 21, No. 3, pg. 321-367, 2010
Heuristic planning for decentralized MDPs with sparse interactions


Francisco S. Melo and Manuela Veloso, Proceedings of the 10th International Symposium on Distributed Autonomous Robotics Systems, 2010
Learning from demonstration using MDP induced metrics


Francisco S. Melo and Manuel Lopes, Proceedings of the 2010 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, pg. 385-401, 2010
Machine optimisation of dynamic gait parameters for bipedal walking


Martin Mason and Peter Gibbons and Manuel Lopes and Francisco S. Melo, Proceedings of the 11th Conference Towards Autonomous Robotic Systems, pg. 154-160, 2010
2009

Manuel Lopes and Francisco S. Melo and Ben Kenward and José Santos-Victor, A computational model of social-learning mechanisms, pg. 467-483, Chap. 6, Vol. 17, Adaptive Behavior, 2009
Active learning for reward estimation in inverse reinforcement learning


Manuel Lopes and Francisco S. Melo and Luis Montesano, Proceedings of the 2009 European Conference on Machine Learning and Practice of Knowledge Discovery in Databases, pg. 31-46, 2009
Learning of coordination: Exploiting sparse interactions in multiagent systems


Francisco S. Melo and Manuela Veloso, Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, pg. 773-780, IFAAMAS, 2009
MODEN: Obstacle-driven elastic network for line-of-sight communication


Francisco S. Melo and Manuela Veloso, Proceedings of the 14th Portuguese Conference on Artificial Intelligence, pg. 254-266, 2009