Francisco S Melo

Scopus Publications

Optimizing 2D Packing Strategies for Autoclave Loading Using Deep Reinforcement Learning
Victor U. Pugliese, Diogo S. Carvalho, Oseias F. Ferreira, Fabio A. Faria, Francisco S. Melo
Lecture Notes in Computer Science, 2026
Psychological, economic, and ethical factors in human feedback for a chatbot-based smoking cessation intervention
Nele Albers, Francisco S. Melo, Mark A. Neerincx, Olya Kudina, Willem-Paul Brinkman
Npj Digital Medicine, 2025
Integrating human support with chatbot-based behavior change interventions raises three challenges: (1) attuning the support to an individual’s state (e.g., motivation) for enhanced engagement, (2) limiting the use of the concerning human resources for enhanced efficiency, and (3) optimizing outcomes on ethical aspects (e.g., fairness). Therefore, we conducted a study in which 679 smokers and vapers had a 20% chance of receiving human feedback between five chatbot sessions. We find that having received feedback increases retention and effort spent on preparatory activities. However, analyzing a reinforcement learning (RL) model fit on the data shows there are also states where not providing feedback is better. Even this “standard” benefit-maximizing RL model is value-laden. It not only prioritizes people who would benefit most, but also those who are already doing well and want feedback. We show how four other ethical principles can be incorporated to favor other smoker subgroups, yet, interdependencies exist.
Centralized training with hybrid execution in multi-agent reinforcement learning via predictive observation imputation
Pedro P. Santos, Diogo S. Carvalho, Miguel Vasco, Alberto Sardinha, Pedro A. Santos, Ana Paiva, Francisco S. Melo
Artificial Intelligence, 2025
We study hybrid execution in multi-agent reinforcement learning (MARL), a paradigm where agents aim to complete cooperative tasks with arbitrary communication levels at execution time by taking advantage of information-sharing among the agents. Under hybrid execution, the communication level can range from a setting in which no communication is allowed between agents (fully decentralized), to a setting featuring full communication (fully centralized), but the agents do not know beforehand which communication level they will encounter at execution time. We contribute MARO, an approach that makes use of an auto-regressive predictive model, trained in a centralized manner, to estimate missing agents' observations at execution time. We evaluate MARO on standard scenarios and extensions of previous benchmarks tailored to emphasize the impact of partial observability in MARL. Experimental results show that our method consistently outperforms relevant baselines, allowing agents to act with faulty communication while successfully exploiting shared information.
Regularization and Two Time Scales for Convergence of Reinforcement Learning
Diogo S. Carvalho, Pedro A. Santos, Francisco S. Melo
Applied Mathematics and Optimization, 2025
Reinforcement learning algorithms aim at solving discrete time stochastic control problems with unknown underlying dynamical systems by an iterative process of interaction. The process is formalized as a Markov decision process, where at each time step, a control action is given, the system provides a reward, and the state changes stochastically. The objective of the controller is the expected sum of rewards obtained throughout the interaction. When the set of states and or actions is large, it is necessary to use some form of function approximation. But even if the function approximation set is simply a linear span of fixed features, the reinforcement learning algorithms may diverge. In this work, we propose and analyze regularized two-time-scale variations of the algorithms, and prove that they are guaranteed to converge almost-surely to a unique solution to the reinforcement learning problem.
Reinforcement learning in convergently non-stationary environments: Feudal hierarchies and learned representations
Diogo S. Carvalho, Pedro A. Santos, Francisco S. Melo
Artificial Intelligence, 2025
We study the convergence of Q -learning-based methods in convergently non-stationary environments, particularly in the context of hierarchical reinforcement learning and of dynamic features encountered in deep reinforcement learning. We demonstrate that Q -learning achieves convergence in tabular representations when applied to convergently non-stationary dynamics, such as the ones arising in a feudal hierarchical setting. Additionally, we establish convergence for Q -learning-based deep reinforcement learning methods with convergently non-stationary features, such as the ones arising in representation-based settings. Our findings offer theoretical support for the application of Q -learning in these complex scenarios and present methodologies for extending established theoretical results from standard cases to their convergently non-stationary counterparts.
Optimize and Coordinate Multiple DMPs under Constraints to Achieve a Collaborative Manipulation Task
Ali H. Kordia, Francisco S. Melo
Proceedings IEEE International Conference on Robotics and Automation, 2025
This paper addresses a significant challenge in achieving collaborative tasks; how can a robot or multiple robots, endowed with a library of pre-learned primitive movements, generate multiple simultaneous coordinated robotic movements, adapting and optimizing those in the library, to complete one collaborative task? This work can thus be seen as a follow-up to the work with a motion presented as dynamic movement primitive (DMP) that now considers collaborative tasks and the existence of multiple robots/manipulators. Specifically, we start with a simple task using one DMP and extend it to accommodate the coordinated execution of multiple DMPs in robots with multiple manipulators or-alternatively-multiple robots with a single manipulator. We investigate mechanisms to jointly optimize multiple DMPs to perform one task in a coordinated fashion. The joint trajectory is built from initial DMPs learned for a single manipulator, and its optimization must comply with task-specific constraints. We illustrate the application of our approach both in a simulated environment and in a simulated and real Baxter robot.
Networked Agents in the Dark: Team Value Learning under Partial Observability
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2025
A Comparative Study of Continual Backpropagation
Jacopo Silvestrin, Francisco S. Melo, Manuel Lopes
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2025
The Number of Trials Matters in Infinite-Horizon General-Utility Markov Decision Processes
Proceedings of Machine Learning Research, 2025
Distributed Value Decomposition Networks with Networked Agents: Extended Abstract
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2025
Implicit Repair with Reinforcement Learning in Emergent Communication
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2025
Preface
Steven Davy, Danyal Aftab
Frontiers in Artificial Intelligence and Applications, 2024
The impact of data distribution on Q-learning with function approximation
Pedro P. Santos, Diogo S. Carvalho, Alberto Sardinha, Francisco S. Melo
Machine Learning, 2024
When a Robot Is Your Teammate
Filipa Correia, Francisco S. Melo, Ana Paiva
Topics in Cognitive Science, 2024
HOTSPOT: An ad hoc teamwork platform for mixed human-robot teams
João G. Ribeiro, Luis Müller Henriques, Sérgio Colcher, Julio Cesar Duarte, Francisco S. Melo, Ruy Luiz Milidiú, Alberto Sardinha
Plos One, 2024
“Guess what I'm doing”: Extending legibility to sequential decision tasks
Miguel Faria, Francisco S. Melo, Ana Paiva
Artificial Intelligence, 2024
Interactively Teaching an Inverse Reinforcement Learner with Limited Feedback
Rustam Zayanov, Francisco Melo, Manuel Lopes
International Conference on Agents and Artificial Intelligence, 2024
NeuralSolver: Learning Algorithms For Consistent and Efficient Extrapolation Across General Tasks
Advances in Neural Information Processing Systems, 2024
Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2024
TEAMSTER: Model-based reinforcement learning for ad hoc teamwork
João G. Ribeiro, Gonçalo Rodrigues, Alberto Sardinha, Francisco S. Melo
Artificial Intelligence, 2023
Theoretical Remarks on Feudal Hierarchies and Reinforcement Learning
Diogo S. Carvalho, Francisco S. Melo, Pedro A. Santos
Frontiers in Artificial Intelligence and Applications, 2023
Making Friends in the Dark: Ad Hoc Teamwork Under Partial Observability
João G. Ribeiro, Cassandro Martinho, Alberto Sardinha, Francisco S. Melo
Frontiers in Artificial Intelligence and Applications, 2023
Pre-training with Augmentations for Efficient Transfer in Model-Based Reinforcement Learning
Bernardo Esteves, Miguel Vasco, Francisco S. Melo
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2023
Learning to Perceive in Deep Model-Free Reinforcement Learning
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2023
Robotic Gaze Responsiveness in Multiparty Teamwork
Filipa Correia, Joana Campos, Francisco S. Melo, Ana Paiva
International Journal of Social Robotics, 2023
“Sequencing Matters”: Investigating Suitable Action Sequences in Robot-Assisted Autism Therapy
Kim Baraka, Marta Couto, Francisco S. Melo, Ana Paiva, Manuela Veloso
Frontiers in Robotics and AI, 2022
Leveraging hierarchy in multimodal generative models for effective cross-modality inference
Miguel Vasco, Hang Yin, Francisco S. Melo, Ana Paiva
Neural Networks, 2022
Preface
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2022
Socially Reactive Navigation Models for Mobile Robots
Francisco Melo, Plinio Moreno
2022 IEEE International Conference on Autonomous Robot Systems and Competitions Icarsc 2022, 2022
FIT: Using Feature Importance to Teach Classification Tasks to Unknown Learners
Carla Guerra, Francisco S. Melo, Manuel Lopes
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2022
Cooperation and Learning Dynamics under Wealth Inequality and Diversity in Individual Risk Perception
Ramona Merhej, Fernando P. Santos, Francisco S. Melo, Francisco C. Santos
Journal of Artificial Intelligence Research, 2022
Perceive, Represent, Generate: Translating Multimodal Information to Robotic Motion Trajectories
Fabio Vital, Miguel Vasco, Alberto Sardinha, Francisco Melo
IEEE International Conference on Intelligent Robots and Systems, 2022
How to Sense the World: Leveraging Hierarchy in Multimodal Perception for Robust Reinforcement Learning Agents
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2022
Cooperation and Learning Dynamics under Risk Diversity and Financial Incentives
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2022
Geometric Multimodal Contrastive Representation Learning
Proceedings of Machine Learning Research, 2022
A Game AI Competition to Foster Collaborative AI Research and Development
Ana Salta, Rui Prada, Francisco S. Melo
IEEE Transactions on Games, 2021
Teaching Multiple Inverse Reinforcement Learners
Francisco S. Melo, Manuel Lopes
Frontiers in Artificial Intelligence, 2021
Understanding robots: Making robots more legible in multi-party interactions
Miguel Faria, Francisco S. Melo, Ana Paiva
2021 30th IEEE International Conference on Robot and Human Interactive Communication Ro Man 2021, 2021
E-covig: A novel mhealth system for remote monitoring of symptoms in covid-19
Afonso Raposo, Luis Marques, Rafael Correia, Francisco Melo, João Valente, Telmo Pereira, Luis Brás Rosário, Filipe Froes, João Sanches, Hugo Plácido da Silva
Sensors, 2021
Movement Recognition and Prediction Using DMPs
Ali H. Kordia, Francisco S. Melo
Proceedings IEEE International Conference on Robotics and Automation, 2021
Revisiting “Recurrent World Models Facilitate Policy Evolution”
Bernardo Esteves, Francisco S. Melo
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2021
Compound Movement Recognition Using Dynamic Movement Primitives
Ali H. Kordia, Francisco S. Melo
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2021
Teaching unknown learners to classify via feature importance
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2021
Interactive Teaching with Groups of Unknown Bayesian Learners
Carla Guerra, Francisco S. Melo, Manuel Lopes
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2021
Cooperation between independent reinforcement learners under wealth inequality and collective risks
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2021
Helping People on the Fly: Ad Hoc Teamwork for Human-Robot Teams
João G. Ribeiro, Miguel Faria, Alberto Sardinha, Francisco S. Melo
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2021
Ad Hoc Teamwork in the Presence of Non-stationary Teammates
Pedro M. Santos, João G. Ribeiro, Alberto Sardinha, Francisco S. Melo
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2021
Preface
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2021
Exploiting Symmetry in Human Robot-Assisted Dressing Using Reinforcement Learning
Pedro Ildefonso, Pedro Remédios, Rui Silva, Miguel Vasco, Francisco S. Melo, Ana Paiva, Manuela Veloso
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2021
An End-to-end Approach for Learning and Generating Complex Robot Motions from Demonstration
Ali H. Kordia, Francisco S. Melo
16th IEEE International Conference on Control Automation Robotics and Vision Icarcv 2020, 2020
ScientIST: Biomedical Engineering Experiments Supported by Mobile Devices, Cloud and IoT
Joana F. Pinto, Hugo Plácido da Silva, Francisco Melo, Ana Fred
Signals, 2020
Optimal action sequence generation for assistive agents in fixed horizon tasks
Kim Baraka, Francisco S. Melo, Marta Couto, Manuela Veloso
Autonomous Agents and Multi Agent Systems, 2020
A new convergent variant of Q-learning with linear function approximation
Advances in Neural Information Processing Systems, 2020
Emergence of Cooperation in N-Person Dilemmas through Actor-Critic Reinforcement Learning
Ala 2020 Adaptive and Learning Agents Workshop at Aamas 2020, 2020
Playing games in the dark: An approach for cross-modality transfer in reinforcement learning
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2020
The Dark Side of Embodiment Teaming Up With Robots VS Disembodied Agents
Filipa Correia, Samuel Gomes, Samuel Mascarenhas, Francisco S. Melo, Ana Paiva
Robotics Science and Systems, 2020
Explainable Agency by Revealing Suboptimality in Child-Robot Learning Scenarios
Silvia Tulli, Marta Couto, Miguel Vasco, Elmira Yadollahi, Francisco Melo, Ana Paiva
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2020
Learning Multimodal Representations for Sample-efficient Recognition of Human Actions
Miguel Vasco, Francisco S. Melo, David Martins de Matos, Ana Paiva, Tetsunari Inamura
IEEE International Conference on Intelligent Robots and Systems, 2019
Walk the Talk Exploring (Mis)Alignment of Words and Deeds by Robotic Teammates in a Public Goods Game
Filipa Correia, Ana Paiva, Shruti Chandra, Samuel Mascarenhas, Julien Charles-Nicolas, Justin Gally, Diana Lopes, Fernando P. Santos, Francisco C. Santos, Francisco S. Melo
2019 28th IEEE International Conference on Robot and Human Interactive Communication Ro Man 2019, 2019
Project INSIDE: towards autonomous semi-unstructured human–robot social interaction in autism therapy
Francisco S. Melo, Alberto Sardinha, David Belo, Marta Couto, Miguel Faria, Anabela Farias, Hugo Gambôa, Cátia Jesus, Mithun Kinarullathil, Pedro Lima, Luís Luz, André Mateus, Isabel Melo, Plinio Moreno, Daniel Osório, Ana Paiva, Jhielson Pimentel, João Rodrigues, Pedro Sequeira, Rubén Solera-Ureña, Miguel Vasco, Manuela Veloso, Rodrigo Ventura
Artificial Intelligence in Medicine, 2019
An ensemble inverse optimal control approach for robotic task learning and adaptation
Hang Yin, Francisco S. Melo, Ana Paiva, Aude Billard
Autonomous Robots, 2019
Group Intelligence on Social Robots
Filipa Correia, Francisco S. Melo, Ana Paiva
ACM IEEE International Conference on Human Robot Interaction, 2019
Exploring Prosociality in Human-Robot Teams
Filipa Correia, Samuel F. Mascarenhas, Samuel Gomes, Patricia Arriaga, Iolanda Leite, Rui Prada, Francisco S. Melo, Ana Paiva
ACM IEEE International Conference on Human Robot Interaction, 2019
Empathic Robot for Group Learning: A Field Study
Patrícia Alves-Oliveira, Pedro Sequeira, Francisco S. Melo, Ginevra Castellano, Ana Paiva
ACM Transactions on Human Robot Interaction, 2019
Interactive robots with model-based 'autism-like' behaviors: Assessing validity and potential benefits
Kim Baraka, Francisco S. Melo, Manuela Veloso
Paladyn, 2019
“I Choose.. YOU!” Membership preferences in human–robot teams
Filipa Correia, Sofia Petisca, Patrícia Alves-Oliveira, Tiago Ribeiro, Francisco S. Melo, Ana Paiva
Autonomous Robots, 2019
Towards Guidelines for Mental State Induction in First-Person Shooters
Tomas Alves, Sandra Gama, Francisco S. Melo
Proceedings Icgi 2018 International Conference on Graphics and Interaction, 2019
Multi-task learning and catastrophic forgetting in continual reinforcement learning
João Ribeiro, Francisco Melo, João Dias
Epic Series in Computing, 2019
Solving motion and action planning for a cooperative agent problem using geometry friends
Ana Salta, Rui Prada, Francisco Melo
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2019
Room usage optimization in timetabling: A case study at Universidade de Lisboa
Alexandre Lemos, Francisco S. Melo, Pedro T. Monteiro, Inês Lynce
Operations Research Perspectives, 2019
An optimization approach for structured agent-based provider/receiver tasks
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2019
A theoretical and algorithmic analysis of configurable MDPs
Rui Silva, Gabriele Farina, Francisco S. Melo, Manuela Veloso
Proceedings International Conference on Automated Planning and Scheduling Icaps, 2019
Online motion concept learning: A novel algorithm for sample-efficient learning and recognition of human actions
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2019
Effects of agents’ transparency on teamwork
Silvia Tulli, Filipa Correia, Samuel Mascarenhas, Samuel Gomes, Francisco S. Melo, Ana Paiva
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2019
For the record - A public goods game for exploring human-robot collaboration
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2019
Flow adaptation in serious games for health
Tomas Alves, Sandra Gama, Francisco S. Melo
2018 IEEE 6th International Conference on Serious Games and Applications for Health Segah 2018, 2018
Group-based Emotions in Teams of Humans and Robots
Filipa Correia, Samuel Mascarenhas, Rui Prada, Francisco S. Melo, Ana Paiva
ACM IEEE International Conference on Human Robot Interaction, 2018
Interactive optimal teaching with unknown learners
Francisco S. Melo, Carla Guerra, Manuel Lopes
Ijcai International Joint Conference on Artificial Intelligence, 2018
Exploring the impact of fault justification in human-robot trust: Socially Interactive Agents Track
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2018
Adaptive indirect control through communication in collaborative human-robot interaction
Rui Silva, Miguel Faria, Francisco S. Melo, Manuela Veloso
IEEE International Conference on Intelligent Robots and Systems, 2017
"me and you together" movement impact in multi-user collaboration tasks
Miguel Faria, Rui Silva, Patricia Alves-Oliveira, Francisco S. Melo, Ana Paiva
IEEE International Conference on Intelligent Robots and Systems, 2017
Monte Carlo tree search experiments in hearthstone
Andre Santos, Pedro A. Santos, Francisco S. Melo
2017 IEEE Conference on Computational Intelligence and Games Cig 2017, 2017
Autonomous Surveillance Robots: A Decision-Making Framework for Networked Muiltiagent Systems
Stefan Witwicki, Jose Carlos Castillo, Joao Messias, Jesus Capitan, Francisco S. Melo, Pedro U. Lima, Manuela Veloso
IEEE Robotics and Automation Magazine, 2017
Data-driven generation of synthetic behavioral feature vectors modeling children with autism spectrum disorders
Kim Baraka, Francisco S. Melo, Manuela Veloso
7th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics ICDL Epirob 2017, 2017
Simulating behaviors of children with autism spectrum disorders through reversal of the autism diagnosis process
Kim Baraka, Francisco S. Melo, Manuela Veloso
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2017
‘Autistic Robots’ for Embodied Emulation of Behaviors Typically Seen in Children with Different Autism Severities
Kim Baraka, Francisco S. Melo, Manuela Veloso
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2017
Learning and Teaching Biodiversity Through a Storyteller Robot
Maria José Ferreira, Valentina Nisi, Francisco Melo, Ana Paiva
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2017
Online learning for conversational agents
Vânia Mendonça, Francisco S. Melo, Luísa Coheur, Alberto Sardinha
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2017
A conversational agent powered by online learning
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2017
Associate latent encodings in learning from demonstrations
31st Aaai Conference on Artificial Intelligence Aaai 2017, 2017
A social robot as a card game player
Proceedings of the 13th Aaai Conference on Artificial Intelligence and Interactive Digital Entertainment Aiide 2017, 2017
Groups of humans and robots: Understanding membership preferences and team formation
Filipa Correia, Sofia Petisca, Patrícia Alves-Oliveira, Tiago Ribeiro, Francisco Melo, Ana Paiva
Robotics Science and Systems, 2017
Just follow the suit! Trust in human-robot interactions during card game playing
Filipa Correia, Patricia Alves-Oliveira, Nuno Maia, Tiago Ribeiro, Sofia Petisca, Francisco S. Melo, Ana Paiva
25th IEEE International Symposium on Robot and Human Interactive Communication Ro Man 2016, 2016
Building a social robot as a game companion in a card game
Filipa Correia, Tiago Ribeiro, Patricia Alves-Oliveira, Nuno Maia, Francisco S. Melo, Ana Paiva
ACM IEEE International Conference on Human Robot Interaction, 2016
Discovering social interaction strategies for robots from restricted-perception wizard-of-oz studies
Pedro Sequeira, Patricia Alves-Oliveira, Tiago Ribeiro, Eugenio Di Tullio, Sofia Petisca, Francisco S. Melo, Ginevra Castellano, Ana Paiva
ACM IEEE International Conference on Human Robot Interaction, 2016
Ad hoc teamwork by learning teammates’ task
Francisco S. Melo, Alberto Sardinha
Autonomous Agents and Multi Agent Systems, 2016
Ad hoc teamwork by learning teammates' task
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2016
Emergence of emotional appraisal signals in reinforcement learning agents
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2016
Adaptive symbiotic collaboration for targeted complex manipulation tasks
Silva Rui, Melo Francisco S., Veloso Manuela
Frontiers in Artificial Intelligence and Applications, 2016
Rapidly-Exploring Random Tree approach for Geometry Friends
Proceedings of the 1st Joint International Conference of Digital Games Research Association and Foundation of Digital Games Digra Fdg 2016, 2016
Me and you together: A study on collaboration in manipulation tasks
Aaai Fall Symposium Technical Report, 2016
Dynamics of fairness in groups of autonomous learning agents
Fernando P. Santos, Francisco C. Santos, Francisco S. Melo, Ana Paiva, Jorge M. Pacheco
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2016
Synthesizing robotic handwriting motion by learning from human demonstrations
Ijcai International Joint Conference on Artificial Intelligence, 2016
Learning to be fair in multiplayer Ultimatum Games
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2016
An interactive tangram game for children with Autism
Beatriz Bernardo, Patrícia Alves-Oliveira, Maria Graça Santos, Francisco S. Melo, Ana Paiva
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2016
Towards table tennis with a quadrotor autonomous learning robot and onboard vision
Rui Silva, Francisco S. Melo, Manuela Veloso
IEEE International Conference on Intelligent Robots and Systems, 2015
The development of cooperation in evolving populations through social importance
Pedro Sequeira, Samuel Mascarenhas, Francisco S. Melo, Ana Paiva
5th Joint International Conference on Development and Learning and Epigenetic Robotics ICDL Epirob 2015, 2015
A reinforcement learning approach for the circle agent of geometry friends
Joao Quiterio, Rui Prada, Francisco S. Melo
2015 IEEE Conference on Computational Intelligence and Games Cig 2015 Proceedings, 2015
'Let's save resources!': A dynamic, collaborative AI for a multiplayer environmental awareness game
Pedro Sequeira, Francisco S. Melo, Ana Paiva
2015 IEEE Conference on Computational Intelligence and Games Cig 2015 Proceedings, 2015
The geometry friends game AI competition
Rui Prada, Phil Lopes, Joao Catarino, Joao Quiterio, Francisco S. Melo
2015 IEEE Conference on Computational Intelligence and Games Cig 2015 Proceedings, 2015
Emergence of emotional appraisal signals in reinforcement learning agents
Pedro Sequeira, Francisco S. Melo, Ana Paiva
Autonomous Agents and Multi Agent Systems, 2015
Modeling students self-studies behaviors
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2015
The "favors game": A framework to study the emergence of cooperation through social importance (extended abstract)
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2015
An empathic robotic tutor for school classrooms: Considering expectation and satisfaction of children as end-users
Patrícia Alves-Oliveira, Tiago Ribeiro, Sofia Petisca, Eugenio di Tullio, Francisco S. Melo, Ana Paiva
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2015
Personalized assistance for dressing users
Steven D. Klee, Beatriz Quintino Ferreira, Rui Silva, João Paulo Costeira, Francisco S. Melo, Manuela Veloso
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2015
It's amazing, we are all feeling it!" emotional climate as a group-level emotional expression in HRI
Aaai Fall Symposium Technical Report, 2015
The influence of social display in competitive multiagent learning
Pedro Sequeira, Francisco S. Melo, Ana Paiva
IEEE ICDL Epirob 2014 4th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics, 2014
Learning by appraising: an emotion-based approach to intrinsic reward design
Pedro Sequeira, Francisco S Melo, Ana Paiva
Adaptive Behavior, 2014
A testbed for autonomous robot surveillance
13th International Conference on Autonomous Agents and Multiagent Systems Aamas 2014, 2014
A flexible approach to modeling unpredictable events in MDPs
Icaps 2013 Proceedings of the 23rd International Conference on Automated Planning and Scheduling, 2013
An associative state-space metric for learning in factored MDPs
Pedro Sequeira, Francisco S. Melo, Ana Paiva
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2013
Towards agents with human-like decisions under uncertainty
Cooperative Minds Social Interaction and Group Dynamics Proceedings of the 35th Annual Meeting of the Cognitive Science Society Cogsci 2013, 2013
Heuristic planning for decentralized MDPs with sparse interactions
Francisco S. Melo, Manuela Veloso
Springer Tracts in Advanced Robotics, 2012
Decentralized multiagent planning for balance control in smart grids
Ceur Workshop Proceedings, 2012
QueryPOMDP: POMDP-based communication in multiagent systems
Francisco S. Melo, Matthijs T. J. Spaan, Stefan J. Witwicki
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2012
Differential Eligibility Vectors for Advantage Updating and Gradient Methods
Francisco Melo
Proceedings of the 25th Aaai Conference on Artificial Intelligence Aaai 2011, 2011
Decentralized MDPs with sparse interactions
Francisco S. Melo, Manuela Veloso
Artificial Intelligence, 2011
Differential eligibility vectors for advantage updating and gradient methods
Francisco Melo
Proceedings of the National Conference on Artificial Intelligence, 2011
Emotion-based intrinsic motivation for reinforcement learning agents
Pedro Sequeira, Francisco S. Melo, Ana Paiva
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2011
Emerging social awareness: Exploring intrinsic motivation in multiagent learning
Pedro Sequeira, Francisco S. Melo, Rui Prada, Ana Paiva
2011 IEEE International Conference on Development and Learning ICDL 2011, 2011
Map-merging-free connectivity positioning for distributed robot teams
Liemhetcharat Somchaya, Veloso Manuela, Melo Francisco, Borrajo Daniel
Intelligent Autonomous Systems 11 IAS 2010, 2010
Abstraction levels for robotic imitation: Overview and computational approaches
Manuel Lopes, Francisco Melo, Luis Montesano, José Santos-Victor
Studies in Computational Intelligence, 2010
Coordinated learning in multiagent MDPs with infinite state-space
Francisco S. Melo, M. Isabel Ribeiro
Autonomous Agents and Multi Agent Systems, 2010
Learning from demonstration using MDP induced metrics
Francisco S. Melo, Manuel Lopes
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2010
Analysis of inverse reinforcement learning with perturbed demonstrations
Melo Francisco S., Lopes Manuel, Ferreira Ricardo
Frontiers in Artificial Intelligence and Applications, 2010
A computational model of object affordances
Luis Montesano, Manuel Lopes, Francisco Melo, Alexandre Bernardino, Jose Santos-Victor
Advances in Cognitive Systems, 2010
A computational model of social-learning mechanisms
Manuel Lopes, Francisco S. Melo, Ben Kenward, José Santos-Victor
Adaptive Behavior, 2009
Learning of coordination: Exploiting sparse interactions in multiagent systems
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2009
Active learning for reward estimation in inverse reinforcement learning
Manuel Lopes, Francisco Melo, Luis Montesano
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2009
Reinforcement learning with function approximation for cooperative navigation tasks
Francisco S. Melo, M. Isabel Ribeiro
Proceedings IEEE International Conference on Robotics and Automation, 2008
Exploiting locality of interactions using a policy-gradient approach in multiagent learning
Melo Francisco S.
Frontiers in Artificial Intelligence and Applications, 2008
Fitted natural actor-critic: A new algorithm for continuous state-action MDPs
Francisco S. Melo, Manuel Lopes
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2008
Interaction-driven Markov games for decentralized multiagent planning under uncertainty
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2008
Emerging coordination in infinite team Markov games
Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems Aamas, 2008
An analysis of reinforcement learning with function approximation
Francisco S. Melo, Sean P. Meyn, M. Isabel Ribeiro
Proceedings of the 25th International Conference on Machine Learning, 2008
Affordance-based imitation learning in robots
Manuel Lopes, Francisco S. Melo, Luis Montesano
IEEE International Conference on Intelligent Robots and Systems, 2007
A unified framework for imitation-like behaviors
Aisb 07 Artificial and Ambient Intelligence, 2007
Q-learning with linear function approximation
Francisco S. Melo, M. Isabel Ribeiro
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2007
Convergence of Q-learning with linear function approximation
Francisco S. Melo, M. Isabel Ribeiro
2007 European Control Conference Ecc 2007, 2007
Learning to coordinate in topological navigation tasks
Francisco S. Melo, Isabel Ribeiro
IFAC Proceedings Volumes IFAC Papersonline, 2007
Convergence of independent adaptive learners
Francisco S. Melo, Manuel C. Lopes
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2007
Transition entropy in partially observable Markov decision processes
Intelligent Autonomous Systems 9 IAS 2006, 2006
Navigation controllability of a mobile robot population
Francisco A. Melo, M. Isabel Ribeiro, Pedro Lima
Lecture Notes in Artificial Intelligence Subseries of Lecture Notes in Computer Science, 2005

Francisco S Melo

RESEARCH INTERESTS

Scopus Publications

RECENT SCHOLAR PUBLICATIONS

MOST CITED SCHOLAR PUBLICATIONS