Machine Learning for Social Interaction Modelling


The field of social human-robot interaction is growing. Understanding how communication between humans (human-human) generalises during human-robot communication is crucial in building fluent and enjoyable interactions with social robots. Everyday, more datasets that feature social interaction between humans and between humans and robots are made freely available online.


In this project we propose to take a data-driven approach to build predictive models of social interactions between humans (HH) ( and between humans and robots (HR) interaction using 3 different datasets. Relevant research questions include:

  • Which multi-modal features can be transferable from HH to HR setups?
  • Are there common features that discriminate human behaviour in HH or HR scenarios (e.g. ‘Do people speak less or slower with robots?’ … )

Goals & Milestones

During this project, the student will:

  • Explore datasets (PinSoRo, MHHRI and P2PSTORY): type of data (video; audio, point cloud), available labels and annotation …
  • Extract relevant features multimodal on each dataset
  • Evaluate predictive models for each dataset (i.e. engagement)
  • Explore transfer learning from one dataset to another

There is also potential to use UNSW’s National Facility for Human-Robot Interaction Research to create a new dataset.


Machine Learning, Human-Robot Interaction


  • Skills: Python, ROS, Git.


Senior Lecturer and ARC DECRA Fellow

My research interests include human-robot interaction, human-compter interaction and intelligent and autonomous systems.