Machine Learning for Social Interaction Modelling

Context

The field of social human-robot interaction is growing. Understanding how communication between humans (human-human) generalises during human-robot communication is crucial in building fluent and enjoyable interactions with social robots. Everyday, more datasets that feature social interaction between humans and between humans and robots are made freely available online.

MHHRI

In this project we propose to take a data-driven approach to build predictive models of social interactions between humans (HH) ( and between humans and robots (HR) interaction using 3 different datasets. Relevant research questions include:

  • Which multi-modal features can be transferable from HH to HR setups?
  • Are there common features that discriminate human behaviour in HH or HR scenarios (e.g. ‘Do people speak less or slower with robots?’ … )

Goals & Milestones

During this project, the student will:

  • Explore datasets (PinSoRo, MHHRI and P2PSTORY): type of data (video; audio, point cloud), available labels and annotation …
  • Extract relevant features multimodal on each dataset
  • Evaluate predictive models for each dataset (i.e. engagement)
  • Explore transfer learning from one dataset to another

There is also potential to use UNSW’s National Facility for Human-Robot Interaction Research to create a new dataset.

Topics

Machine Learning, Human-Robot Interaction

Prerequisites

  • Skills: Python, ROS, Git.

References

Related