I Spy - An AR Game with a Robot in Home Environment

Context

In the context of the (HURL project)[https://wafa.johal.org/project/hurl/], we investigate various ways to enable novice users to train robots to perform everyday life tasks. In particular, we employ gamification and Augmented Reality (AR) to make the training more engaging for the human trainer.

In this project, your goal will be to implement a game “I spy” to be played with a robot in a home environment. For this game, we would like to use AR with the Hololens 2

Goals & Milestones

During this project, the student will:

  • Learn about Hololens and Augmented Reality development
  • Develop a game such as “I spy” using object recognition models (such as YOLO) for both the human and robot to play
  • Evaluate the game in the robotics lab at UNSW

Topics

Augmented Reality, Object Recognition, Machine Learning, Human-Robot Interaction

Prerequisites or willing to learn

Skills: C#, Python, Git D or above in a machine learning or AI course

References

Related