TeleOp Game - Explore Gamification of Robot Teleoperation


Service robots are becoming more common in households. Tasks that they may be assigned include vacuuming the floor or grabbing objects from the top shelf. Augmented reality (AR) is one of the many interfaces the robot and the human user may communicate through. During teleoperation or robot training, human disengagement can be an issue (the task demonstrated by the human often repetitive).

As a preliminary step, we aim to explore how gamification can affect human engagement in an AR game setting where the objective of the game is to complete a common household task (such as clearing the table). The AR application will be developed for the Microsoft HoloLens 2 headset. The game can be thought of as encouraging the user to complete the task.

The student will in particular explore how game mechanics and appealing visuals could be inserted through AR to engage the user in repetitive tasks.

Goals & Milestones:

  • Design the game
  • Implement teleoperation module for the Kinova Gen 3 Robot.
  • Implement gamification elements
  • Evaluate the impact of the gamification elements via a user study


Augmented Reality, Games, Gamification, Robot Teleoperation

Prerequisites and Skills (demonstrated or willing to learn):

Python, C#, Unity Framework, ROS (Robotics OS) Game Design, Computer Vision