Find Your Voice: Use of Voice Assistant for Learning

FYV

Context

Many children struggle to find their voice in social situations. They may be shy, suffer from social anxiety, be new to a culture (migrants) or have impairments in communication due to atypical development (e.g., autism spectrum disorder, speech or hearing disorders).

The voices of these children often go unheard, as they find it hard to contribute to a conversation.

The Find your Voice (FyV, http://wafa.johal.org/project/fyv/) project was initiated to investigate how joke telling could help children to speak up and gain confidence. We are also interested in story telling and general conversation. Improvements in communication can have a significant impact in confidence, and help children:

  • reduce stress
  • improve self-confidence
  • ease social interactions
  • make friends more easily
  • improving literacy and language

To help children develop the ability to communicate, tell jokes or stories to their peers, we propose leveraging social robots (e.g. NAO) and voice assistants (e.g., Alexa, Olly and Google Home) to:

  • Model how to tell jokes/stories and respond to other children during conversations .
  • Practice joke/story telling with a ‘friendly’ and ‘non-judgmental’ audience.
  • Practice turn taking during conversation.
  • TLearn jokes, stories and interesting facts to tell other children.

The overall goals of the project are:

  • To enable children to improve their social communication skills by learning intonation and timing, through interacting with voice assistants
  • To learn to how to perform in front of peers and family
  • To make children more confident in social situations

The FyV project involves partners in London and California.

Goals & Milestones

At UNSW, our main goal will be to develop a ‘Learning by Teaching’ application using a robot or voice assistant. This application will allow the user to teach a virtual agent (robot or voice assistant) a joke/story. As the agent learns by demonstration, the user can practice and refine how the story/joke is told until the voice assistant (and the child) is able to tell the joke/story in a satisfactory way.

  • Design the Learning Scenario
  • Explore TTS software for speech conversion
  • Implement a new Alexa Skill
  • Run a Pilot demonstrating the learning of joke/story telling features (e.g. pauses and intonations)

Topics

Voice Assistant, Machine Learning, HCI

Prerequisites

  • Skills: Python or C++. Git.

References

Related