Project overview:

The project aims to provide innovative speech pathology services for children with Autism Spectrum Disorder (ASD) and their families in rural areas. The Kinect sensor plays a key role, being used as a novel remote feedback and assessment tool for the quality of parent-child interactions.

This project is developing an automated tracking and analysis system. The software provides meaningful statistics based on the quality of the parent-child interaction with a prototype dashboard display developed that takes the output of the Kinect sensor and displays both real-time and cumulative measurements alongside avatar skeleton figures. Measurements include head-height offset, proximity, number and position of touches, voice recognition, real-time static pose recognition, as well as a rudimentary overall ‘Q’ factor for the session.

It is envisaged that as the speech-pathology intervention progresses the automatically generated quality factors from the toolbox will show a marked improvement. This would also independently validate the intervention process. The toolbox could be used in future iterations as an ‘expert system’ that would provide speech-pathology support in areas that are typically underserved.

Project team

Zaher Joukhadar, Ken Clarke, Robyn Granett, Tricia Eadie, and Bronwyn Davidson

Technologies:

• C#
• C++
• Machine Learning
• Image Processing

My role in the project:

I led the entire technical and software development side of the project. I developed machine learning algorithms to detect and recognise the child and caregiver movements. I build the software using C# programming language.