Project overview:

Physiotherapy students often struggle to translate anatomical knowledge from textbooks into a dynamic understanding of the mechanics of body movements in real life patients. Augmented Studio is an augmented reality system that uses body tracking to project anatomical structures and annotations over moving bodies for physiotherapy education. Augmented Studio enables augmentation through projection mapping to display anatomical information such as muscles and skeleton in real time on the body as it moves. We created a technique for annotation to create projected hand-drawing on the moving body, to enable explicit communication of the teacher’s clinical reasoning strategies to the students. Augmented Studio helped facilitating a more engaging learning and teaching experience and increased communication between teacher and students.

Augmented Studio uses several projectors, Microsoft Kinect to track the human body movements, and Microsoft RoomAlive Toolkit for spatial mapping. We combine spatial mapping information with Kinect body tracking to transform images projected by the projectors so they correctly align in real time on the moving human body.

Project team

Thuong Hoang, Martin Reinoso, Zaher Joukhadar, Frank Vetere, and David Kelly

Technologies:

• C#
• C++
• Machine Learning
• Image Processing

My role in the project:

I developed the first prototype using Microsoft RoomAlive, Microsoft Kinect, and Unity3D