HappyFeet: Recognizing and Assessing Dance on the Floor

The widespread availability of Internet-of-Thing (IoT) devices, wearable sensors and smart watches have been promoting innovative activity recognition applications in our everyday lives. Recognizing dance steps with fine granularity using wearables is one of those exciting applications. In a typical dance classroom scenario where the instructors are frequently outnumbered by the students, accelerometer sensors can be utilized to automatically compare the performance of the dancers and provide informative feedback to all the stakeholders, for example, the instructors and the learners. However, owing to the complexity of the movement kinematics of human body, building a sufficiently accurate and reliable system can be a daunting task. Utilization of multiple sensors can help improve the reliability, however most wearable sensors do not boast sufficient resolution for such tasks and often suffer from various data sampling, device heterogeneity and instability issues. To address these challenges, we introduce \emph{HappyFeet}, a convolutional neural network based deep, self-evolving feature learning model that accurately recognizes the micro steps of various dance activities. We show that our model consistently outperforms feature engineering based shallow learning approaches by a margin (approximately 7%) accuracy on data collected from dance routines (Indian classical) performed by a professional dancer. We also posit a Body Sensor Network model and discuss the underpinning challenges and possible solutions associated with multiple sensors’ signal variations.

HotMobile ‘18: Proceedings of the 19th International Workshop on Mobile Computing Systems & Applications