The Nose Knows: Using Thermal Imaging to Approximate Children's Engagement with Robots

Abstract

We explore the integration of a visual and thermal camera to approximate physiological changes as markers of cognitive load and child's engagement with a robot. The aim of our data pipeline is to enable non-invasive engagement tracking for a desktop social robot developed by Honda Research Institute named Haru. From utilizing these two cameras we can recognize engagement during child-robot interactions (CRI) using changes in nose-tip temperature. We tested our algorithm on data collected while a child participant interacted with Haru during a passive activity as well as an active activity. Then, we did a preliminary modeling of engagement with Hidden Markov models. This paper describes our experimental setup, our data collection, multi-modal pipeline, and some preliminary results from modeling the data.