Attentional Mechanism Based on a Microphone Array for Embedded Devices and a Single Camera

Abstract

This work presents an attentional mechanism with the capability of detecting the localization of a speaker for interaction purposes, based on audio and video information. The localization is computed in terms of azimuth and elevation angles, to be used as input values for controlling mobile systems such as a pan-tilt videocamera or a robotic head. For this purpose the SRP-PHAT algorithm has been implemented with a commercial array of microphones for embedded devices, in order to estimate the localization of a sound source in the surroundings of the array. In order to improve the limitations of the SRP-PHAT algorithm in the estimation of the z coordinate, the elevation angle is corrected via video information by using Haar cascade classifiers for face detection. Simulations and experiments show the accuracy of the system, as well as the application for controlling a pan-tilt videocamera in a real scenario with speakers and ambient noise.