PhD Defense: Multimodal Social Cue Integration for Attention Modeling and Robot Gaze Control

16 December 2024

PhD Colloquium by Fares Abawi, Informatics Dept., University of Hamburg

Abstract:

Social cues influence our attention leading to shifts in our gaze. In this thesis, we integrate non-verbal head-related social cues such as gaze direction, facial expressions, and localized sound into saliency models. We deploy these models on robots, enabling gaze control during social interactions. Using neural early fusion and late integration, we enhance audiovisual saliency prediction models. We then extend these models to predict personalized scanpath patterns. Finally, we introduce a distributed framework for deploying and evaluating our models on robotic platforms, allowing us to design robots with humanlike gaze for conducting human-robot interaction studies and cognitive robotic simulations.

The presentation will be given in English.

Date and time: 16 December 2024, 12.00 p.m.
Seminar Room B-201, Informatikum, Vogt-Kölln-Str. 30, 22527 Hamburg.