PhD Defense: Human-Robot Interaction in Augmented Virtuality: Perception, Cognition and Action in 360° Video-based Robotic Telepresence System

8 November 2021

Abstract: Telepresence refers to a set of technologies, which enables humans to visit a remote environment (RE) and interact with people there without physical travel. Ideally, users would feel a sense of presence in the RE during the teleoperation tasks. However, current telepresence systems restrict this sensation as well as other relevant user experience severely. For example, the usage of single webcam and desktop-based user interface could only provide users with restricted field of view and limited sense of presence. Moreover, movement control of mobile platforms are often limited to traditional input devices like keyboards or mice, which lacks natural methods to map the movements of human users to motions of the robotic platform.

In order to address these challenges, this thesis proposes a novel solution by integrating 360° cameras and virtual reality (VR) head-mounted displays (HMDs) into telepresence systems and exploiting the human user’s natural walk as the control method to drive the robot in remote environments. In addition, in order to allow a human user to explore a RE, which is significantly larger than the tracking area, redirected walking (RDW) method is applied to compress the human user’s locomotion. Furthermore, we use a mixed reality avatar attached on top of the robot and relevant animations to indicate the robot state, in order to improve the process of human-robot interaction.

With these solutions, we implemented corresponding prototypes and performed a series of user studies to explore the availability and evaluate the performance. The findings of this thesis provide novel insights into the telepresence system design and the interaction between humans and telepresence systems.

To participate in the PhD defense, please contact Jingxin via email. The Zoom link will then be mailed to you shortly before the meeting.

Wednesday, 08 November 2021, 14:00, Online Meeting (zoom)

Speaker: Jingxin Zhang, Informatics Dept., Univ. Hamburg