Integrating multimodal signals in human-robot interaction. Marta Romeo, Heriot-Watt University

Session 1: Conversational HRI
Chair: Laurent Besacier, NAVER LABS Europe

Title: Integrating multimodal signals in human-robot interaction: progress and future directions.

Effective communication in human interactions is facilitated by our ability to seamlessly integrate various non-verbal signals with spoken language. These signals help manage interactions, such as indicating when to take turns, and enhance our understanding of social aspects like emotions and intentions. This capacity to perceive, comprehend, and integrate a wide range of signals is a key trait that defines us as social beings. Therefore, when developing social robots intended for integration into our social environments, it is crucial to equip them with similar capabilities. Over the years, human-robot interaction has utilised techniques from linguistics, natural language processing, computer vision, and social signal processing to try and achieve comparable levels of integration in robotic companions. In this talk, we will explore the progress made in this field, particularly focusing on multi-party interactions, considering human factors and advancements in AI solutions.