United States

Understanding human emotions is paramount for effective human-human interactions. As technology advances, social robots are increasingly being developed with the capability to discern and respond to human emotions, with the ultimate aim of providing assistance and companionship. However, existing research on emotion recognition for human-robot interaction predominantly focuses on facial expressions or verbal speech, neglecting other potential mediums of emotional expression. In this study, we shed light on the significance of considering various forms of emotional expression, mainly non-verbal vocalization known as vocal bursts, which have been overlooked in the context of emotion modeling for human-robot interaction. Vocal bursts, characterized by brief and intense vocal utterances, represent a rich source of emotional cues that can significantly enhance the capabilities of social robots in understanding and responding to human emotions. Driven by the increasing interest in vocal bursts within speech and affective computing research, we propose a baseline model for affective vocal burst recognition that can outperform large audio models. The proposed baseline model achieves weighted F1 scores of 0.606, 0.342, and 0.287 on 10, 24, and 30 emotion classes, respectively. Additionally, we identify challenges that must be addressed to enhance affective vocal burst recognition for human-robot interaction. Code available at github.com/rahullabs/Sigh

IEEE RO-MAN 2024

Sigh!!! There Is More Than Just Faces and Verbal Speech to Recognize Emotion in Human-Robot Interaction

affective vocal burst

non-verbal communication

affective computing

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

technical paper

Telepresence robots can enhance communication experiences by providing a sense of physical presence, embodiment and may evoke co-presence. In spite of that, telepresence robots have not made it fully to consumer markets. In this paper, we investigate how different levels of controlling a telepresence robot (teleoperation, shared control, and no control) influence presence. To this aim, we conducted a study (N=45) where participants were evenly distributed to one of the robot control conditions. The task involved navigating an unknown room and listening to stories told by a person co-located with the robot. We collected subjective impressions of presence using the temple presence inventory and performed a thematic content analysis on a post-experiment interview. Our results suggest nuances in perceived presence under different levels of robot control after performing a thematic content analysis. Co-presence can be experienced during teleoperation and shared control, and teleoperation may evoke negative sentiments if it does not provide enough spatial information during navigation. However, our results did not point to significant differences in spatial or social presence. We consider that these findings encourage further discussions on how presence is perceived in robot-mediated communication.

An Exploratory Study on the Impact of Varying Levels of Robot Control on Presence in Robot-Mediated Communication

Looking at state-of-the-art robots, the majority have a white surface. This raises the questions of why robots are designed predominantly white, and how surface color affects the robots social perception. This not only reinforces human stereotypes but also influences the acceptance and usage of robots. Accordingly, this online study with a 2 (black vs. white) x 3 (Pepper robot vs. Temi robot vs. Kuka robot) within-subjects design (N = 100) investigated the effect on racist stereotypical perception and the role of anthropomorphism using implicit (Implicit Association Test) and explicit (Robotic Social Attributes Scale) measures and questionnaires on behavioral intentions. Our results show that robot color influences the implicit perception of robots but rarely affects explicit perception and behavior measures. Moreover, no effects of the robots' anthropomorphism on racist perceptions were found. As more effects of anthropomorphism on our outcome variables were found than for robot color, we assume that robot color activates stereotypes in terms of color rather than race.

All Too White? Effects of Anthropomorphism on the Stereotypical Perception of Robot Color

The rapid advancements of drones have demonstrated the versatility and promising potential of human-swarm systems (HSS) across various domains. However, human performance within these systems may be impaired by factors such as limited domain knowledge and mental stress, often leading to cognitive overload and hindering the efficiency and effectiveness of human-swarm teaming. Consequently, the accurate monitoring of cognitive load levels is crucial for optimizing HSS performance. To address the challenges of existing measurement methods, which are often expensive, time-consuming, or lack real-time capabilities, we propose a Large Language Model driven cognitive load prediction framework. This framework integrates comprehensive task context, domain knowledge, and behavior analysis to provide fast and cost-effective predictions in complex scenarios. By leveraging the capabilities of Large Language Models and employing reinforcement learning to model the cognitive load generation, our framework aims to offer real-time insights into human-related factors causing high cognitive load and predict cognitive levels over time, ultimately enhancing the performance of HSS teaming.

Large Language Model Driven Interactive Learning for Real-Time Cognitive Load Prediction in Human-Swarm Systems

Mobile robots are being introduced to industrial workplaces in roles that require copresence with humans. To develop effective robots that do not negatively impact humans, including their subjective experience and ability to get their work done, we must understand humans needs for working near these robots. In this paper, we examine what human workers need from copresent robots motion during work at a large warehouse. To do so, report and synthesize findings about robot motion from across five studies (e.g., focus group, observation, experiment). Results indicate that workers were most focused on robot movement, including consistency, distance, prioritizing people, and indicating when it sensed people. Researchers and practitioners can use these findings to prioritize what aspects of mobile robots to develop to improve human worker experiences around robots and team efficiency.

But Where Are You Going?! Motion Is What Is Most Important for Real-World Co-Present Mobile Robots

In real-world scenarios, robots come closer to humans in many applications, sharing the same workspace or even manipulating the same objects. To ensure safe and intuitive collaboration, it is crucial to have an accurate knowledge of the human's 3D position in space, which should be estimated with high precision, high frequency and low latency. However, individual sensors such as inertial measurement units (IMUs) or cameras cannot meet all requirements for reliable human pose estimation under conditions such as long operating times, large distances and occlusions. In this study, we highlight the limitations of different visual pose methods and present a fused approach for real-time estimation of the 3D position of the human pelvis using machine learning-based visual pose from a monocular camera and an IMU sensor suit. The multimodal fusion is based on the Invariant Extended Kalman filter (InEKF) on Lie Groups, which fuses drift-free visual poses with high-frequency inertial measurements in a loosely-coupled manner. The evaluation is performed on a recorded dataset of multiple subjects performing various experimental scenarios. The results show that the fused approach can increase the accuracy and robustness of the estimates, taking a step closer towards smooth human-robot collaboration.

Fusion of Inertial Sensor Suit and Monocular Camera for 3D Human Pelvis Pose Estimation

Frailty assessment plays a pivotal role in providing older adults care. However, the current process is time-consuming and only measures patients' completion time for each test. This paper introduces a set of algorithms to be used in robots to autonomously perform frailty assessments. In doing so we aim at reducing therapists' burden and provide additional frailty-related metrics that can enhance the effectiveness of diagnosis. We conducted a pilot study with 22 elderly participants and compared our system's performance with that of medical professionals to assess its precision. The results demonstrate that our approach achieved performances close to that of its human counterpart. This research represents an important step forward in the integration of social robotics in healthcare, offering potential benefits for patient care and clinical decision-making.

Exploring the Potential of a Robot-Assisted Frailty Assessment System for Elderly Care

This study explores the complex dynamics between humans and robots, focusing on how emotional states influence proxemics. We conducted a user study using a standard mobile robot to investigate whether emotions elicited from a loudspeaker, affect human perception of robot proximity. Based on previous research on Human-Human Interaction (HHI), we analysed participants responses to robots displaying different behaviours. Participants observed the robots approach while experiencing positive or negative emotions. Our findings suggest that emotional states induced by external stimuli can affect participants perception of robot proximity. In detail, the results indicate that while comfortable stopping distances were unaffected by participants emotional state, individuals who experienced positive emotions judged the same proxemics distance used while performing an avoidance behaviour to be more acceptable compared to the case of negative emotions. This study describes the extent to which our emotions can alter the perception of robot behaviours, ultimately affecting our acceptance of these novel social agents.

Are Emotions Important? a Study on Social Distances for Path Planning Based on Emotions

The development of social robots with advanced conversational capabilities aiming to engage humans in natural interactions has recently surged. This paper investigates the dynamic aspect of Human-Robot Interaction (HRI), focusing on the regulation of interpersonal distance based on human emotion. Through a user study with a humanoid robot, we explore how participants perceive and respond to rule-based versus randomly generated robot behaviours in adjusting interpersonal space during an unconstrained conversation. Results suggest that participants perceive the robot using rule-based behaviours as more socially competent and adaptable to human behaviour and emotions compared to the random ones. These findings highlight the importance of considering subtle nonverbal cues and adapting robot behaviour based on human emotions to improve the quality of HRI, and consequently facilitate the successful integration of human natural nuances in robots.

Too Close to You? a Study on Emotion-Adapted Proxemics Behaviours

Complex manipulation tasks often require robots with complementary capabilities to collaborate. We introduce a benchmark for LanguagE-Conditioned Multi-robot MAnipulation (LEMMA) focused on task allocation and long-horizon object manipulation based on human language instructions in a tabletop setting. LEMMA features 8 types of procedurally generated tasks with varying degree of complexity, some of which require the robots to use tools and pass tools to each other. For each task, we provide 800 expert demonstrations and human instructions for training and evaluations. LEMMA poses greater challenges compared to existing benchmarks, as it requires the system to identify each manipulators limitations and assign sub-tasks accordingly while also handling strong temporal dependencies in each task. To address these challenges, we propose a modular hierarchical planning approach as a baseline. Our results highlight the potential of LEMMA for developing future language-conditioned multi-robot systems.

LEMMA: Learning Language-Conditioned Multi-Robot Manipulation

Recent advancements in medical science and technology have led to a remarkable increase in the lifespan of the global elderly population. However, this demographic often struggles with mobility issues, leading to a sedentary lifestyle fueled by concerns over their physical capabilities. Traditional treadmill gait training, although beneficial, often becomes monotonous and lacks the real-world feedback necessary for engaging and effective rehabilitation. Addressing this gap, research into virtual walking systems utilizing avatar robots has gained traction. Despite the progress, several challenges remain where the systems prioritize visual feedback without considering the crucial need for alerting users to potential dangers and obstacles. This lack of comprehensive environmental awareness and feedback undermines both user engagement and safety. To address this problem, this paper proposes an algorithm that employs B-splines for precise free space detection, integrated with a safety stop mechanism for an avatar robot. This novel approach enhances user awareness of their surroundings through a sophisticated Graphic User Interface (GUI) that leverages Augmented Reality (AR) technology. By superimposing free space boundaries and warning messages directly onto a real-time camera feed, the system provides an intuitive and immersive navigation aid. The efficacy of our proposed GUI was rigorously tested across a series of realistic scenarios, comparing teleoperation control performance with and without the augmented interface. Our findings reveal that our GUI markedly enhances user safety and navigational effectiveness, fostering a deeper understanding of and interaction with the surrounding environment, thereby redefining user experience in virtual mobility assistance.

Premium content

Downloads

Next from IEEE RO-MAN 2024

An Exploratory Study on the Impact of Varying Levels of Robot Control on Presence in Robot-Mediated Communication

Similar lecture

A Speech and Facial Information Based Emotion Recognition System of Collaborative Robot for Empathic Human-Robot Collaboration