United States

This article investigates mixed reality (MR) to enhance human-robot collaboration (HRC). The proposed solution adopts MR as a communication layer to convey a mobile manipulator&#39;s intentions and upcoming actions to the humans with whom it interacts, thus improving their collaboration. A user study involving 20 volunteers demonstrated the effectiveness of this MR-focused approach in facilitating collaborative tasks, with a positive effect on overall collaboration performances and human satisfaction.

IEEE RO-MAN 2024

Investigating Mixed Reality for Communication between Humans and Mobile Manipulators

communication channels

mixed reality

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

This article investigates mixed reality (MR) to enhance human-robot collaboration (HRC). The proposed solution adopts MR as a communication layer to convey a mobile manipulator's intentions and upcoming actions to the humans with whom it interacts, thus improving their collaboration. A user study involving 20 volunteers demonstrated the effectiveness of this MR-focused approach in facilitating collaborative tasks, with a positive effect on overall collaboration performances and human satisfaction.

technical paper

The growing implementation of robots in societal contexts necessitates a deeper exploration of the dynamics of trust between humans and robots. This exploration should expand beyond traditional viewpoints that primarily emphasize the influence of robot performance. In the burgeoning area of social robotics, fine-tuning a robot's personality traits is increasingly recognized as a crucial element in shaping users' experiences during human-robot interaction (HRI). Research in this field has led to the creation of trust scales that encompass various trust dimensions in HRI. These scales include aspects related to performance as well as moral dimensions. Our previous study revealed that breaches of moral trust by robots impact human trust more negatively than performance trust breaches, and humans take retaliatory approaches in response to morality breaches by robots. In the present study, our main aim was to explore if trust loss and retaliation tendencies differ based on the identity of the teammates following the violations of these different trust aspects. Through multiple versions of an online search task, we examined our research questions and found that breaches of morality by robotic teammates cause a significantly higher trust loss in humans compared to human teammates. These findings highlight the importance of a robot's morality in determining how humans view a robot's trustworthiness. For effective robot design, robots must meet ethical and moral standards, which are higher than the ethical and moral standards expected from humans.

Do Humans Have Different Expectations Regarding Humans and Robots' Morality?

Autistic individuals, or individuals diagnosed with autism spectrum disorder (ASD), often experience challenges in social relations and may experience increased stress or anxiety. Recent studies highlight the potential of socially assistive robots (SARs) as tools for robot-assisted interventions. However, current human-robot interaction (HRI) designs lack consideration for the diverse anxiety issues in individuals with ASD and the need for personalized, empathetic assistance. The question that persists is whether the most recent individualized and modular intervention program for autism can also be effectively implemented through the use of SARs. This study presents a novel HRI framework that incorporates psychological factors into a modular intervention aimed at reducing users anxiety levels. The developed system integrates techniques such as human pose estimation and motion imitation while also leveraging conversational agents. During the user study, users engaged in interactions with a humanoid robot equipped with multiple intervention modules, and their anxiety and perceptions of the robots empathy were assessed through questionnaires in a within-participants study design. Results show the implemented system effectively mitigated users anxiety by assessing the differences between pre-session and post-session scores. Additionally, our findings suggest that the observed change in empathetic scores may be a contributing factor to the reduction in anxiety levels. The study offers significant insights into the use of SARs for mental health support in autistic adolescents.

An Empathetic Social Robot with Modular Anxiety Interventions for Autistic Adolescents

The deployment of social robots in public spaces has received increased interest over the past years. To execute various tasks, these robots need to process a wide array of personal data to offer services that are tailored to users requirements. While much research has been carried out regarding the creation of explainable content, little research has dealt with how data transparency - as a way to address uncertainty and concerns regarding the handling of personal data - is conveyed towards users. To examine the impact on trust, performance, and robot evaluation of modalities of transparency declarations, we conducted a virtual reality (VR) supported laboratory experiment with N = 53 participants who interacted with a robot in a public setting (a library). The interaction between users and robots is accompanied by information on the handling of users personal data using three different modalities (via posters, the robots tablet, or verbally). The results imply that, while all modalities are understandable and perceived as useful, there is no preference for any modality. This research contributes to HRI research by examining different modalities for transparency declarations in an effort to foster understandable and transparent processing of data.

Towards Understandable Transparency in Human-Robot-Interactions in Public Spaces

Human-Autonomy Teaming (HAT) is a multi-disciplinary domain with a diverse set of research needs and goals stemming from fields such as computer science, robotics, and human factors. This melting pot of fields generates a unique challenge in that there exist many disjoint research methods (measures and tasks) that cause issues with knowledge transfer and comparison between researchers. One way to address this issue is by providing researchers with a testbed containing a standardized suite of analysis tools and tasks that allow direct comparison between different approaches. Therefore, this study attempts to bring the HAT community together in a collaborative discussion to collect and organize their research needs for the future development of these testbeds. Specifically, through thematic analysis, our work reveals three emergent prongs that underpin testbed needs of HAT experts: task, AI, and technical requirements. Also, we organize our thematic analysis by priority to suggest possible paths for HAT testbed development to maximize its immediate and continued utility. Our research indicates that the HAT community places significant importance on both the pre-established, standardized functions available within the testbed and the freedom to tailor and develop their unique tasks or AI solutions.

Research Needs in Human-Autonomy Teaming: Thematic Analysis of Priority Features for Testbed Development

The efficiency of collaborative mobile robot applications is influenced by the inherent uncertainty introduced by humans' presence and active participation. This uncertainty stems from the dynamic nature of the working environment, various external factors, and human performance variability. The observed makespan of an executed plan will deviate from any deterministic estimate. This raises questions about whether a calculated plan is optimal given uncertainties, potentially risking failure to complete the plan within the estimated timeframe. This research addresses a collaborative task planning problem for a mobile robot serving multiple humans through tasks such as providing parts and fetching assemblies. To account for uncertainties in the durations needed for a single robot and multiple humans to perform different tasks, a probabilistic modeling approach is employed, treating task durations as random variables. The developed task planning algorithm considers the modeled uncertainties while searching for the most efficient plans. The outcome is a set of the best plans, where no plan is better than the other in terms of stochastic dominance. Our proposed methodology offers a systematic framework for making informed decisions regarding selecting a plan from this set, considering the desired risk level specific to the given operational context.

Risk-Aware Planning of Collaborative Mobile Robot Applications with Uncertain Task Durations

Visual planning methods are promising to handle complex settings where extracting the system state is challenging. However, none of the existing works tackles the case of multiple heterogeneous agents which are characterized by different capabilities and/or embodiment. In this work, we propose a method to realize visual action planning in multi-agent settings by exploiting a roadmap built in a low-dimensional structured latent space and used for planning. To enable multi-agent settings, we infer possible parallel actions from a dataset composed of tuples associated with individual actions. Next, we evaluate feasibility and cost of them based on the capabilities of the multi-agent system and endow the roadmap with this information, building a capability latent space roadmap (C-LSR). Additionally, a capability suggestion strategy is designed to inform the human operator about possible missing capabilities when no paths are found. The approach is validated in a simulated burger cooking task and a real-world box packing task.

Visual Action Planning with Multiple Heterogeneous Agents

In robotics, understanding human interaction with autonomous systems is crucial for enhancing collaborative technologies. We focus on human-swarm interaction (HSI), exploring how active robot groups of varying sizes affect operators' cognitive and perceptual reactions over distinct durations. We analyze the impact of different numbers of active robots within a 15-robot swarm on operators' time perception, emotional state, flow experience, and task difficulty perception. Our findings indicate that managing multiple active robots when compared to one active robot significantly alters time perception and flow experience, leading to a faster passage of time and increased flow. More active robots and extended durations cause increased emotional arousal and perceived task difficulty, highlighting the interaction between the number of active robots and human cognitive processes. These insights inform the creation of intuitive human-swarm interfaces and aid in developing swarm robotic systems aligned with human cognitive structures, enhancing human-robot collaboration.

From One to Many: How Active Robot Swarm Sizes Influence Human Cognitive Processes

In this paper, we set out to explore how people judge the personality of a non-anthropomorphic virtual agent during group interactions. Using a Wizard of Oz (WoZ) based approach, we observed that people judged the acceptability of a virtual agent's behavior from a tool-based lens, that is, if this robot and its behavior are useful to the team or not. Furthermore, we found that while people were able to acknowledge the virtual agent's personality and recognize its identity through social cues, the tool-based framing impacts these perceptions into a normative judgment of the robots' utility. We present two case studies that we think highlight this tool-based interpretation of robotic personality: robots expressing non-factual opinions and robots expressing humor. Finally, we suggest that researchers should consider the impact of this tool-based framing on people's perceptions of a robot's identity when designing robots for social interaction.

A Tool but Not a Peer: How Framing Affects People's Perceptions of AI Agents in Teams

Humanrobot work collaboration can be comple- mentary where both the robot and employee use their strengths to complete the task at hand. But who is ultimately responsible for the task? Employees may assume more or less responsibility when working with robots, which can have a direct impact on job outcomes. In this paper the authors use the job characteristics model as a framework to investigate changes in job characteristics and their impact on job responsibility and worker outcomes as a result of robot implementation in restaurants. The results identify changes in task significance, job autonomy, and feedback from the job as important predictors of changes in experienced responsibility toward work. The study also found a significant positive relationship between job responsibility and motivation to work.

Working with Robots in Restaurants: Job Responsibility and Work Motivation Changes

In human-robot collaboration, legible intent of the robot is critical to success as it enables the human to more effectively work with and around the robot. Environments where humans and robots collaborate are widely varied and in the real world are most often cluttered. However, prior work in legible motion utilizes primarily environments which are uncluttered. Success in these environments does not necessarily guarantee success in more cluttered environments. Furthermore, the prior work has been primarily performed based on results from robot-human studies and the problem has not been studied from the prospective of what people do to express intent to each other. Therefore, this work addresses a gap in current research into legible robot arm motion in the following ways: first we perform a human-human study in order to establish the factors which humans use to express their intent through body language, and second we perform the study in a cluttered and varied environment. Through the study we showed that the primary factors which people considered are: timing, kinematic parameters, hand gestures, object proximity, etc. The results also showed that legibility is correlated with perceived safety, perceived social intelligence, the collaborators contribution, and trust which further speaks to the importance of legible motion. Future work will utilize the pose data extracted from the study's video recordings to develop a model for legible motion.

Premium content

Downloads

Next from IEEE RO-MAN 2024

Do Humans Have Different Expectations Regarding Humans and Robots' Morality?

Similar lecture

Kinesthetic Teaching in Robotics: A Mixed Reality Approach