United States

Research on underwater human-robot teaming is particularly promising because of complementary sensory skills that may overcome unique challenges associated with the combination of limited connectivity and low visibility. Nevertheless, testing human-robot collaboration under water, especially in complex, real-world scenarios, poses severe safety, cost, and time constraints that significantly hinder academic research in this space. This paper presents a novel cyber-physical extended-reality (XR) testbed, referred to as Underwater Real-Time Human Autonomous Systems Collaboration (UnRealTHASC), designed to enable human-robot interactions in simulated photorealistic environments with mixed real and virtual wearables, and advanced autonomous underwater sensors in-the-loop. Novel sensor interfaces are designed to integrate real and virtual sensors for measuring physiological and cognitive human states underpinning decision-making abilities. Physics-based human and robot motion models are developed along with new sensor simulations in order to capture the couplings between underwater behaviors and perception based on measurements from optical and sonar sensors. Real-time data acquisition pipelines are created to access and share data from both real and virtual sensors and robots, such that new methods for online planning and collaboration may be tested via human-in-the-loop demonstrations.

IEEE RO-MAN 2024

UnRealTHASC – a Cyber-Physical XR Testbed for Underwater Real-Time Human Autonomous Systems Collaboration

cyber-physical extended reality testbed

human-centric human-robot collaboration

underwater human-robot teaming

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

technical paper

Quadruped robots are showing impressive abilities to navigate the real world. If they are to become more integrated into society, social trust in interactions with humans will become increasingly important. Additionally, robots will need to be adaptable to different humans based on individual preferences. In this work, we study the social interaction task of learning optimal handshakes for quadruped robots based on user preferences. While maintaining balance on three legs, we parameterize handshakes with a Central Pattern Generator consisting of an amplitude, frequency, stiffness, and duration. Through 10 binary choices between handshakes, we learn a belief model to fit individual preferences for 25 different subjects. Our results show that this is an effective strategy, with 76% of users feeling happy with their identified optimal handshake parameters, and 20% feeling neutral. Moreover, compared with random and test handshakes, the optimized handshakes have significantly decreased errors in amplitude and frequency, lower Dynamic Time Warping scores, and improved energy efficiency, all of which indicate robot synchronization to the user's preferences

Learning Human-Robot Handshaking Preferences for Quadruped Robots

To facilitate human-robot interaction (HRI) tasks in real-world scenarios, service robots must adapt to dynamic environments and understand the required tasks while effectively communicating with humans. To accomplish HRI in practice, we propose a novel indoor dynamic map, task understanding system, and response generation system. The indoor dynamic map optimizes robot behavior by managing an occupancy grid map and dynamic information, such as furniture and humans, in separate layers. The task understanding system targets tasks that require multiple actions, such as serving ordered items. Task representations that predefine the flow of necessary actions are applied to achieve highly accurate understanding. The response generation system is executed in parallel with task understanding to facilitate smooth HRI by informing humans of the subsequent actions of the robot. In this study, we focused on waiter duties in a restaurant setting as a representative application of HRI in a dynamic environment. We developed an HRI system that could perform tasks such as serving food and cleaning up while communicating with customers. In experiments conducted in a simulated restaurant environment, the proposed HRI system successfully communicated with customers and served ordered food with 90% accuracy. In a questionnaire administered after the experiment, the HRI system of the robot received 4.2 points out of 5. These outcomes indicated the effectiveness of the proposed method and HRI system in executing waiter tasks in real-world environments.

Unified Understanding of Environment, Task, and Human for Human-Robot Interaction in Real-World Environments

This paper addresses the challenge of selecting explanations for XAI (Explainable AI)-based Intelligent Decision Support Systems (IDSSs). IDSSs have shown promise in improving user decisions through XAI-generated explanations along with AI predictions, and the development of XAI made it possible to generate a variety of such explanations. However, how IDSSs should select explanations to enhance user decision-making remains an open question. This paper proposes X-Selector, a method for selectively presenting XAI explanations. It enables IDSSs to strategically guide users to an AI-suggested decision by predicting the impact of different combinations of explanations on a users decision and selecting the combination that is expected to minimize the discrepancy between an AI suggestion and a user decision. We compared the efficacy of X-Selector with two naive strategies (all possible explanations and explanations only for the most likely prediction) and two baselines (no explanation and no AI support) in virtual stock-trading support scenarios. The results suggest the potential of X-Selector to guide users to AI-suggested decisions and improve task performance under the condition of a high AI accuracy.

User Decision Guidance with Selective Explanation Presentation from Explainable-AI

Apology behavior design is becoming important for social robots that work in daily environments because of their widespread use. Robotics researchers have reported that multiple robots can effectively achieve more acceptance and trust in apology situations. Unfortunately, such effects have only been confirmed in a single country: Japan. Some studies investigated cultural differences in apologies between Japan and other cultures and reported how the former influences the perceived function and meaning of apologies. Therefore, we conducted a web-based survey to investigate whether using multiple robots in apology situations is effective in another country. We compared such perceived feelings as forgiveness and trust toward a robots apologies between Japan and the U.S. by using the visual stimuli of one and two robots. The experiment results showed that U.S. people felt that multiple robot apologies are more acceptable than apologies from just one robot, similar to results with Japanese participants. Perceived trust did show a different phenomenon between the two countries.

Two Is Better Than One: Cultural Differences in the Number of Apologizing Robots in the U.S. and Japan

This study investigates the development of a robotic walker for gait rehabilitation that integrates social assistive robotics and principles of Rhythmic Auditory Stimulation (RAS), a form of music therapy. The robotic system is intended to supplement, rather than replace, the work of professional music therapists by enabling therapeutic interventions to continue outside of therapy sessions. Robot-facilitated Music Therapy is an evolutionary step of telerehabilitation, providing patients with greater autonomy while still benefiting from the expertise and guidance of therapists. Contributions of the research include 1) Robot-facilitated Music Therapy through a robotic walker as a platform for gait rehabilitation. 2) Remote Music Therapy through a robotic walker as a telerehabilitation platform for music therapists to remotely monitor and consult with patients. 3) Intelligent companionship integrated with a robotic walker for comprehensive user assistance. The proposed system, tested for its functionalities including personified RAS, social interaction, user monitoring, mobility control, and emergency response yielded promising results for real world application.

Smart Music Therapist 1.0: Rhythmic Auditory Stimulation Integrated Robotic Walker As a Therapeutic Companion for Gait Rehabilitation

As virtual spaces continue to attract more and more attention, the development of virtual agents is also gaining momentum. Agents capable of expressing emotions are being developed, and various methods exist through which they express such feelings. Although much research has focused on the implementation of human actions in agents, no research has yet implemented facial self-touch for agents used in emotional expressions, even though such transmission of human emotions is a critical tool for them. In this study, we developed a system that enriches agents' emotional expressions by implementing self-touches in virtual agents. From an evaluation of our system, we found that it enhances the emotions of virtual agents. The naturalness of their emotional expressions varied depending on the emotion.

What Kinds of Facial Self-Touches Strengthen Expressed Emotions?

Effective collaboration between humans and AIs hinges on transparent communication and alignment of mental models. However, explicit, verbal communication is not always feasible in many scenarios. Under such circumstances, human-human teams often depend on implicit, nonverbal cues to glean important information about their teammates such as future intentions and expertise, thereby bolstering team alignment and adaptability. Among these cues a humans actions in the environment and their visual attention are the most salient. In this paper, we propose a novel method to combine eye gaze data and behavioral data and evaluate their respective predictive power for human proficiency, trust, and intentions. We first collect a dataset of paired eye gaze and gameplay data in the fast-paced collaborative Overcooked environment. We then train variations of our models on this dataset to compare how the predictive powers differ between gaze data, gameplay data, and their combination. We additionally compare our method to prior works that aggregate eye gaze data and demonstrate how these aggregation methods can substantially reduce the predictive ability of eye gaze relative to our approach. Our results indicate that although there are certain situations where eye gaze data or gameplay data yield superior results, the model that integrates both types of data consistently outperforms all baselines. This work paves the way for developing intuitive and responsive agents that can efficiently adapt themselves to new and unseen teammates.

Eyes on the Game: Deciphering Implicit Human Signals to Infer Human Proficiency, Trust, and Intent

With the increasing presence of social robots in various environments and applications, there is an increasing need for these robots to exhibit socially-compliant behaviors. Legible motion, characterized by the ability of a robot to clearly and quickly convey intentions and goals to the individuals in its vicinity, through its motion, holds significant importance in this context. This will improve the overall user experience and acceptance of robots in human environments. In this paper, we introduce a novel approach to incorporate legibility into local motion planning for mobile robots. This can enable robots to generate legible motions in real-time and dynamic environments. To demonstrate the effectiveness of our proposed methodology, we also provide a robotic stack designed for deploying legibility-aware motion planning in a social robot, by integrating perception and localization components.

Legibot: Generating Legible Motions for Service Robots Using Cost-Based Local Planners

Co-speech gesture generation on artificial agents has gained attention recently, mainly when it is based on data-driven models. However, end-to-end methods often fail to generate co-speech gestures related to semantics with specific forms, i.e., Symbolic and Deictic gestures. In this work, we identify which words in a sentence are contextually related to Symbolic and Deictic gestures. Firstly, we appropriately chose 12 gestures recognized by people from the Italian culture, which different humanoid robots can reproduce. Then, we implemented two rule-based algorithms to label sentences with Symbolic and Deictic gestures. The rules depend on the semantic similarity scores computed with the RoBerta model between sentences that heuristically represent gestures and sub-sentences inside an objective sentence that artificial agents have to pronounce. We also implemented a baseline algorithm that assigns gestures without computing similarity scores. Finally, to validate the results, we asked 30 persons to label a set of sentences with Deictic and Symbolic gestures through a Graphical User Interface (GUI), and we compared the labels with the ones produced by our algorithms. For this scope, we computed Average Precision (AP) and Intersection Over Union (IOU) scores, and we evaluated the Average Computational Time (ACT). Our results show that semantic similarity scores are useful for finding Symbolic and Deictic gestures in utterances.

Labeling Sentences with Symbolic and Deictic Gestures Via Semantic Similarity

Using multiple robots in presentation effectively attracts and provides information to people compared to a single robot. Many robotics researchers employed this approach to realize effective robot-presentation systems in real environments. These systems implicitly equalize the amount of speech in conversational content between multiple robots; however, it is not known whether that rule is actually important. For example, the effects of the second robot that provides only a back-channel have not yet been investigated. If we unveil the relationships between the presentation effects and the amount of speech in multiple robot presentations, it is crucial in designing the content of multiple robots' conversations. For this purpose, we conducted a video-based web-survey. Our finding showed that product recommendations by multiple robots make them more positive, and a part of the advantages of equalizing the amount of speech compared to biasing the amount of speech in presentation.

Premium content

Downloads

Next from IEEE RO-MAN 2024

Learning Human-Robot Handshaking Preferences for Quadruped Robots

Similar lecture

Extended Reality for Enhanced Human-Robot Collaboration: A Human-In-The-Loop Approach