United States

Addressing multi-label action recognition in videos represents a significant challenge for robotic applications in dynamic environments, especially when the robot is required to cooperate with humans in tasks that involve objects. Existing methods still struggle to recognize unseen actions or require extensive training data. To overcome these problems, we pro- pose Dual-VCLIP, a unified approach for zero-shot multi-label action recognition. Dual-VCLIP enhances VCLIP, a zero-shot action recognition method, with the DualCoOp method for multi-label image classification. The strength of our method is that at training time it only learns two prompts, and it is therefore much simpler than other methods. We validate our method on the Charades dataset that includes a majority of object-based actions, demonstrating that  despite its simplicity  our method performs favorably with respect to existing methods on the complete dataset, and promising performance when tested on unseen actions. Our contribution emphasizes the impact of verb-object class-splits during robots training for new cooperative tasks, highlighting the influence on the performance and giving insights into mitigating biases. Dataset splits and code are publicly available on the project&#39;s repository https://github.com/hsp-iit/DualVCLIP.

IEEE RO-MAN 2024

The Impact of Compositionality in Zero-Shot Multi-Label Action Recognition for Object-Based Tasks

multi-label action recognition

human-robot interaction

compositionality

zero-shot

ICPR 2020 is the premier international conference on Pattern Recognition, gathering leading researchers from around the world to present their latest achievements in Pattern Analysis and Recognition, applied Artificial Intelligence and Machine Learning. The topic of the year is " Putting Artificial intelligence to work on Patterns". 
For the first time, ICPR has employed a two-round review process similar to journal submissions, with new papers allowed to be submitted in either the first or second round and papers submitted in the first round and not accepted allowed to be revised and re-submitted for second round review. 

In this edition of the **ICPR 2020** Conference, there are 1407 Oral/Poster presentations organized in five Tracks: 
- Artificial Intelligence, Machine Learning for Pattern Analysis; 
- Biometrics, Human Analysis and Behavior Understanding; Computer Vision, Robotics and Intelligent Systems; 
- Document and Media Analysis; and 
- Image and Signal Processing. 

The **ICPR 2020** Main Conference program also includes 7 Keynotes from prominent world-renowned scientists, 8 Tutorials, 28 Demos, 8 Challenges, 2 Panels, as well as an Industry/Sponsor Exhibition. In addition, 41 Workshops on a range of important current and emerging research topics complete the ICPR 2020 program. 


ICPR 2020

Addressing multi-label action recognition in videos represents a significant challenge for robotic applications in dynamic environments, especially when the robot is required to cooperate with humans in tasks that involve objects. Existing methods still struggle to recognize unseen actions or require extensive training data. To overcome these problems, we pro- pose Dual-VCLIP, a unified approach for zero-shot multi-label action recognition. Dual-VCLIP enhances VCLIP, a zero-shot action recognition method, with the DualCoOp method for multi-label image classification. The strength of our method is that at training time it only learns two prompts, and it is therefore much simpler than other methods. We validate our method on the Charades dataset that includes a majority of object-based actions, demonstrating that  despite its simplicity  our method performs favorably with respect to existing methods on the complete dataset, and promising performance when tested on unseen actions. Our contribution emphasizes the impact of verb-object class-splits during robots training for new cooperative tasks, highlighting the influence on the performance and giving insights into mitigating biases. Dataset splits and code are publicly available on the project's repository https://github.com/hsp-iit/DualVCLIP.

technical paper

[![](https://assets.underline.io/markdown_image/1/image/f25361a85b7c83a7476c0b3c1c0bb1b2.png)](https://drive.google.com/file/d/18jcAcciCBQHj6R07Dbpm2HCOBEIwAOCU/view?usp=sharing)


Dear Colleagues and Friends,

The 33rd IEEE International Conference on Robot and Human Interactive Communication, IEEE RO-MAN 2024, was held at Pasadena Convention Center in Pasadena, USA.

The theme of the IEEE RO-MAN 2024 was **“Embracing Human-Centered HRI”**.

With the continuous technological effort of understanding human needs, creating human-desired robotic systems, and developing real human user experiences, we are marching together towards building a promising and excellent human quality of life, all for our wellness and our next generation.

The conference provided a forum for researchers and practitioners to discuss a variety of topics related to the broad theme of Robot and Human Interactive Communication involving theories, methodologies, technologies, empirical and experimental studies, and generated more insights for the future of Robot and Human Interactive Communication!

We hope you continue to enjoy the post-conference content on Underline!
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

A registration is required to access this event page. 



Registration Is Required

The 33rd IEEE International Conference on Robot and Human Interactive Communication was held in Pasadena, California, USA. The theme of the IEEE RO-MAN 2024 was “Embracing Human-Centered HRI”.

Multiple modalities can boost accuracy in the difficult task of Sign Language Recognition (SLR), however, each modality does not necessarily contribute the same quality of information. Current multi-modal approaches assign the same importance weightings to each modality, or set weightings based on unproven heuristics. This paper takes a systematic approach to find the optimal weights by performing grid search. Firstly, we create a multi-modal version of the RGB only WLASL100 data with additional hand crop and skeletal pose modalities. Secondly, we create a 3D CNN based weighted multi-modal sign language network (WMSLRnet). Finally, we run various grid searches to find the optimal weightings for each modality. We show that very minor adjustments in the weightings can have major effects on the final SLR accuracy. On WLASL100, we significantly outperform previous networks of similar design, and achieve high accuracy in SLR without highly complex pre-training schemes or extra data.

Weighted Multi-Modal Sign Language Recognition

Parent-Child Interaction Therapy (PCIT) is a well established method used by psychologists to improve and to maintain a healthy and good parent-child relationship. PCIT requires trained and certified therapists, and the therapy sessions are conducted in a special room, but participants can also practice and later perform sessions in their homes with our without therapist supervision. In order to facilitate the therapy for the parents, here we present a possible approach to introduce social robotics into the PCIT therapy. We developed a behavior model to control a mobile robot and interact with both the parent and the child during a therapy session with the goal of stimulating interaction and communication between the parent and child. The behavior of this social robot depends on the contents of the conversation between the participants, which is analyzed in real-time. For speech analysis we have developed a framework for integrating various already available components to achieve real-time Speech-to-Text transcription, and also developed our own Natural Language Understanding (NLU) engine. The evaluation of the conversation is performed by this NLU engine, which was inspired by the Dyadic Parent- Child Interaction Coding System (DPICS) used in the PCIT therapy. Depending on the evaluation results, our mobile robot can exhibit different behaviors, which were inspired by Etho-Robotics, such as approaching, getting attention, playing fetch with a toy. Besides constructing a functional real-time human conversation analysis system and a behavior model, we also performed preliminary HRI experiments with a real mobile robot controlled by the proposed model in real-time.

Towards an Interaction Stimulator Social Robot in the Parent-Child Interaction Therapy Based on Real-Time Speech Processing

Research has shown the potential of social robots to support learning in science, technology, and language. We contribute to this field by exploring how robots can support music learning. We report on a within-subjects experiment where 50 young learners practiced the piano in the presence of a robot assuming a non-evaluative and a self-assessment enhancing role implemented in a Wizard-of-Oz fashion. We examined whether the robot can make piano practice more fun, and whether initiating self-assessment to support self-regulated learning is a useful strategy for the robot. We collected quantitative self-report data to assess fun, learning, interest, engagement, and effort. We found a direct positive effect of fun on learning in the context of musical instrument practice. Path modeling showed a positive influence of having fun on learners' attitudes, interests, and learning outcomes in music education, particularly with the self- assessment robot role exhibiting superiority.

Can Robots Enhance the Learning Experience by Making Music More Fun?

Social robots are increasingly finding applications in sectors such as industry, retail, and healthcare. They employ a combination of verbal and non-verbal cues in order to allow an empathetic and efficient interaction with the humans. In contrast to traditional robots, social robots possess contextual awareness, enabling intelligent responses to human interactions. Within this context, in this paper we propose a framework for social robots, integrating advanced audio and video analytics capabilities with a novel engagement algorithm, designed and developed to facilitate effective communication in multi-user settings. Furthermore, other than conversation skills, the proposed social robot is enhanced with the capability to interactively play games with the human. The effectiveness of the proposed system was tested in real-world scenarios, during two trade shows in Italy, providing valuable insights into its performance and adaptability to different contexts and different audiences.

Empowering Human Interaction: A Socially Assistive Robot for Support in Trade Shows

With more robots being deployed in the world, users will likely interact with multiple robots sequentially when receiving services. In this paper, we describe an exploratory field study in which unsuspecting participants experienced a ``person transfer'' -- a scenario in which they first interacted with one stationary robot before another mobile robot joined to complete the interaction. In our 7-hour study spanning 4 days, we recorded 18 instances of person transfers with 40+ individuals. We also interviewed 11 participants after the interaction to further understand their experience. We used the recorded video and interview data to extract interesting insights about in-the-field sequential human-robot interaction, such as mobile robot handovers, trust in person transfer, and the importance of the robots' positions. Our findings expose pitfalls and present important factors to consider when designing sequential human-robot interaction.

Person Transfer in the Field: Examining Real World Sequential Human-Robot Interaction between Two Robots

Science fiction has long promised a future within which robots assist humans in many facets of their daily lives, and robot technology is advancing at a pace which suggests that the necessary technology already exists, or may exist, in the near future. But, once the technology is in place, how accepting will humans be to autonomous machines performing tasks traditionally performed by humans? Are we designing and developing robots that are human centric? In a study involving 357 undergraduate students, we found that acceptance of robots was dependent upon previous exposure to different forms of technology (i.e., robots, avatars, video games). Men were more likely to have previous exposure to technology, and were therefore more likely to accept robots and avatars in different tasks compared to women. Enhancing the acceptability of robots by both men and women will require an increased exposure to technology, and women may require additional experience with technology to close the technology acceptance gap.

Technology Exposure Elicits Increased Acceptance of Autonomous Robots and Avatars

Robots are inherently imperfect, and collaborating with an error-prone robotic teammate can deteriorate perceptions of trust and the willingness of users to continue working with the robot. Evidence-based trust repair strategies can be implemented into a robots design to mitigate the decline of trust in human-robot relationships following errors. It is not yet clear what trust repair strategies are most effective. To address this shortcoming, this study investigates two novel trust repair strategies: offered and automatic physical repair. A between-subjects repeated measures study was performed to determine the extent to which each type of physical trust repair was successful in restoring participants perceptions of trust. The results indicated that, where the no-repair condition experienced a significant decrease in trust score, only the automatic repair was consistently successful in bypassing the trust decline. Detailed analysis showed that participants from the offered repair condition did not view the robot as providing the appropriate information, meaning that the offer itself may have confused them. Participants response rate to the Multi- Dimensional Measure of Trust also revealed that users were less willing to associate moral terms with robotic teammates, though this hesitancy may reduce over time. These results contribute to research on human-robot trust repair by uncovering that physical repair is effective when it is automatic, but not when it is offered. This finding will help to further elucidate what repair strategies work to mitigate trust decline and thus help inform robot design.

Robots That Use Physical Repair Strategies after Repeated Errors to Mitigate Trust Decline in Human-Robot Interaction: A Repeated Measures Experiment

This paper demonstrates the effectiveness of learning based models for accurate, and reliable robot to human handoffs in various HRI scenarios. Specifically we bench marked a neuromorphic spiking neural network and a time series k-nearest neighbors classifier against traditional hand crafted force threshold methods. These models use linear force in the x, y, and z direction, as well as torque about the x, y, and z axis at the end effector of the robot arm to make handoff predictions. This paper demonstrates that these learning based methods are more robust to noise which occurs during operational use. We applied our algorithms to both stationary handoffs (stationary robot) and moving handoffs (robot walking). We believe that our evaluation is the first to examine walking handoffs. We evaluated all models in tests which determined the accuracy, precision, recall, f1, and average execution time for handoff events, noise events, and no event tests. We find that the SLAYER spiking neural network model performed the best across both walking and stationary handoffs for the majority of the evaluation criteria. Our results suggest that neuromorphic spiking neural networks are strong contenders for applications in time series, event based HRI applications.

Spiking Neural Networks for Improved Robot-Human Handoffs

Using a dual-task paradigm, we explore how robot actions, performance, and the introduction of a secondary task influence human trust and engagement. In our study, a human supervisor simultaneously engages in a target-tracking task while supervising a mobile manipulator performing an object collection task. The robot can either autonomously collect the object or ask for human assistance. The human supervisor also has the choice to rely on or interrupt the robot. Using data from initial experiments, we model the dynamics of human trust and engagement using a linear dynamical system (LDS). Furthermore, we develop a human action model to define the probability of human reliance on the robot. Our model suggests that participants are more likely to interrupt the robot when their trust and engagement are low during high-complexity collection tasks. Using Model Predictive Control (MPC), we design an optimal assistance-seeking policy. Evaluation experiments demonstrate the superior performance of the MPC policy over the baseline policy for most participants.

Assistance-Seeking in Human-Supervised Autonomy: Role of Trust and Secondary Task Engagement

While robots have advanced in understanding their environments, collaborative tasks demand a deeper comprehension of human intentions to mitigate uncertainty. Anticipatory and proactive behaviours are pivotal in enhancing Human-Robot Interactions (HRI), yet literature often conflates these terms. This study elucidates the distinction between anticipation and proactivity, offering clear definitions and exemplifying their implications through a handover scenario. Through a user study with 24 volunteers performing a total of 72 experiments, we have found that humans are able to distinguish both behaviours and that there is a statistically significant increase in the anthropomorphism of the robot when it behaves proactively. Additionally, both anticipation and proactivity show statistically significant increases in multiple aspects of effective HRI (fluency, comfort, performance, etc.). However, no clear preference for either has been detected.

Premium content

Downloads

Next from IEEE RO-MAN 2024

Weighted Multi-Modal Sign Language Recognition

Similar lecture

Attention-Oriented Action Recognition for Real-Time Human-Robot Interaction