Italy

Robotic comedians (and social robots generally) need to recognize and adapt to human responses during playful
dialog. To support this ability, we determined design guidelines
via a survey of 20 human comedians and developed a machine learning pipeline to support comedian-like behaviors by our robotic system. Based on comedian input, we identified that discerning laughter vs. no laughter during a joke setup and big laugh vs. so-so response vs. no laugh after a punchline were important skills for a comedian. To enable these abilities in a robotic system, we used an existing dataset of robot comedy performance audio to train classifiers for audience
responses during the setup and after the punchline of jokes. Top-performing models for the above types of discernment performed similarly to human raters who completed the same
classification task. Comparison of the current results to our past efforts of a similar nature reveal repeatability of top-performing approaches and generalizability of the approaches to new parts of robot comedy routines. The social intelligence supported by this work can promote the likability and acceptance of robots.

IEEE RO-MAN 2022

How long does it take to learn trimanipulation?

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. This theme also touches upon people's expectations and perceptions of robots, as well as other social, cognitive and cultural factors that will influence human-robot interactions. We will have a special session/track on this topic, invite targeted submissions and include plenary speakers who are specialist in this area. We sincerely hope that this focus will help enlighten the design-thinking of all of us as we fabricate and program our own various systems.

To access RO-MAN Virtual 2021, [please register here.](https://ro-man2021.org/registration/)

IEEE RO-MAN 2021

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. 

technical paper

**31st IEEE International Conference on Robot & Human Interactive Communication**
 This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.
 
 The conference covers a wide range of topics related to Robot and Human Interactive Communication, involving theories, methodologies, technologies, empirical and experimental studies. Papers related to the study of robotic technology, psychology, cognitive science, artificial intelligence, human factors, ethics and policies, interaction-based robot design and other topics related to human-robot interaction are welcome.
 
 **CONFERENCE THEME:**
 Social, asocial, and antisocial robots
 
 Robots are starting to populate our lives: at work, in our homes, hotels, shops. However, not everybody is comfortable with robots finding a place in society. For example, people may be afraid that robots will replace humans in their jobs. This belief is exacerbated by the media that, on one side, raise expectations of robots’ capabilities, while amplifying fears and worries on the other side.
 
 In the next years, due to the increasing request for interaction with people in social environments, academic research and market analyses predict that robots will become more and more “social”. But, at the same time, robots may incur the risk of being perceived as “asocial” as their pervasiveness in everyday environments may become an obstacle to social interaction between humans. Or they may even be rejected as “antisocial” if researchers do not take care to develop robots that behave according to the customs and habits of the communities where they are expected to operate.
 
 To prevent this from happening, robotic researchers can no longer take the luxury of being mere technicians. They must be capable of addressing all the challenges raised by future robot-populated communities in a multidisciplinary way. They need to share their knowledge with the public to make people aware of what robots really are, what expectations we can have from them, the risks we may incur, and the possible mitigation plans.
 
 While being open to any kind of contribution related to Human-Robot Interactive Communication, the RO-MAN 2022 conference will emphasize these aspects of robotic research. How can we create robots capable of being welcomed as our assistants and companions in society while, at the same time, not being perceived as “asocial” or “antisocial?” How can we create robots perceived as facilitators of the natural development of human beings and communities, rather than obstacles?
 
 Contributions from different disciplines, not exclusively related to technological development, will be welcomed as crucial to pave the way to global acceptance of future robots with an informed attitude.
 
 [![](https://assets.underline.io/uploads/markdown_image/1/image/af259abe105b3791c498ae1552d75710.png)](https://drive.google.com/file/d/18w73BZPB_JVOgcrcdZDqYpEupwVt4CnU/view?usp=sharing)

A registration is required to access this event page. If you have not already done so, please use the below link to register for the event. 

[Register Here](http://www.smile.unina.it/ro-man2022/welcome-to-ieee-ro-man-2022/)

Registration Is Required

31st IEEE International Conference on Robot & Human Interactive Communication
This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.

Face spoofing detection could be used in conjunction with face recognition or facial expression recognition in social robots for better interaction with the users. Recent works have employed thermal image-based spoofing detection due to its competitive spoofing detection performance. However, existing methods utilized indoor thermal images which put a limit on using thermal image-based spoofing detection in outdoor environments. Moreover, several thermal image-based spoofing detection datasets consist of facial images at a frontal viewpoint, which would be put a limit on the usage in social robots with short heights. In this paper, we present our novel thermal image dataset for spoofing detection in outdoor environments. Unlike most existing datasets, our dataset is collected in outdoor environments with the camera facing the upper view to simulate the viewpoint of a social robot. To cover diverse outdoor environments, the dataset was collected at varying locations, times, and weather. Furthermore, we trained an event recognition-based spoofing detection on our collected dataset and also have provided benchmarking results for several baselines. Our spoofing detection method based on MobileNetV2 yielded a spoofing detection accuracy of 96.04\% and GFLOPs of 0.88. Our dataset will be open-sourced for users pursuing academic research and will be available upon request to the authors.

Towards evaluating the impact of swarm robotic control strategy on operators' cognitive load

Robots are becoming increasingly prominent in the entertainment sphere, where they interact with guests in themed environments to tell stories, often in place of human characters. To evaluate the potential benefits of robots in these contexts compared to humans, we created an interactive puzzle game where either a robot or a human actor serves as a diegetic "game guide" character that is both a cooperative partner and an omniscient game master. In the game, participants solve a crime mystery by asking the game guide for information to complete tasks and for hints to solve puzzles. We conducted a between-subjects study (n = 42) to investigate how players' game experiences differed when the game guide was a human compared to an embodied robot. Our results show that participants playing with a robot had more fun, felt less judged, and felt more connected with the robot while solving tasks compared to those playing with a human. These results suggest that robots can be effective alternatives to human actors in broader immersive entertainment contexts such as escape rooms to provide greater enjoyment and promote more social interaction with in-game characters.

Talk to Kotaro: a web crowdsourcing study on the impact of phoneme and prosody choice for synthesized speech on human impression

The use of multi-robot systems is increasing in
disaster response, industry, transport, and logistics. Humans
will remain indispensable to control and manage these fleets
of robots, particularly in safety-critical applications. However,
a human operator’s cognitive capacities can be challenged and
exceeded as the sizes of autonomous fleets grow, and more
sophisticated AI techniques can lead to opaque robot control
programs. In a user study (n = 40), we explore how autonomous
swarm intelligence algorithms and novel tangible interaction
modalities relate to subjective and physiological indices of
operator cognitive load (i.e., NASA Task Load Index, heart
rate variability). Our findings suggest that there are differences
in workload across conditions; however, subjective and cardiac
measures appear to be sensitive to different aspects of cognitive
state. The results hint at the potential of both tangible interfaces
and automation to engage operators and reduce cognitive load,
yet show the need for further validation of workload measures
for use in studying and optimizing human-swarm interactions.

Examining Audio Communication Mechanisms for Supervising Fleets of Agricultural Robots

During the COVID-19 pandemic, many research areas that require in person experiments with human volunteers have been impacted due to lockdowns and other activity-restricting policies. The field of robotics is no exception, and specially human-robot interaction research has been severely impacted. In order to circumvent the difficulty of gathering volunteers in person to interact with a robot, we have decided to build a novel crowdsourcing web platform for hosting our "Talk to Kotaro" experiment. The experiment consists of volunteers talking to a robot avatar and reacting to its semantic-free utterances. The developed web platform, which was built using the Python Flask framework, allows for such interactions while recording audio and video and other relevant data, which will be used for studying human impression estimation on gibberish speech. This paper describes not only the experiment and its preliminary results, but the developed platform itself; such tool is essential during pandemics and very useful for regular times, because it enables crowdsourcing data from all over the world.

How a Social Robot's Vocalization Affects Children's Speech, Learning, and Interaction

Social robots are used to perform mobile manipulation tasks, such as tidying up and carrying, based on instructions provided by humans.
A mobile manipulation planner, which is used to exploit the robot's functions, requires a better understanding of the feasible actions in real space based on the robot's subsystem configuration and the object placement in the environment.
This study aims to realize a mobile manipulation planner considering the world state, which consists of the robot state (subsystem configuration and their state) required to exploit the robot's functions.
In this paper, this study proposes a novel environmental representation called a world state-dependent action graph (WDAG).
The WDAG represents the spatial and temporal order of feasible actions based on the world state by adopting the knowledge representation with scene graphs and a recursive multilayered graph structure.
The study also proposes a mobile manipulation planning method using the WDAG.
The planner enables the derivation of many effective action sequences to accomplish the given tasks based on an exhaustive understanding of the spatial and temporal connections of actions.
The effectiveness of the proposed method is evaluated through practical machine experiments performed.
The experimental result demonstrates that the proposed method facilitates the effective utilization of the robot's functions.


A Modular Interface for Controlling Interactive Behaviors of a Humanoid Robot for Socio-Emotional Skills Training

As human–agent collaborations become more prevalent, it is increasingly important for an agent to be able to adapt to their collaborator and explain their own behavior. In order to do so, they need to be able to identify critical states during the interaction that call for proactive clarifications or behavioral adaptations. In this paper, we explore
whether the agent could infer such states from the human's eye gaze for which we compare gaze patterns across different situations in a collaborative task. Our findings show that the human's gaze patterns significantly differ between times at which the user is confused about the task, times at which the agent makes an error, and times of normal workflow. During errors the amount of gaze towards the agent increases, while during confusion the amount towards the environment increases. 
We conclude that these signals could tell the agent what and when to explain.

Analysing Eye Gaze Patterns during Confusion and Errors in Human–Agent Collaborations

In human-robot conversation, the human partner may ask any question within a specific task and expects an answer, ideally alongside a reason for that answer. For example, in a care home for the elderly, a human user might ask a companionship robot, "Where is my mug?", to which the robot would respond, "It is on the counter." For a given interaction to be smooth, a robot should respond correctly and be able to motivate its response, particularly in the cases of ambiguities. When providing answers to a user's question about a given scene, depth plays a critical role as spatial relations within the scene might change with depth as opposed to viewing the scene in 2D. For instance, as shown in Figure 1, when a user asks which object is closest to the white chair, 2D data might be insufficient, but incorporating 3D information can lead to a higher degree of correct responses provided by the robot in response to a human, and thus improve the quality of the interaction.

Improving Visual Question Answering by Leveraging Depth and Adapting Explainability

In the current state-of-the-art, robots are bound to make errors in a human-robot interaction (HRI). Trust is one of the important concepts in HRI that is often lowered by these errors. Fortunately, research has shown there are strategies that can help rebuild trust. An apology made by the robot is one of those strategies. However, apologies can take different forms.
We designed a study in which Nao first built trust with the users, then violated that trust by making a speech recognition error, and then tried to restore it by either an apology with
display of remorse, without remorse, or no apology at all. The results showed expected trends; an apology with remorse in most cases rebuilt trust the strongest. Although the effect of
the type of apology on the trusting beliefs were not significant, the effect on the trusting behaviours was found to be just significant. Suggestions for future research include repeating
the study without its current limitations (more participants, offline) and investigating the accuracy of the portrayed remorse by the robot.

The Robot That Showed Remorse: Repairing Trust with a Genuine Apology

Contemporary approaches to grounded language communication accept an utterance and current world representation as input and produce symbols representing the meaning as output. Since modern approaches to language understanding for human-robot interaction use techniques rooted in machine learning, the quality or sensitivity of the solution is often opaque relative to small changes in input. Although it is possible to sample and visualize solutions over a large space of inputs, naive application of current techniques is often prohibitively expensive for real-time feedback. In this paper we address this problem by reformulating the inference process of Distributed Correspondence Graphs to only recompute subsets of spatially dependent constituent features over a space of sam- pled environment models. We quantitatively evaluate the speed of inference in physical experiments involving a tabletop robot manipulation scenario. We demonstrate the ability to visualize configurations of the environment where symbol grounding produces consistent solutions in real-time and illustrate how these techniques can be used to identify and repair gaps or inaccuracies in training data.

Premium content

Downloads

Next from IEEE RO-MAN 2022

Towards evaluating the impact of swarm robotic control strategy on operators' cognitive load

Similar lecture

Augmented Reality interface to verify Robot Learning