Italy

In this study, we extend our robot trust model into
a multimodal setting in which the Nao robot leverages audiovisual data to perform a sequential multimodal pattern recalling
task while interacting with a human partner who has different
guiding strategies: reliable, unreliable, and random. Here, the
humanoid robot is equipped with a multimodal auto-associative
memory module to process audio-visual patterns to extract
cognitive load (i.e., computational cost) and an internal reward
module to perform cost-guided reinforcement learning. After
interactive experiments, the robot associates a low cognitive load
(i.e., high cumulative reward) yielded during the interaction
with high trustworthiness of the guiding strategy of the partner.
At the end of the experiment, we provide a free choice to the
robot to select a trustworthy instructor. We show that the robot
forms trust in a reliable partner. In the second setting of the
same experiment, we endow the robot with an additional simple
theory of mind module to assess the efficacy of the instructor
in helping the robot perform the task. Our results show that
the performance of the robot is improved when the robot bases
its action decisions on factoring in the instructor assessment.

IEEE RO-MAN 2022

Trustworthiness assessment in multimodal human-robot interaction based on cognitive load

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. This theme also touches upon people's expectations and perceptions of robots, as well as other social, cognitive and cultural factors that will influence human-robot interactions. We will have a special session/track on this topic, invite targeted submissions and include plenary speakers who are specialist in this area. We sincerely hope that this focus will help enlighten the design-thinking of all of us as we fabricate and program our own various systems.

To access RO-MAN Virtual 2021, [please register here.](https://ro-man2021.org/registration/)

IEEE RO-MAN 2021

RO-MAN 2021's theme is trustworthiness and cooperation. We chose this term well before the pandemic put the world on edge and added a new reason for us to question our own personal safety when in the company of others. But what about in the company of social robots? What would cause us to trust, or not trust, them? How do we as researchers create that feeling, that critical component of any relationship? With social robots, it is of course largely in our power to design the physical form, the control code and the interface software to maximize trustworthiness, facilitate cooperation and enhance the quality of human-robot interaction. 

technical paper

**31st IEEE International Conference on Robot & Human Interactive Communication**
 This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.
 
 The conference covers a wide range of topics related to Robot and Human Interactive Communication, involving theories, methodologies, technologies, empirical and experimental studies. Papers related to the study of robotic technology, psychology, cognitive science, artificial intelligence, human factors, ethics and policies, interaction-based robot design and other topics related to human-robot interaction are welcome.
 
 **CONFERENCE THEME:**
 Social, asocial, and antisocial robots
 
 Robots are starting to populate our lives: at work, in our homes, hotels, shops. However, not everybody is comfortable with robots finding a place in society. For example, people may be afraid that robots will replace humans in their jobs. This belief is exacerbated by the media that, on one side, raise expectations of robots’ capabilities, while amplifying fears and worries on the other side.
 
 In the next years, due to the increasing request for interaction with people in social environments, academic research and market analyses predict that robots will become more and more “social”. But, at the same time, robots may incur the risk of being perceived as “asocial” as their pervasiveness in everyday environments may become an obstacle to social interaction between humans. Or they may even be rejected as “antisocial” if researchers do not take care to develop robots that behave according to the customs and habits of the communities where they are expected to operate.
 
 To prevent this from happening, robotic researchers can no longer take the luxury of being mere technicians. They must be capable of addressing all the challenges raised by future robot-populated communities in a multidisciplinary way. They need to share their knowledge with the public to make people aware of what robots really are, what expectations we can have from them, the risks we may incur, and the possible mitigation plans.
 
 While being open to any kind of contribution related to Human-Robot Interactive Communication, the RO-MAN 2022 conference will emphasize these aspects of robotic research. How can we create robots capable of being welcomed as our assistants and companions in society while, at the same time, not being perceived as “asocial” or “antisocial?” How can we create robots perceived as facilitators of the natural development of human beings and communities, rather than obstacles?
 
 Contributions from different disciplines, not exclusively related to technological development, will be welcomed as crucial to pave the way to global acceptance of future robots with an informed attitude.
 
 [![](https://assets.underline.io/uploads/markdown_image/1/image/af259abe105b3791c498ae1552d75710.png)](https://drive.google.com/file/d/18w73BZPB_JVOgcrcdZDqYpEupwVt4CnU/view?usp=sharing)

A registration is required to access this event page. If you have not already done so, please use the below link to register for the event. 

[Register Here](http://www.smile.unina.it/ro-man2022/welcome-to-ieee-ro-man-2022/)

Registration Is Required

31st IEEE International Conference on Robot & Human Interactive Communication
This conference is a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.

Sex dolls are passively jointed human-like structures. Their limited but non negligible dissemination may suggest future new modes of sexual and affective relationships with sex robots. Understanding the quality of the relationship between a owner and his/her sex doll could help us to anticipate the social consequences of future sex robots. In this framework, we propose, from available testimonies reporting a large range of behaviors including sexual relationships, destructive behaviors, or installation of the doll in the daily life of the owner, or romance with the doll generally expressed by photographies, to interpret the sex doll as a new form of transitional object in the words of Winnicott. Moreover, the hyper-sexualized appearance of the doll makes easier its fetishization as a “labor of signification” according to Baudrillard’s semiotic theory. If to cherish a transitional object in adulthood can be an indicator of depressive syndromes or borderline personality disorder, no emotional disorder is necessary linked to its private use. Further studies about sex dolls and robots could so take advantage of work that was already done with other forms of transitional objects for adults.


We are all Individuals: The Role of Robot Personality and Human Traits in Trustworthy Interaction

Enabling robots to identify when humans need
assistance is key to being able to provide help that is both
proactive and efficient. This challenge is particularly difficult
for humans eating a meal in a restaurant, a context which is
dense with interlaced social elements such as conversation in
addition to functional tasks such as eating. We investigated the
challenge of identifying human dining activities from singleviewpoint
footage by collecting and annotating the individual
activities of five two-person meals. From this process, we found
that addressing the question of identifying meal phases and
overall neediness requires identifying an underlying group state
for the table as a whole. We report on the individual activities
and group states, as well as the interdependencies between these
factors that can be leveraged to both provide and measure
effective robotic restaurant service. In addition to the insights
revealed by this dataset, we describe preliminary attempts to
create an automated classification system for these activities.

How Users, Facility Managers, and Bystanders Perceive and Accept a Navigation Robot for Visually Impaired People in Public Buildings

his study explores how to best utilize service robots when deploying them in customer facing contexts for information requests. Many companies investing in such robots unfortunately fail to integrate them successfully in a way where they facilitate meaningful interactions with users. Copenhagen Visitor Service is one of those companies who have tried to integrate a Pepper robot into their service environment but failed due to several factors relating to both the environment, the robot and lack of a strategical implementation. Through an exploratory field study with in-situ feedback from the participants, it became evident that the robot in this specific setting needed a more focused purpose and an interaction flow that was not reliant on verbal communication. A prototype was developed consisting of an informative quiz targeting families. A subsequent field deployment demonstrated a better and more meaningful user experience, indicating that careful consideration of the use context is paramount for a successful integration as well as a strategical implementation.

Ankle Intention Detection Algorithm with HD-EMG Sensor

This paper reports on a field study of more than three hundred ‘incidental’ human-robot encounters in a public outdoor space. We have used an ethnomethodologically informed ethnographic method, employing breaching experiments and membership categorization analysis to reveal how Incidentally Co-present Persons (InCoPs) interact with and make sense of robots in an urban environment.

Proactive Robot Movements in a Crowd by Predicting and Considering the Social Influence

Children with Autism Spectrum Disorder (ASD) often struggle with visual perspective taking (VPT) skills and the understanding that others might have viewpoints and perspectives that are different from their own; i.e., the ability to understand that two or more people looking at the same object from different positions might not see the same thing. The understanding of VPT can be improved by introducing explicit causal explanations in the interactions involving autistic children. Moreover, the use of social robots can help autistic children improve their social skills. We present a retrospective study with Kaspar, a humanoid social robot specifically designed to interact with children with ASD, which aims to define the initial protocol for a study on the effect of causal explanation in VPT provided by Kaspar. To this end, we investigate in which scenarios causal explanations, provided either by researchers or by Kaspar, contribute substantially to the child’s understanding of VPT. The results have helped us identify multiple interaction categories that benefit from causal explanation. We have used these results in order to define new interaction games that benefit from causal explanations. These are now progressing through usability assessment experiments.

Listen and tell me who the user is talking to: Automatic detection of the interlocutor’s type during a conversation

Thailand, as well as many other countries worldwide, is facing a shortage of medical staff. We propose a solution to improve medical services in health centres: a robot teleoperation
system to allow patients to consult with doctors from public hospitals, and for doctors to examine and make decisions about their required care. To develop such a system, a user-centred design (UCD) process is followed. Here we present an important first step in this process to establish user requirements for such a system. Hence, we have conducted a focus group with Thai
medical staff from Banphaeo General Hospital and an online survey with potential patients. An online collaborative board has been setup to facilitate running the focus group virtually and provide an effective tool to gather data. A qualitative data is then analysed using a framework analysis. Based on this work, we present a list of user requirements for doctors, patients
and assistants and discuss how the collected requirements can be transferred into technical specifications of the system. Our study found that communication among different user groups is the most important requirement.

Drowsiness prevention using a social robot

Expressiveness is a critical feature for the communication between humans and robots, and it helps humans to better understand and accept a robot. Emotions can be expressed through a variety of modalities: kinesthetic (via facial expression), body posture and gestures, auditory, thus the acoustic features of speech, and semantic, thus the content of what is said. One of the most effective modalities to communicate emotions is through facial expressions. Social robots often show facial expressions with coded animations. However, the robot must be able to express appropriate emotional responses according to the interaction with people. In this work, we consider verbal interactions between humans and robots and propose a system composed of two modules for the generation of facial emotions by recognising the arousal and valence values of a written sentence. The first module, based on Bidirectional Encoder Representations from Transformers, is deployed for emotion recognition in a sentence. The second, an Auxiliary Classifier Generative Adversarial Network, is proposed for the generation of facial movements for expressing the recognised emotion in terms of valence and arousal.

Action Unit Generation through Dimensional Emotion Recognition from Text

This paper examines how conversational agents (CAs) can communicate emotions non-verbally using light as communication modality. Therefore, we manipulated the CA Alexa to demonstrate emotions (joy and sorrow) using different light modalities (Echo Dot ring, a Hue lamp and the combination) with either a congruent verbal context (party or funeral) or no verbal context. In an online study 167 participants evaluated the perceived emotion of Alexa, their own emotional state as well as the perception of Alexa after watching a video with a user interacting with Alexa. Although the perceived emotions of Alexa were not affected by the experimental conditions, the results indicate that the perception of Alexa as well as the user’s emotion is affected by the displayed communication modality. As external light can be used to manipulate the users’ perception of CAs, the findings give relevant implications for the design of CAs.

Evaluating the Effect of Theory of Mind on Peoples Trust in a Faulty Robot

From RViz to augmented reality (AR), a wide variety of robot \textcolor{black}{signal visualizations exist for conveying robot capabilities. Many of the visualizations designed for AR, however, have not isolated multiple salient Virtual Design Elements (VDEs) for a given signal and comparatively evaluated combinations of those VDEs. To address this, we identify multiple VDEs for AR signaling of the following core robot capabilities:} \textit{navigation}, \textit{light detection and ranging (LiDAR)}, \textit{camera}, \textit{face detection}, \textit{audio localization}, and \textit{natural language processing}. We evaluated each signal's VDE combinations with an Amazon Mechanical Turk study ($n$=150) where participants watched 4 videos for each signal (consisting of 2 independent VDE choices) and rated the clarity and visual appeal of each signal. The results define a set of the most clear and visually appealing signal visualization designs and inform about interaction effects among VDEs. The resulting VDEs offer design insights and a baseline for continued research into AR robot capability signalling.

Human-Robot Handovers using Task-Space Quadratic Programming

Mobile robots must navigate efficiently, reliably, and appropriately around people when acting in shared social environments. For robots to be accepted in such environments, we explore robot navigation for the social contexts of each setting. Navigating through dynamic environments solely considering a collision-free path has long been solved. In human-robot environments, the challenge is no longer about efficiently navigating from one point to another. Autonomously detecting the context and adapting to an appropriate social navigation strategy is vital for social robots' long-term applicability in dense human environments. As complex social environments, museums are suitable for studying such behavior as they have many different navigation contexts in a small space.

Our prior Socially-Aware Navigation model considered context classification, object detection, and pre-defined rules to define navigation behavior in more specific contexts, such as a hallway or queue. This work uses environmental context, object information, and more realistic interaction rules for complex social spaces. In the first part of the project, we convert real-world interactions into algorithmic rules for use in a robot's navigation system. Moreover, we use context recognition, object detection, and scene data for context-appropriate rule selection. We introduce our methodology of studying social behaviors in complex contexts, different analyses of our text corpus for museums, and the presentation of extracted social norms. Finally, we demonstrate applying some of the rules in scenarios in the simulation environment.

Premium content

Downloads

Next from IEEE RO-MAN 2022

We are all Individuals: The Role of Robot Personality and Human Traits in Trustworthy Interaction

Similar lecture

Trust Me! I Am a Robot: An Affective Computational Account of Scaffolding in Robot-Robot Interaction