Canada

One of the ways to make reinforcement learning (RL) more ef-
ficient is by utilizing human advice. Because human advice is
expensive, the central question in advice-based reinforcement
learning is, how to decide in which states the agent should ask
for advice. To approach this challenge, various advice strate-
gies have been proposed. Although all of these strategies dis-
tribute advice more efficiently than naive strategies (such as
choosing random states), they rely solely on the agent’s inter-
nal representation of the task (the action-value function, the
policy, etc.) and therefore, are rather inefficient when this rep-
resentation is not accurate, in particular, in the early stages of
the learning process. To address this weakness, we propose an
approach to advice-based RL, in which the human’s role is not
limited to giving advice in chosen states, but also includes hint-
ing apriori (before the learning procedure) which sub-domains
of the state space require more advice. Specifically, we sug-
gest different ways to improve any given advice strategy by
utilizing the concept of critical states: states in which it is very
important to choose the correct action. Finally, we present ex-
periments in 2 environments that validate the efficiency of our
approach.

CogSci 2022

Criticality-Based Advice in Reinforcement Learning

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible. The experience for on-site participants would closely approximate a normal pre-COVID *ACL conference, with 5-6 thematically organized parallel sessions and live Q/A and interactive discussion immediately after the talks. Presentations by virtual participants will be equitably interleaved with those of on-site participants, projected on the auditorium screens as if on-site, and also followed immediately by live Q/A and interactive discussion at a time during reasonable waking hours for the virtual presenter. For all participants, on-site and virtual, who are unable to attend a session due to either time-zone issues or because they are participating in another session live, talk recordings and slides will be available online at a minimum after the live presentation (and in many cases before as well), and questions may be submitted in advance on session-specific discussion boards and answered live in session with the usual visual aids if desired.

<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/ceby/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

Please Note: The EMNLP registration system is not currently connected to the underline site as we are still in the process of building out EMNLP 2021. You will receive access instructions from underline the week of November 1st. 

Access is given only to EMNLP upon registration, if you have not registered please do so [here](https://2021.emnlp.org/registration).

Registered attendees will receive access the week of November 1st.

EMNLP 2021

EMNLP 2021 is planned to be a hybrid event in Punta Cana, Dominican Republic, with both on-site and fully virtual participation possible.

technical paper

Join us for CogSci2022! Our 44th Annual Conference will be centered around the theme of diversity, with an accompanying range of keynotes and invited speakers who themselves highlight the diversity of our field. Our three symposia are Cognition across Diverse Populations, Cognition within Diverse Societies, and Global Challenges for Cognitive Science. The conference is being offered in a fully hybrid environment for onsite and online attendees.
<br>
<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/07/Cogsci-2022-PAAG-Detailed-July-12final-1.pdf)
<br>[**The Detailed Program can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/08/DeArmond-CogSci2022-program-v113-final.pdf)
<br>
<br>

This conference requires a registration to attend. Please use the link below to secure your ticket. The website will be available to all registered attendees on July 25th. 

[Register to Attend](https://cognitivesciencesociety.org/registration/)

Registration Is Required

The 44th Annual Meeting of the Cognitive Science Society highlighted research on the theme Cognitive Diversity in addition to the full breadth and diversity of research topics offered by the society’s membership. 





Human communication usually exhibits two fundamental and essential characteristics under environmental pressure, efficiency, i.e., using less communication frequency to achieve comparable performances in cooperation; and robustness, i.e., maintaining a relative performance when communicating in a complicated environment. Since a critical goal of designing artificial agents is making them human-like in many scenarios. How artificial agents could learn a human-like communication mechanisms in terms of efficiency and robustness is a long-existing problem which has not yet been solved. Reinforcement Learning, due to its trail-and-error paradigm, provides a promising framework in solving the above research problem. With reinforcement learning, this paper develops architectures that help agents learn an efficient and robust communication, and carries out extensive experiments which uncover that artificial agents are cognitively capable to learn such human-like communication protocol in various environments (tasks).

Achieving Consensus to Learn an Efficient and Robust Communication via Reinforcement Learning

Deep learning (DL) makes it possible to automate the work of theorists, developers, and programmers in creating intelligent agents. At the same time, it has limitations related to the statistical nature of the method, which largely ignores available domain knowledge and requires large training datasets. These limitations become a barrier in the field of modeling and characterization of human social-emotional behavior. Biologically inspired cognitive architectures (BICA) are based on facts from cognitive and brain sciences and therefore can be successful here, although at the great cost of intellectual human labor. The challenge therefore is to integrate the two approaches, combining their strengths and compensating for their weaknesses. Here a particular form of such integration is presented, which involves a scaffolding of DL by BICA. Experimental paradigms include a virtual registrar, a virtual partner dance, a virtual clownery, and more. This work was supported by the Russian Science Foundation Grant #22-11-00213.

Socially emotional intelligent agents based on BICA and deep learning

We follow up on recent work demonstrating clear advantages of lexical-to-sublexical feedback in the TRACE model of spoken word recognition. The prior work compared accuracy and recognition times in TRACE with feedback on or off as progressively
more noise was added to inputs. Recognition times were faster with feedback at every level of noise, and there was an accuracy advantage for feedback with noise added to inputs. However, a recent article claims that those results must be an artifact of
converting activations to response probabilities, because feedback could only reinforce the “status quo.” That is, the claim is that given noisy inputs, feedback must reinforce all inputs equally, whether driven by signal or noise. We demonstrate that the feedback advantage replicates with raw activations. We also demonstrate that lexical feedback selectively reinforces lexically-coherent input patterns – that is, signal over noise –
and explain how that behavior emerges naturally in interactive activation.

How Feedback in Interactive Activation Improves Perception

Deep reinforcement learning agents such as AlphaZero have achieved superhuman strength in complex combinatorial games. By contrast, the cognitive science of planning has mostly focused on simple tasks for experimental and computational tractability. Using a board game that strikes a balance between complexity and tractability, we find that AlphaZero agents improve in value function quality and planning depth through learning, similar to human in previous modeling work. In addition, these metrics reflect causal contributions to AlphaZero's playing strength. Yet the strongest contributor is the policy quality. The decrease in policy entropy also drives the increase in planning depth. The contribution of planning depth to performance is lessened in late training. These results contribute to a joint understanding of machine and human planning, providing an interpretable way of understanding the learning and strength of AlphaZero, while generating novel hypothesis on human planning.

Comparing Machine and Human Learning in a Planning Task of Intermediate Complexity

Language reflects our construal of the world which is grounded in everyday sensorimotor experience. Thus, when we experience a particular world including our real physical world, linguistic explanations about it induce particular ways to see it which sometimes influences our behaviors. This study explores such effects of language, especially when the world is virtually presented on a head-mounted display (HMD). We showed university student participants an on-campus view, which was familiar to them but captured from a height of three meters, via a HMD with either of the following two linguistic instructions, “You gradually grow huge.” and “You gradually float up in the air.” Then, they were asked to produce some physical actions while wearing the HMD and answer questions on their bodily images and imagination levels. We discuss a possible interrelationship between language, visual perception, imagination and bodily action, including how they facilitate or restrict each other.

Our construal and bodily actions are impacted by changes in language and virtual viewpoints

Subway systems in large cities witness high volumes of commuter traffic, with crowded coaches and limited seats. In such scenarios, commuters often carefully position themselves in strategic locations with the aim of maximizing their chances of getting a place to sit. While user behavior in subways around the world have been the focus of multiple studies in the past, these everyday acts of ‘optimal decision making’ is of particular interest to the cognitive scientist. This paper inquires into commuter positioning choices in simulated subway coaches, within the framework of Bayesian probabilistic modelling. Data on preferred standing positions were collected across 20 subjects for 30 co-passenger configurations, through an interactive computer game. A generative model based on a Bayesian network involving three key spatial parameters was constructed, and used for inferring preferred positions conditioned on the specific configurations. The model was able to accurately simulate the quick and intuitive decisions made by the players under constraints of time, and also effectively capture noise in responses across subjects

Where would you stand on the subway? A Bayesian framework for modeling commuter positioning choices in simulated subway coaches

Reasoners consider some differences in geometric features as more similar than other differences in geometric features. For example, humans and other animals consider horizontal mirror images more similar to each other than vertical mirror images or objects that have different orientations around a point in the vertical plane. Previous theories that explain sensitivity to geometry in terms of Euclidean principles cannot account for these categorization tendencies because in Euclidean geometry, rotation is an invariant transformation. There is thus no difference between the vertical and horizontal axes in Euclidean geometry. We propose a new theory that can account for these tendencies because it considers the role that gravity plays in how minds categorize geometric objects. Our theory makes new predictions about how reasoners categorize geometric objects in different contexts: 3D contexts with gravity, 3D contexts with gravity where objects were manipulated, and 2D contexts. Five experiments with adults support our theory.

When Newton beats Euclid: intuitive physics underlies sensitivity to geometry

This work analyzes how we can understand science and technology as enactive practices, and how that characterization helps promoting an epistemology that does not rely only over the epistemological processes of science and technology, but rather brings into play other categories that help other types of reflections, such as science and technology in the face of cultural diversity. The idea that cognitive technologies can be understood as scaffoldings for developments and innovations within enactive practices is used and developed, to lead to the understanding that cultural variety plays an essential role in understanding the diversity of practices based on differentiation of the specialization of skills in relation to the media through affordances. This allows proposing a proposal for critical intercultural ethics based and understood from enactive practices.

Intercultural enactive ethics: an approach from science and technology understood as enactive practices

Body perception can be influenced by internal (body satisfaction) and external pressures (social comparison). In this study, 120 young women (18-to-30-year-olds) performed two tasks: they estimated the weight of 15 female body images and they compared their weight to that of the images presented in random order. For each estimation, they provided confidence judgments using a Likert scale. Afterward, they completed the Sociocultural Attitudes Questionnaire Towards Appearance (SATAQ-4R) and the Body Shape Questionnaire (BSQ-21) to evaluate the perceived social pressure on the body image. Estimation accuracy in the first task was predicted by confidence judgments. Estimation accuracy in the weight comparison tasks was predicted by Body Measuring Index (BMI), weight concern, body dissatisfaction, and peer pressure. Results show that when comparing our weight to others, our own weight, how we feel about our body, and the perceived pressure to change it determines how accurately we compare it to other people’s bodies.

How heavy do you think you are? Self-weight estimation is modulated by body dissatisfaction and weight concerns.

Research suggests individual differences in motor skills may be associated with reading ability in school-age children. This meta-analysis explored whether deficits in motor skills are evident in developmental dyslexia, using estimates from 33 studies (k) with 122 effects (m) from dyslexic and age-matched neurotypical samples (N dyslexia = 1248, M age = 11.6 years, range 7.0 to 25.3 years). An overall effect of moderate magnitude, g = –.52 [95% CI –.74; –.29], confirmed that motor skills are impaired in dyslexia. Meta-regression analyses indicated no significant effects of participant age or language. Subgroup analyses revealed significant group differences for tasks with nonlinguistic stimuli (k = 28, m = 83), e.g., pegboard task, g = –.46 [–.71; –.22], or linguistic stimuli (k = 10, m = 39), e.g., word copying task, g = –.66 [–1.09; –.24]. Effects were significant for groups with confirmed dyslexia diagnoses (k = 27, m = 103), g = –.56 [–.81; –.30], but not for groups identified as poor readers, (k = 6, m = 19), g = –.32 [–.93; .28]. Effects were significant for fine motor skills (k = 29, m = 106), g = –.61 [95% CI –.83; –.39], but not for gross motor or composite measures (k = 10, m = 16), g = –.26 [95% CI –.76; .24]. The results suggest that fine motor tasks might help to identify children at risk of dyslexia. Longitudinal research may further elucidate relations between motor and reading skills.

Premium content

Downloads

Next from CogSci 2022

Achieving Consensus to Learn an Efficient and Robust Communication via Reinforcement Learning

Similar lecture

Detecting Health Advice in Medical Research Literature