United States

Individuals teach others about the world through language and demonstration. When might one of these modalities be more effective than the other? In this work, we study the factors that modulate the effectiveness of language vs. demonstration through the lens of multi-agent systems. We train neural network agents to teach via language or demonstration in a grounded communication task, manipulating 1) the inherent difficulty of the task and 2) the competence of the teacher. We find that teaching by demonstration is more effective in the simplest settings, but language is more effective as task difficulty increases, due to its ability to generalize more effectively to unseen scenarios. Overall, these results provide converging evidence for a tradeoff between language and demonstration as teaching modalities, and make the novel predictions that demonstration may be optimal for easy tasks, while language enables generalization in more challenging settings.

CogSci 2023

Characterizing tradeoffs between teaching via language and demonstrations in multi-agent systems

### Welcome!
The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. Our Virtual Poster Sessions will take place online Thursday, August 22, 2024.

You are required to register for this event. **Please register [here](https://2024.aclweb.org/registration). **

If you have already registered, please check your inbox for an email from Underline granting you access to ACL 2024 content.

Please register!

ACL 2024

The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. More information will be announced soon.

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference. 
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
 [**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
 

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Active learning techniques are educational tools that engage students with hands-on and interactive learning experiences. However, despite proven pedagogical advantages, active learning activities can be difficult to implement in traditional university classroom setups. We present a newly developed template for an educational escape room that can be used in a class of any size or discipline, does not require any special equipment, and is supplemented by free and accessible technology. We describe implementation of the escape room template in a plot-driven, dynamic sleep laboratory escape room, using puzzles to reinforce lecture content about circadian rhythms and sleep cycles in a large introductory neuroscience course of 300 students. This proof-of-concept innovative educational escape room framework combines escape room principles, technology, and physical paper elements to provide a new way of engaging students with complex course concepts in an interactive way, connecting best practices in education, applied neuroscience, and computer science.

An Innovative Educational Escape Room Framework for Active Learning in Classrooms

Computer programs correspond to instructions that tell a computer how to perform a task. Because computers do not understand human languages, programs have to be written in a specialized programming language. Such languages have unique syntax and semantics that on the surface appear quite different from human languages. To foster understanding what a program means, novices need to translate the programming language instructions into a familiar human language. We developed a tutoring system that provides translation activities, from the language Python into plain English, and feedback on student answers. One version of the tutoring system provides open-ended input boxes that students type translations into. A second version instead asks students to generate answers to the translation using drop-down menus. The latter provides more scaffolding but also may encourage shallow behaviors such as guessing and checking. To shed light on how these designs influence learning, we evaluate them with novice programmers. Here, we report on the results of the evaluation and next steps.

Supporting Learning of Programming through Translation Activities in a Computer Tutor

One possibility that may facilitate learning difficult tasks is building a specific curriculum, i.e., to start with simpler tasks and continue with increasing task difficulty. While such curriculum learning has long been considered and investigated in educational settings, little research has been conducted on curriculum learning in well-controlled experimental settings. To fill this gap, we investigated curriculum learning in a perceptual decision-making task. Across two experiments, we tested whether different training conditions (ascending difficulty, descending difficulty, random difficulty, easy difficulty, and difficult difficulty) affected participants' performance in a subsequent testing phase. We found that participants trained on a sequence of varying difficulty (i.e., ascending difficulty, descending difficulty, and random difficulty) outperformed others from easy-only and difficult-only training. However, we found no differences during testing between the conditions with varying difficulty. These results suggest that varying the difficulty during training yields advantages when learning the random-dot-motion task.

Does a Curriculum Improve Perceptual Decision Making?

In classrooms and media, research results are often reported with explanations of the outcome. However, past research suggests that presenting one outcome to participants reduces surprise and increases expectation of replication (hindsight bias), and explanations may enhance these effects (explanation bias). We tested whether actual outcomes and explanations of them increased difficulty of explaining alternative outcomes, and, if so, whether it biased people towards the actual outcome. Amazon Mechanical Turk (AMT) workers and introductory psychology students were divided into three groups: 1. Receiving details of four psychological studies, 2. additional receiving outcomes, 3. additionally receiving explanations of outcomes. We did not find reliable effects on difficulty of explaining alternative outcomes, and correspondingly, little evidence for hindsight or explanation biases. We speculate that this may be due to our participants' having explained alternative outcomes immediately after considering the actual outcomes.

Considering Alternative Outcomes of Research: Does Knowing the Actual Outcome Create Bias?

The establishment of international cooperative universities presents a unique challenge for educators and administrators due to cultural differences in practices around academic integrity. This is especially relevant in the context of Sino-foreign universities due to increased tolerance and decreased training concerning plagiarism in Chinese education. For many students enrolled in these universities, knowledge of academic integrity may be bound to the English language. Informed by research on the Foreign Language Effect, the present study investigated whether Mandarin-English bilingual university students (n = 106) respond differently to academic integrity dilemmas based on the language of presentation. Most interestingly, we identified a modulatory role of English proficiency with higher levels associated with a lower likelihood of plagiarism when dilemmas were presented in English. Additional findings suggest that separable dimensions of English experience interact to modulate responses to academic integrity dilemmas.

Is There a Foreign Language Effect on Academic Integrity?

Humans systematically underestimate exponential growth, which directly impacts their real-world behavior. Recent research yielded conflicting results as to the origins of this cognitive bias. In this study, we present an experiment with a short educational intervention, in which we further examine factors modulating the exponential bias in graphs. We test the hypothesis that the use of logarithmic vs. linear scales can induce misperceptions in a specific context. Moreover, we explore the effect of mathematical education by testing two groups of participants (humanities vs. formal sciences). The results confirm that when used in an inadequate context, these scales can strongly impact the interpretation of visualizations. While the log scale leads to errors in graph description, the linear scale misleads people in prediction tasks. Our educational intervention significantly reduced these difficulties, although the learning effect was greater for mathematically-skilled participants. These findings are discussed in light of a dual-process model.

The exponential growth bias in graphs: how to avoid contextual pitfalls

Past studies have suggested associations between children’s mindset and Achievement Orientation, but most research was conducted with children from Western countries. This study aims to examine how Failure and Intelligence Mindset relate to children’s Achievement Orientation across culture. An online interview was conducted with 113 Singaporean children and 122 U.S. children aged between 4 to 9, during which Achievement Orientation was measured with a hypothetical scenario, and mindset was measured with child-friendly Likert scale items adapted from previous surveys. Results showed that after controlling for age, the correlation between Intelligence Mindset and Achievement Orientation was not significant for either culture. Failure Mindset correlated with Achievement Orientation for Singaporean children but not for U.S. children. These findings suggest that Failure Mindset maybe especially important in Asian culture, where a failure-is-motivating mindset can encourage children to face challenging tasks.

Failure Mindset is Associated with Achievement Orientation Among Singaporean Children

Resources and examples of building equity into the design of rigorous education studies of cognitive interventions is lacking. Here we describe an early-stage attempt to build equity into a large-scale randomized trial of a cognitive intervention - i.e., interleaved practice - for middle-school mathematics learning. Our approach is informed by WestEd's Anti-Racist Evaluation Strategies (WestEd, 2021) which is informed by literature on equitable research and evaluation. It incorporates three overarching themes: self-reflection and learning, equitable partnerships, and understanding context. Our approach incorporates the themes into two major strands of equitable research practices: (1) engaging the study team in learning and self-reflection around issues of diversity, equity, and inclusion and (2) Engaging relevant parties in the research processes. Though early stage in the project, we will report both successes and challenges of these efforts and aim to foster discussion about how to best incorporate equity-based approaches in cognitive science research.


Embedding Equitable Research Practices into the Rigorous Study of a Cognitive Learning Intervention

In the U.S., Heritage language learners (HLLs) are bilinguals who grew up in homes where a non-English language was spoken but received English instruction from school. We used resting-state functional connectivity analysis to examine the differences in the reading networks of HLL and matched bilingual children who are biliterate in both languages (bilingual-biliterates). Compared to bilingual-biliterates, HLLs built a wider and more distributed reading network. This more distributed network included connections to regions known to be involved in motor control, verbal memory, speech production, and language control. Moreover, even though no significant difference in English reading was found, HLLs continued to show differentiated intrinsic connectivity associated with English reading scores. For HLLs only, better English reading scores were related to stronger functional connectivity between reading and language control areas. This suggests that HLLs are more likely to recruit brain regions involved in language control to achieve advanced English word reading.

Speaking a Language is Not Enough! Distinct Neurobiological Reading Networks Support Heritage Language and Bilingual-Biliterate Children’s Reading

Textbooks often consist not only of textual information but also of supplementary information. In this study, we investigated whether the comprehension of supplementary information differs when it is placed around the text or inserted into the text. 81 university students were asked to read texts with supplementary information around the text and texts with supplementary information inserted into the text, and to answer questions related to the content. They were asked to answer questions about the content. They were also asked about their reading skills and the ease with which they could understand each text. The results showed that comprehension scores were higher when additional information was inserted into the text. In addition, only participants who think they are poor readers rated the supplementary information as easier to understand and remember when it was inserted into the text. Based on these results, the better layout of the textbook was discussed.

Premium content

Downloads

Next from CogSci 2023

An Innovative Educational Escape Room Framework for Active Learning in Classrooms

Similar lecture

Revisiting Demonstration Selection Strategies in In-Context Learning