United States

Decisions under risk have been classically studied with tasks involving lotteries with explicit monetary rewards and uncertain gambles. More recently, sensorimotor decisions, specifically single movements to targets yielding rewards and losses, have been conceptualized as decisions under risk. While human choices between gambles have long been known not to maximize expected gains, sensorimotor decisions have been well described by statistical decision theory in many tasks. However, because many naturalistic scenarios of sensorimotor decisions are inescapably governed by the laws of physics, the question arises, how people act under such circumstances. Here, participants slid pucks to target areas, providing gains and losses in a Pre record environment so that the uncertainty inherent in motor control interacts with the physical relationships governing objects&#39; motion. Using model comparison with several generative models of participants&#39; sliding actions, we find evidence, that humans use Newtonian physics in their motor decisions for scenarios with prospective economic outcomes.


CogSci 2023

People use Newtonian physics in intuitive sensorimotor decisions under risk

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Decisions under risk have been classically studied with tasks involving lotteries with explicit monetary rewards and uncertain gambles. More recently, sensorimotor decisions, specifically single movements to targets yielding rewards and losses, have been conceptualized as decisions under risk. While human choices between gambles have long been known not to maximize expected gains, sensorimotor decisions have been well described by statistical decision theory in many tasks. However, because many naturalistic scenarios of sensorimotor decisions are inescapably governed by the laws of physics, the question arises, how people act under such circumstances. Here, participants slid pucks to target areas, providing gains and losses in a Pre record environment so that the uncertainty inherent in motor control interacts with the physical relationships governing objects' motion. Using model comparison with several generative models of participants' sliding actions, we find evidence, that humans use Newtonian physics in their motor decisions for scenarios with prospective economic outcomes.


technical paper

When we exercise sequences of actions, their execution becomes more fluent and precise. Here, we consider the possibility that exercised action sequences can also be used to make planning faster and more accurate by focusing expansion of the search tree on paths that have been frequently used in the past, and by reducing deep planning problems to shallow ones via multi-step jumps in the tree. To capture such sequences, we use a flexible Bayesian action chunking mechanism which finds and exploits statistically reliable structure at different scales. This gives rise to shorter or longer routines that can be embedded into a Monte-Carlo tree search planner. We show the benefits of this scheme using a physical construction task patterned after tangrams.


Habits of Mind: Reusing Action Sequences for Efficient Planning

Compositionality is a central component of the human faculty for generalization and flexibility. 
However, the computations involved are poorly understood, especially in terms of their cognitive costs. 
On one hand, compositionality requires searching combinatorially large hypothesis spaces, raising issues of tractability.
On the other hand, compositional representations afford efficient and compact compression. 
To shed light on the cognitive resource required for compositionality, we used a within-subject time pressure manipulation to study how participants navigated a series of mazes, generated using recursive operations over spatial primitives. 
We find evidence that behavior is guided by the use of primitives and abstract operations over them, where the degree of compositional structure increases performance and speeds up decisions.
And while time pressure led to more random errors, it did not impair the capacity for compositionality. 
Rather, participants increased their reliance on reusing and recombining previous computations, suggesting a remarkable robustness of human compositional reasoning.


Compositionality under time pressure

Previous computational models of early language acquisition have shown how linguistic structure of speech can be acquired using auditory or audiovisual learning mechanisms. However, real infants have sustained access to both uni- and multimodal sensory experiences. Therefore, it is of interest how the uni- and multimodal learning mechanisms could operate in concert, and how their interplay might affect the acquisition dynamics of different linguistic representations. This paper explores these questions with a computational model capable of simultaneous auditory and audiovisual learning from speech and images. We study how the model’s latent representations reflect phonemic, lexical, and semantic knowledge as a function of language experience. We also test how the findings vary with differential emphasis on the two learning mechanisms. As a result, we find phonemic learning always preceding lexical learning, followed by semantics. This pattern also emerges when the learner is all the time prioritizing audiovisual associative learning instead of lexical or sublexical representations. The result illustrates how the acquisition order of linguistic representations is decoupled from the primary learning objectives (mechanisms) of the learner, and how the emergence of phonemes and words can be facilitated by both auditory and audiovisual learning in a synergetic manner.


Computational Insights to Acquisition of Phonemes, Words, and Word Meanings in Early Language: Sequential or Parallel Acquisition?

Representational gestures are co-speech gestures that carry semantic content related to the content of speech. Previous studies focusing on adult-adult conversation have investigated the temporal alignment of gestures and speech finding that the overwhelming majority of representational gestures are produced right before the lexical content they refer to (their lexical affiliate, LA). However, nothing is yet known about whether caregivers would also time their gestures in the same way in naturalistic interactions. We annotated representational gestures from a large corpus (anonymized) of semi-naturalistic conversations between caregivers and their 3-4 year old children (n = 899 gestures from n=36 caregivers). We found that, just as in adult-directed language (ADL), representational gestures in child-directed language (CDL) were more tightly linked to the onset of LAs than the onset of the utterance in which Las were produced (hence planned when full events are encoded); with the overall majority of the representational gestures starting before their LAs. We further found that age of acquisition (AoA) rating of the LA had a significant effect on the speech-gesture latency. We found that for words acquired earlier, the gesture’s stroke (the meaningful part of a gesture) tended to be produced before the LA’s onset; for the later acquired word, the stroke tended to be produced at the same time or after the onset of the LA in speech. Our findings suggest that: (1) Regardless of their addressee, speakers always time the production of representational gestures to specific conceptual/linguistic units, rather than the full event/utterance. (2) In contrast to ADL, caregivers’ gestures may support addresses’ linguistic processing not only by supporting word prediction (of likely better known words), but also by supporting the learning of conceptual features (of likely less well-known words).


How Speech and Representational Gestures Align in Child-Directed Language: a Corpus-based Study

While naturalistic daylong audio recordings of children’s auditory environments have the potential to reveal key insights about the input children receive and inform our theories of language development, it also presents various methodological hurdles. In the present work, we used three fully transcribed daylong audio recordings to investigate the challenge of manually extrapolating aggregate statistics and quantify the kinds of sampling choices daylong researchers can make. Our findings highlight sampling choices that maximize sampling from the full distribution of the day and potential tradeoffs between human effort and obtaining accuracy.


Optimizing Random Sampling of Daylong Audio

Using evidential expressions to indicate one’s source of information for an utterance tends to convey uncertainty on the speaker’s part. Previous accounts of this uncertainty inference attribute it to either extralinguistic reasoning about evidence directness, or to pragmatic reasoning about alternative utterances. Here we present a novel hybrid account, and introduce a set of utterances which allows us to tease apart the three accounts’ predictions. We test these predictions in two studies by manipulating the directness of evidence indicated by an evidential expression. Exp. 1 shows that listeners infer more uncertainty with extreme values of directness. Exp. 2 shows that speakers are more likely to indicate evidence in contexts where the evidence is unreliable. We argue that these findings support an account which involves both extralinguistic and pragmatic reasoning, and develop a formal implementation of such an account within the Rational Speech Act framework.


Evidential uncertainty involves both pragmatic and extralinguistic reasoning: a computational account

Projection inferences are inferences about speaker commitment to a content embedded under an entailment-canceling operator, such as polar interrogatives with clause-embedding predicates (e.g., Does John know that Julian dances salsa?) Inferred speaker belief regarding whether Julian dances salsa is modulated by multiple factors, including the predicate used, interlocutors' prior beliefs about the content, and the content's at-issueness. We propose a first step towards a unified probabilistic model of projection inferences about content embedded under clause-embedding predicates, couched in the RSA framework. In a behavioral study, we investigate the beliefs about contents embedded under the predicates ``think'' and ``know'' that listeners ascribe to the speaker and a potential attitude holder (John). The empirical data were used to inform the model. The resulting model predictions mirror some, but not all, of the qualitative empirical patterns. This is the first step toward a systematic analysis of projection inferences using probabilistic pragmatic models.


Towards a computational account of projection inferences in clause-embedding predicates

When we reason about what could have been, some possibilities seem plausible, and others far-fetched. According to a recent theory, counterfactual possibilities are plausible if they can be generated by making local, probabilistic adjustments to the causes of what actually happened. We provide evidence that people think about counterfactuals in this way even when they have to infer the causes of what happened. We told participants about the diet of a fictional animal, and then asked them simple counterfactual questions. For example, given that the animal has eaten 1 berry today, how much food could it plausibly have eaten instead? When the amount of food eaten by the animal licensed an inference about a causally upstream variable, participants inferred the state of this variable and used it to guide their counterfactual plausibility judgments. 
More generally, the distribution over counterfactual values derived from participants' judgments was remarkably similar to the distribution predicted by the model.


Causal inference shapes counterfactual plausibility

Not too long ago most headlines talked about our fear of AI. Today, AI is ubiquitous, and the conversation has moved on from whether we should use AI to how we can trust the AI systems that we use in our daily lives. In this talk I look at some key technical ingredients that help us build confidence and trust in using intelligent technology. I argue that intuitiveness, interaction, explainability and inclusion of human domain knowledge are essential in building this trust. I present some of the techniques and methods we are building for making AI systems that think and interact with humans in more intuitive and personalised ways, enabling humans to better understand the solutions produced by machines, and enabling machines to incorporate human domain knowledge in their reasoning and learning processes.

Premium content

Downloads

Next from CogSci 2023

Habits of Mind: Reusing Action Sequences for Efficient Planning

Similar lecture

Effects of learning and feedback on risk preferences