United States

Automatic prompt optimization (APO) has emerged as a powerful paradigm for improving LLM performance without manual prompt engineering. Reflective APO methods such as GEPA iteratively refine prompts by diagnosing failure cases, but the optimization process remains black-box and label-free, leading to uninterpretable trajectories and systematic failure. We identify and empirically demonstrate four limitations: on GSM8K with a defective seed, GEPA degrades accuracy from 23.81\% to 13.50\%. We propose VISTA, a multi-agent APO framework that decouples hypothesis generation from prompt rewriting, enabling semantically labeled hypotheses, parallel minibatch verification, and interpretable optimization trace. A two-layer explore-exploit mechanism combining random restart and epsilon-greedy sampling further escapes local optima. VISTA recovers accuracy to 87.57\% on the same defective seed and consistently outperforms baselines across all conditions on GSM8K and AIME2025.

ACL 2026

Reflection in the Dark: Exposing and Escaping the Black Box in Reflective Prompt Optimization

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

AAAI 2026

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

technical paper

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Understanding idiomatic and figurative language in images remains a fundamental challenge for vision–language models, as it requires reasoning beyond literal image–text alignment. Although large pretrained models such as CLIP and BLIP-2 perform well on literal recognition, they consistently fail on multimodal figurative benchmarks, often favoring visually salient but semantically literal interpretations. We show that this failure arises from a systematic literal alignment bias rather than limited model capacity. Motivated by this observation, we reformulate multimodal figurative understanding as a contrastive semantic deviation problem, where figurative images must be distinguished from visually plausible literal alternatives. We introduce a parameter-efficient adaptation of CLIP using Low-Rank Adaptation (LoRA) with hard literal negative mining, achieving targeted reshaping of multimodal representations without full fine-tuning. Experiments on the IRFL benchmark across idioms, metaphors, and similes demonstrate substantial improvements over zero-shot CLIP, BLIP- 2, ensemble-based, and knowledge-augmented baselines. Finally, we introduce FIGMENT, a multilingual figurative grounding evaluation spanning five idiom-rich languages, and show that the adapted model generalizes across languages despite being trained exclusively on English supervision.

Semantic Contrastive Adaptation for Multimodal Figurative Language Understanding

Chain-of-Thought (CoT) prompting is the dominant strategy for eliciting step-by-step reasoning in LLMs, yet its effect on model behavior remains poorly understood. We present a controlled study of when reasoning augmentation helps versus hurts in code generation — a task requiring deductive reasoning under formal constraints — spanning three architectures: a 2×2 design with Qwen2.5-Coder-1.5B and DeepSeek-Coder-1.3B (base and instruction-tuned variants) on HumanEval, MBPP, and LiveCodeBench, plus CodeLlama-7B. Our key finding is that instruction tuning reverses CoT's effect on the same base architecture: CoT significantly improves Qwen base (+13.4%, $p<0.001$) but significantly degrades Qwen instruct ($-15.2\%$, $p<0.001$). DeepSeek remains insensitive regardless of training regime (all $p>0.2$). To explain why, we apply layer-wise linear probing and find that all models encode prompt type (Direct vs. CoT) by Layer 1–4 ($>$90% accuracy) — universally, whether CoT helps or hurts. This reveals a key mechanistic insight: representation does not determine interpretation. The same internal signal for "this is a reasoning prompt" drives opposite behavioral outcomes depending on training regime. Building on this, we develop a probe-guided style router that selects from 12 prompt styles per problem via a single forward pass (84ms). The router is statistically indistinguishable from the best fixed style in 7/8 settings (McNemar's, all $p_{\text{Best}}>0.1$) and significantly outperforms CoT where CoT is most harmful ($p=0.012$, $h=+0.40$). Our findings argue for model-aware prompt strategies informed by internal representations rather than one-size-fits-all reasoning augmentation.

Think Less, Code Better: Probing When Chain-of-Thought Hurts and How to Route Around It

Garden-path sentences offer a controlled probe of incremental sentence processing because they require a reader to revise an initially plausible parse when a later region disambiguates the structure. We present an architecture-aware comparison of garden-path recovery in causal and masked language models using 100 ambiguous--control pairs (200 sentences) spanning three classic constructions: NP/Z, NP/S, and MV/RR. Causal models are evaluated with left-to-right word surprisal, whereas masked models are evaluated with pseudo-surprisal derived from masked language model scoring. Beyond the disambiguating region, we analyze cumulative excess surprisal, area-under-curve recovery summaries, and layer-wise hidden-state divergence between ambiguous and control counterparts. Across the headline-valid model set, causal models show larger disruption than masked models overall, with the clearest family-level difference on NP/Z constructions. GPT-2-medium yields the strongest NP/Z spike, while DeBERTa-v3-base shows little evidence of disruption on the same construction. Hidden-state analyses further reveal a qualitative family difference: excluding the embedding layer, masked models peak in very shallow layers, whereas GPT-2 and GPT-2-medium peak substantially deeper. These results suggest that decoder-only models exhibit stronger online disruption at the point of syntactic revision, while bidirectional encoders appear comparatively buffered at the disambiguator, consistent with their access to right context. More broadly, the findings argue that garden-path evaluation should emphasize recovery dynamics, not merely end-state plausibility or task accuracy.

Garden Path Recovery in Causal and Masked Language Models

Counterspeech offers a way to tackle harmful content online without restricting freedom of expression. This work explores counterspeech generation using small language models (SLMs) as lightweight and cost-effective alternatives to large language models. We evaluate SLMs ranging from 100 million to 3 billion parameters using simple prompting strategies as well as fine-tuning, combining automatic and robust human evaluations. Our findings demonstrate that small language models can generate relevant, coherent, and high-quality counterspeech, suggesting their potential suitability for efficient and responsible deployments.

Counterspeech Generation using Small Language Models

Shortcut learning remains a major obstacle to robust NLP systems: models can achieve high in-distribution accuracy by relying on surface cues that fail under distribution shift. We study whether shortcut reliance can be diagnosed and mitigated in small instruction-tuned language models using a simple representation-level quantity. We fine-tune Gemma 3 1B Instruct and Llama 3.2 1B on two synthetic sentiment shortcuts in SST-2 and one natural shortcut in MNLI based on lexical overlap. During training, we fit linear probes for the task label and the shortcut attribute at every layer and define CDRE as the absolute cosine similarity between the two probe directions. Across settings, increasing shortcut prevalence produces a sharp rise in the robustness gap between shortcut-aligned and shortcut-free test sets, and higher deep-layer CDRE tracks this degradation. At a 99\% shortcut ratio, Llama's clean accuracy on capitalization-biased SST-2 drops from 93.2\% at 0\% bias to 49.0\%, while Gemma drops from 91.8\% to 60.2\%. A CDRE-regularized objective substantially improves robustness for capitalization and lexical-overlap shortcuts, but offers little benefit for a speaker-prefix shortcut whose learned directions are already nearly orthogonal. These results show that probe-derived representation entanglement provides a reliable signal of harmful shortcut reliance and offers a practical criterion for determining when shortcut mitigation is likely to be effective.

Measuring and Mitigating Shortcut Reliance in Language Models with Probe-Based Representation Entanglement

Automatic Speech Recognition (ASR) has achieved strong performance for high-resource languages, but dense intra-sentential code-switched speech in African low-resource settings remains underexplored. Existing multilingual and pretrained ASR systems improve general recognition accuracy, yet they remain weak at switch regions, are sensitive to language imbalance during adaptation, and are typically evaluated with metrics that obscure switching-specific errors. This thesis proposes a self-adaptive and epistemic uncertainty-guided framework for African low-resource code-switched ASR, using Hausa--English (Engausa) and Hausa--Yor\`ub\'a as case studies. The work investigates three linked questions: (1) how to design a linguistically informed code-switched corpus with explicit switch-region annotation and labeled/unlabeled partitions for adaptive learning, (2) whether epistemic uncertainty is systematically elevated around switch regions and can guide pseudo-label selection in semi-supervised training, and (3) whether switch-aware adaptation with auxiliary language identification and boundary supervision can reduce recognition errors without increasing catastrophic forgetting. The long-term goal is to develop scalable and data-efficient ASR systems that model code-switching as a structured linguistic phenomenon rather than as noise in multilingual African speech.

Thesis Proposal: Self-Adaptive and Epistemic Uncertainty-Guided ASR of Dense Intra-Sentential Code-Switched Speech for African Low-Resource Languages

Low-resource agglutinative languages, characterized by rich morphological inflection and severe vocabulary sparsity in corpora, have long posed numerous challenges in the field of representation learning. Word-level representations preserve semantic integrity but struggle to handle sparse surface forms, whereas morpheme-level representations, though easier to learn, often lack holistic semantic information. Existing multi-granularity methods are typically modeled at the word and phrase levels, with very limited application to low-resource agglutinative languages. Focusing on the morphemes of agglutinative languages, this paper proposes MAGNet, a morphology-aware gated multi-granularity pre-training framework. At the morpheme granularity, this framework leverages morphological knowledge and integrates morpheme segmentation with morphological tagging to construct fine-grained representations. It further introduces a morphology-aware masked language modeling objective to facilitate the model in learning functional morphological regularities. Meanwhile, at the word granularity, a word-level encoder is employed to capture contextual semantics and maintain its semantic coherence.Finally, a gated fusion mechanism dynamically fuses representations of different granularities according to the context. Experiments conducted on two low-resource agglutinative languages, Mongolian and Turkish, for the tasks of dependency parsing and named entity recognition (NER) demonstrate that our method achieves consistent performance improvements over strong baseline models. Ablation studies further validate the complementary roles of morphological tagging and whole-word modeling in efficient representation learning.

Morphology-Aware Multi-Granularity Representation Learning for Agglutinative Languages

Most languages lack labeled evaluation benchmarks for large language models (LLMs). Creating such benchmarks requires native speakers, domain expertise, and answer annotation - resources unavailable for the vast majority of languages. We investigate whether a model's internal processing signals - such as generation entropy and tokenizer statistics - correlate with its actual accuracy on a language, with the long-term goal of estimating language competence without labeled data. Our key observation is that for languages a model does not know, both tokenizer segmentation and generation entropy become highly variable across questions, whereas for known languages they remain consistent. We call this the inconsistency hypothesis and test it on 11 instruction-tuned LLMs (1B-70B parameters) across 14 language-script varieties (12 Turkic plus English and Russian controls). Standard deviation outperforms the mean as an accuracy predictor in 21 of 25 greedy-pass family $\times$ measure combinations (binomial $p < 0.001$, though the measures are not fully independent); sampling-based measures show no such pattern (4 of 20). A three-feature proxy combining a tokenizer coverage ratio chars\_std / fertility), entropy standard deviation, and the model's EN/RU benchmark accuracy achieves leave-one-language-out $R^2 = 0.75$ and Kendall $\tau = 0.72$ on 154 model-language pairs, with the tokenizer ratio contributing most of the language-level signal ($\Delta R^2 = 0.34$). Removing the EN/RU anchor isolates an anchor-free component at $R^2 = 0.49$, which exceeds the model-mean baseline and captures language coverage but not model capability.

Processing Inconsistency Predicts Language Competence: LLM Evaluation Without Answer Labels on Turkic Languages

Machine unlearning aims to remove the influence of specific training data from a model while preserving reliable behavior on the remaining data, making reliable prediction and uncertainty estimation essential for evaluation. Calibration is commonly used as a proxy for reliability in language models, but low calibration error does not necessarily imply reliable decision rules, as models may rely on spurious correlations while remaining well calibrated. We investigate this gap in generative language models using the multiple-choice question-answering evaluation protocol on the TOFU benchmark, measuring probabilistic reliability with calibration metrics (ECE, MCE, Brier) and decision-rule reliability via attribution-based shortcut detection with Integrated Gradients and Local Mutual Information. We find that fine-tuned models achieve low calibration error (ECE ≈ 0.04) compared to pretrained models (ECE > 0.5), and models after unlearning retain similarly low calibration despite reduced accuracy on the forget split, while attribution analysis shows increased reliance on correlation-based tokens. These results demonstrate that good calibration can coexist with shortcut-based decision rules after unlearning, extending the reliability paradox to the machine unlearning setting.

Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models

Automated ICD coding is a critical task for standardizing clinical information from electronic health records (EHRs) and supporting downstream healthcare administration.
However, existing automated ICD coding systems face several fundamental challenges. First, the majority of existing research focuses on English ICD tasks, with limited attention to Chinese-language clinical contexts due to the scarcity of publicly available Chinese ICD datasets. 
Second, most approaches primarily target disease coding, overlooking procedure coding as well as the multi-stage workflows followed in real-world clinical practice. 
Moreover, many recent methods rely heavily on closed-source large language models or substantial computational resources, which limits their scalability and deployability in clinical environments.
To address these gaps, this paper proposes \textit{JointCoder}, which includes a real-world Chinese ICD coding dataset and a multi-agent framework that reformulates automated ICD coding as a joint disease-procedure coding task. JointCoder explicitly models real-world clinical coding workflows through stage-wise agent collaboration.
All agents are instantiated using locally deployed 1.7B-parameter models, enabling scalable and privacy-preserving deployment.
Extensive experiments on real-world Chinese ICD coding datasets demonstrate JointCoder’s superiority over state-of-the-art baselines across all evaluation metrics. Our Github repository is available at https://github.com/KJOrigin/JointCoder. Our demo website is available at http://159.89.222.214:9028/. The demo video can be accessed at https://yunpan.qlu.edu.cn:443/link/CE86D849DD24EA9EE3D68AF2FD90D539.

Downloads

Next from ACL 2026

Semantic Contrastive Adaptation for Multimodal Figurative Language Understanding

Similar lecture

MARS: Multi-Agent Adaptive Reasoning with Socratic Guidance for Automated Prompt Optimization

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

Semantic Contrastive Adaptation for Multimodal Figurative Language Understanding

Similar lecture

MARS: Multi-Agent Adaptive Reasoning with Socratic Guidance for Automated Prompt Optimization

Downloads