United States

This paper reports findings from the BEA 2026 Shared Task on Vocabulary Difficulty Prediction for English Learners across three L1s (Spanish, German and Mandarin). The task featured open and closed tracks, using data from the British Council&#39;s Knowledge-based Vocabulary Lists (KVL). Submissions were received from 23 teams employing diverse modelling approaches, including transformers, Large Language Models, feature-based approaches and ensembles. Results were evaluated using RMSE, with winning systems significantly exceeding the baseline and establishing new state-of-the-art benchmarks. This paper offers an examination of the participating systems, performance across tracks and L1s, and the factors that can affect prediction accuracy.

ACL 2026

Findings of the BEA 2026 Shared Task on Vocabulary Difficulty Prediction for English Learners

knowledge-based vocabulary lists

british council

difficulty prediction

mandarin

difficulty

glmm

english

spanish

shared task

vocabulary

german

lexical

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. <br>
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) <br><br><br>
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>
<br><br><br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

This paper reports findings from the BEA 2026 Shared Task on Vocabulary Difficulty Prediction for English Learners across three L1s (Spanish, German and Mandarin). The task featured open and closed tracks, using data from the British Council's Knowledge-based Vocabulary Lists (KVL). Submissions were received from 23 teams employing diverse modelling approaches, including transformers, Large Language Models, feature-based approaches and ensembles. Results were evaluated using RMSE, with winning systems significantly exceeding the baseline and establishing new state-of-the-art benchmarks. This paper offers an examination of the participating systems, performance across tracks and L1s, and the factors that can affect prediction accuracy.

workshop paper

This paper presents a Closed Track system for the BEA 2026 Shared Task on Vocabulary Difficulty Prediction for English Learners. The task requires predicting GLMM-calibrated difficulty scores for English vocabulary items across Spanish, German, and Mandarin Chinese L1 backgrounds. Our approach studies how engineered linguistic features can complement contextual multilingual transformer representations under the Closed Track setting. We evaluate 33 phonological, morphological, semantic, contextual, and cross-lingual features through feature-only regressors, Solo transformer models, Hybrid transformer models, and prediction-level ensembling. The official Closed Track submissions used XLM-RoBERTa-large Solo and Hybrid models and improved over the official baseline across all three L1 groups. We also report an extended post-submission Closed Track analysis using mDeBERTa-v3-base components and a Ridge stacking ensemble, which achieved the lowest RMSE among our evaluated systems: 1.037 for Spanish, 0.997 for German, and 0.913 for Mandarin Chinese, with a mean RMSE of 0.982. The results suggest that engineered features provide useful complementary linguistic signal, while prediction-level ensembling offers the most robust improvement for L1-aware vocabulary difficulty prediction in the Closed Track setting.

A Hybrid Transformer–Feature Ensemble for L1-Aware English Vocabulary Difficulty Prediction: System Description for the BEA 2026 Shared Task

Large language models are increasingly used to provide automated feedback on open-ended student responses. However, most current systems generate content-level corrections without explicitly modeling learners' cognitive states. This may limit pedagogical effectiveness, particularly in domains where personalized feedback is scarce. We propose a cognitively-aware generative feedback framework that integrates lightweight learner state modeling with LLM-based response generation. The system infers latent cognitive signals from student responses and conditions feedback generation on these signals. We evaluate our approach on three publicly available datasets spanning science, computing, and technical domains, comprising 2,847 responses. Compared to a static LLM baseline, cognitively-aware feedback receives higher expert ratings (+0.38 on a 5-point scale). Ablation results indicate that multiple cognitive dimensions contribute to feedback quality. The approach maintains competitive performance with reduced training data, achieving 94% of full-model performance with 200 training examples. These findings suggest that integrating cognitive signals into generative feedback is a promising direction for adaptive automated assessment.

Cognitively-Aware Generative Feedback: Integrating Learner State Modeling with Large Language Models for Adaptive Educational Support

We introduce a thermal–visual fusion approach to improve the robustness of
non-invasive Voice Activity Detection (VAD) for classroom engagement
monitoring. In noisy multi-speaker classrooms using a single microphone, acoustic-only methods fail to reliably isolate individual speakers. Our method integrates facial thermal signatures—capturing
respiratory and speech-related heat patterns—with visual lip-motion
cues to provide an acoustic-independent speech signal. This provides a localized, privacy-preserving, and
acoustic-independent indicator of speech activity.

This system acts as a visual-diarization front-end,
informing Automatic Speech Recognition (ASR) and Natural Language
Processing (NLP) systems not only when speech occurs, but precisely
which student is speaking. Using 19 engineered features, our
Thermal-Only Random Forest classifier achieved a Recall of 0.9234 and an
F1-score of 0.8105 in subject-independent evaluations, outperforming
visual-only baselines. The system was successfully deployed on a
Raspberry Pi 5, demonstrating real-time performance. These results
demonstrate that thermal–visual fusion enables more reliable linguistic
analysis of collaborative learning and provide critical input for AI
agents to facilitate group participation in real-world educational
settings that lead to more successful learning outcomes.

Inferring Student Engagement via Real-Time Thermal–Visual Voice Activity Detection

With the emergence of large language models (LLMs) as a powerful class of generative artificial intelligence (AI), their use in tutoring has become increasingly prominent. Prior works on LLM-based tutoring typically learn a single tutor policy and do not capture the diversity of tutoring styles. In real-world tutor–student interactions, pedagogical intent is realized through adaptive instructional strategies, with tutors varying the level of scaffolding, instructional directiveness, feedback, and affective support in response to learners' needs. These differences can all impact dialogue dynamics and student engagement. In this paper, we explore how tutor personas embedded in human tutor-student dialogues can be used to guide LLM behavior without relying on explicitly prompted instructions. We modify Bidirectional Preference Optimization (BiPO) to learn a steering vector, an activation-space direction that steers model responses towards certain tutor personas. We find that this steering vector captures tutor-specific variation across dialogue contexts, improving semantic alignment with ground-truth tutor utterances and increasing preference-based evaluations, while largely preserving lexical similarity. Analysis of the learned directional coefficients further reveals interpretable structure across tutors, corresponding to consistent differences in tutoring behavior. These results demonstrate that activation steering offers an effective and interpretable way for controlling tutor-specific variation in LLMs using signals derived directly from human dialogue data.

Letting Tutor Personas Speak Up for LLMs: Learning Steering Vectors from Dialogue via Preference Optimization

Large language models have strong potential for use in intelligent tutoring systems, but they often fail to follow effective pedagogical strategies, such as guiding students without revealing final answers. We propose a two-stage alignment pipeline for math mistake remediation, combining supervised fine-tuning on tutoring dialogs with Direct Preference Optimization on synthetic preference pairs. We construct a dataset that integrates existing tutoring corpora with synthetic data generated along pedagogical dimensions, such as scaffolding and factuality, and study different input configurations that incorporate solution correctness and gold answers. Experiments show that our approach improves both factual accuracy and pedagogical quality over base models and existing tutoring models. Human evaluation further indicates that our best model is competitive with a strong proprietary baseline, while providing additional benefits in terms of openness, transparency, and reproducibility. Our results highlight the effectiveness of preference-based pedagogical alignment, while also revealing challenges in reliably evaluating tutoring quality.

Towards Pedagogically Aligned LLM Tutors for Math Mistake Remediation

Intelligent Tutoring Systems (ITS) possess the ability to record learner interactions in fine-grained detail at scale. This opens the door to data-driven methods for investigating system performance and identifying points for improvement. 
In this paper, we draw on authentic log data from an English language ITS (N_logs = 5646, N_students = 368) to investigate the performance of its feedback algorithm.
In step 1 of our analysis, we profiled feedback accuracy by exploring how well the system provided error-specific feedback to malformed student answers in gap-filling grammar exercises using an expert-created set of feedback generation rules. We then identified frequently occurring student errors that triggered incorrect or unspecific feedback and refined the rule set to detect and respond to these errors with correct specific feedback.
In step 2, we validated the rule modifications on an unseen dataset. Comparing the performance of the initial and updated rule sets, we find significant improvement that generalizes to unseen data. Our study thus illustrates how an empirical evaluation of authentic data can complement feedback creators' expertise by informing rule refinement decisions that yield significant and generalizable performance improvements to feedback in ITS systems.

Using Interaction Log Data to Evaluate and Improve Feedback Accuracy in an Intelligent Language Tutoring System

Most existing math benchmarks for LLMs focus on evaluating correct solutions. In educational settings, however, it is equally important to understand whether LLMs grasp the pedagogical intent behind math problems, beyond producing the right solution. Tagging curriculum standards is challenging for the same reason: distinguishing fine-grained standards requires understanding subtle pedagogical distinctions. In this paper, we use the MathFish benchmark, which frames alignment as multi-label prediction over 385 Common Core State Standards, to evaluate a three-stage pipeline inspired by baseline failure modes in retrieval and structural reasoning: curriculum-informed hard negatives (M1), a cross-encoder re-ranker (M2), and a ReAct agent paired with an LLM-as-a-judge critic (M3). We additionally evaluate a training-free alternative (A1) combining hybrid sparse-dense retrieval with curriculum graph reranking. M3 achieves 31.3% exact match, roughly 6.5 times the three-shot GPT-4-Turbo baseline, with the largest gains from deliberative multi-step reasoning. Error analysis shows that even with these improvements, the pipeline struggles with missing predictions, grade-level misalignment, and sibling confusion, reinforcing that precise curriculum alignment is a fundamentally hard problem in educational NLP.

A Bigger Catch: Fine-Grained Curriculum Standards Alignment on the MathFish Benchmark

Beyond performance, model transparency is a crucial factor in Automated Essay Scoring, yet current systems often lack explainability, limiting their pedagogical value and users' trust. Existing explainability methods, such as gradient-based attribution or feature-importance approaches, either produce counterintuitive explanations or are too complex for classroom use. To address this limitation, we make use of fine-grained prediction at the sentence level as a way to enhance explainability. We propose ablation strategies to derive sentence-level pseudo scores from essay-level gold scores and use them to train sentence-level models. We evaluate their performance against essay-level baselines on two datasets (ASAP and MEWS), and compare their sentence-level output to a human baseline. Results indicate a trade-off between essay-level performance and sentence-level granularity. For the language quality trait, most sentence-level models achieve performance comparable to the essay-level baseline, whereas for content, the approach yields more positive results on prompts with shorter

Through the Sentence Lens: Explainable Essay Scoring through Fine-Grained Predictions

Grammatical Error Correction (GEC) requires models to make edit decisions under competing objectives: correcting errors while either minimizing changes or maximizing fluency.
However, we lack a principled characterization of how instruction-following Large Language Models (LLMs) shift their edit decisions across such editing modes, and whether standard evaluation setups faithfully reflect these shifts.
We address this gap by defining three modes—Neutral, Minimal-Edit, and Fluency-Edit—and measuring neutral-anchored performance shifts to quantify instructional sensitivity.
We benchmark seven LLMs, including proprietary and open-weight models, in a unified zero-shot prompting schema on CoNLL-2014, BEA-2019, and JFLEG datasets.
The Minimal-Edit instruction mitigates over-editing and typically boosts precision; in some settings, strong models also improve recall, suggesting more selective and effective corrections.
In contrast, the Fluency-Edit instruction often encourages broader paraphrastic rewriting that may improve perceived fluency while lowering GLEU, suggesting both a metric-objective mismatch and a shift away from targeted local correction.
Notably, Claude-Sonnet-4.5 demonstrates superior zero-shot capabilities, outperforming previously reported scores and matching or even exceeding few-shot results across CoNLL-2014 (F_0.5: 67.05), BEA-2019 (F_0.5: 64.91), and JFLEG (GLEU: 66.09).

Instruction-Following LLMs for Grammatical Error Correction: Analyzing Neutral-Anchored Instructional Sensitivity Across Editing Modes

Automated short answer grading (ASAG) with large language models (LLMs) is commonly evaluated with aggregate metrics such as macro-F1 and Cohen's kappa. However, these metrics provide limited insight into how grading performance varies across student responses of differing grading difficulty. We introduce an evaluation framework for LLM-based ASAG based on item response theory (IRT), which models grading correctness as a function of latent grader ability and response grading difficulty. This formulation enables response-level analysis of where LLM graders succeed or fail and reveals robustness differences that are not visible from aggregate scores alone. We apply the framework to 17 open-weight LLMs on the SciEntsBank and Beetle benchmarks. The results show that even models with similar overall performance differ substantially in how sharply their grading accuracy declines as response difficulty increases. In addition, confusion patterns show that errors on difficult responses concentrate disproportionately on the \texttt{partially\_correct\_incomplete} label, indicating a tendency toward intermediate-label collapse under ambiguity. To characterize difficult responses, we further analyze semantic and linguistic correlates of estimated difficulty. Across both datasets, higher difficulty is associated with weaker semantic alignment to the reference answer, stronger contradiction signals, and greater semantic isolation in embedding space. Overall, these results show that item response theory offers a useful framework for evaluating LLM-based ASAG beyond aggregate performance measures.

Content not yet available

Downloads

Next from ACL 2026

A Hybrid Transformer–Feature Ensemble for L1-Aware English Vocabulary Difficulty Prediction: System Description for the BEA 2026 Shared Task

Similar lecture

A Hybrid Transformer–Feature Ensemble for L1-Aware English Vocabulary Difficulty Prediction: System Description for the BEA 2026 Shared Task

Content not yet available

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

A Hybrid Transformer–Feature Ensemble for L1-Aware English Vocabulary Difficulty Prediction: System Description for the BEA 2026 Shared Task

Similar lecture

A Hybrid Transformer–Feature Ensemble for L1-Aware English Vocabulary Difficulty Prediction: System Description for the BEA 2026 Shared Task

Downloads