China

Repetitive practice with isomorphic math problems is crucial for mathematics education. To automate the creation of such problems at scale, recent techniques using Large Language Models(LLM) have shown promise, yet they remain unsuitable for production environments due to their failure to ensure mathematical correctness and structural consistency. To bridge this gap, we introduce the task of Isomorphic Math Problem Generation (IMPG) and propose Computational Blueprints for Isomorphic Twins (CBIT) to tackle it. In this framework, the LLM authors a reusable program that generates problems, rather than the problem text itself. Extensive experiments demonstrate that CBIT achieves mathematical accuracy, structural consistency, and efficiency. Furthermore, problems generated via CBIT shows competitive performance against those from human experts when deployed in a real-world educational setting.

EMNLP 2025

Computational Blueprints: Generating Isomorphic Math Problems with Large Language Models

problem generation

prompt engineering

large language model

math

education

## Welcome!
"I am excited to welcome you to this year’s edition of the Conference on Empirical Methods in Natural Language Processing! Importantly, it marks the 30th edition of EMNLP. With over 8,000 submissions, more than 3,000 accepted papers, and thousands of attendees, we have come a long way from that first
workshop, which had 14 accepted papers. As the field looks ahead, Suzhou is the fitting location for celebrating this milestone: rooted in a long literary tradition, yet modern and forward-looking, and home to a large share of the NLP community."<br>

*Message from the General Chair, Dirk Hovy*





<br>

Celebrate 30 Years of EMNLP! 
EMNLP 2025 will be held in Suzhou, China from November 5th to November 9th, 2025.

poster

Recent advances in Large Language Models (LLMs) have brought significant improvements to various service domains, including chatbots and medical pre-consultation applications. In the healthcare domain, the most common approach for adapting LLMs to multi-turn dialogue generation is Supervised Fine-Tuning (SFT). However, datasets for SFT in tasks like medical pre-consultation typically exhibit a skewed turn-count distribution. Training on such data induces a novel failure mechanism we term **Format Inertia**, where models tend to generate repetitive, format-correct, but diagnostically uninformative questions in long medical dialogues. To mitigate this observed failure mechanism, we adopt a simple, data-centric method that rebalances the turn-count distribution of the training dataset. Experimental results show that our approach substantially alleviates Format Inertia in medical pre-consultation.

Format Inertia: A Failure Mechanism of LLMs in Medical Pre-Consultation

Developing document understanding models at enterprise scale requires large, diverse, and well-annotated datasets spanning a wide range of document types. However, collecting such data is prohibitively expensive due to privacy constraints, legal restrictions, and the sheer volume of manual annotation needed - costs that can scale into millions of dollars. We introduce FlexDoc, a scalable synthetic data generation framework that combines Stochastic Schemas and Parameterized Sampling to produce realistic, multilingual semi-structured documents with rich annotations. By probabilistically modeling layout patterns, visual structure, and content variability, FlexDoc enables the controlled generation of diverse document variants at scale. Experiments on Key Information Extraction (KIE) tasks demonstrate that FlexDoc-generated data improves the absolute F1 Score by up to 11% when used to augment real datasets, while reducing annotation effort by over 90% compared to traditional hard-template methods. The solution is in active deployment, where it has accelerated the development of enterprise-grade document understanding models while significantly reducing data acquisition and annotation costs.

FlexDoc: Parameterized Sampling for Diverse Multilingual Synthetic Documents for Training Document Understanding Models

Input errors in question-answering (QA) systems often lead to incorrect responses. Large language models (LLMs) struggle with this task, frequently failing to interpret user intent (misinterpretation) or unnecessarily altering the original question's structure (over-correction).
We propose QuestionRAG, a framework that tackles these problems. To address misinterpretation, it enriches the input with external knowledge (e.g., search results, related entities). To prevent over-correction, it uses reinforcement learning (RL) to align the model's objective with precise correction, not just paraphrasing.
Our results demonstrate that knowledge augmentation is critical for understanding faulty questions. Furthermore, RL-based alignment proves significantly more effective than traditional supervised fine-tuning (SFT), boosting the model's ability to follow instructions and generalize. By integrating these two strategies, QuestionRAG unlocks the full potential of LLMs for the question correction task.

Knowledge-Augmented Question Error Correction for Chinese Question Answer System with QuestionRAG

Creating robust occupation taxonomies, vital for applications ranging from job recommendation to labor market intelligence, is challenging.
Manual curation is slow, while existing automated methods are either not adaptive to dynamic regional markets (top-down) or struggle to build coherent hierarchies from noisy data (bottom-up). We introduce CLIMB (CLusterIng-based Multi-agent taxonomy Builder), a framework that fully automates the creation of high-quality, data-driven taxonomies from raw job postings. CLIMB uses global semantic clustering to distill core occupations, then employs a reflection-based multi-agent system to iteratively build a logically coherent hierarchy. On three diverse, real-world datasets, we show that CLIMB produces taxonomies that are more coherent and scalable than existing methods and successfully capture unique regional characteristics. We release our code and datasets at \url{https://anonymous.4open.science/r/CLIMB }.

Building Data-Driven Occupation Taxonomies: A Bottom-Up Multi-Stage Approach via Semantic Clustering and Multi-Agent Collaboration

Evaluating long-form answers in high-stakes domains such as law or medicine remains a fundamental challenge. Standard metrics like BLEU and ROUGE fail to capture semantic correctness, and current LLM-based evaluators often reduce nuanced aspects of answer quality into a single undifferentiated score. We introduce DeCE, a decomposed LLM evaluation framework that separates precision (factual accuracy and relevance) and recall (coverage of required concepts), using instance-specific criteria automatically extracted from gold answer requirements. DeCE is model-agnostic and domain-general, requiring no predefined taxonomies or handcrafted rubrics. We instantiate DeCE to evaluate different LLMs on a real-world legal QA task involving multi-jurisdictional reasoning and citation grounding. DeCE achieves substantially stronger correlation with expert judgments (r=0.78), compared to traditional metrics (r=0.12) and pointwise LLM scoring (r=0.35). It also reveals interpretable trade-offs: generalist models favor recall, while specialized models favor precision. Importantly, only 11.95% of LLM-generated criteria required expert revision, underscoring DeCE’s scalability. DeCE offers an interpretable and actionable LLM evaluation framework in expert domains.

Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responses

Telemarketing towards merchants is considerably more complex than traditional dialogue systems. 
Given a user utterance, the response must not only follow the context but also strategically and naturally guide the conversation toward marketing objectives. 
A common approach is to fine-tune LLMs using high-quality dialogue data from top sales. 
However, we find that even after careful data cleaning, these data cannot be used directly due to two issues:
(1) Poor strategy-following: Real-world conversations are highly random with much chit-chat topics, easily leading deviation from intended strategy.
(2) Insufficient expert knowledge learning: Expert knowledge appears infrequently or not at all in limited collected corpus.
To this end, we introduce a hybrid data synthesis framework with two main innovations. 
First, we unify the input schema with profile and strategy designed by top sales, and extract them via a Multi-task paradigm.
In addition, we propose Role-playing Simulation and Session Prefix Completion to complementarily improve the strategy-following and inject long-tail expert knowledge into our fine-tuned model -- TeleBot.
Comprehensive online and offline evaluations demonstrate its effectiveness.
In particular, in terms of the final marketing results -- High Intention Rate, TeleBot reaches the performance level of the top 25% of human sales.

Advancing E-commerce Merchants Telemarketing with Synthetic Data-Driven LLMs

Limited access to mental healthcare resources hinders timely depression diagnosis.
Social media platforms present a valuable data source for early detection, yet this task faces two significant challenges: 1) the need for medical knowledge to distinguish clinical depression from transient mood changes, and 2) the dual requirement for high accuracy and model explainability.
To address this, we propose DORIS, a framework that leverages Large Language Models (LLMs).
To integrate medical knowledge, DORIS utilizes LLMs to annotate user texts against established medical diagnostic criteria and to summarize historical posts into temporal \textit{mood courses.}
These medically-informed features are then used to train an accurate Gradient Boosting Tree (GBT) classifier.
Explainability is achieved by generating justifications for predictions based on the LLM-derived symptom annotations and mood course analyses.
Extensive experimental results validate the effectiveness and interpretability of our method, highlighting its potential as a supportive tool for mental health professionals.

Medical Knowledge-Guided Depression Detection on Social Media with Large Language Models

Recent investigations into effective context lengths of modern flagship large language models (LLMs) have revealed major limitations in effective question answering (QA) and reasoning over long and complex contexts for even the largest and most impressive cadre of models. While approaches like retrieval-augmented generation (RAG) and chunk-based re-ranking attempt to mitigate this issue, they are sensitive to chunking, embedding and retrieval strategies and models, and furthermore, rely on extensive pre-processing, knowledge acquisition and indexing steps. In this paper, we propose Tagging-Augmented Generation (TAG), a lightweight data augmentation strategy that boosts LLM performance in long-context scenarios, without degrading and altering the integrity and composition of retrieved documents. We validate our hypothesis by augmenting two challenging and directly relevant question-answering benchmarks --- NoLima and NovelQA --- and show that tagging the context or even just adding tag definitions into QA prompts leads to consistent performance gains over the baseline --- up to 17\% for 32K token contexts, and 2.9\% in complex reasoning question-answering for multi-hop queries requiring knowledge across a wide span of text.

Tagging-Augmented Generation: Assisting Language Models in Finding Intricate Knowledge In Long Contexts

Recent advancements in slow-thinking reasoning models have shown exceptional performance in complex reasoning tasks. However, their tendency for "overthinking" on simple problems leads to excessive computational resource usage and increased inference latency, which hinders their widespread industrial adoption. While current mitigation strategies uniformly reduce reasoning tokens, they risk degrading performance on challenging tasks that require extended reasoning. This paper introduces Difficulty-Adaptive Slow-Thinking (DAST), a novel framework that enables models to autonomously adjust Chain-of-Thought (CoT) length based on problem difficulty. We propose a Token Length Budget (TLB) metric and leverage budget-aware preference optimization to implement DAST, which penalizes inefficiency on simple problems while incentivizing deep reasoning for complex ones. Experiments demonstrate DAST's significant value for practical application: it effectively mitigates overthinking, substantially lowering costs and latency—while crucially preserving high accuracy on complex problems, paving the way for the efficient application of advanced reasoning models.

DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models

Embedding retrieval systems learn a shared semantic representation space for queries and items, enabling efficient retrieval through an approximate nearest-neighbor search. However, current industrial implementations face a critical limitation: using a fixed retrieval cutoff for all queries inevitably compromises performance, yielding insufficient recall for high-frequency (head) queries and reduced precision for low-frequency (tail) queries. This persistent challenge stems fundamentally from the frequentist paradigms dominating existing loss function designs. In this work, we introduce a novel framework, probabilistic Embedding-Based Retrieval (pEBR): maximum likelihood estimation-based and contrastive estimation-based, which learns the underlying probability distribution of relevant items for each query, compute adaptive cosine similarity cutoffs via probabilistic cumulative distribution functions (CDF), and automatically adapts to the distinct characteristics of head vs. tail queries. Experiments and ablation studies demonstrate that pEBR simultaneously improves precision and recall while maintaining computational efficiency.

Downloads

Next from EMNLP 2025

Format Inertia: A Failure Mechanism of LLMs in Medical Pre-Consultation

Similar lecture

From A and B to A+B: Can Large Language Models Solve Compositional Math Problems?

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from EMNLP 2025

Format Inertia: A Failure Mechanism of LLMs in Medical Pre-Consultation

Similar lecture

From A and B to A+B: Can Large Language Models Solve Compositional Math Problems?

Downloads