Singapore

Training large neural network models requires extensive computational resources, often distributed across several nodes and accelerators. Recent findings suggest that it may be sufficient to only exchange the fast moving components of the gradients, while accumulating momentum locally (Decoupled Momentum, or DeMo). However, DeMo assumes that models fit on a single accelerator. We relax this assumption and introduce FlexDeMo, whereby nodes fully shard model parameters locally between different accelerators, while inter-node communication is reduced by synchronizing only fast-moving components instead of the full gradients -- resulting in a hybrid sharded data parallel training strategy. We further introduce a framework, denoted as DeToNATION, that generalizes DeMo, FlexDeMo, and other popular distributed training schemes such as DiLoCo -- introducing new variations of replication schemes and challenging choices made in DeMo. Our results across language and vision domains show that FlexDeMo attains similar validation loss as hybrid sharded data parallel training employing AdamW and full gradient synchronization, while being substantially faster. FlexDeMo is thus a promising distributed training scheme for the largest machine learning models.

AAAI 2026

DeToNATION: Decoupled Torch Network-Aware Training on Interlinked Online Nodes

efficient ml / green ai

scalability of ml systems

distributed machine learning & federated learning

The purpose of the AAAI conference is to promote research in artificial intelligence (AI) and scientific exchange among AI researchers, practitioners, scientists, and engineers in affiliated disciplines. 

AAAI 2021

poster

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Conformal prediction (CP) is a general framework to quantify the predictive uncertainty of machine learning models that uses a set prediction to include the true label with a valid probability.
To align the uncertainty measured by CP, conformal training methods minimize the size of the prediction sets.
A typical way is to use a surrogate indicator function, usually Sigmoid or Gaussian error function.
However, these surrogate functions do not have a uniform error bound to the indicator function, leading to uncontrollable learning bounds.
In this paper, we propose a simple cost-sensitive conformal training algorithm that does not rely on the indicator approximation mechanism.
Specifically, we theoretically show that minimizing the expected size of prediction sets is upper bounded by the expected rank of true labels.
To this end, we develop an importance weighting strategy that assigns the weight using the rank of true label on each data.
Our analysis provably demonstrates the tightness between the proposed weighted objective and the expected size of conformal prediction sets.
Extensive experiments verify the validity of our theoretical insights, 
and superior empirical performance over other conformal training in terms of predictive efficiency with $21.38\\%$ reduction for average prediction set size.

Cost-Sensitive Conformal Training with Provably Controllable Learning Bounds

Prompt tuning enables Vision-Language Models (VLMs) to efficiently adapt to new tasks through learnable prompt vectors. This naturally raises a question: do these prompts leak private information about their training data? While Membership Inference Attacks (MIAs) can quantify this risk, current methods rely on access to model outputs or internal gradients. This limitation prevents a clear assessment of a prompt’s standalone privacy leakage, particularly in deployment scenarios where such information is inaccessible. 
In this paper, we propose Prompt Intrinsic Privacy Risk Analyzer (PIPRA) to address this gap. As the first output-free MIA, PIPRA leverages open-source pre-trained VLMs to extract features from both prompts and samples within a shared cross-modal semantic space. By employing a contrastive learning-based feature projector to enhance these representations, PIPRA enables a subsequent discriminator to effectively perform membership inference.
Extensive experiments across nine benchmark datasets and multiple VLMs show PIPRA achieves an average AUC of 87.58\%, significantly outperforming traditional output-dependent methods (77.05\%). These findings reveal that prompts pose a substantially greater privacy risk than previously recognized, highlighting the urgent need for prompt-level privacy protection.

Your Prompts Are Not Safe: Output-Free Membership Inference via Prompt Vectors in Vision-Language Tuning

Federated learning (FL) enables collaborative model training
without centralizing data. In multi-domain scenarios with
non-identically and independently distributed (non-IID) data,
prediction performance is often hindered by catastrophic forgetting
of specialized local knowledge and negative transfer
from conflicting client updates. To address these challenges,
we propose a personalized FL framework with dual-branch
(pFedDB) structure and a two-phase training protocol. The
dual-branch architecture separates the model into a shared
branch for cross-client aggregation and a private branch that
remains on each local client. The private branch is never overwritten
by server updates, which prevents the catastrophic
forgetting of domain-specific knowledge. This structure also
significantly reduces communication overhead per round as
only the shared branch is transmitted. To mitigate negative
transfer, our two-phase protocol first establishes a personalized
knowledge anchor by training a single-branch expert
model on each client’s local data. In the second phase, the locally
trained model is cloned to initialize private and shared
branches. Only the shared branch is aggregated in federated
training. This process enables the shared branch to learn a
general representation that complements the established local
expertise. This design consistently improves the performance
of every client over its single-domain baseline, overcoming
the challenge of negative transfer among clients. Experiments
on our new Chest-X-Ray-4 suite and three public benchmarks
show that the proposed pFedDB method obtains 30% saving
in communication overhead per round and competitive or better
accuracy performance than recent FL methods.

Decoupling Shared and Personalized Knowledge: A Dual-Branch Federated Learning Framework for Multi-Domain with Non-IID Data

The escalating global demand for mental health services highlights the potential of Large Language Models (LLMs) in psychological counseling. However, current LLM-based approaches, particularly fine-tuned models, are constrained by data distribution biases, leading to limited therapeutic diversity and personalization. Crucially, they often lack anticipatory empathetic reasoning, struggling to foresee patient emotional responses beyond immediate dialogue history, and incur substantial computational costs.
To address these limitations, we propose PsyPARSE, a novel training-free framework for psychological counseling that emulates the deliberate and empathetic reasoning of human counselors. PsyPARSE integrates Multi-Therapy Retrieval-Augmented Generation (RAG) to overcome data biases and provide highly personalized therapeutic approaches tailored to individual patient attributes. Pioneering the first multi-stage slow-thinking engine in mental health LLMs, PsyPARSE employs Multi-Turn Rollouts to identify optimal therapeutic paths and through anticipating patient reactions, optimizes empathetic responses, thereby ensuring genuinely empathetic and impactful responses in complex, long-dialogue interactions.
Operating as a plug-and-play solution, PsyPARSE avoids the computational burden of fine-tuning. We establish a comprehensive LLM-based patient-therapist agent simulation framework for evaluation. Extensive experiments demonstrate that PsyPARSE significantly enhances the capabilities of various LLM baselines, achieving superior personalization and deeper empathy compared to both fine-tuned and other training-free methods. This work offers an efficient, adaptable, and scalable solution for advancing mental health support.

PsyPARSE: Retrieval-Augmented Slow Thinking for Personalized Empathetic Counseling

Fair incomplete multi-view clustering (FIMVC) confronts a critical yet unresolved challenge, as existing methods often fail to address the intertwined issues of data missingness and algorithmic bias simultaneously. In this paper, we propose a novel FIMVC method named Adversarial Fair Incomplete Multi-View Clustering (AFIMVC). The core of AFIMVC is a new adaptive adversarial disentanglement mechanism. This mechanism trains the feature encoder to produce representations that are invariant to sensitive attributes by adversary learning, where the adversarial intensity is dynamically controlled by the model's real-time bias. Additionally, we develop a probabilistic cross-view contrastive learning strategy to achieve semantic consistency in latent space. To handle missing data, AFIMVC employs a context-aware fusion strategy that leverages cross-sample attention to robustly synthesize a unified representation from incomplete views. Extensive experiments demonstrate that AFIMVC achieves a state-of-the-art balance between clustering accuracy and fairness, significantly outperforming existing methods.

Adversarial Fair Incomplete Multi-View Clustering

Catastrophic forgetting remains a fundamental barrier to artificial continual learning (CL) - a capability innate to humans. Existing CL methods often incur prohibitive computational costs in resource-constrained scenarios. Spiking neural networks (SNNs), with their biological plausibility and energy efficiency, offer distinct advantages for CL. Inspired by cortico-hippocampal memory mechanisms, we propose a spiking neural network framework integrating Hebbian plasticity with meta-learning, named HLML-SNN. This architecture emulates a dual-phase CL process: (1) In the short-term phase, sample-level Hebbian learning rapidly adapts to new inputs through local synaptic updates; (2) In the long-term phase, task-level meta-learning optimizes cross-task parameters using consolidated synaptic weights, mimicking cortical memory integration to refine shared representations and initialize subsequent Hebbian learning. HLML-SNN incrementally transforms short-term adaptations into stable long-term knowledge, where the synergy of rapid synaptic updates and meta-driven global optimization enables efficient continual learning while balancing stability and plasticity. Empirical results establish HLML-SNN's state-of-the-art performance across split-MNIST/CIFAR10/CIFAR100/TinyImageNet while markedly reducing training time compared to existing methods, demonstrating substantial practical potential for rapid deployment scenarios.

HLML-SNN：Fast Continual Learning in Spiking Neural Networks Achieved via Hebbian Learning-Driven Meta-Learning

Leveraging the Wasserstein distance—a summation of sample-wise transport distances in data space—is advantageous in many applications for measuring support differences between two underlying density functions. 
However, when supports significantly overlap while densities exhibit substantial pointwise differences, it remains unclear whether and how this transport information can accurately identify these differences, particularly their analytical characterization in finite-sample settings.
We address this issue by conducting an analysis of the information processing capabilities of the one-dimensional Wasserstein distance with finite samples. 
By utilizing the Poisson process and isolating the rate factor, we demonstrate the capability of capturing the pointwise density difference with Wasserstein distances and how this information harmonizes with support differences.
The analyzed properties are confirmed using neural spike train decoding and amino acid contact frequency data. The results reveal that the one-dimensional Wasserstein distance highlights meaningful density differences related to both rate and support.

On the Information Processing of One-Dimensional Wasserstein Distances with Finite Samples

Hierarchical representations provide powerful and principled approaches for analyzing many musical genres. Such representations have been broadly studied in music theory, for instance via Schenkerian analysis (SchA). Hierarchical music analyses, however, are highly cost-intensive; the analysis of a single piece of music requires a great deal of time and effort from trained experts. The representation of hierarchical analyses in a computer-readable format is also a further challenge. Given recent developments in hierarchical deep learning and increasing quantities of computer-readable data, there is great promise in extending such work for an automatic hierarchical representation framework. This paper thus introduces a novel approach, AutoSchA, which extends recent developments in graph neural networks (GNNs) for hierarchical music analysis. AutoSchA features three key contributions: 1) a new graph learning framework for hierarchical music representation, 2) a new graph pooling mechanism based on node isolation that directly optimizes learned pooling assignments, and 3) a state-of-the-art architecture that integrates such developments for automatic hierarchical music analysis. We show, in a suite of experiments, that AutoSchA performs comparably to human experts when analyzing Baroque fugue subjects.

AutoSchA: Automatic Hierarchical Music Representations via Multi-Relational Node Isolation

Recent advances in large-scale code generation models, trained in a self-supervised manner on extensive unlabeled code corpora, have led to notable progress in generating high-quality code. Despite their success in generative tasks, these decoder-only models often underperform on code understanding tasks such as code search and clone detection, due to the generation-oriented nature of their training objectives. While training a large encoder-only model from scratch on massive code data may enhance understanding performance, this approach is typically resource-intensive and time-consuming. In this paper, we explore a more efficient alternative by transferring knowledge from pre-trained decoder-only code generation models to code understanding tasks. We investigate effective strategies for enabling decoder-only architectures to learn meaningful code representations suitable for comprehension. To this end, we propose CL4D, a contrastive learning framework tailored to strengthen the representation capabilities of decoder-only models. Extensive experiments on benchmark datasets demonstrate that our approach achieves competitive or superior performance compared to existing methods on tasks such as code search and clone detection. The results indicate that CL4D improves the semantic alignment of code representations by reducing the distance between semantically similar code snippets.

Towards Better Code Understanding in Decoder-Only Models with Contrastive Learning

*Counterfactual regret minimization (CFR)* algorithms are a foundational class of methods for solving imperfect-information games, with the time average of their iterates converging to a Nash equilibrium in two-player zero-sum games. Prior state-of-the-art variants, *Discounted CFR (DCFR)* and *Predictive CFR$^+$ (PCFR$^+$)*, achieve the fastest known practical performance by improving convergence rates over vanilla CFR through discounting early iterations, with a fixed discounting scheme. More recently, *Dynamic DCFR (DDCFR)* introduced agent-learned dynamic discounting schemes to further accelerate convergence, at the cost of increased complexity. To address this, we propose *Hyperparameter Schedules (HSs)*, a remarkably simple, training-free framework that dynamically adjusts CFR discounting over time. HSs aggressively downweight early updates and gradually transition to trusting late-stage strategies, leading to substantially faster convergence with less than 15 lines of code modification. We show that HSs derived from just three small extensive-form games generalize effectively to 17 diverse games (including large-scale realistic poker) in both extensive-form and normal-form settings, without any game-specific tuning. Our method establishes a new state of the art for solving two-player zero-sum games.

Downloads

Next from AAAI 2026

Cost-Sensitive Conformal Training with Provably Controllable Learning Bounds

Similar lecture

Step-Ahead Error Feedback for Distributed Training with Compressed Gradient

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAAI 2026

Cost-Sensitive Conformal Training with Provably Controllable Learning Bounds

Similar lecture

Step-Ahead Error Feedback for Distributed Training with Compressed Gradient

Downloads