United States

The Villain BOT and the Large Fraud are bundled microservices that are deployed in the Noblis Edge Environment on a Docker Container. The capability has two different components: an adversarial or &quot;Abliterated&quot; Chat BOT that has had its safety features removed to enable analysts to explore cyber/fraud scenarios from a criminal&#39;s perspective, and a fraud detection model with a novel LLM-driven Text-to-SQL interface that allows non-technical analysts to ask plain language questions. The villain BOT address a security use case and highlights a microservice can provide automation to complex task to save time and resources and meet the goals enabling AI-Ready Data for Decision Superiority.

The Large Fraud Model uses raw log transaction data and transformed ML-scored feature data, vectorizing it for LLM consumption and allowing each record to be evaluated against known fraud signatures. 

This is presented to analysts to explore via a Text-to-SQL interface that greatly increases analytical insight without having to engage expensive data scientists, enabling a much more rapid query-building experience and saving time for the analyst.

The BOT uses a local LLM that is isolated from the public internet to allow analysts to explore fraud schemes, data explorations, insider threat scenarios, and vulnerabilities. The ability of the bot to answer questions that are not allowed by standard LLM safety protocols opens opportunities for research, product workflow artifacts, and technical analysis, making analysts much more efficient and insightful. 

The ability of analyst to move back and forth between raw data and plain text discussions around fraud schemes allows for analyst to analyze the data, identify the threat and then compose detection countermeasure scripting within the same application is leap forward in insight, efficiency and ability to inject new fraud detection capabilities. 

This effort is part of the Noblis Sponsored Research Program for a large civilian agency customer and has broad application to other agencies&#39; cyber programs and analyst-based environments.


NAML 2026

Villain BOT / Large Fraud Model

criminal perspective

adversarial llm

fraud research

gen ai

fraud detection

text to sql

### Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

Message from the General Chair: 
*It is my great pleasure and honor to welcome you to the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), held in beautiful Vienna, Austria, from July 27 to August 1, 2025. ACL2025continues our field’s tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.* 
[Read more](https://drive.google.com/file/d/1GI_hvOpjswAuYdUTromfeDiPpCcqidwg/view?usp=sharing)

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2025". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you need to log in with the **email address you registered with**. 

ACL 2025

Welcome to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

The Villain BOT and the Large Fraud are bundled microservices that are deployed in the Noblis Edge Environment on a Docker Container. The capability has two different components: an adversarial or "Abliterated" Chat BOT that has had its safety features removed to enable analysts to explore cyber/fraud scenarios from a criminal's perspective, and a fraud detection model with a novel LLM-driven Text-to-SQL interface that allows non-technical analysts to ask plain language questions. The villain BOT address a security use case and highlights a microservice can provide automation to complex task to save time and resources and meet the goals enabling AI-Ready Data for Decision Superiority.

The Large Fraud Model uses raw log transaction data and transformed ML-scored feature data, vectorizing it for LLM consumption and allowing each record to be evaluated against known fraud signatures. 

This is presented to analysts to explore via a Text-to-SQL interface that greatly increases analytical insight without having to engage expensive data scientists, enabling a much more rapid query-building experience and saving time for the analyst.

The BOT uses a local LLM that is isolated from the public internet to allow analysts to explore fraud schemes, data explorations, insider threat scenarios, and vulnerabilities. The ability of the bot to answer questions that are not allowed by standard LLM safety protocols opens opportunities for research, product workflow artifacts, and technical analysis, making analysts much more efficient and insightful. 

The ability of analyst to move back and forth between raw data and plain text discussions around fraud schemes allows for analyst to analyze the data, identify the threat and then compose detection countermeasure scripting within the same application is leap forward in insight, efficiency and ability to inject new fraud detection capabilities. 

This effort is part of the Noblis Sponsored Research Program for a large civilian agency customer and has broad application to other agencies' cyber programs and analyst-based environments.


technical paper

### Welcome to the 10th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. 
Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.
#### On-Demand Access

* This is an on-demand site and all the recorded content from the workshop is available here. 
* Access to the recordings is exclusively available to registered participants.
* Each registrant has received an email notification, with instructions on how to access the content.

[![](https://assets.underline.io/markdown_image/1/image/60cd12aa376c191d3467ea2ee60944ea.png)](https://naml2026.org/)

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications
* To connect machine learning researchers with experts in military requirements
* To build and strengthen collaborations within the defense research community

#### *"What we do as a Navy is a human business. Don’t pass up an opportunity to come together as a team. Don’t pass up leadership opportunities, and don’t outsource them to a PowerPoint. If you have an opportunity to come together as a team, to increase belonging, do it."*
Admiral John M. Richardson, 31st Chief of Naval Operations

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Please log in using the email address you registered with.** If you have any issues logging in, please contact us at <naml2026@underline.io>.

Let’s celebrate the past decade by investing in the next. 🚀

Please log in to explore this event.

This is an on-demand site. All recorded content from NAML 2026 is available here and access to it is exclusively available to registered participants.

**Explore the latest advancements in machine learning tailored to Naval applications 🔎**

With on-demand access to NAML 2026, you can watch all lectures & browse poster presentations at your convenience, while also engaging directly with presenters via the Underline platform - ask questions, exchange ideas, and stay connected.

**If you still haven’t registered, you can do so now.**
👇
**Take advantage of a special one-time offer: full access for just $99, with complimentary registration for students, government, and military employees.**

**Get access here: https://naml2026-ondemand.paperform.co/**

Please check your spam folder, or contact our support team at **<naml2026@underline.io>** if you registered, but still haven't received an access email.

Let’s celebrate the past decade by investing in the next. 🚀

Please register!

The 10th Annual Workshop on Naval Applications of Machine Learning (NAML 2026) was held in San Diego, California, March 2-5, 2026. NAML is a forum for interdisciplinary experts from government, industry, and academia to discuss results of ongoing research on theory and applications of machine learning to address some of the most challenging problems faced by our naval forces. 



Distributed maritime operations (DMO) demand unprecedented resilience, endurance, and autonomous decision-making from both manned and unmanned naval platforms. To address these challenges, Integer Technologies is developing DIGIT CORE—a next-generation machinery controls, health monitoring, and decision-support platform leveraging real-time, multi-domain digital twins to optimize power and energy systems. DIGIT CORE integrates high-fidelity physics-based models with operational sensor data to create a dynamic, predictive representation of the physical machinery ecosystem.
DIGIT CORE employs a rigorous, iterative development pathway: (1) Component-level validation of high-fidelity models against empirical data, (2) Generation of optimized reduced-order models for edge-deployable compute efficiency, and (3) Deployment of a Minimum Viable Digital Twin (MVDT) interfacing directly with the DIGIT CORE machinery autonomy stack. This MVDT enables continuous system health assessment, anomaly prediction, performance optimization, and prescriptive action recommendations based on first principles modeling, real-time, during mission execution. By fusing physics-based simulation with machine learning-driven analytics, DIGIT CORE transforms raw sensor data into actionable operational intelligence, ensuring data-driven decisions drive mission success.
For naval applications, DIGIT CORE delivers critical advantages: extended platform endurance through optimized energy management, enhanced resilience via predictive failure mitigation, and maximized ""time in the fight"" by minimizing unplanned downtime. Its architecture is inherently adaptable across diverse manned/unmanned platforms and mission profiles, providing a unified framework for power system resilience in contested, distributed environments. By setting a new standard for machinery control and health management, DIGIT CORE directly addresses the Navy’s need for persistent, reliable, and intelligent power solutions—ensuring platforms operate at peak effectiveness when and where it matters most. This work demonstrates the transformative potential of validated digital twins in advancing naval operational readiness through machine learning.

Physics-Driven Digital Twin Controls for Resilient Naval Power Systems

High Altitude Balloons (HABs) can leverage stratospheric wind layers for limited horizontal control, enabling applications in reconnaissance, environmental monitoring, and communications networks. Existing multi-agent HAB coordination approaches use deterministic methods like Voronoi partitioning and extremum seeking control for large global constellations, which perform poorly for smaller teams and localized missions. While single-agent HAB control using reinforcement learning has been demonstrated on HABs, coordinated multi-agent reinforcement learning (MARL) has not yet been investigated. This work presents the first systematic application of multi-agent reinforcement learning (MARL) to HAB coordination for distributed area coverage. We extend our previously developed reinforcement learning simulation environment (RLHAB) to support cooperative multi-agent learning, enabling multiple agents to operate simultaneously in realistic atmospheric conditions. We adapt QMIX for HAB area coverage coordination, leveraging Centralized Training with Decentralized Execution to address atmospheric vehicle coordination challenges. Our approach employs specialized observation spaces providing individual state, environmental context, and teammate data, with hierarchical rewards prioritizing coverage while encouraging spatial distribution. We demonstrate that QMIX achieves similar performance to the theoretically optimal geometric deterministic method for distributed area coverage, validating the MARL approach and providing a foundation for more complex autonomous multi-HAB missions where deterministic methods become intractable.

Distributed Area Coverage with High Altitude Balloons Using Multi-Agent Reinforcement Learning

State-of-the-art autonomous systems often suffer from being size, weight, and power (SWaP)-prohibitive, coupled with a heavy reliance on extensive datasets for robust learning. Such limitations underscore the critical need for artificial intelligence (AI) to enable robots to learn from environmental interactions, adapt to dynamic conditions, and make informed decisions in novel situations without explicit programming. Neuromorphic computing, a specialized branch of AI, offers a promising solution to address the SWaP constraints of conventional autonomous systems. In this work, we showcase the engineering of a neuromorphic system tailored for autonomous exploration and adaptive learning, leveraging biologically inspired associative mechanisms to improve the performance of autonomous systems and reduce the amount of training data required. This computational approach demonstrates superior efficiency compared to conventional AI frameworks, exhibiting reduced computational overhead, enhanced power optimization, and improved adaptive response capabilities for robotic applications. In particular, our methodology utilizes both virtual simulation environments and physical experimental testbeds, both featuring a controlled arena equipped with chromatic visual markers and tactile perturbation elements to strengthen the sensory association. The hardware platform is made of an AI-ready NVIDIA processor and multiple sensor modalities, including LiDAR for spatial mapping and obstacle detection, depth imaging systems for chromatic pattern recognition, and inertial measurement units for tactile feedback acquisition. The associative learning algorithm facilitates a shift from rigid and preprogrammed robotic behavior towards adaptive and experiential learning. Such a technique reduces the need for exhaustive rule-setting and allows for more flexible responses to uncertain environments.

Neuromorphic Systems to Enable AI for Autonomous Systems

Internet access has transformed Large Language Models (LLMs) into powerful tools that enable intelligent systems capable of understanding and generating up-to-date, real-time information. This significantly augments LLMs capabilities by accelerating and, in some cases, supplanting human-driven approaches. Paradoxically, this very internet access is the primary obstacle to the widespread adoption of LLMs within the Department of Defense (DoD) due to security concerns surrounding sensitive information and the lack of domain-specific context causing hallucinations. To this point, DoD systems that utilize LLMs employ various security controls limiting access to information beyond their initial training and effectiveness in specialized or mission-critical environments. This work explores a Retrieval-Augmented Generation (RAG) framework designed to align a localized LLM's reasoning and communication with specific operational contexts. By integrating curated domain knowledge into the model’s generation process, the approach allows for contextually relevant responses that extend beyond the static limits of pretraining. We focus on methods for constructing knowledge bases tailored to specific mission objectives, optimizing information retrieval pipelines, and balancing contextual domain precision with the hardware efficiency needed for real-time human–AI collaboration with open-source software and pretrained models. Ultimately, this research highlights how the implementation of RAG can evolve LLMs from general-purpose into reliable, mission-aware partners capable of relevant information retrieval and synthesis allowing responses with operational nuance without the vulnerability of internet access.
Our demonstration will provide a live look at a disconnected RAG-enabled LLM for two scenarios tailored to DoD system lifecycles and audiences. For the first demonstration we will prompt the RAG-enabled LLM with common questions from the warfighting community in the context of operating, maintaining, and troubleshooting a Navy Command-and-Control system. We will additionally demonstrate technical questions that engineers and scientists unfamiliar with tactical messaging standards may ask when trying to implement or integrate novel capabilities with MIL-STD formats (e.g. MIL-STD-6016x, MIL-STD-3011E).


GenAI.mil and AI Cattle Driving

Large Language Models (LLMs) are transforming how we interact with computational tools, enabling natural language interfaces for complex workflows. The Model Context Protocol (MCP) provides a standardized way to connect LLMs with external services, allowing for execution of tasks without writing code. This session will introduce MCP and demonstrate how it can be used to perform data analysis through conversational interaction. We’ll explore the benefits of MCP and showcase a live example of connecting an LLM to a computational engine via MATLAB MCP Server to analyze and visualize data.

Highlights include:
-Overview of MCP and its role in LLM integration
-How LLMs enable code-free workflows for technical computing
-Live demo of a MATLAB MCP-enabled data analysis workflow

How to Talk to Your Data: Leveraging MCP Servers with LLMs for Code-Free Data Analysis

This presentation will introduce a new paradigm for testing and evaluation of AI-enabled autonomous systems. In a world where AI is everywhere and is changing how research is conducted, collapsing expected timescales for innovation, and increasing efficiencies across domains how can AI help you? Industry-developed products are being rolled out as an unprecedented pace with shiny new capabilities or improved performance on specific tasks. However, translation of these performance gains to your missions is not guaranteed and introduces distinct tradeoffs. An inability to rapidly evaluate thesetradeoffs can delay delivery timelines and compromise our intelligence posture by chasing, rather than driving, state-of-the-art AI innovation. High-consequence missions present different constraints that influence Testing, Resources, Assurance, Data, and Expressivity (TRADE) considerations for development of viable solutions. Developing the technical capability, and associated testbeds, to evaluate models against mission TRADE requirements will position the US to assess and integrate AI-enabled autonomous systems at mission timescales.

Evaluating TRADE-Constrained AI for Development of Deployable Autonomous Systems

Verification and Validation (V&V) is a critical component in making Machine Learning (ML) enabled systems usable in the real world. This is even more so the case when these tools are used in mission-critical scenarios, such as potentially used by the United States Navy. Unfortunately, most tools must be evaluated as black-box systems, where proprietary architectures and sensitive training data prevent full access to data and/or models. This work introduces a practical black-box verification framework designed specifically for ML-enabled systems that are relevant to maritime perception pipelines, with tools that expose failure modes highly relevant to naval missions.
Our approach integrates four components: (1) subset- and superset-based performance evaluation (performance for classes within classes), enabling focused assessment of mission-relevant classes; (2) image-quality–conditioned testing, where controlled variations in brightness, hue, and blur simulate operational conditions such as low light, haze, sea spray, and sensor degradation; (3) black-box drift detection, using standard image-quality assessment (IQA) metrics, Laplacian-pyramid features, and model-output statistics to flag changing environmental or sensor conditions; and (4) poison detection, using STRIP entropy and spatial patch sweeps to identify potential backdoor triggers.
Across 75,000 COCO images and 20,000 maritime images from a Navy-relevant Kaggle Ships dataset, this framework reveals vulnerabilities critical to mission success. Most notably, the YOLOv8 detector experiences an F1 drop from 0.7 to 0.15 for small pixel-on-target (POT) detections under minimal brightness reduction—conditions where a trained operator can still correctly classify the object. These results highlight significant risks in using general performance statistics as a surrogate for mission-specific testing. 
Drift-detection modules further demonstrate strong sensitivity to operational shifts, detecting changes in lighting, compression, sharpness, and scene composition before major performance degradation occurs. Poison-detection and vulnerability assessment techniques can help identify if specific ML tools are compromised, and how easily they can be poisoned. It was found that Yolo style architectures, even without model access or specific version knowledge, can be exploited due to common architectural attributes. 
Overall, this work delivers a deployable, black-box verification capability tailored to maritime sensing missions, enabling operators and system integrators to more realistically assess model performance, robustness, and identify risks even when internal model details are inaccessible. The results underscore the importance of conditional, domain-aware verification for ensuring detector reliability in high-consequence naval and maritime operations.


Practical, Mission-Specific Verification and Validation of MLEnabled Systems

While the increasing power and complexity of Artificial Intelligence (AI) models has been a boon for automating rote human tasks (e.g., image classification) and identifying valuable patterns in massive amounts of data, the growing pervasive use of these models has also generated new attack-vectors which adversaries can exploit to compromise underlying model integrity to create adverse outcomes. Concern over these emerging AI-based vulnerabilities has given rise to new fields in cyber and software security that aim to harden and assure the resilience and robustness of AI algorithms prior to their deployment to production systems.

Noblis researchers are designing novel methods to exploit and test the robustness and resilience of various AI models, and using the results from said methods as part of an overall AI Risk Management Framework (AI-RMF) to assess a model’s suitability for deployment to mission-production systems.  

Our approach is novel, in part, in that it provides an extendable, scalable cloud architecture that accepts new model interrogation methods and applies them according to model type (e.g., transformer models for text classification, image-based models). The impact of this work is threefold. First (1), we aim to automate the risk-evaluation of AI models prior to their promotion to mission-production systems. Second (2), we seek to offer a repeatable, common method for assessing and comparing vulnerability risk across models to inform applied practitioners of potential model vulnerabilities and weaknesses.  Third (3), we aim to create an AI evaluation suite that can be used by multiple agencies and government to help them understand and assess the weaknesses and potential vulnerabilities of AI algorithms prior to their deployment to production.


Artificial Intelligence Assurance Implementation

In today's volatile geopolitical landscape, our adversaries are increasingly weaponizing global supply chains as modern warfare tactics. Despite coordinated efforts by the Department of War, Department of Commerce, Department of Homeland Security, and the Department of Navy to strengthen our Defense and Maritime Industrial Base, critical vulnerabilities remain in our transoceanic supply lines. 

This presentation explores an emerging simulation technology that creates digital twins of strategic maritime supply routes, with a particular focus on pathways adjacent to adversarial spheres of influence and contested regions. The conceptual framework offers new capabilities for: 

- Predicting and visualizing potential supply chain disruptions in real-time 

- Advancing modeling of potential disruption scenarios 

- Creation of flexible contingency plans, including potential existing alternative sources of supply 

- Evaluation of countermeasures prior to implementation 

- Converting traditional tabletop wargaming into dynamic digital simulation scenarios 

Examine how these digital twins and simulations can enhance wargaming by enabling scenario-based planning for industrial base resilience, potentially offering new insights for identifying vulnerabilities in critical supply chains. Further insights can be uncovered through stress-testing alternative supply routes, mitigation tactics, alternative supply sources, and countermeasures. 

We will engage the audience in an interactive discussion about each of the parameters that may be useful as part of this type of simulation framework. The audience will learn the importance of conceptual scenario-based simulation as part of strategic risk mitigation to achieve supply chain security.

Securing U.S. Lifelines: Rapid Simulation for Strategic Supply Chain Defense

Modern naval operations increasingly demand high-precision analytic judgment, large-scale information processing, and rapid anticipation of adversary behavior. Human analysts excel at nuanced reasoning, rapid adaptation to changing scenarios, and the application of deep domain knowledge. However, their work is inherently limited in scale. Technology solutions like large-scale computational systems offer speed and capacity but lack the flexibility and contextual sensitivity required for complex and dynamic operational problems. Recent advances in AI agent architectures powered by LLMs and multimodal models offer a pathway to delivering highly precise analysis at never before seen scale while simultaneously enabling more faithful modeling of adversary cognition and decision-making.

We present the concept of Precision at Scale, a framework for understanding how AI agents can augment naval analysts and operators by combining expert-level reasoning with massive, policy-driven task decomposition across thousands of concurrent analytic threads. A key contribution of this framework is in the integration of adversary-aware modeling techniques. Agents can be grounded in adversary doctrine, cultural framing, linguistic priors, and historical patterns, allowing them to approximate the planning environment and strategic logic of competitors.

Our approach integrates four core elements: 

1. Adaptive agent workflows that allow agents to understand baseline expectations grounded in adversary doctrine while retaining the flexibility to shift strategies as mission context evolves.
2. Integrating multimodal data ingest pipelines spanning long-form text, geospatial data, open-source feeds, and additional high-fidelity modalities to adapt to the full range of relevant data.
3. Applying self-verification and proactively escalating to human operators to improve reliability and reduce analyst cognitive load.
4. Leveraging distributed agent teams to handle massive problem sets without sacrificing fine-grained reasoning quality.

This framework enables both precise outputs and scaleable adversary-behavior simulation, supporting use cases such as information environment analysis, network targeting, geospatial analysis, and wargaming. We report early experimental results and lessons learned from an operational deployment during Talisman Sabre 2025, during which the system produced multiple scenario predictions that later materialized. These findings highlight both the promise and limitations of deploying agentic systems in realistic operational settings.

We conclude by outlining open research challenges in modeling adversary cognition, scaling multimodal reasoning, and building distributed infrastructure capable of supporting real-time decision workflows. Together, these contributions establish an empirical foundation and technical roadmap for AI-driven systems that augment human planners and enhance quick-turn strategic foresight in contested environments.

Premium content

Downloads

Next from NAML 2026

Physics-Driven Digital Twin Controls for Resilient Naval Power Systems

Similar lecture

Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements