United States

&quot;Most AI/ML practitioners would agree that one of the top challenges they face is a lack of real-world data to use for research, training, and development. Scientists and engineers from NIWC Atlantic have been working to address this problem by supporting Task Force 59 to develop an infrastructure for collecting raw sensor data during maritime exercises and moving that data into a secure cloud store for later use by the fleet, S&amp;T, and a variety of other stakeholders.
This presentation will provide an overview of the NIWC Atlantic involvement with Task Force 59 and Digital Horizon 2022, with specific focus on data collection and data lake creation. Engineering for this event was performed on a highly compressed timeline with numerous challenges and unknowns, and we will share the lessons learned which reflect an approach of pragmatism and initiative. 
&quot;

NAML 2023

Digital Horizon 2022 - Creating a Data Lake in the Persian Gulf

### Welcome to the 8th annual workshop on Naval Applications of Machine Learning!

NAML is the premier annual event showcasing current machine learning research relevant to Naval applications. We welcome participation from the defense community, industry, and academia. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific<br><br>

**[More about NAML 2024](https://sites.google.com/go.spawar.navy.mil/naml/)**
<br><br>

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this workshop.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

NAML 2024

The 8th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


"Most AI/ML practitioners would agree that one of the top challenges they face is a lack of real-world data to use for research, training, and development. Scientists and engineers from NIWC Atlantic have been working to address this problem by supporting Task Force 59 to develop an infrastructure for collecting raw sensor data during maritime exercises and moving that data into a secure cloud store for later use by the fleet, S&T, and a variety of other stakeholders.
This presentation will provide an overview of the NIWC Atlantic involvement with Task Force 59 and Digital Horizon 2022, with specific focus on data collection and data lake creation. Engineering for this event was performed on a highly compressed timeline with numerous challenges and unknowns, and we will share the lessons learned which reflect an approach of pragmatism and initiative. 
"

technical paper

### Welcome to the 7th annual workshop on Naval Applications of Machine Learning!

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

**General Chairs:**

Katie Rainey, Naval Information Warfare Center Pacific

Nicole Isoda, Naval Information Warfare Center Pacific<br><br>

<button name="button" onclick="window.open('https://sites.google.com/go.spawar.navy.mil/naml/','blank')">**NAML 2023 Website**</button>

<br><br><br>

Please log in to explore this event.

The 7th annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


The future success of the Navy will depend, in part, on artificial intelligence. In practice, many artificially intelligent algorithms, and in particular deep learning models, rely on continual learning to maintain performance in dynamic environments. The software requires adaptation to maintain its initial level of performance in unseen situations. However, if not monitored properly, continual learning may lead to several issues including catastrophic forgetting in which a trained model forgets previously learned tasks when being retrained on new data. The authors created a new framework for safely performing continual learning with the goal of pairing this safety framework with a deep learning computer vision algorithm to allow for safe and high-performing automatic deck tracking on carriers and amphibious assault ships. The safety framework includes several features, such as an ensemble of convolutional neural networks to perform image classification, a manager to record confidences and determine the best answer from the ensemble, a model of the environment to predict when the system may fail to meet minimum performance metrics, a performance monitor to log system and domain performance and check against requirements, and a retraining component to update the ensemble and manager to maintain performance. The authors validated the proposed method using extensive simulation studies based on dynamic image classification. The authors showed the safety framework could probabilistically detect out of distribution data. The results also show the framework can detect when the system is no longer performing safely and can significantly extend the image classifier’s working envelope. Future follow-on work will include increasing the fidelity of the images to mimic views from the Integrated Launch and Recovery Television.

Assurance for Continual Learning Systems

Small autonomous underwater vehicles operating in complex shallow-water environments must have the ability to find passageways, determine their location, and accomplish their mission goals without external control. Sonar offers the right range of sensing distances for such systems even in the most turbid waters. However, operation in shallow waters and among complex natural and man-made obstacles is very different from the open-water sensing scenarios for which typical sonars have been designed. Unlike large vessels operating in open waters, small autonomous vehicles operating in complex underwater environment have to extract the sensory information they need from clutter echoes, i.e., echoes that consist of many contributions from unresolved scatterers and have hence to be treated as random waveforms for lack of knowledge. This sensing scenario is not unlike that of certain bat species that have mastered hunting their prey in dense vegetation based on biosonar echoes. All bats have a poor angular resolution due to their small sizes and comparatively long ultrasonic wavelengths and must hence be able to obtain all the sensory information that is required by their active, predatory lifestyles without resolving all the individual scatterers in their environments. Biomimetic robotics inspired by bats combined with deep learning offers hence a unique opportunity to revisit the problem of extracting sensory information from clutter echos without resolving the individual scatterers. Following this approach, the authors have created robots that mimic the periphery of bat biosonar systems, including their unique dynamics where the emitting (so-called "noseleaves") and receiving structures (i.e., outer ears) change their shapes while interacting with the outgoing and returning waveforms. Based on data sets consisting of biomimetic echoes from natural environments (esp. foliages), deep-learning methods successfully demonstrated that it is possible to detect narrow passageways and identify locations in natural environments without resolving the scatterers that have contributed to the echoes. For the passageway-finding problem, it was not only possible to detect narrow gaps reliably from a distance, but transparent AI methods (class-activation mapping) were able to pinpoint the rising flank of the echoes as the region that contained most of the relevant sensory information. Based on this insight, it was possible to create a neuromorphic spike code for the shape of the rising echo flank that performed almost as good as a deep neural network in detecting the gaps. The problem of identifying a location without GPS is particularly relevant for autonomous underwater navigation. Based on biomimetic sonar echoes, it has been possible to identify locations in a natural forest with a precision that was limited by the employed recreation-grade GPS reference. Finally, the impact of bats' biosonar peripheral dynamics on clutter echoes is currently under investigation.

Bats, Biomimetic Robots, Machine Learning, and Making Sense of Clutter Echoes

The lack of automation for model improvement, validation, and performance monitoring is a critical limitation on the delivery rate of AI/ML models and data products to the warfighter. Currently, the DoD fields machine learning models through traditional analysis, development, training and test processes akin to existing block cycle software update model, which can take as long as two years. In the age of software-defined systems and AI/ML driven behavior, when enemy hardware capabilities can be updated and radically altered within hours, the difference in delivery time creates a strategic vulnerability across many domains. Our research has focused on the ability to operationalize Machine Learning Operations (MLOps) at the edge, fog, and cloud with Infrastructure as Code (IaC) to accelerate delivery and provide a standard operating model in theater and for analysts back at home. With access to Electronic Warfare (EW) and flight test data analysis subject matter experts (SME), we have had the unique opportunity to create representative use cases to analyze the delivery time between data gathering, data labelling, model training, and model testing. 

Edge Deployable MLOps for Accelerated Model Delivery

The ability to detect and categorize very small unmanned aerial vehicles (UAV) is of crucial importance for situational awareness and defense of vulnerable assets. However, the ability to process timeseries data on man-portable hardware without impacting mobility is an ongoing issue in the machine learning community. We present a novel approach that leverages developments from modern neurophysiology to convert timeseries information to highly separable spectral representations, which are well suited to machine learning techniques. Specifically, we use multitaper spectral estimation (Thompson, 1982) to average spectral content over orthogonal tapers and across data-defined cyclostationary timescales. This results in a better estimate of the central spectral frequency of each bin, while simultaneously averaging out noise that is not physically related to the source. We demonstrate this technique by comparing naive networks operating on spectral data (both multi-layer perceptrons and convnets) with those operating on multitaper data. These networks are trained on experimentally acquired recording of very small consumer drones (DJI and Parrot, <5 kg gross weight) and show very high specificity and accuracy. We show that there is a statistically significant performance advantage for the multitaper method, and which holds under large training data restrictions (i.e. 90%). Our novel ML pipeline thus provides a suitable basis for continuously retrainable edge deployment of UAV detectors.

Multitaper Spectral Preprocessing Improves Performance of Lightweight Classifiers for UAV detection

Cyber Agreements for Resilient Machines Using Augmented AI

The US Navy is committed to providing top-of-the-line care as required by SECNAV INSTRUCTION 3900.41H to its valued marine mammal assets. Recent advances in machine and deep learning have made it possible to predict various disease states in humans from voice recordings. Pattern analysis of dolphin whistles has traditionally relied on human operators to differentiate between call types and measure acoustic characteristics. By using machine and deep learning for audio analysis, we can reduce human bias and improve feature pattern detection. The sound and health team has created a vocal catalogue of individual whistle recordings from 20 of the Navy’s bottlenose dolphins. Additionally, an extensive health history database for each animal is maintained as part of a preventive medicine program. Together, these result in a unique, labelled dataset comprised of tens of thousands of whistles emitted during defined health states. These data were leveraged to train a gradient-boosted classifier to classify the health status of a dolphin from audio files of a recorded whistle. Preliminary results suggest that health information is encoded in dolphin whistle characteristics, similar to changes in the human voice. Applied goals for optimizing, testing and implementing this tool to aid in the early classification of illness in Navy dolphins are discussed. 

Utilizing Machine Learning to Detect Health Changes in Navy Dolphins from Their Whistles

"Rapid advances in Artificial Intelligence, Machine Learning, and Robotics have led to the unprecedented adoption and acceptance of machine intelligence in complex domains that were previously limited to human experts.  As these innovations have continued, especially in the area of neuro-symbolic intelligence, there has been a increased interest in cognitive capabilities in machine intelligence that will allow an AI system to function less like an application or tool, and more like an interdependent teammate.  Symbiotic Artificial Intelligence holds the promise of providing both autonomous and interdependent decision support to stakeholders in society with little or no technical experience, if AI systems can better adapt to users, explain their reasoning using emotionally vulnerable methods to build relationship equity and establish trust, and understand their purpose in a mission and task based manner, and their relationship with other performers human or machine.

In this work, we review the challenges that must be addressed in order to achieve Symbiotic Intelligence, and present some basic building blocks of a novel, open-source, cognitive architecture that has resulted from our work in this space.  Our methods are designed to be flexible, to interoperate with a variety of other AI and ML components, and have been demonstrated effectively in the field in both simulated crisis decision making contexts and during the COVID-19 crisis.  In this work we present: (1) the use of interdependence analysis methods and workflow orchestration to establish a task- and mission-based sense of self for the Symbiotic AI, and theory of mind for its co-performers; (2) a trust-based relationship equity model that evaluates trust- and trust-adjacent signals to understand the comportment and state of other performers; (3) a novel manifold learning technique that improves our ability to elicit commander's intent, refine, and iterate on tasks, and converge on loyal and performant behaviors; (4) a novel method for lexical entrainment that allows the extension of neuro-symbolic methods to lexical spaces, estimating the common ground with other performers; and (5) novel methods for convergent and divergent reasoning, using lexical commands, to generate strategic surprisal.  We envision these components not as a monolithic architecture, but as more of a services architecture to provide the necessary functions for interdependence analysis.  We discuss the theory behind our methods, their practical concerns, and the direct experimental results, and their use in human-subjects experiments that provide evidence of their efficacy, especially in crisis decision making."

Leveraging Manifold Learning and Relationship Equity Management to build Symbiotic and Explainable Artificial Intelligence

The Navy is looking for new machine learning methods to improve fleet management, conduct passive surveillance, and perform target identification. Conventional machine learning methods leverage neural networks, SVMs, or a random forest approach. These are not the only methods. Oracle has developed an industry-leading prognostic algorithm called the Multivariate State Estimation Technique (MSET) to perform prognostic analysis used in fleet management, passive surveillance, and target identification. Oracle’s algorithm can be deployed on-premises, in the cloud, or at the edge, and it can be delivered in a user-friendly package for operators without the need for formal training. MSET is an advanced prognostic pattern recognition method that was originally developed by the DOE for high-sensitivity prognostic fault monitoring applications in commercial nuclear power and aerospace applications. Over the last 20 years, Oracle has pioneered a suite of intelligent-data-preprocessing (IDP) algorithms and automated tuning and sensitivity-optimization algorithms for a second generation MSET2, which possesses significant advantages over conventional ML algorithmic approaches, including neural nets. MSET2 advantages include higher prognostic accuracy, earlier warning of incipient anomalies in complex/dynamic/chaotic time-series signatures, lower false-alarm and missed-alarm probabilities, and much lower overhead compute cost, which is crucial for real-time dense-sensor streaming prognostics. MSET2 has been used in a broad range of safety-critical, mission-critical, and business applications for anomaly detection, early warning, counterfeit-component detection, and passive surveillance. Applied examples include NASA space shuttles, naval ship-propulsion prognostics, Oil-and-Gas exploration predictive and prescriptive maintenance, human-in-the-loop supervisory control, prognostic cyber security for SCADA assets, Utility grid assets, and enterprise IT assets in commercial datacenters. 

Naval Prognostic Applications for MSET2 Algorithm

Current AI systems lack adaptability: They excel in well-defined, structured environments but become unreliable in highly dynamic ones. Novelty-robust AI aims to resolve these inconsistencies by focusing on methods that maintain performance under unexpected and unforeseen conditions. Despite its promising results, however, novelty-robust research is still removed from the open-world disruptions commonplace in military operations. Our objective is to evaluate the effectiveness of this research on existing military simulations. The contributions of this presentation include an explanation of different novelty-robust planning prototypes, a comparison of their approaches and integration requirements, and a report of each prototype’s results.

Novelty-Robust AI in Military Domains

The presentation will provide an overview of what is ChatGPT.  There is much talk in the media about ChatGPT, the generative pre-trained transformer chatbot, which was launched in November 2022 by openAI. The talk will also introduce openAI as well as potential usecases of ChatGPT.  A demonstration of ChatGPT will also be provided.   

Premium content

Downloads

Next from NAML 2023

Assurance for Continual Learning Systems

Similar lecture

NIWC PAC Code 584 Machine Learning & Natural Language Processing: Review of Accomplishments & Future Direction