United States

Machine Learning (ML) systems are able to complete many tasks more efficiently and accurately than humans, but their competency often drops when operating in untrained conditions.  The opaqueness of how most ML agents arrive at their output reduces trust of human operators or teammates even for conditions where the ML agent’s average capability exceeds that of their human counterparts.  The MindfuL™ System has been developed on DARPA’s Competency Aware Machine Learning (CAML) program and aims to solve both of these issues.
The MindfuL™ system characterizes incoming data and environmental conditions that drive competency of the ML system through Hierarchical Dirichlet Processes.  This very compact representation focuses on the elements of the ML agent’s experience that leads to specific approaches in the ML model to completing the task (strategies) and performance of the ML agent.  Leveraging input data as well as past performance and strategies of the ML agent, MindfuL™ software is able to predict future performance and strategy implementation and provides users with the context necessary to understand if an ML agent is suited to its task given the current environment and why.
The MindfuL™ system performance exceeded the Year 2 DARPA Competency Aware Machine Learning program metrics.  When applied to a Convolutional Neural Network aiming to predict obstacle depth from 2-D imagery, the MindfuL™ system was able to correctly identify more than 95% of competency controlling conditions present in the environment, to characterize the behavior of the neural network correctly 94% of the time, and to predict performance of the ML agent relevant to a human for 99% of images.
Continued development on MindfuL™ software aims to provide more semantically meaningful conditions and specific suggestions for how best an operator should use an ML agent under the current or proposed operating conditions.

NAML 2022

MindfuL™ Software: Competency Aware Machine Learning

competency aware machine learning

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=580 height=700 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

NAML 2021

The annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


poster

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community


<figure class="video_container">
  <iframe src="https://s3.amazonaws.com/pf-user-files-01/u-59356/uploads/2022-03-22/hk13mxo/NAML_ReceptionWithtitle.mp4" width=826  height=526  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop. The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

To spread awareness of current machine learning research relevant to Naval applications

To connect machine learning researchers with experts in military requirements

To build and strengthen collaborations within the defense research community



The Digital Aviation Readiness Technology Engine (DARTE) has proven highly successful in monthly predictions for readiness metrics, including the number of Mission Capable (MC) F/A-18 aircraft at the squadron level across the Naval Aviation Enterprise. We show that the DARTE method can be applied on a more granular timescale by using the method to predict squadron-level F/A-18 readiness on a daily basis, up to one week in advance. We quickly train multiple models that have different architectures and provide the model outputs to tree-based ensemble meta-learners, creating an ensemble model that can accurately predict the number of MC aircraft that a Naval F/A-18 squadron will have each day for the next week. Additionally, we interrogate the predictions with explainable AI (XAI) to provide decision makers and subject matter experts with information that they can use to derive actionable insights. Finally, these predictions and insights are displayed via dashboards.


Readiness Predictions for the MH-60 SeaHawk with the Digital Aviation Readiness Technology Engine

PMW/ISEA teams are dealing with thousands of trouble tickets daily, by prioritizing the tickets as they are submitted based upon how long the ticket is estimated to resolve will enable the help desk and the ISEA groups to make quicker decisions and prioritize their resources. Delta Duration Model was built with an ensemble structure to predict the time it will take for a ticket to resolve as the ticket ages. The Delta Duration Model uses up to 4 different AI/ML algorithms working together to determine duration of ticket upon submission. As the ticket ages, additional models are used to more accurately predict a resolution date. A model maintenance pipeline was created out of a need to track various accuracy metrics on the deployed models for the Delta Duration Model. Each week when new tickets have been submitted and predicted upon the maintenance pipeline tracks accuracy for each set of models within the pipeline. This monitors changes in the model’s accuracy over time and allows transparency when model drift has occurred.

Machine Learning to Predict and Monitor Trouble Ticket Duration

Research shows that genders are affected differently by length and word choices in job descriptions. Addition, research exhibits the existence of a gender gap where women tend to not apply unless they are 100% qualified for the job. Our hypothesis was that women do not know what the job description specifically states if it is filled with jargon, therefore do not feel like they are 100% qualified so they tend to be less likely to apply. To investigate this potential unconscious gender bias further, an experiment was performed that surveyed both Navy and non-Navy people to view three job descriptions in their respective STEM field. The survey asked them if they were interested in the job and if they would send this job to someone else. Additionally, they were asked if they would share the ad.

How Long, Jargon-filled Job Descriptions Affect Potential Candidate Interest Level

The U.S. maritime operations, particularly in the littorals, have been contested and dangerous. It is imperative to apply advanced analytics to help operational or tactical units execute distributed maritime operations (DMO) and expeditionary advanced base operations (EABO). The DMO and EABO concepts require flexible configurations and combinations of capabilities and tactics across multiple domains to win in complex contested environments. The DMO and EABO concepts also require to use advanced detection and counter-detection technologies.

Distributed and collaborative computing and networking have been associated with peer-to-peer models. The researchers apply the peer-to-peer and self-organizing collaborative learning agents (CLAs) to meet the requirements of DMO and EABO. As an example of EABO, the researchers show a use case for distributed Marine transportation units to handle transportation movement requests (TMRs). The researchers show how to data-mine historical data, discover capability and behavior patterns, then apply them to design new peer groups to balance the load and maintain much lower signatures. 

The researchers map the need of lowering operation signatures to a load balancing problem and applies lexical link analysis (LLA) for CLAs. The researchers further apply the principle of quantum entanglement and superposition into a framework of LLA quantum intelligence game (LLAQIG). LLAQIG helps a peer simultaneously achieve the Nash Equilibrium for itself and the optimal total social welfare for its peer-to-peer network. 

The LLAQIG is used to find the intrinsic value of a unit, (i.e., a unit possesses a requested capability and has the potential to become a high-value unit). The LLA groups and LLQIG quantum effect impact factor help a peer-to-peer network self-organize its units and make them appear more randomly connected, which therefore balances the load, lowers signatures, and avoids detection. The DoD operational or tactical units can potentially leverage the results for shaping flexible C2, organizational structures, and modernization.




Self-organizing and Load Balancing with Quantum Effect for Peer-to-Peer Collaborative Learning Agents and Flexible Command and Control and Organizational Structures

There is a pressing need to exploit fleeting opportunities continually--real time anticipatory ecosystem entropy offers a lens for these kinds of exploitations. We want to measure ecosystem entropy continually to make better, faster data-driven decisions which advantage one player over another. Combining entropy measures with process models, we seek to understand how dissipative structures interact with one another. Continual sliding window entropy measurements identify exploitation opportunities when a player’s ecosystem is especially chaotic or orderly. Process models provide turnkey simulation parameters which predict ecosystem next-states and identify exploitation actions. This concept may be applied to space, social media, critical infrastructure, urban, supply chain, macro-economic, natural, and geo-political ecosystems, as well. Efforts to date have neither examined ecosystems in a temporal process mining context using categorical data event logs nor analyzed interactions between ecosystem players in order to anticipate outcomes and improve decision making. A highly adaptable, domain agnostic information maneuver capability is possible. It will be possible to drive entropy purposefully by anticipating ecosystem changes or inflection points and make decisions which favor one party over the other.

Measuring and Exploiting Entropy Shifts in Time Series Ecosystems

NAVPLAN 2021 calls for distributed maritime operations (DMO), littoral operations in a contested environment (LOCE), and expeditionary advanced base operations (EABO). This project informs DMO, EABO, and LOCE objectives with technical designs for hardware, software, processing, and artificial intelligence/machine learning (AI/ML) at the tactical edge. The selected micro-service mesh architecture may additionally support naval tactical grid (NTG) objectives. Chassis and servers were optimized for tactical cloud far edge deployment with high performance in a low space, weight, power, and cost configuration (SWaP-C). AI/ML for the tactical edge operating environment was evaluated through a scenario for forward deployed sensor assessment for systems maintenance, situational awareness, and far edge decision support.

AI/ML in the Tactical Cloud for Forward Deployed Decision Support

Military tactical decision-making requires the ability to determine efficient optimized actions across multiple entities in complex environments, potentially with limited or denied communications. In recent years, researchers have applied various approaches to wargaming problems, including game theory, optimization, and more recently, deep reinforcement learning. Most deep reinforcement learning algorithms treat agents as decentralized with rewards that do not require cooperation across agents. In this research, we model a blue versus red war game scenario in an AI-gym environment. We add a stochastic game algorithm (COCO-Q) to a decentralized reinforcement learning algorithm (PRIMAL) to incorporate cooperation as a part of a goal-oriented model for a blue versus red simulation. 

PRIMAL is a decentralized pathfinding model used to implement pathways for warehouse drones. Each drone has a drop-off location, and it makes pathway decisions on its moves within a certain radius. This approach will provide a locally optimal pathway for each individual drone. COCO-Q is a stochastic game model with side payments. The side payments provide incentive for agents to work together for a common goal. These incentives are included when calculating the COCO value. The advantage of COCO-Q is that it considers both competitive and cooperative objectives, which allow the model to be shaped as a team advantage solution versus individual agents playing the same game independently. In this work, we introduce an extension of COCO-Q to a blue/red wargame that looks for a cooperative pathway for the given scenario and compare solutions to PRIMAL’s individual pathway solution.

Stochastic Game with Multi-agents in a competitive-cooperative game theory environment with an altered reward structure.

We investigate the use of artificial intelligence-based techniques in learning to play a 2-player, real-time strategy game called Hunting-of-the-Plark. The game is challenging to play for both humans and artificial intelligence-based techniques because neither player can directly observe its opponent's moves at any time during the game, and has to rely on post hoc analyses of game moves and outcomes to infer the opponent's game-playing strategies and learn to play effectively against them. We analyze the performance of different deep reinforcement algorithms to train software agents that can play the game. We also investigate different techniques that can be used by players to improve their play including self-play, where each player first builds a model of the opponent from the history of opponent's perceived actions and then uses it to refine their own game-playing strategies, and, dynamic strategy evolution, where the players strategically adapt their own strategies to gain advantage over the opponent. Our experimental results show that the actor-critic learning algorithm is the most efficient in terms of training time and rewards received by both players. Our experiments also showed that while one of the players starts with a disadvantage due to the inherent asymmetric nature of the game rules, it can increase its score considerably by using improved techniques like self-play and dynamic strategy evolution.

Learning to Play Simulated War-Games using AI: A Case Study of Deep Reinforcement Learning for Hunting of the Plark Game

R-DEX Systems (R-DEX) was formed in 2014 to develop and transition innovative technologies to Government and commercial clients. The R-DEX mission is to bring cutting edge Artificial Intelligence (AI), Machine Learning (ML), and robotics technologies to Government and commercial customers. R-DEX is actively engaged in multiple federally-funded efforts to develop a suite of technologies to achieve resilient AI systems that can be trusted, validated, and on par with human-like performance. These efforts focus in multiple modalities non-traditional sensor modalities, including Synthetic Aperture Radar (SAR), Inverse SAR, panchromatic electro-optical (EO), Electronic Warfare (EW), RF, and radar. Throughout the course of these efforts, R-DEX has developed and continues to develop a suite of Quality Assurance (QA) tools for AI/ML models with the goal of improving confidence in AI/ML results and to improve performance. In this talk, we will discuss R-DEX QA tools for AI/ML that focus in three general areas: 1) stress testing, 2) preventative tools, and 3) treatment tools. This discussion will include a summary of R-DEX’s proprietary Adversarial Input Detector (AID) models to detect and classify adversarial inputs as well as methods for prediction correction. These models hold for adversarial inputs that are caused by either bad actors or natural distortions. R-DEX’s tools also include techniques to combat epistemic uncertainty, namely, the uncertainty that AI/ML models face that arise from a lack of knowledge (e.g., lack of training examples, corrupted data, etc.). Common issues include data drift, unseen samples arising, and model not learning proper signals. R-DEX tools for combatting epistemic uncertainty include SDENets, data drift detection and retraining, and AI/ML forensics and meta-learning analysis. In addition, R-DEX tools to improve model robustness include DefenseGAN, R-DEX’s AID models, Jacobian Regularization, and Segmentation Masks.

Building AI Resiliency

While the Navy benefits tremendously from cloud-based, big data systems, they have yet to realize the benefits of artificial intelligence throughout the denied, disconnected, intermittent and low-bandwidth (DIL) settings in which they operate. In order to help the Navy apply artificial intelligence to missions in DIL settings—including predictive maintenance and early warning, JADC2, and ISR—Booz Allen developed the Beacon platform for edge AI. Using the Beacon platform, Booz Allen AI engineers have been able to dramatically reduce size and power consumption of algorithms while increase throughput and holding accuracy relatively constant. Ultimately, Booz Allen’s Beacon platform is a key enabler for bringing AI to DIL mission environments.

Premium content

Downloads

Next from NAML 2022

Readiness Predictions for the MH-60 SeaHawk with the Digital Aviation Readiness Technology Engine

Similar lecture

Mindful™: Competency Aware Machine Learning