United States

We discuss issues in trying to apply AI to procurement tasks. We conducted surveys with procurement professionals who had implemented AI methods in procurement to determine their goals with the AI, the obstacles they encountered, and their assessment of the success of their projects.

NAML 2021

A survey of adoption of commercial-style artificial intelligence for procurement

obstacles

commercial tools

applications

procurement

software

artificial intelligence

Track A participant information packet [PDF](https://drive.google.com/file/d/1dQtZHSkIzuXHT-TFNRaY_UTi5FTk_Kcb/view?usp=sharing)

Track B participant information packet [PDF](https://drive.google.com/file/d/1HtBtLsmb4ST392VF7K22St5ZJqaSsSZ9/view?usp=sharing)


Dear B15 cohort members,

On behalf of the University of California and OA2020 Initiative, we are pleased to share with you the attendee information packet for the 15th Berlin Open Access Conference (B15): Adapt and Advance.

Since 2003, the Berlin Open Access Conference has explored the potentials of our digital environment to transform scholarly communication and “build a future-proof, flexible, open, and high-quality scholarly and scientific publishing system.” Over the years, the persistence of subscription paywalls prompted the creation of a global alliance dedicated to accelerating the transition to open access; the focus of the Berlin Open Access Conference series now centers on viable strategies to replace the existing subscription business model with transparent and sustainable open access models.

The international cohort of the 14th Berlin Open Access Conference (2018) affirmed the importance of transformative agreements in accelerating the open access transition and today we can observe the progress made with transformative agreements in just a few short years. As we progress in our strategies, each step elevates us to a new vantage point revealing a new set of challenges. At this juncture, in 2021, we are therefore convening various stakeholders and decision-makers—including higher education and research administrators; library leaders; academic communities and faculty governance; research funders; and ministries of research, education and innovation—involved in further adapting, improving and advancing transformative open access agreements for the betterment of the research enterprise. As detailed in the agenda that follows, the B15 plenaries will consider transformative open access agreements in the context of heightened budget pressures, equity and scholarly communications, and the growing variety of transformative models being implemented worldwide.

As a B15 attendee, we ask that you use this conference as a springboard for further action, partnership and change.
We look forward to seeing you at B15.

Many thanks,

Gerard Meijer, Director and Scientific Member, Fritz Haber Institute, Max Planck Society, Berlin (B15 Track B Conference Chair)
 Günter Waibel, Associate Vice Provost and Executive Director, California Digital Library, University of California (B15 Track A Conference Chair)

### **Schedule at a glance:**

To facilitate global participation, the B15 conference will be offered twice (to confirm your track, please refer to your invitation from the B15 coordinators or the participant information packet that was distributed via email in advance of the conference).

The conference program will comprise four plenary sessions and include opportunities for discussion (breakout sessions) to reflect together on the insights presented in plenaries 1-3.

Day 01
 -Welcome & opening remarks
 -Plenary 1 - Open access and cost reduction: Competing priorities?
 -Panel
 			-Breakout discussions
 -Plenary 2 - Equal opportunity to publish open access
 -Panel 
 -Breakout discussions
 -Closing Remarks
 -Social event
 
 Day 02
 -Opening remarks
 -Plenary 3 - The range of open access transition models
 -Panel 
 -Breakout discussions
 -Poster session
 -Plenary Panel 4 - The B15 Journey
 -Closing remarks
 
To view the full schedule, please click [here](https://underline.io/events/186/schedule?day=2021-09-28T04%3A00%3A00.000Z) (be sure to sort by your track).

### **B15 message board: Please share your insights, feedback and questions**
In advance of and during B15, we invite you to share your insights, feedback and questions around transformative open access on the B15 message board. Please access the message board [here](https://underline.io/events/186/posters/7945/poster/35472-message-board).

### **Technical issues or questions?**
If you have any questions (before or during B15) about the underline event platform, Zoom or Gather Town, please don’t hesitate to contact the underline help desk [here](https://underline.io/events/186/help-desk).

This event requires a registration to attend, please register for the event [here](https://oa2020.org/b15-conference/). 

Registration Required

The 15th Berlin Open Access Conference: Adapt and Advance

Welcome letter:

Dear B15 cohort members,
On behalf of the University of California and OA2020 Initiative, we are pleased to welcome you to the 15th Berlin Open Access Conference (B15): Adapt and Advance.

technical paper

Naval Information Warfare Center Pacific is partnering with AFCEA San Diego chapter to host the workshop.

The NAML workshop is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team.

The goals of NAML are:

* To spread awareness of current machine learning research relevant to Naval applications

* To connect machine learning researchers with experts in military requirements

* To build and strengthen collaborations within the defense research community

**Watch this video to hear about the AFCEA purpose.**

<iframe width=580 height=700 frameborder="0" scrolling="no" src="https://screencast-o-matic.com/player/creleVVcvRh?v=6&ff=1&title=0&controls=1" allowfullscreen="true"></iframe>

Please log in to explore this event. You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

Registration for this workshop has now closed. We appreciate your interest and hope to welcome you at a future event. If you’d like to hear about similar upcoming events, please check our website or join our mailing list.

Registration for this workshop has now closed.

The annual workshop on Naval Applications of Machine Learning is an opportunity for researchers from Navy research and development centers and our partner organizations to exchange technical ideas and to come together as a team. The goals of NAML are to spread awareness of current machine learning research relevant to Naval applications, to connect machine learning researchers with experts in military requirements and to build and strengthen collaborations within the defense research community.


Mr. Sasala provides a compelling arugment on why data readiness drives warfighting readiness, and how the Department of the Navy will be using to drive organizational performance across the DON, including the USN and USMC.

DON Data Transformation

The Department of Defense (DoD) needs methods to be able to quickly ingest and extract relevant information from large quantities of unstructured documents that may vary greatly in length, content, variety, theme or topic, formality, and language. For example, Special Operations Forces (SOF) are often called in to participate in so-called “smash-and-grab” missions, whereby large quantities of previously unviewed and undocumented Captured Enemy Material (CEM) are obtained and must be examined for relevant intelligence information; however, searching and sifting through the potentially wide range and quantity of content obtained as CEM can be difficult and time consuming via manual inspection. Supervised methods of searching for standard labels are insufficient, since new threats may surface at any time and retraining supervised models introduces extra overhead for the pipeline. Automated methods are required for quickly parsing, separating, and identifying relevant or irrelevant sources of information. We outline an ongoing research and development effort aimed at building an automated, modular text pipeline that would ingest documents, translate them into English, perform unsupervised topic modeling to generate relevant topics and themes in messy data, and pass results on to the analyst so that they can avoid the ‘noise’ and more immediately inform their missions. The pipeline utilizes a tri-gram based naïve Bayes classifier for language detection, an NSA language pair translator, hierarchical non-negative matrix factorization rank 2 (HierNMF2) for topic modeling, and a salience-based two-dimensional (2D) rendering algorithm to help visualize the results. This unique workflow is aimed at addressing the “first-pass” problem with examining raw, messy, unstructured, text-based intelligence such as CEM.

Disparate Data Distillation

There is a great deal of interest in Naval applications of reinforcement learning (RL) for decision making and autonomy. RL models typically need full information about an environment to make good decisions and successfully maximize the decision making policy. Some work has been done on partially observable RL, particularly through the study of partially observable Markov decision processes (POMDPs), but POMDPs assume that the part of the environment that can be observed is pre-determined. This work focuses on developing models that can excel at tasks and also choose which part of the environment to view in order to accomplish their task. The model is only rewarded for its success in the task but it also learns to choose which part of the environment to view. We demonstrate that RL agents learn where to look and are able to reach similar performance in Atari games such as Pong and Seaquest as when the models are trained in fully observable environments.

Learning to Both Observe and Act

Machine learning is a key application of artificial intelligence that can be used to solve complex tasks for optimizing workload, improving safety, and increasing the efficiency of our sailors onboard aircraft carriers in the Navy. Identification of aircraft, people, and equipment for awareness on the flight deck is currently performed manually through visual inspection by several sailors. This task is labor intensive, requires focus for extended periods of time, and requires multiple personnel stationed during all flight operations. PATRIOT is a research endeavor to apply computer vision techniques to aircraft launch and recovery (ALRE) to reduce data delays and errors, and increase situational awareness. PATRIOT tracks aircraft, personnel, and vehicles through existing camera feeds, and integrates the information with the Ouija board, the interface sailors currently use to view and modify the status of aircraft for flight planning. This work includes an overview and comparison of the current methods and preliminary results for the overall pipeline for PATRIOT, as well as the components for data collection and processing, image classification, pose regression, and logical analysis

Panoramic Asset Tracking of Real-time Information for the Ouija Tabletop
(PATRIOT)

Marines in combat require a rapid and flexible logistics capability responsive to the 21st century battlefield. The USMC’s supply and maintenance chain receives data feed from a long chain, including: maintenance, requisition, transportation, and finance processes, activities, and decisions.
Such a complex enterprise needs trusted deep analytics, including machine learning (ML) and artificial intelligence (AI) methods to achieve automation, improve readiness, and win in unexpected environment. 

In this paper, we first review an overall framework of leveraging artificial Intelligence to learn, optimize, and win (LAILOW) for a complex enterprise, and then show how the LAILOW framework is applied to the USMC supply and maintenance chain as a test case. We first show a collection of supervised machine-learning algorithms, from an open-source based on the python ML library scikit-learn and the Soar reinforcement learning algorithm, to predict business metrics such as predict probability of fail (POF). We then show an unsupervised machine learning method lexical link analysis (LLA) to discover associative and sequential patterns, combined with predictive models in a game-theoretic set up using the coevolutionary algorithms in the LAILOW framework, where demand perturbations introduced by high-impact and low-occurrence items propagate to the whole system. We show the LAILOW framework as predictive and simulation tool using data sets extracted from operational databases to improve total readiness, agility, and resilience for the USMC logistics enterprise.

To show the feasibility of the LAILOW, we have sampled data for a specific Table of Authorized Material Control Number (TAMCN, 2020) from the database Global Combat Support System-Marine Corps (GCSS-MC), including service ticket number level attributes related to maintenance data, supply data, and equipment usage data. The sample data set contains 2065 service numbers/tickets, where 599 (29%) have more than 65 days (65 days is the mean value of the days between the open and close dates for the data set), between their opened and closed date, which is the target variable or measure of performance (MOP) for prediction. 

We found Soar-RL results comparable in predictive accuracy for predicting the MOP. Since Soar-RL is also rule-based and explainable, it was selected and used in a simulation phase integrated with the coevolutionary algorithms. The simulation shows that the logistics solutions, on average, worsens in evolution (in terms of the fitness value), while the opponent, representing logistics tests, on average, improves in evolution in terms of the MOP. The algorithms systematically simulate and discover possible new tests or “vulnerability”, and evolved solutions or “resiliency” are also discovered. Therefore, the LAILOW framework provides a holistic predictive and simulation platform to improve total readiness of a resilient and agile USMC logistics enterprise.



Leverage Artificial Intelligence to Learn, Optimize, and Win (LAILOW) for the Marine Maintenance and Supply Complex System

Accountability is widely understood as a goal for well governed computer systems, and is a sought-after value in many governance contexts. But how can it be achieved? Recent work on standards for governable artificial intelligence systems offers a related principle: traceability. Traceability requires establishing not only how a system worked but how it was created and for what purpose, in a way that explains why a system has particular dynamics or behaviors. It connects records of how the system was constructed and what the system did mechanically to the broader goals of governance, in a way that highlights human understanding of that mechanical operation and the decision processes underlying it. We examine the various ways in which the principle of traceability has been articulated in AI principles and other policy documents from around the world, distill from these a set of requirements on software systems driven by the principle, and systematize the technologies available to meet those requirements. From our map of requirements to supporting tools, techniques, and procedures, we identify gaps and needs separating what traceability requires from the toolbox available for practitioners. This map reframes existing discussions around accountability and transparency, using the principle of traceability to show how, when, and why trans- parency can be deployed to serve accountability goals and thereby improve the normative fidelity of systems and their development processes.

This work was recently published at the 2021 ACM Conference on Fairness, Accountability, and Transparency.

Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems

Fully analog neuromorphic computing systems offer tremendous potential for the future of edge-AI applications, with the possibility of mimicking the functionality and efficiency of the human brain under considerable energy and spatial constraints. However, training such neuromorphic systems with conventional machine learning methods has proven rather challenging. In recent years, novel approaches relying on so called Energy Based Models provide an alternative paradigm for training and inference in analog neural networks. A major breakthrough in that direction was achieved with the introduction of the Equilibrium Propagation method in a paper by Benjamin Scellier and Yoshua Bengio published in 2017. Since then multiple extensions and generalization have been been proposed, including an Equilibrium Propagation algorithm for nonlinear resistive networks by Kendall, Bengio et al, and an associated extension for Spiking Neural Networks by Martin et al, both published in 2020. The goal of this tutorial is to provide a gentle introduction to Energy Based Models and the Equilibrium Propagation method, both as a general framework and in the context of training analog neural networks. The self-contained exposition will be based on a survey of recent papers and workshop presentations, as well as basic review of relevant background material. We will also discuss ideas for possible research directions and applications to realistic edge-AI problems.

Energy Based Models for Analog Neural Networks

The session will introduce what quantum computing is and further define what are quantum-inspired algorithms. Problems where quantum-inspired optimizations are well suited are introduced. The methods on how quantum-inspired optimization solves these problems will be presented. The methods such as simulated annealing and quantum annealing are two common methods that will be presented. The methods will then be applied to a real-world problem such as transportation.

Using Quantum-inspired Algorithms to Solve Optimization Problems

Coding medical data involves analyzing unstructured text fields, including lengthy clinical narratives, physician and nurse notes, lab reports, discharge summaries, scanned documents, and medications. Unstructured text data typically requires extensive resources to annotate, describe, analyze, and convert into meaningful and actionable information. These clinical data are often found in the Subjective, Objective, Assessment, and Plan (SOAP) notes. This presentation will demonstrate and discuss various machine learning algorithms using natural language processing (NLP) to parse SOAP notes that reside in the Theater Medical Data Store (TMDS)—specifically to estimate the injury subcategories (e.g., fractures of the lower limb), the three-digit International Classification of Diseases, 9th revision (ICD-9) code (e.g., 824, fracture of ankle), or the principal ICD-9-Clinical Modification code (e.g., 824.1, fracture of medial malleolus, open). The machine learning algorithms will be trained using manually coded diagnostic data obtained from the Expeditionary Medical Encounter Database and compared to the TMDS for accuracy. Sensitivity and accuracy will be the primary performance metrics that determine model efficiency. This research project’s long-term objective is to transition the translated coded data into a clinical encounter repository, which then can be used in conjunction with Department of Defense medical data repositories. 
Disclaimer: I am a military service member or employee of the U.S. Government. This work was prepared as part of my official duties. Title 17, U.S.C. §105 provides that copyright protection under this title is not available for any work of the U.S. Government. Title 17, U.S.C. §101 defines a U.S. Government work as work prepared by a military service member or employee of the U.S. Government as part of that person’s official duties.
This work was supported by Defense Medical Research and Development Program under work unit no. N1214. The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the Department of the Navy, Department of Defense, nor the U.S. Government. 
The study protocol was approved by the Naval Health Research Center Institutional Review Board in compliance with all applicable Federal regulations governing the protection of human subjects. Research data were derived from an approved Naval Health Research Center, Institutional Review Board protocol number NHRC.2003.0025.


Developing Natural Language Processing Algorithms to Medically Code Clinical Notes in Electronic Health Records

Advanced Persistent Threat (APT) and cyber criminals are known to utilize DNS protocol for command and control or beaconing or resolution of attacker domains, like the most recent Solarwinds breach. This is why academic researchers and industry groups advise security teams to collect and analyze DNS events to hunt, detect, investigate and respond to threats. 

We will discuss how to detect a remote access trojan using passive DNS (pDNS) and cyber threat intel at scale using ML/AI. We’ll review how to use open source technologies such as Delta and Spark to store and analyze TBs of passive DNS data. Additionally, we'll review ML to develop a model to detect domain generating algorithms (DGAs) and manage the ML lifecycle, from experiment tracking to training to production, using the open source project MLflow.

Premium content

Downloads

Next from NAML 2021

DON Data Transformation

Similar lecture

The Road to Open Through Strategic Procurement