New Zealand

Kunikazu Kobayashi, Koji Nakano, Takashi Kuremoto and Masanao Obayashi (February 1st 2010). Objective-based Reinforcement Learning System for Cooperative Behavior Acquisition, Application of Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/8615. 

Yoshiyuki Tajima and Takehisa Onisawa (February 1st 2010). Model-based Reinforcement Learning with Model Error and Its Application, Application of Machine Learning, Yagang Zhang, IntechOpen, DOI: 10.5772/8607. 

Paolo Villa, Giovanmaria Lechi and Mario A. Gomarasca (October 1st 2009). Multivariate Differencing Techniques for Land Cover Change Detection: the Normalized Difference Reflectance Approach, Geoscience and Remote Sensing, Pei-Gee Peter Ho, IntechOpen, DOI: 10.5772/8312. 

&quot;Reinforcement learning (RL) methods learn optimal decisions in the presence of a stationary environment. However, the stationary assumption on the environment is very restrictive. In many real world problems like traffic signal control, robotic applications, etc., one often encounters situations with non-stationary environments and in these scenarios, RL methods yield sub-optimal decisions. In this presentation, I talk about the problem of developing RL methods that obtain optimal decisions in a non-stationary environment. The goal of this problem is to maximize the long-term discounted reward accrued when the underlying model of the environment changes over time. To achieve this, we first adapt a change point algorithm to detect change in the statistics of the environment and then develop an RL algorithm that maximizes the long-run reward accrued. I illustrate that our change point method detects change in the model of the environment effectively and thus facilitates the RL algorithm in maximizing the long-run reward. In our work, we further validate the effectiveness of the proposed solution on non-stationary random Markov decision processes, a sensor energy management problem and a traffic signal control problem.
&quot;


AAMAS 2020

Reinforcement Learning Algorithms for Autonomous Adaptive Agents

non-stationary environments

change detection

reinforcement learning

The purpose of the AAAI conference is to promote research in artificial intelligence (AI) and scientific exchange among AI researchers, practitioners, scientists, and engineers in affiliated disciplines. 

AAAI 2022

"Reinforcement learning (RL) methods learn optimal decisions in the presence of a stationary environment. However, the stationary assumption on the environment is very restrictive. In many real world problems like traffic signal control, robotic applications, etc., one often encounters situations with non-stationary environments and in these scenarios, RL methods yield sub-optimal decisions. In this presentation, I talk about the problem of developing RL methods that obtain optimal decisions in a non-stationary environment. The goal of this problem is to maximize the long-term discounted reward accrued when the underlying model of the environment changes over time. To achieve this, we first adapt a change point algorithm to detect change in the statistics of the environment and then develop an RL algorithm that maximizes the long-run reward accrued. I illustrate that our change point method detects change in the model of the environment effectively and thus facilitates the RL algorithm in maximizing the long-run reward. In our work, we further validate the effectiveness of the proposed solution on non-stationary random Markov decision processes, a sensor energy management problem and a traffic signal control problem.
"


technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

"Our PhD research is concerned with the task of achieving cooperation in a system of competitive agents which cannot be explicitly controlled. To this end, it examines the problem from the system’s point of view, without restricting the agents’ behavior or requiring specific knowledge about their decision-making.
The governance of the MAS will be achieved via a dynamically adaptive governing policy based on a set of rules, which leaves full autonomy to the individual agents, but reacts to their actions via suitable changes of the environment. The mechanism is designed to lead to system-level cooperation while only assuming that the agents follow their own self-interested motives."


Achieving Emergent Governance in Competitive Multi-Agent Systems

This is an overview of my doctoral work studying inequality in social networks. The focus of this work is to design fairer algorithms that take into account the networks structure as well as community affiliations and individual incentives. I'll show my work in this field in three applications: recommendation algorithms, information diffusion, and partition algorithms. 


Algorithmic Fairness for Networked Algorithms

"Recent years have seen a push towards deploying fully autonomous robots in large, complex domains such as autonomous driving, space exploration, and service robots. However, legal, ethical, or technical constraints have limited the extent of these systems' employable autonomy. In order to successfully achieve their intended goals, these systems must utilize assistance from humans to compensate for their limitations. For such systems to be successful over the course of a long-term deployment, they must both be cognizant of their own competence and have the ability to improve this competence over time in a safe way. 
Motivated by practical concerns faced in industry, 
this thesis provides a formal model for such a human-agent system to reason about its own competence and aims in future work to provide effective ways of safely improving the competence of the system over the course of its deployment."


Competence-Aware Systems for Long-Term Autonomy

Many contributions to the field of computational social choice employ a variety of techniques developed in computer science to study preference aggregation mechanisms. However, there has been little research on how the availability of such techniques can help us reason about collective decision making. In this positional paper I discuss how computer-aided methods can be used to —automatically— reason about the outcomes of a collective decision making process. I also lay down several research directions one could explore to further develop the field in this direction.


Computer-aided Reasoning about Collective Decision Making

The process of determining the appropriate set of norms, referred to as synthesis, for a multiagent system has predominantly been carried out offline by the designers of the system. Of recent, there have been a few approaches that synthesise norms online utilising a centralised mechanism. The research presented here aims to propose a mechanism for decentralised runtime (online) norm synthesis through the use of agents dedicated to synthesising norms based on participating agents' requests.


Decentralised Runtime Norm Synthesis

The problem of multi-agent resource allocation is important and well-studied within AI and economics. The general assumption is that the amount of each resource is known beforehand. However, many real-world problems, the exact amount of each resource may not be known at the time of decision making, e,g. in the case of weather dependent renewable energy production. This work considers a homogeneous divisible resource where the available amount is given by a probability distribution. In general, a model for efficient usage under fairness and the possibilities of manipulation is studied. Firstly, the notion of ex-ante envy-freeness, where, in expectation, agents weakly prefer their allocation over every other agent's allocation is introduced. For this case the tension between fairness and social welfare is considered. The price of envy-freeness is at least $\Omega(n)$, where $n$ is the number of agents and the problem of optimising ex-ante social welfare subject to ex-ante envy-freeness is strongly NP-hard. Additionally, the possibility for an integer program to calculate the optimal ex-ante envy-free allocation for linear satiable valuation functions is presented.


Efficiency and Fairness of Resource Utilisation under Uncertainty

Cost Allocation in Ridesharing

We formalize an allocation model that is more gen- eral than the well-studied Shapley-Scarf housing market. In our model, the agents do not just care which house or resource they get but also care about who gets their own resource. This assump- tion is especially important when considering tem- porary exchanges in which each resource is eventu- ally returned to the owner. We show that several positive axiomatic and computational results that hold for housing markets do not extend to the more general setting. We then identify natural restric- tions on the preferences of agents for which several positive results do hold. One of our central results is a general class of algorithms that return any allo- cation that is individually rational and Pareto opti- mal with respect to the responsive set extension.


The Temporary Exchange Problem

One of the challenging problems of impartial combinatorial games (ICGs) is to construct generalized winning strategies for possibly infinitely many states. In this paper, we investigate synthesizing generalized winning strategies for ICGs. To this end, we first propose a logical framework to formalize ICGs based on linear integer arithmetic. We then propose an approach to generating the winning strategy for ICGs. Experimental results on several games demonstrate that our approach is effective in most of these games.


Automatic Synthesis of Generalized Winning Strategy of Impartial Combinatorial Games

In many tasks, classifiers play a fundamental role in the way an
agent behaves. Most rational agents collect sensor data from the environment,
classify it, and act based on that classification. Recently,
deep neural networks (DNNs) have become the dominant approach
to develop classifiers due to their excellent performance. When
training and evaluating the performance of DNNs, it is normally
assumed that the cost of all misclassification errors are equal. However,
this is unlikely to be true in practice. Incorrect classification
predictions can cause an agent to take inappropriate actions. The
costs of these actions can be asymmetric, vary from agent-to-agent,
and depend on context. In this paper, we discuss the importance of
considering risk and uncertainty quantification together to reduce
agents’ cost of making misclassifications using deep classifiers.

Downloads

Next from AAMAS 2020

Achieving Emergent Governance in Competitive Multi-Agent Systems

Similar lecture

Structure Learning-Based Task Decomposition for Reinforcement Learning In Non-Stationary Environments

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Achieving Emergent Governance in Competitive Multi-Agent Systems

Similar lecture

Structure Learning-Based Task Decomposition for Reinforcement Learning In Non-Stationary Environments

Downloads