New Zealand

Takao Hinamoto, Keijiro Kawai, Masayoshi Nakamoto andWu-Sheng Lu (March 1st 2010). Roundoff Noise Minimization for State-Estimate Feedback Digital Controllers Using Joint Optimization of Error Feedback and Realization, Signal Processing, Sebastian Miron, IntechOpen, DOI: 10.5772/8521.

Shuang Cong, Yanyang Liang and Weiwei Shang (January 1st 2009). Function Approximation-based Sliding Mode Adaptive Control for Time-varying Uncertain Nonlinear Systems, Frontiers in Adaptive Control, Shuang Cong, IntechOpen, DOI: 10.5772/6427.

P.C. Nayak, K.P. Sudheer and S.K. Jain (March 21st 2012). Fuzzy Nonlinear Function Approximation (FNLLA) Model for River Flow Forecasting, Water Resources Management and Modeling, Purna Nayak, IntechOpen, DOI: 10.5772/35378. 

Function approximation is a powerful approach for structuring large decision problems that has facilitated great achievements in the areas of reinforcement learning and game playing. Regression counterfactual regret minimization (RCFR) is a simple algorithm for approximately solving imperfect information games with normalized rectified linear unit (ReLU) parameterized policies. In contrast, the more conventional softmax parameterization is standard in the field of reinforcement learning and yields a regret bound with a better dependence on the number of actions. We derive approximation error-aware regret bounds for $(\Phi, f)$-regret matching, which applies to a general class of link functions and regret objectives. These bounds recover a tighter bound for RCFR and provide a theoretical justification for RCFR implementations with alternative policy parameterizations ($f$-RCFR), including softmax. We provide exploitability bounds for $f$-RCFR with the polynomial and exponential link functions in zero-sum imperfect information games and examine empirically how the link function interacts with the severity of the approximation. We find that the previously studied ReLU parameterization performs better when the approximation error is small while the softmax parameterization can perform better when the approximation error is large.


AAMAS 2020

Alternative Function Approximation Parameterizations for Solving Games: An Analysis of f-Regression Counterfactual Regret Minimization

extensive-form games

function approximation

regret minimization

The purpose of the AAAI conference is to promote research in artificial intelligence (AI) and scientific exchange among AI researchers, practitioners, scientists, and engineers in affiliated disciplines. 

AAAI 2021

technical paper

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

Improved Algorithms for Learning Equilibria in Simulation-Based Games

In the field of informed decision-making, the usage of a single diagnostic expert system has limitations when dealing with complicated circumstances. The usage of a multi-agent information fusion (MAIF) system can mitigate this situation, as it allows multiple agents collaborating to solve the problems in a complex environ- ment. However, the MAIF system needs to handle the uncertainty problem between different agents objectively at the same time. Target to this goal, this study reconstructs the generation of basic probability assignments (BPAs) based on the framework of evidence theory, and presents the uncertainty relationship between recognition sets, which are beneficial to the applications of the MAIF system. On the basis of evidence distance measurement, our method demonstrates the effectiveness and extendibility in numerical examples, and improves the accuracy and anti-interference ability during the identification process in the MAIF system.


Uncertainty Modeling in Multi-agent Information Systems

In many multiagent domains, and particularly in tightly coupled domains, teasing an agent's contribution to the system performance based on a single episodic return is difficult. This well-known difficulty hits state-to-action mapping approaches such as neural networks trained by evolutionary algorithms particularly hard. This paper introduces fitness critics, which leverage the expected fitness to evaluate an agent's performance. This approach turns a sparse performance metric (policy evaluation) into a dense performance metric (state-action evaluation) by relating the episodic feedback to the state-action pairs experienced during the execution of that policy. In the tightly-coupled multi-rover domain (where multiple rovers have to perform a particular task simultaneously), only teams using fitness critics were able to demonstrate effective learning on tasks with tight coupling while other coevolved teams were unable to learn at all.


Multi-level Fitness Critics for Coevolution

We consider exploration tasks in which an autonomous mobile robot incrementally builds maps of initially unknown indoor environments. In such tasks, the robot makes a sequence of decisions on where to move next that, usually, are based on knowledge about the observed parts of the environment. 
In this paper, we present an approach that exploits a prediction of the geometric structure of the unknown parts of an environment to improve exploration performance. In particular, we leverage an existing method that reconstructs the layout of an environment starting from a partial grid map and that predicts the shape of partially observed rooms on the basis of geometric features representing the regularities of the indoor environment. Then, we originally employ the predicted layout to estimate the amount of new area the robot would observe from candidate locations in order to inform the selection of the next best location and to early stop the exploration when no further relevant area is expected to be discovered. Experimental activities show that our approach is able to effectively predict the layout of partially observed rooms and to use such knowledge to speed up the exploration.

Exploration of Indoor Environments Predicting the Layout of Partially Observed Rooms

Using Cognitive Models to Train Big Data Models with Small Data

With nearly every new swarm robotic platform built, the designers develop its software stack, from low-level drivers to high-level algorithmic implementations. And while the different software stacks frequently share components, especially in robot-to-robot communication, these common components are also developed from scratch time and again. We present SwarmTalk, a new communication library that can be quickly ported to new and existing swarm hardware. SwarmTalk adopts a publish-subscribe communication model that satisfies the severe hardware constraints found in many swarms, and provides an easy-to-use programming interface. We port our SwarmTalk prototype to two hardware swarm platforms and two simulator-based platforms, and implement commonly-used swarm algorithms on these four platforms. We present the design and implementation of SwarmTalk, discuss some of the system challenges in implementation and cross-platform porting, and report on our initial experiences as a common communication abstraction for a community benchmarking suite.

SwarmTalk - Towards Benchmark Software Suites for Swarm Robotics Platforms

ELESSAR: Ethics in Norm-Aware Agents

Interest in semi-autonomous systems (SAS) is growing rapidly as a paradigm to deploy autonomous systems in domains that require occasional reliance on humans. This paradigm allows service robots or autonomous vehicles to operate at varying levels of autonomy and offer safety in situations that require human judgment. We propose an introspective model of autonomy that is learned and updated online through experience and dictates the extent to which the agent can act autonomously in any given situation. We define a competence-aware system (CAS) that explicitly models its own proficiency at different levels of autonomy and the available human feedback. A CAS learns to adjust its level of autonomy based on experience to maximize overall efficiency, factoring in the cost of human assistance. We analyze the convergence properties of CAS and provide experimental results for robot delivery and autonomous driving domains that demonstrate the benefits of the approach.


Learning to Optimize Autonomy in Competence Aware Systems

In this work, we present a competitive and cooperative heterogeneous deep reinforcement learning framework called C2HRL. C2HRL aims to learn a superior agent that exceeds the capabilities of the individual agent in an agent pool through two agent management mechanisms: one competitive, the other cooperative.

Competitive and Cooperative Heterogeneous Deep Reinforcement Learning (C2HRL)

Fair division is the problem of allocating a set of items among a set of agents in a fair and efficient way. It arises naturally in a wide range of real-life settings. Competitive equilibrium (CE) is a central solution concept in economics to study markets, and due to its remarkable fairness and efficiency properties (e.g., envy-freeness, proportionality, Pareto optimality), it is also one of the most preferred mechanisms for fair division even though there is no money involved.

The vast majority of work in fair division focuses on the case of disposable goods, which all agents like or can throw away at no cost. In this paper, we consider the case of mixed manna under linear utilities where some items are positive goods liked by all agents, some are bads (chores) that no one likes, an the remaining some agents like and others dislike. The recent work of Bogomolnaia et al. Econometrica '17, initiated the study of CE in mixed manna. They establish that a CE always exists and maintains all nice properties found in the case of all goods. However, computing a CE of mixed manna is genuinely harder than in the case all goods due to the non-convex and disconnected nature of the CE set. Our main result is a polynomial-time algorithm for computing a CE of mixed manna when the number of agents or items is constant.


Downloads

Next from AAMAS 2020

Improved Algorithms for Learning Equilibria in Simulation-Based Games

Similar lecture

Model-Free Online Learning in Unknown Sequential Decision Making Problems and Games

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Improved Algorithms for Learning Equilibria in Simulation-Based Games

Similar lecture

Model-Free Online Learning in Unknown Sequential Decision Making Problems and Games

Downloads