New Zealand

Yi-Ting Tsao, Ke-Ting Xiao, Von-Wun Soo and Chung-Cheng Chiu (June 1st 2010). Graph Laplacian Based Transfer Learning Methods in Reinforcement Learning, Autonomous Agents, Vedran Kordic, IntechOpen, DOI: 10.5772/9654.

Maria Luisa Sein-Echaluce, Dolores Leris, Joaquin Castello, Vicente Martinez, Jose Manuel Correas and Natalia Boal (October 1st 2009). Cooperation between Universities to Develop an Online Learning Environment in Mathematics, Technology, Education and Development, Aleksandar Lazinica and Carlos Calafate, IntechOpen, DOI: 10.5772/7284.

Punnarumol Temdee (April 22nd 2020). Smart Learning Environment: Paradigm Shift for Online Learning, Multi Agent Systems - Strategies and Applications, Ricardo López - Ruiz, IntechOpen, DOI: 10.5772/intechopen.85787.

We propose a market-inspired bidding scheme for the assignment of paper reviews in large academic conferences. The primary contribution is an analysis of the incentives of reviewers during the bidding phases, when reviewers have private costs and some information about the demand for each paper, and want to obtain the best possible k papers for a predetermined k. We show that by assigning budgets to reviewers and a `price&#39; for every paper that is (roughly) inversely proportional to its demand, the best response of a reviewer is to bid sincerely, i.e., on her most favorite papers, and match the budget even when it is not enforced. Finally, we show via extensive simulations on bidding data from real conferences, that our suggested bidding scheme would substantially improve the assignment, under several common assignment algorithms.

AAMAS 2020

A Market-Inspired Bidding Scheme for Peer Review Paper Assignment

allocations

bidding

social choice

AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). The aim of the joint conference is to provide a single, high-profile, internationally-respected archival forum for scientific research in the theory and practice of autonomous agents and multi-agent systems.

Browse keynotes, discussions, panels and over 300 presentations.


AAMAS is the leading scientific conference for research in autonomous agents and multi-agent systems. The AAMAS conference series was initiated in 2002 as the merging of three respected scientific meetings: the International Conference on Multi-Agent Systems (ICMAS), the International Workshop on Agent Theories, Architectures, and Languages (ATAL), and the International Conference on Autonomous Agents (AA). 

We propose a market-inspired bidding scheme for the assignment of paper reviews in large academic conferences. The primary contribution is an analysis of the incentives of reviewers during the bidding phases, when reviewers have private costs and some information about the demand for each paper, and want to obtain the best possible k papers for a predetermined k. We show that by assigning budgets to reviewers and a `price' for every paper that is (roughly) inversely proportional to its demand, the best response of a reviewer is to bid sincerely, i.e., on her most favorite papers, and match the budget even when it is not enforced. Finally, we show via extensive simulations on bidding data from real conferences, that our suggested bidding scheme would substantially improve the assignment, under several common assignment algorithms.

technical paper

Equitable Allocations of Invisible Chores

Reinforcement Learning (RL) has been widely used to solve sequential decision-making problems. However, RL algorithms suffer from poor sample efficiency and require a long time to learn a suitable policy, especially when multiple agents are learning without prior knowledge. This problem can be alleviated through reusing knowledge from other agents during the learning process. One notable approach is advising actions based on a teacher-student relationship, where the decision of a student agent during learning is aided by an experienced teacher agent. A critical assumption in teacher-student paradigm is that the communication may be limited, so that a student may wait for a while and learn by itself before receiving the next advice. More importantly, in some noisy or stochastic environments, the student may not be able to mas- ter the advised actions when they are only performed once. We propose three methods for agents choosing between learning by exploration, asking for advice and reusing previous advice. The results show that our approaches significantly outperform existing advising methods without reusing advice.


Learning by Reusing Previous Advice in Teacher-Student Paradigm

A Deliberate BIAT Logic for Modelling Manipulations

We introduce and analyze new envy-based fairness concepts for agents with weights that quantify their entitlements in the allocation of indivisible items. We propose two variants of weighted envy-freeness up to one item (WEF1): strong, where the envy can be eliminated by removing an item from the envied agent’s bundle, and weak, where the envy can be eliminated either by removing an item as in the strong version or by replicating an item from the envied agent’s bundle in the envying agent’s bundle. We prove that for additive valuations, an allocation that is both Pareto optimal and strongly WEF1 always exists; however, an allocation that maximizes the weighted Nash social welfare may not be strongly WEF1 but always satisfies the weak version of the property. Moreover, we establish that a generalization of the round-robin picking sequence algorithm produces in polynomial time a strongly WEF1 allocation for an arbitrary number of agents; for two agents, we can efficiently achieve both strong WEF1 and Pareto optimality by adapting the adjusted winner procedure. Our work exhibits several aspects in which weighted fair division is richer and more challenging than its unweighted counterpart.


Weighted Envy-Freeness in Indivisible Item Allocation

Mean field theory provides an effective way of scaling multiagent reinforcement learning algorithms to environments with many agents that can be abstracted by a virtual mean agent. In this paper, we extend mean field multiagent algorithms to multiple types. The types enable the relaxation of a core assumption in mean field games, which is that all agents in the environment are playing almost similar strategies and have the same goal. We conduct experiments on three different testbeds for the field of many agent reinforcement learning, based on the standard MAgents framework. We consider two different kinds of mean field games: a) Games where agents belong to predefined types that are known a priori and b) Games where the type of each agent is unknown and therefore must be learned based on observations. We introduce new algorithms for each type of game and demonstrate their superior performance over state of the art algorithms that assume that all agents belong to the same type and other baseline algorithms in the MAgent framework.


Multi Type Mean Field Reinforcement Learning

Multi-agent reinforcement learning has generally been studied under an assumption inherited from classical reinforcement learning: that the reward function is the exclusive property of the environment, and is only altered by external factors. In this work, we break free of this assumption and introduce peer rewarding, in which agents can deliberately influence each others' reward function. We formalize this more general setting and discuss its properties in depth. We also empirically study gifting, a peer rewarding mechanism which allows agents to reward other agents as part of their action space. We demonstrate that this approach can greatly improve learning progression in a resource appropriation setting and provide a preliminary analysis of the complex effects of gifting on the learning dynamics.


Gifting in Multi-Agent Reinforcement Learning

Model-based reinforcement learning algorithms make decisions by building and utilizing a model of the environment. However, none of the existing algorithms attempts to infer the dynamics of any state-action pair from known state-action pairs before meeting it for sufficient times. We propose a new model-based method called Greedy Inference Model (GIM) that infers the unknown dynamics from known dynamics based on the internal spectral properties of the environment. In other words, GIM can "learn by analogy". We further introduce a new exploration strategy which ensures that the agent rapidly and evenly visits unknown state-action pairs. GIM is much more computationally efficient than state-of-the-art model-based algorithms, as the number of dynamic programming operations is independent of the environment size. Lower sample complexity could also be achieved under mild conditions compared against methods without inferring. Experimental results demonstrate the effectiveness and efficiency of GIM in a variety of real-world tasks.


Can Agents Learn by Analogy? An Inferable Model for PAC Reinforcement Learning

While game-theoretic models and algorithms have been developed to combat illegal activities, such as poaching and over-fishing, in green security domains, none of the existing work considers the crucial aspect of community engagement: community members are recruited by law enforcement agencies as informants and can provide valuable tips, e.g., the location of ongoing illegal activities, to assist patrols. We fill this gap and (i) introduce a novel two-stage security game model for community engagement, with a bipartite graph representing the informant-attacker social network and a level-$\kappa$ response model for attackers inspired by cognitive hierarchy; (ii) provide complexity results and exact, approximate, and heuristic algorithms for selecting informants and allocating patrollers against level-$\kappa$ ($\kappa<\infty$) attackers; (iii) provide a novel algorithm to find the optimal defender strategy against level-$\infty$ attackers, which converts the problem of optimizing a parameterized fixed-point to a bi-level optimization problem, where the inner level is just a linear program, and the outer level has only a linear number of variables and a single linear constraint. We also evaluate the algorithms through extensive experiments.


Green Security Game with Community Engagement

We study a general multi-dueling bandit problem, where an agent compares multiple options simultaneously and aims to minimize the regret due to selecting suboptimal arms. This setting generalizes the traditional two-dueling bandit problem and finds many real-world applications involving subjective feedback on multiple options. We start with the two-dueling bandit setting and propose two efficient algorithms, DoublerBAI and MultiSBM-Feedback.DoublerBAI provides a generic schema for translating known results on best arm identification algorithms to the dueling bandit problem, and achieves a regret bound of $O(\ln T)$. MultiSBM-Feedback not only has an optimal $O(\ln T)$ regret, but also reduces the constant factor by almost a half compared to benchmark results. Then, we consider the general multi-dueling case and develop an efficient algorithm MultiRUCB. Using a novel finite-time regret analysis for the general multi-dueling bandit problem, we show that MultiRUCB also achieves an $O(\ln T)$ regret bound and the bound tightens as the capacity of the comparison set increases. Based on both synthetic and real-world datasets, we empirically demonstrate that our algorithms outperform existing algorithms. 


Dueling Bandits: From Two-dueling to Multi-dueling

Signalized intersections are managed by controllers that assign right of way (green, yellow, and red lights) to non-conflicting directions. Optimizing the actuation policy of such controllers is expected to alleviate traffic congestion and its adverse impact. Given such a safety-critical domain, the affiliated actuation policy is required to be interpretable in a way that can be understood and regulated by a human. This presentation demonstrates and gives an overview of how we approached solving such a problem.

Downloads

Next from AAMAS 2020

Equitable Allocations of Invisible Chores

Similar lecture

A Market-Inspired Bidding Scheme for Peer Review Paper Assignment

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from AAMAS 2020

Equitable Allocations of Invisible Chores

Similar lecture

A Market-Inspired Bidding Scheme for Peer Review Paper Assignment

Downloads