If you have any problems related to the accessibility of any content (or if you want to request that a specific publication be accessible), please contact us at scholarworks@unr.edu.
Attention-Enabled Reinforcement Learning for Control of Scalable Multi-Agent Systems
Date
2024Type
ThesisDepartment
Electrical Engineering
Degree Level
Master's Degree
Abstract
Multi-agent reinforcement learning has been the subject of considerable interest and effort for its potential as a means of specifying behavior policies for multi-agent systems. Specifically, on-policy algorithms based on gradient estimation have achieved state-of-the-art performance on end-to-end control problems once thought beyond the scope of machine learning methods.In seeking to apply the benefits of MARL to practical control of physical autonomous systems, we must begin to account for three factors: (1) the presence of other autonomous elements in the environment configuration space, which may or may not be amenable to coordination; (2) non-idealities in sensing the configuration of the environment (e.g. locality and limited observability); and (3) variability in the number of sensed dynamical elements.
The attention head, a relational ML structure originally designed for extraction of abstract natural language features, is structurally well suited to addressing these challenges. This work presents a systematic argument and framework for the use of attention as an input layer to enable learning of neural policy models in changing multi-agent environments which are not well-suited to other representations.
In benchmark physical simulations, it is shown that such models achieve competitive performance on cooperative and mixed cooperative/competitive MAS control tasks as the agent cohort is arbitrarily changed. Prospective advantages of attention-based architectures for physical autonomous systems in select applications are discussed, as well as drawbacks associated with explainability and potential for emergent behavior.
Permanent link
http://hdl.handle.net/11714/10912Additional Information
Committee Member | Fadali, M. Sami; La, Hung |
---|---|
Rights | Creative Commons Attribution-ShareAlike 4.0 United States |
Rights Holder | Author(s) |