Effective reinforcement learning for collaborative multi-agent domains

Abstract

Online reinforcement learning (RL) in collaborative multi-agent domains is difficult in general. The number of available actions is exponential in the number of agents. This poses serious problems for online learning as exploration requirements are massive. Consequently, the learning system has lesser opportunities to exploit. Furthermore, the learning models for multiple agents can quickly become complex as the number of agents increase, and agents may have fluctuating communication links.

To improve online exploration, we devise coordination guided RL (CGRL). CGRL employs expert knowledge of coordination as coordination constraints to dynamically guide exploration. Next, we present distributed CGRL for domains where communication links among agents fluctuate. Then, we introduce distributed relational temporal difference learning that greatly reduces the number of learning parameters for distributed RL. These methods improve learning performance in various domains. Last, we investigate an application of multi-agent learning to the real-world domain of automating retinal image analysis.