Please use this identifier to cite or link to this item: https://scholarbank.nus.edu.sg/handle/10635/230311
Title: The Dynamics of Q-learning in Population Games: a Physics-Inspired Continuity Equation Model
Authors: Hu, S 
Leung, CW
Leung, HF
Soh, H 
Keywords: Q-Learning
Mathematical Modeling
Population Game
Issue Date: 1-Jan-2022
Citation: Hu, S, Leung, CW, Leung, HF, Soh, H (2022-01-01). The Dynamics of Q-learning in Population Games: a Physics-Inspired Continuity Equation Model 1 : 615-623. ScholarBank@NUS Repository.
Abstract: Although learning has found wide application in multi-agent systems, its effects on the temporal evolution of a system are far from understood. This paper focuses on the dynamics of Q-learning in large-scale multi-agent systems modeled as population games. We revisit the replicator equation model for Q-learning dynamics and observe that this model is inappropriate for our concerned setting. Motivated by this, we develop a new formal model, which bears a formal connection with the continuity equation in physics. We show that our model always accurately describes the Q-learning dynamics in population games across different initial settings of MASs and game configurations. We also show that our model can be applied to different exploration mechanisms, describe the mean dynamics, and be extended to Q-learning in 2-player and n-player games. Last but not least, we show that our model can provide insights into algorithm parameters and facilitate parameter tuning.
URI: https://scholarbank.nus.edu.sg/handle/10635/230311
ISBN: 9781713854333
ISSN: 1548-8403
1558-2914
Appears in Collections:Staff Publications
Elements

Show full item record
Files in This Item:
File Description SizeFormatAccess SettingsVersion 
p615.pdfPublished version2.61 MBAdobe PDF

OPEN

NoneView/Download

Page view(s)

32
checked on Sep 29, 2022

Download(s)

4
checked on Sep 29, 2022

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.