A combined tactical and strategic hierarchical learning framework in multiagent games

Abstract

This paper presents a novel approach to modeling a generic cognitive framework in game agents to provide tactical behavior generation as well as strategic decision making in modern multi-agent computer games. The core of our framework consists of two characterization concepts we term as the tactical and strategic personalities, embedded in each game agent. Tactical actions and strategic plans are generated according to the weights defined in their respective personalities. The personalities are constantly improved as the game proceeds by a learning process based on reinforcement learning. Also, the strategies selected at each level of the agents' command hierarchy affect the personalities and hence the decisions of other agents. The learning system improves performance of the game agents in combat and is decoupled from the action selection mechanism to ensure speed. The variability in tactical behavior and decentralized strategic decision making improves realism and increases entertainment value. Our framework is implemented in a real game scenario as an experiment and shown to outperform various scripted opponent team tactics and strategies, as well as one with a randomly varying strategy. © 2008 ACM.