Please use this identifier to cite or link to this item:
Title: Distributionally robust markov decision processes
Authors: Xu, H. 
Mannor, S.
Keywords: Distributional robustness
Markov decision process
Parameter uncertainty
Issue Date: May-2012
Citation: Xu, H., Mannor, S. (2012-05). Distributionally robust markov decision processes. Mathematics of Operations Research 37 (2) : 288-300. ScholarBank@NUS Repository.
Abstract: We consider Markov decision processes where the values of the parameters are uncertain. This uncertainty is described by a sequence of nested sets (that is, each set contains the previous one), each of which corresponds to a probabilistic guarantee for a different confidence level. Consequently, a set of admissible probability distributions of the unknown parameters is specified. This formulation models the case where the decision maker is aware of and wants to exploit some (yet imprecise) a priori information of the distribution of parameters, and it arises naturally in practice where methods for estimating the confidence region of parameters abound. We propose a decision criterion based on distributional robustness: the optimal strategy maximizes the expected total reward under the most adversarial admissible probability distributions. We show that finding the optimal distributionally robust strategy can be reduced to the standard robust MDP where parameters are known to belong to a single uncertainty set; hence, it can be computed in polynomial time under mild technical conditions. © 2012 INFORMS.
Source Title: Mathematics of Operations Research
ISSN: 0364765X
DOI: 10.1287/moor.1120.0540
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.


checked on Feb 9, 2019


checked on Jan 30, 2019

Page view(s)

checked on Nov 17, 2018

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.