Please use this identifier to cite or link to this item: https://doi.org/10.1287/moor.1120.0540
Title: Distributionally robust markov decision processes
Authors: Xu, H. 
Mannor, S.
Keywords: Distributional robustness
Markov decision process
Parameter uncertainty
Issue Date: May-2012
Source: Xu, H., Mannor, S. (2012-05). Distributionally robust markov decision processes. Mathematics of Operations Research 37 (2) : 288-300. ScholarBank@NUS Repository. https://doi.org/10.1287/moor.1120.0540
Abstract: We consider Markov decision processes where the values of the parameters are uncertain. This uncertainty is described by a sequence of nested sets (that is, each set contains the previous one), each of which corresponds to a probabilistic guarantee for a different confidence level. Consequently, a set of admissible probability distributions of the unknown parameters is specified. This formulation models the case where the decision maker is aware of and wants to exploit some (yet imprecise) a priori information of the distribution of parameters, and it arises naturally in practice where methods for estimating the confidence region of parameters abound. We propose a decision criterion based on distributional robustness: the optimal strategy maximizes the expected total reward under the most adversarial admissible probability distributions. We show that finding the optimal distributionally robust strategy can be reduced to the standard robust MDP where parameters are known to belong to a single uncertainty set; hence, it can be computed in polynomial time under mild technical conditions. © 2012 INFORMS.
Source Title: Mathematics of Operations Research
URI: http://scholarbank.nus.edu.sg/handle/10635/59978
ISSN: 0364765X
DOI: 10.1287/moor.1120.0540
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

SCOPUSTM   
Citations

13
checked on Dec 7, 2017

WEB OF SCIENCETM
Citations

10
checked on Nov 29, 2017

Page view(s)

19
checked on Dec 11, 2017

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.