Uncertainty in Artificial Intelligence
First Name   Last Name   Password   Forgot Password   Log in!
    Proceedings   Proceeding details   Article details         Authors         Search    
Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation
Marek Petrik, Dharmashankar Subramanian, Janusz Marecki
Abstract:
We propose solution methods for previously-unsolved constrained MDPs in which actions can continuously modify the transition probabilities within some acceptable sets. While many methods have been proposed to solve regular MDPs with large state sets, there are few practical approaches for solving constrained MDPs with large action sets. In particular, we show that the continuous action sets can be replaced by their extreme points when the rewards are linear in the modulation. We also develop a tractable optimization formulation for concave reward functions and, surprisingly, also extend it to non- concave reward functions by using their concave envelopes. We evaluate the effectiveness of the approach on the problem of managing delinquencies in a portfolio of loans.
Keywords:
Pages: 518-526
PS Link:
PDF Link: /papers/13/p518-petrik.pdf
BibTex:
@INPROCEEDINGS{Petrik13,
AUTHOR = "Marek Petrik and Dharmashankar Subramanian and Janusz Marecki",
TITLE = "Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation",
BOOKTITLE = "Proceedings of the Twenty-Ninth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-13)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "2013",
PAGES = "518--526"
}


hosted by DSL   •   site info   •   help