|
|
|
|
Metrics for Markov Decision Processes with Infinite State Spaces
Norman Ferns, Prakash Panangaden, Doina Precup
Abstract:
We present metrics for measuring state similarity in Markov decision processes (MDPs) with infinitely many states, including MDPs with continuous state spaces. Such metrics provide a stable quantitative analogue of the notion of bisimulation for MDPs, and are suitable for use in MDP approximation. We show that the optimal value function associated with a discounted infinite horizon planning task varies continuously with respect to our metric distances.
Keywords:
Pages: 201-208
PS Link:
PDF Link: /papers/05/p201-ferns.pdf
BibTex:
@INPROCEEDINGS{Ferns05,
AUTHOR = "Norman Ferns
and Prakash Panangaden and Doina Precup",
TITLE = "Metrics for Markov Decision Processes with Infinite State Spaces",
BOOKTITLE = "Proceedings of the Twenty-First Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-05)",
PUBLISHER = "AUAI Press",
ADDRESS = "Arlington, Virginia",
YEAR = "2005",
PAGES = "201--208"
}
|
|
