Uncertainty in Artificial Intelligence
First Name   Last Name   Password   Forgot Password   Log in!
    Proceedings   Proceeding details   Article details         Authors         Search    
Value Function Approximation in Noisy Environments Using Locally Smoothed Regularized Approximate Linear Programs
Gavin Taylor, Ron Parr
Abstract:
Recently, Petrik et al. demonstrated that L1Regularized Approximate Linear Programming (RALP) could produce value functions and policies which compared favorably to established linear value function approximation techniques like LSPI. RALP's success primarily stems from the ability to solve the feature selection and value function approximation steps simultaneously. RALP's performance guarantees become looser if sampled next states are used. For very noisy domains, RALP requires an accurate model rather than samples, which can be unrealistic in some practical scenarios. In this paper, we demonstrate this weakness, and then introduce Locally Smoothed L1-Regularized Approximate Linear Programming (LS-RALP). We demonstrate that LS-RALP mitigates inaccuracies stemming from noise even without an accurate model. We show that, given some smoothness assumptions, as the number of samples increases, error from noise approaches zero, and provide experimental examples of LS-RALP's success on common reinforcement learning benchmark problems.
Keywords:
Pages: 835-842
PS Link:
PDF Link: /papers/12/p835-taylor.pdf
BibTex:
@INPROCEEDINGS{Taylor12,
AUTHOR = "Gavin Taylor and Ron Parr",
TITLE = "Value Function Approximation in Noisy Environments Using Locally Smoothed Regularized Approximate Linear Programs",
BOOKTITLE = "Proceedings of the Twenty-Eighth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-12)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "2012",
PAGES = "835--842"
}


hosted by DSL   •   site info   •   help