Uncertainty in Artificial Intelligence
First Name   Last Name   Password   Forgot Password   Log in!
    Proceedings   Proceeding details   Article details         Authors         Search    
Exploring compact reinforcement-learning representations with linear regression
Thomas Walsh, Istvan Szita, Carlos Diuk, Michael Littman
Abstract:
This paper presents a new algorithm for online linear regression whose efficiency guarantees satisfy the requirements of the KWIK (Knows What It Knows) framework. The algorithm improves on the complexity bounds of the current state-of-the-art procedure in this setting. We explore several applications of this algorithm for learning compact reinforcement-learning representations. We show that KWIK linear regression can be used to learn the reward function of a factored MDP and the probabilities of action outcomes in Stochastic STRIPS and Object Oriented MDPs, none of which have been proven to be efficiently learnable in the RL setting before. We also combine KWIK linear regression with other KWIK learners to learn larger portions of these models, including experiments on learning factored MDP transition and reward functions together.
Keywords: null
Pages: 591-598
PS Link:
PDF Link: /papers/09/p591-walsh.pdf
BibTex:
@INPROCEEDINGS{Walsh09,
AUTHOR = "Thomas Walsh and Istvan Szita and Carlos Diuk and Michael Littman",
TITLE = "Exploring compact reinforcement-learning representations with linear regression",
BOOKTITLE = "Proceedings of the Twenty-Fifth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-09)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "2009",
PAGES = "591--598"
}


hosted by DSL   •   site info   •   help