Uncertainty in Artificial Intelligence
First Name   Last Name   Password   Forgot Password   Log in!
    Proceedings   Proceeding details   Article details         Authors         Search    
Annealing Paths for the Evaluation of Topic Models
James Foulds, Padhraic Smyth
Abstract:
Statistical topic models such as latent Dirich- let allocation have become enormously popu- lar in the past decade, with dozens of learning algorithms and extensions being proposed each year. As these models and algorithms continue to be developed, it becomes increasingly impor- tant to evaluate them relative to previous tech- niques. However, evaluating the predictive per- formance of a topic model is a computationally difficult task. Annealed importance sampling (AIS), a Monte Carlo technique which operates by annealing between two distributions, has pre- viously been successfully used for topic model evaluation (Wallach et al., 2009b). This tech- nique estimates the likelihood of a held-out doc- ument by simulating an annealing process from the prior to the posterior for the latent topic as- signments, and using this simulation as an im- portance sampling proposal distribution. In this paper we introduce new AIS annealing paths which instead anneal from one topic model to another, thereby estimating the relative perfor- mance of the models. This strategy can exhibit much lower empirical variance than previous ap- proaches, facilitating reliable per-document com- parisons of topic models. We then show how to use these paths to evaluate the predictive perfor- mance of topic model learning algorithms by effi- ciently estimating the likelihood at each iteration of the training procedure. The proposed method achieves better held-out likelihood estimates for this task than previous algorithms with, in some cases, an order of magnitude less computation.
Keywords:
Pages: 220-229
PS Link:
PDF Link: /papers/14/p220-foulds.pdf
BibTex:
@INPROCEEDINGS{Foulds14,
AUTHOR = "James Foulds and Padhraic Smyth",
TITLE = "Annealing Paths for the Evaluation of Topic Models",
BOOKTITLE = "Proceedings of the Thirtieth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-14)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "2014",
PAGES = "220--229"
}


hosted by DSL   •   site info   •   help