Uncertainty in Artificial Intelligence
First Name   Last Name   Password   Forgot Password   Log in!
    Proceedings   Proceeding details   Article details         Authors         Search    
Modeling Documents with Deep Boltzmann Machines
Nitish Srivastava, Ruslan Salakhutdinov, Geoffrey Hinton
Abstract:
We introduce a Deep Boltzmann Machine model suitable for modeling and extracting latent semantic representations from a large unstructured collection of documents. We overcome the apparent difficulty of training a DBM with judicious parameter tying. This parameter tying enables an efficient pretraining algorithm and a state initialization scheme that aids inference. The model can be trained just as efficiently as a standard Restricted Boltzmann Machine. Our experiments show that the model assigns better log probability to unseen data than the Replicated Softmax model. Features extracted from our model outperform LDA, Replicated Softmax, and DocNADE models on document retrieval and document classification tasks.
Keywords:
Pages: 616-624
PS Link:
PDF Link: /papers/13/p616-srivastava.pdf
BibTex:
@INPROCEEDINGS{Srivastava13,
AUTHOR = "Nitish Srivastava and Ruslan Salakhutdinov and Geoffrey Hinton",
TITLE = "Modeling Documents with Deep Boltzmann Machines",
BOOKTITLE = "Proceedings of the Twenty-Ninth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-13)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "2013",
PAGES = "616--624"
}


hosted by DSL   •   site info   •   help