Uncertainty in Artificial Intelligence
First Name   Last Name   Password   Forgot Password   Log in!
    Proceedings         Authors   Author's Info   Article details         Search    
Stochastic Discriminative EM
Andres Masegosa
Abstract:
Stochastic discriminative EM (sdEM) is an online-EM-type algorithm for discriminative training of probabilistic generative models be- longing to the natural exponential family. In this work, we introduce and justify this algorithm as a stochastic natural gradient descent method, i.e. a method which accounts for the informa- tion geometry in the parameter space of the sta- tistical model. We show how this learning algo- rithm can be used to train probabilistic genera- tive models by minimizing different discrimina- tive loss functions, such as the negative condi- tional log-likelihood and the Hinge loss. The re- sulting models trained by sdEM are always gen- erative (i.e. they define a joint probability distri- bution) and, in consequence, allows to deal with missing data and latent variables in a principled way either when being learned or when making predictions. The performance of this method is illustrated by several text classification problems for which a multinomial naive Bayes and a latent Dirichlet allocation based classifier are learned using different discriminative loss functions.
Keywords:
Pages: 573-582
PS Link:
PDF Link: /papers/14/p573-masegosa.pdf
BibTex:
@INPROCEEDINGS{Masegosa14,
AUTHOR = "Andres Masegosa ",
TITLE = "Stochastic Discriminative EM",
BOOKTITLE = "Proceedings of the Thirtieth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-14)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "2014",
PAGES = "573--582"
}


hosted by DSL   •   site info   •   help