Uncertainty in Artificial Intelligence
First Name   Last Name   Password   Forgot Password   Log in!
    Proceedings         Authors   Author's Info   Article details         Search    
Reversible Jump MCMC Simulated Annealing for Neural Networks
Christophe Andrieu, Nando de Freitas, Arnaud Doucet
Abstract:
We propose a novel reversible jump Markov chain Monte Carlo (MCMC) simulated annealing algorithm to optimize radial basis function (RBF) networks. This algorithm enables us to maximize the joint posterior distribution of the network parameters and the number of basis functions. It performs a global search in the joint space of the parameters and number of parameters, thereby surmounting the problem of local minima. We also show that by calibrating a Bayesian model, we can obtain the classical AIC, BIC and MDL model selection criteria within a penalized likelihood framework. Finally, we show theoretically and empirically that the algorithm converges to the modes of the full posterior distribution in an efficient way.
Keywords: Model selection, Model averaging, MCMC, Reversible jump MCMC, MDL, AIC, BIC, Bayesian
Pages: 11-18
PS Link: http://www.cs.berkeley.edu/~jfgf/rjpaper.ps
PDF Link: /papers/00/p11-andrieu.pdf
BibTex:
@INPROCEEDINGS{Andrieu00,
AUTHOR = "Christophe Andrieu and Nando de Freitas and Arnaud Doucet",
TITLE = "Reversible Jump MCMC Simulated Annealing for Neural Networks",
BOOKTITLE = "Proceedings of the Sixteenth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-00)",
PUBLISHER = "Morgan Kaufmann",
ADDRESS = "San Francisco, CA",
YEAR = "2000",
PAGES = "11--18"
}


hosted by DSL   •   site info   •   help