Hierarchical MixturesofExperts for Exponential Family Regression Models with Generalized Linear Mean Functions: A Survey of Approximation and Consistency Results
Wenxin Jiang, Martin Tanner
Abstract:
We investigate a class of hierarchical mixturesofexperts (HME) models where exponential family regression models with generalized linear mean functions of the form psi(ga+fx^Tfgb) are mixed. Here psi(...) is the inverse link function. Suppose the true response y follows an exponential family regression model with mean function belonging to a class of smooth functions of the form psi(h(fx)) where h(...)in W_2^infty (a Sobolev class over [0,1]^{s}). It is shown that the HME probability density functions can approximate the true density, at a rate of O(m^{2/s}) in L_p norm, and at a rate of O(m^{4/s}) in KullbackLeibler divergence. These rates can be achieved within the family of HME structures with no more than slayers, where s is the dimension of the predictor fx. It is also shown that likelihoodbased inference based on HME is consistent in recovering the truth, in the sense that as the sample size n and the number of experts m both increase, the mean square error of the predicted mean response goes to zero. Conditions for such results to hold are stated and discussed.
Keywords: Approximation, consistency, exponential family, generalized linear models,hierarchica
Pages: 296303
PS Link:
PDF Link: /papers/98/p296jiang.pdf
BibTex:
@INPROCEEDINGS{Jiang98,
AUTHOR = "Wenxin Jiang
and Martin Tanner",
TITLE = "Hierarchical MixturesofExperts for Exponential Family Regression Models with Generalized Linear Mean Functions: A Survey of Approximation and Consistency Results",
BOOKTITLE = "Proceedings of the Fourteenth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI98)",
PUBLISHER = "Morgan Kaufmann",
ADDRESS = "San Francisco, CA",
YEAR = "1998",
PAGES = "296303"
}

