Satisfaction of Assumptions is a Weak Predictor of Performance
This paper demonstrates a methodology for examining the accuracy of uncertain inference systems (UIS), after their parameters have been optimized, and does so for several common UIS's. This methodology may be used to test the accuracy when either the prior assumptions or updating formulae are not exactly satisfied. Surprisingly, these UIS's were revealed to be no more accurate on the average than a simple linear regression. Moreover, even on prior distributions which were deliberately biased so as give very good accuracy, they were less accurate than the simple probabilistic model which assumes marginal independence between inputs. This demonstrates that the importance of updating formulae can outweigh that of prior assumptions. Thus, when UIS's are judged by their final accuracy after optimization, we get completely different results than when they are judged by whether or not their prior assumptions are perfectly satisfied.
Keywords: Uncertain Inference Systems, UIS
PDF Link: /papers/87/p45-wise.pdf
AUTHOR = "Ben Wise
TITLE = "Satisfaction of Assumptions is a Weak Predictor of Performance",
BOOKTITLE = "Uncertainty in Artificial Intelligence 3 Annual Conference on Uncertainty in Artificial Intelligence (UAI-87)",
PUBLISHER = "Elsevier Science",
ADDRESS = "Amsterdam, NL",
YEAR = "1987",
PAGES = "45--54"