Ranking Under Uncertainty
Or Zuk, Liat Ein-Dor, Eytan Domany
Ranking objects is a simple and natural pro- cedure for organizing data. It is often per- formed by assigning a quality score to each object according to its relevance to the prob- lem at hand. Ranking is widely used for ob- ject selection, when resources are limited and it is necessary to select a subset of most rel- evant objects for further processing. In real world situations, the object's scores are often calculated from noisy measurements, casting doubt on the ranking reliability. We intro- duce an analytical method for assessing the influence of noise levels on the ranking re- liability. We use two similarity measures for reliability evaluation, Top-K-List overlap and Kendall's tau measure, and show that the for- mer is much more sensitive to noise than the latter. We apply our method to gene selec- tion in a series of microarray experiments of several cancer types. The results indicate that the reliability of the lists obtained from these experiments is very poor, and that ex- periment sizes which are necessary for attain- ing reasonably stable Top-K-Lists are much larger than those currently available. Simu- lations support our analytical results.
PDF Link: /papers/07/p466-zuk.pdf
AUTHOR = "Or Zuk
and Liat Ein-Dor and Eytan Domany",
TITLE = "Ranking Under Uncertainty",
BOOKTITLE = "Proceedings of the Twenty-Third Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-07)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "2007",
PAGES = "466--473"