Relative Entropy, Probabilistic Inference and AI
Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of probability distributions. In this paper, I will review some basic properties of relative entropy as well as its role in probabilistic inference. I will also mention briefly a few existing and potential applications of relative entropy to so-called artificial intelligence (AI).
PDF Link: /papers/85/p43-shore.pdf
AUTHOR = "John Shore
TITLE = "Relative Entropy, Probabilistic Inference and AI",
BOOKTITLE = "Proceedings of the First Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-85)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "1985",
PAGES = "43--47"