Latent Kullback Leibler Control for Continuous-State Systems using Probabilistic Graphical Models
Takamitsu Matsubara, Vicenc Gomez, Hilbert Kappen
Kullback Leibler (KL) control problems al-
low for efficient computation of optimal con-
trol by solving a principal eigenvector prob-
lem. However, direct applicability of such
framework to continuous state-action sys-
tems is limited. In this paper, we propose
to embed a KL control problem in a proba-
bilistic graphical model where observed vari-
ables correspond to the continuous (possi-
bly high-dimensional) state of the system
and latent variables correspond to a dis-
crete (low-dimensional) representation of the
state amenable for KL control computation.
We present two examples of this approach.
The first one uses standard hidden Markov
models (HMMs) and computes exact opti-
mal control, but is only applicable to low-
dimensional systems. The second one uses
factorial HMMs, it is scalable to higher di-
mensional problems, but control computa-
tion is approximate. We illustrate both ex-
amples in several robot motor control tasks.
PDF Link: /papers/14/p583-matsubara.pdf
AUTHOR = "Takamitsu Matsubara
and Vicenc Gomez and Hilbert Kappen",
TITLE = "Latent Kullback Leibler Control for Continuous-State Systems using Probabilistic Graphical Models",
BOOKTITLE = "Proceedings of the Thirtieth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-14)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "2014",
PAGES = "583--592"