Rank/Norm Regularization with Closed-Form Solutions: Application to Subspace Clustering
Yao-Liang Yu, Dale Schuurmans
When data is sampled from an unknown subspace, principal component analysis (PCA) provides an effective way to estimate the subspace and hence reduce the dimension of the data. At the heart of PCA is the Eckart-Young-Mirsky theorem, which characterizes the best rank k approximation of a matrix. In this paper, we prove a generalization of the Eckart-Young-Mirsky theorem under all unitarily invariant norms. Using this result, we obtain closed-form solutions for a set of rank/norm regularized problems, and derive closed-form solutions for a general class of subspace clustering problems (where data is modelled by unions of unknown subspaces). From these results we obtain new theoretical insights and promising experimental results.
PDF Link: /papers/11/p778-yu.pdf
AUTHOR = "Yao-Liang Yu
and Dale Schuurmans",
TITLE = "Rank/Norm Regularization with Closed-Form Solutions: Application to Subspace Clustering",
BOOKTITLE = "Proceedings of the Twenty-Seventh Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-11)",
PUBLISHER = "AUAI Press",
ADDRESS = "Corvallis, Oregon",
YEAR = "2011",
PAGES = "778--785"