Principled Approaches for Learning Latent Variable Models

In any learning task, it is natural to incorporate latent or hidden variables which are not directly observed. For instance, in a social network, we can observe interactions among the actors, but not their hidden interests/intents, in gene networks, we can measure gene expression levels but not the detailed regulatory mechanisms, and so on. I will present a broad framework for unsupervised learning of latent variable models, addressing both statistical and computational concerns. We show that higher order relationships among observed variables have a low rank representation under natural statistical constraints such as conditional-independence relationships. We also present efficient computational methods for finding these low rank representations. These findings have implications in a number of settings such as finding hidden communities in networks, discovering topics in text documents and learning about gene regulation in computational biology. I will also present principled approaches for learning overcomplete models, where the latent dimensionality can be much larger than the observed dimensionality, under natural sparsity constraints. This has implications in a number of applications such as sparse coding and feature learning.