Stochastic Neural Networks

Will machines one day be as creative as humans?

When we write a letter, have a conversation, or draw a picture, we exercise a uniquely human skill by creating complex artifacts that embody information. Current AI technology cannot yet match human ability in this area because it fails to have the same understanding of the world in terms of independent causal factors. Without such understanding AI systems cannot compose information in the rich ways a human could.

Instead, current successful AI technology requires a large amount of supervised data in order to learn different plausible combinations of independent variations. In the stochastic neural network project we aim to build the next generation of deep learning models which are more data-efficient and can enable machines to learn more efficiently and eventually to be truly creative.

Research Direction

Research in the stochastic neural networks project addresses this research challenge along three lines:

1. Developing novel algorithms for deep probabilistic models;
2. Learning disentangled representations of complex data;
3. Applications of deep probabilistic models to applications.

Novel Algorithms for Deep Probabilistic Models

We analyze and improve important algorithms for generative models such as methods for generative adversarial networks, models based on integral probability metrics, and variational autoencoders.

Variational characterization of f-divergences in f-GAN.

Learning Disentangled Representations of Complex Data

We develop novel models to learn latent factor representations of complex data, including models that can learn disentangled representations using minimal supervision.

Disentangled representation of faces, allowing style transfer between people.

Applications of Deep Probabilistic Models to Applications

We apply deep probabilistic models to challenging applications with heteroscedastic uncertainty; application areas include computer vision, reinforcement learning, and natural language models.

ResNet GAN with and without stabilization.