Room:

Abstracts

Keynote 1: Autoencoders -- From Genomics to Self-Regularization

The ability of deep neural networks to generalize well in the overparameterized regime has become a subject of significant research interest. In genomics, autoencoders in particular are emerging as a unique approach to integrate and translate between different data modalities. We study overparameterized autoencoders and show that they exhibit a strong form of self-regularization that constrains the function learned through the optimization process to concentrate around the training examples, although the network could in principle represent a much larger function class. In particular, we prove that fully-connected autoencoders project data onto the (nonlinear) span of the training examples. In addition, we show that autoencoders learn a map that is locally contractive at the training examples, and hence iterating the autoencoder results in convergence to the training examples. This has important implications for privacy, provides a new mechanism for memory and recall through gradient descent in neural networks, and leads to practical guidelines with respect to the selection of network architecture.