Abstract: Deep generative models are probabilistic generative models where the generator is parameterized by a deep neural network. They are popular models for modeling high-dimensional data such as texts, images and speeches, and have achieved impressive empirical success. Despite demonstrated success in  empirical performance,  theoretical understanding of such models  is largely lacking . We investigate statistical properties of deep generative models from a nonparametric distribution estimation viewpoint. In the considered model, data are assumed to be observed in the some high-dimensional ambient space but concentrate around some low-dimensional structure such as a lower-dimensional manifold. Estimating the distribution supported on this low-dimensional structure is challenging due to its singularity with respect to the Lebesgue measure in the ambient space. We obtain convergence rates with respect to the Wasserstein metric of  distribution estimators based on two methods: a sieve MLE based on the perturbed data and a GAN type estimator. Such an analysis  provides  insights into  i) how deep generative models can avoid the curse of dimensionality and outperform classical nonparametric estimates,  and  ii) how likelihood approaches work for singular distribution estimation, especially in adapting to the intrinsic geometry of the data.

Lizhen Lin is the Robert and Sara Lumpkins Associate Professor of statistics at the University of Notre Dame. Her areas of expertise are Bayesian modeling and theory, statistics on manifolds, statistical network analysis and statistical foundations of deep neural network models.