An Overview of Deep Generative Models

ABSTRACT

As an important category of deep models, deep generative model has attracted more and more attention with the proposal of Deep Belief Networks (DBNs) and the fast greedy training algorithm based on restricted Boltzmann machines (RBMs). In the past few years, many different deep generative models are proposed and used in the area of Artificial Intelligence. In this paper, three important deep generative models including DBNs, deep autoencoder, and deep Boltzmann machine are reviewed. In addition, some successful applications of deep generative models in image processing, speech recognition and information retrieval are also introduced and analysed.

Keywords:

• Deep autoencoder
• Deep belief networks
• Deep boltzmann machine
• Deep generative model
• Restricted boltzmann machine

Additional information

Funding

This work was supported by the National Natural Science Foundation of China [grant number 61372171]; the National Key Technology R&D Program of China [grant number 2012BAH23B03].

Notes on contributors

Jungang Xu

Jungang Xu is an associate professor of the School of Computer and Control Engineering, University of Chinese Academy of Sciences. He received the PhD degree in computer applied technology from Graduate University of Chinese Academy of Sciences in 2003. During 2003–2005, he was a post-doctor of Tsinghua University. His current research interests include deep learning, parallel computing, big data management, etc.

Email: [email protected].

Hui Li

Hui Li is an MS student of School of Computer and Control Engineering, University of Chinese Academy of Sciences. She received the BS degree in software engineering from Jilin University in 2011. Her current research interests include deep learning theory and its application.

Email: [email protected].

Shilong Zhou

Shilong Zhou is an MS student of School of Computer and Control Engineering, University of Chinese Academy of Sciences. He received the BS degree in software engineering from Northeast University in 2012. His current research interests include deep learning and information retrieval.

Email: [email protected].