A Particular Character Speech Synthesis System Based on Deep Learning

ABSTRACT

The speech synthesis system of a particular character is a TTS (text-to-speech) synthetic system, which can obtain voice with the specific speaker’s voice characteristics. The traditional method, based on machine learning, requires a great amount of training samples and large iterations. In this paper, we proposed a novel TTS system based on fully convolutional neural networks and attention mechanism. The system can be trained start from scratch with random initialization and realize end-to-end output. By adding the attention layer and the loss of attention, it can better adapt to the features of the pronunciation, intonation and accent of a specific speaker. Experimental results show that our speech synthesis framework demonstrates a stronger model performance by synthesizing higher quality forged specific character audio with a smaller training set and lesser iterations.

KEYWORDS:

• Convolutional neural networks
• deep learning
• guided attention
• speech synthesis
• text-to-speech
• the loss of attention

Acknowledgements

This work was partially supported by the National Key Research Development Program of China (2019QY(Y)0206, 2016QY01W0200), the National Natural Science Foundation of China NSFC (U1736211, 62072343).

Additional information

Funding

This work was partially supported by the National Key Research Development Program of China [grant numbers 2019QY(Y)0206, 2016QY01W0200], the National Natural Science Foundation of China NSFC [grant numbers U1736211, 62072343].

Notes on contributors

Yuan Mei

Yuan Mei was born in Wuhan, China. She received the Master degree from Huazhong University of Science and Technology in 2010. She is currently a PhD candidate of Wuhan University. Her research interests include Multimedia security, Speech synthesis and forensics and mobile crowd sensing.

Deng-pan Ye

Deng-pan Ye was born in Hubei, China. He received the B.A.Sc in automatic control from SCUT in 1996 and PhD. at NJUST in 2005, respectively. He worked as a Post-Doctoral Fellow in Information System School of Singapore Management University. And since 2012 he has been a professor in the school of computer science at Wuhan University. His research interests include information hiding and multimedia security. He is the author or co-author of more than 30 refereed journal and conference papers.Corresponding author. Email: [email protected]

Shun-zhi Jiang

Shun-zhi Jiang received the MS degree in automatic control from CAUC, Tianjin, China, in 2014. Currently, he is pursuing the PhD degree with the School of Cyber Science and Engineering, Wuhan University. His research interest focuses on steganalysis, steganography and machine learning.

Jia-rui Liu

Jia-rui Liu is currently an undergraduate student of Wuhan University. His research interests include data mining, Speech synthesis and security.