HierTTS

End-to-end text-to-speech models that directly generate waveforms from the text are gaining popularity. However, existing end-to-end models are still not natural enough in their prosodic expressiveness. And previous studies on improving the expressiveness of TTS mainly focus on acoustic models. There is a lack of research on enhancing expressiveness in an end-to-end framework. Therefore, we propose HierTTS, a highly expressive end-to-end text-to-waveform generation model. It deeply couples the hierarchical properties of speech with hierarchical variational autoencoders and models multi-scale latent variables, including frame, phone, subword, word, and sentence levels. The hierarchical encoder encodes the speech signal from fine-grained features into coarse-grained latent variables. In contrast, the hierarchical decoder generates fine-grained features conditioned on the coarse-grained latent variables. We propose a staged KL-weighted annealing strategy to prevent hierarchical posterior collapse. Furthermore, we employ a hierarchical text encoder to extract linguistic information at different levels and act on both the encoder and decoder. Experiments results show that our model is closer to natural speech in prosody expressiveness and has better generative diversity.