Aliasing Artifacts

In recent years, with the application of deep learning in speech synthesis, waveform generation models based on generative adversarial networks achieve high quality comparable to natural speech. In most waveform generators, neural upsampling unit plays an essential role as it is employed to upsample acoustic features to sample level. However, we observe significant aliasing artifacts in the generated speech from non-idea upsampling-based waveform generators. According to the Shannon-Nyquist sampling theorem, non-ideal upsampling filters such as nearest neighbor, bilinear, subpixel or zero-padding upsampling result in aliasing. In this paper, we aim to systematically analyze how aliasing artifacts are produced in the non-idea upsampling-based waveform generators. We explore commonly used neural upsampling unit including transposed convolution, nearest neighbor interpolation and subpixel convolution in HiFi-GAN and VITS and find that the high-frequency spectral details are generated based on low-frequency structure utilizing the nonlinear transformation. But the nonlinear transformation could not perfectly eliminate the low-frequency spectral imprint and finally manifested as spectral artifacts in generated waveforms. To suppress the aliasing artifacts, we first apply a low-pass filter after upsampling layer but find significant performance drops. Experimental results indicate that the aliasing makes the training process converge faster by filling the high-frequency empty. In this regard, we propose to mix high-frequency noise into low-pass filtered features so that it can converge quicker and naturally avoid artifacts. In addition, we design an artifact-detection algorithm based on structural similarity to evaluate the effectiveness of our method.