Journal Article

PUBDB-2024-06733

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Causal Diffusion Models for Generalized Speech Enhancement

Richter, J. (Corresponding author) ; Welker, S.CFEL*DESY* ; Lemercier, J.-M. ; Lay, B. ; Peer, T. ; Gerkmann, T. (Corresponding author)

2024
IEEE [New York, NY]

This record in other databases:      

Please use a persistent id in citations: doi:10.1109/OJSP.2024.3379070  doi:10.3204/PUBDB-2024-06733

Abstract: In this work, we present a causal speech enhancement system that is designed to handledifferent types of corruptions. This paper is an extended version of our contribution to the “ICASSP 2023Speech Signal Improvement Challenge”. The method is based on a generative diffusion model which hasbeen shown to work well in scenarios beyond speech-in-noise, such as missing data and non-additivecorruptions. We guarantee causal processing with an algorithmic latency of 20 ms by modifying the networkarchitecture and removing non-causal normalization techniques. To train and test our model, we generate anew corrupted speech dataset which includes additive background noise, reverberation, clipping, packet loss,bandwidth reduction, and codec artifacts. We compare the causal and non-causal versions of our method toinvestigate the impact of causal processing and we assess the gap between specialized models trained on aparticular corruption type and the generalized model trained on all corruptions. Although specialized modelsand non-causal models have a small advantage, we show that the generalized causal approach does not sufferfrom a significant performance penalty, while it can be flexibly employed for real-world applications wheredifferent types of distortions may occur.

---

Contributing Institute(s):

1. FS-CFEL-1 (Group Leader: Henry Chapman) (CFEL-I)
2. CFEL-Coherent X-Ray Imaging (FS-CFEL-1)

Research Program(s):

1. 633 - Life Sciences – Building Blocks of Life: Structure and Function (POF4-633) (POF4-633)
2. HIDSS-0002 - DASHH: Data Science in Hamburg - Helmholtz Graduate School for the Structure of Matter (2019_IVF-HIDSS-0002) (2019_IVF-HIDSS-0002)

Experiment(s):

1. No specific instrument

Appears in the scientific report 2024

---

Database coverage:
; ; ; ; Article Processing Charges ; Clarivate Analytics Master Journal List ; DOAJ Seal ; Emerging Sources Citation Index ; Fees ; IF < 5 ; JCR ; SCOPUS ; Web of Science Core Collection

---