Contribution to a conference proceedings/Contribution to a book

PUBDB-2025-02243

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Speaker Recognition in Realistic Scenario Using Multimodal Data

Shah, S. H. ; Saeed, M. S. ; Nawaz, S.Extern*DESY* ; Yousaf, M. H.

2023
IEEE

This record in other databases:  

Please use a persistent id in citations: doi:10.1109/ICAI58407.2023.10136626

Abstract: In recent years, an association is established between faces and voices of celebrities leveraging large scale audio-visual information from YouTube. The availability of large scale audio-visual datasets is instrumental in developing speaker recognition methods based on standard Convolutional Neural Networks. Thus, the aim of this paper is to leverage large scale audio-visual information to improve speaker recognition task. To achieve this task, we proposed a two-branch network to learn joint representations of faces and voices in a multimodal system. Afterwards, features are extracted from the two-branch network to train a classifier for speaker recognition. We evaluated our proposed framework on a large scale audio-visual dataset named VoxCelebl. Our results show that addition of facial information improved the performance of speaker recognition. Moreover, our results indicate that there is an overlap between face and voice.

---

Contributing Institute(s):

1. FS-Detektor Systeme (FS-DS)

Research Program(s):

1. 622 - Detector Technologies and Systems (POF4-622) (POF4-622)

Experiment(s):

1. No specific instrument

---