Speech Wikimedia: A 77 Language Multilingual Speech Dataset

Speech research has witnessed remarkable advancements in recent years, largely driven by the availability of vast amounts of data. Tasks such as automatic speech recognition (ASR), speaker recognition, and speech translation have reached robust results even in the presence of background noise, jargon, and different accents. Nevertheless, one of the fundamental challenges in speech research is the scarcity of multilingual datasets.

In this paper, we introduce the Speech Wikimedia Dataset, a supervised dataset consisting of 1780 hours of audio files, each with one one or more transcripts. The last point bears repeating: much (approximately 25%) of the dataset has multiple associated transcripts, each having its own language, which is rare in datasets sourced from the internet. The dataset is specifically curated from Wikimedia Commons to address some of the key challenges in the speech recognition, machine translation, and speech translation spaces, particularly the need for diverse multilingual data and appropriate licensing for academic and commercial usage.

The paper is organized as follows. In section 2, we describe the dataset itself and training tasks that it can support; in section 3, we describe related work; in section 4, we describe some limitations of the dataset today; in section 5, we conclude with some future work that this dataset can enable.

To construct the Speech Wikimedia dataset, we downloaded raw video and audio from Wikimedia Commons (“https://commons.wikimedia.org”), which allows only content that is ”free” (wik, ). For our purposes, this means data under CC-BY or CC-BY-SA license, or otherwise public domain. After downloading, the data was converted to 16kHz monochannel FLAC using ffmpeg. The data is uploaded to Huggingface at https://huggingface.co/datasets/MLCommons/speech-wikimedia

We give statistics for three possible tasks that this dataset can be used for: speech recognition, speech translation, and machine translation.

In this section we provide an overview of previous, similar datasets.

Mozilla Common Voice (Ardila et al., 2020) is a CC0-licensed 17,690-hour public domain corpus of single speaker read speech in 108 languages created by volunteers. In contrast, Speech Wikimedia has much more diverse audio sources.

Multilingual Librispeech (Pratap et al., 2020) is a CC-BY dataset of 50,500 hours of transcribed read speech in eight languages; 6,000 of its hours are non-English. Meanwhile, our dataset contains 77 languages, and the majority of the data is also in English.

VoxPopuli (Wang et al., 2021) is CC0-licensed dataset containing an unsupervised set of 400,000 hours in 23 languages, and 1,500 hours of transcribed audio in 15 languages. Like our dataset, it also contains a subset suitable for a speech translation task.

Multilingual Spoken Words Corpus (Mazumder et al., 2021) is a CC-BY licensed 6,000-hour dataset, containing more than 340,000 keywords in 50 different languages. It is for training keyword spotting models, not speech recognition models.

opensubtitles (Lison & Tiedemann, 2016) is a machine translation dataset containing 1,782 language pairings extracted from movie subtitles in 62 languages. Given the data source, it is not licensed for commercial usage. In contrast, the Speech Wikimedia Dataset has 929 language pairings from 77 languages.

The raw data is available publicly online on Hugging Face as mentioned before; however, this data is not yet processed via forced alignment of audio to transcript and bitext word alignment for transcript to transcript, and thus not able to be used immediately in training models.

We removed all video data when converting to FLAC. In future work, this data could be helpful for a multimodal task.

While collecting this dataset, we realized that there is also a collection of audio data in Wikimedia Commons without any transcripts. We have not explored this subset and have not made it available at this time, however.

Given the small size of the dataset, we are not providing a training-test split.

We introduce the Speech Wikimedia Dataset, a collection of audio files with transcriptions in multiple languages extracted form Wikimedia Commons. The dataset encompasses over 1,780 hours of transcribed speech in multiple languages. The CC-BY-SA license enables commercial usage. This is the first non-read multilingual speech dataset allowing for commercial usage that we are aware of other than VoxPopuli.