Khairil Ilmi, a master’s student in the Artificial Intelligence program at the Faculty of Mathematics and Natural Sciences (FMIPA), Universitas Syiah Kuala, is developing an automatic speech recognition (ASR) system for Acehnese and Gayo, two low-resource regional languages of Indonesia. Carried out at the Data Science and Artificial Intelligence Research Center (DSAI-RC), the research, supervised by Prof. Dr. Taufik Fuadi Abidin and Afnan, Ph.D., addresses a gap that has long held back language technology in the province: neither language has a speech-to-text system, nor an adequately labelled speech corpus on which one could be built. The absence carries a second risk, that any system eventually developed would falter against the emotional variation of everyday speech, since emotion measurably alters pronunciation and speaking patterns.
Because no suitable corpus existed, Ilmi built one from the ground up. The completed collection comprises 7,096 Acehnese clips — roughly 12.13 hours recorded from 35 speakers — alongside 23,911 Gayo clips, some 15.04 hours
from 49 speakers. The corpus was processed and divided by stratified sampling, then used to fine-tune the XLS-R-300M acoustic model under three training strategies: direct, staged, and augmentation-based. Decoding was evaluated using beam search coupled with a five-gram KenLM language model, with Whisper-small serving as the comparison baseline.
Direct fine-tuning proved the strongest of the three strategies for both languages. It reached a word error rate (WER) of 27.53 percent for Acehnese and 18.69 percent for Gayo, outperforming the Whisper-small baseline, which recorded 31.19 percent and 23.59 percent respectively. Ilmi is currently working to reduce WER of Acehnese.
The work now turns to a harder question — whether such a system holds up when its speakers are angry, grieving, or delighted. Rather than rely on synthetic augmentation, which represents genuine emotional variation poorly, Ilmi is gathering authentic emotional speech through a web-based recording application, targeting some 3,000 recordings for each of three categories — happy, angry, and sad — in Acehnese. The direction reflects a broader conviction at DSAI-RC: that language technology should meet speakers as they actually speak, and that Indonesia’s regional languages merit the same computational attention long reserved for the world’s dominant few.