5 papers
Attention Sinks in Massively Multilingual Neural Machine Translation:Discovery, Analysis, and Mitigation
Hillary Mutisya, John Mugane
Cross-attention patterns in neural machine translation (NMT) are widely used to study how multilingual models align linguistic structure. We report a systematic artifact in cross-a…
Neural Recovery of Historical Lexical Structure in Bantu Languages from Modern Data
Hillary Mutisya, John Mugane
We investigate whether neural models trained exclusively on modern morphological data can recover cross-lingual lexical structure consistent with historical reconstruction. Using B…
Zero-Shot Morphological Discovery in Low-Resource Bantu Languages via Cross-Lingual Transfer and Unsupervised Clustering
Hillary Mutisya, John Mugane
We present a method for discovering morphological features in low-resource Bantu languages by combining cross-lingual transfer learning with unsupervised clustering. Applied to Gir…
The Thiomi Dataset: A Large-Scale Multimodal Corpus for Low-Resource African Languages
Hillary Mutisya, John Mugane, Gavin Nyamboga +2
We present the Thiomi Dataset, a large-scale multimodal corpus spanning ten African languages across four language families: Swahili, Kikuyu, Kamba, Kimeru, Luo, Maasai, Kipsigis,…
Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data
Hillary Mutisya, John Mugane
We investigate continued pretraining (CPT) for adapting wav2vec2-bert-2.0 to Swahili automatic speech recognition (ASR). Our approach combines unlabeled audio with limited labeled…