4 papers
CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognition
Tianchi Liu, Nicholas Moratelli, Khurram Azeem Hashmi +5
Feature distillation is usually expected to add to logit distillation. We find that this expectation does not hold for two common feature losses in compact multilingual automatic s…
VisionPsy-Nano: Improving Accuracy, Efficiency, and Reliability in On-Device Vision-Language Models
Khurram Azeem Hashmi, Mohammadreza Zolfaghari, Changdae Park +6
Sub-billion-parameter Vision-Language Models are increasingly viable for on-device deployment, yet compact model size alone does not guarantee usability. On a phone, such a model c…
QVAC Genesis III: A Large-Scale, High-Quality Open Synthetic STEM Corpus for Efficient Language Model Pre-Training
Davide Vitabile, N. Ranjan, Akshay Nambiar +2
High-quality pre-training data is a critical bottleneck for educational and STEM-specific language models targeting edge AI and on-device deployment where token budgets are tightly…
TranslatePsy-AfriSLM: High-Quality Data Scaling For Low-Resource Machine Translation
Milan Gritta, Patrik Lambert, Jihye Back +1
The rapid progress in Artificial Intelligence has largely bypassed African languages, creating a digital divide that limits AI adoption on the continent. Recent open-source LLMs sy…