4 citations · 6 across the 9 of their papers we have counts for
3 papers · 1 filter
Weight subcloning: direct initialization of transformers using larger pretrained ones
Mohammad Samragh, Mehrdad Farajtabar, Sachin Mehta +5
Training large transformer models from scratch for a target task requires lots of data and is computationally demanding. The usual practice of transfer learning overcomes this chal…
eDKM: An Efficient and Accurate Train-time Weight Clustering for Large Language Models
Minsik Cho, Keivan A. Vahid, Qichen Fu +5
Since Large Language Models or LLMs have demonstrated high-quality performance on many complex language tasks, there is a great interest in bringing these LLMs to mobile devices fo…
Improving vision-inspired keyword spotting using dynamic module skipping in streaming conformer encoder
Alexandre Bittar, Paul Dixon, Mohammad Samragh +2
Using a vision-inspired keyword spotting framework, we propose an architecture with input-dependent dynamic depth capable of processing streaming audio. Specifically, we extend a c…