2 citations · 4 across the 12 of their papers we have counts for
4 papers · 1 filter
Scaling Parameter-Constrained Language Models with Quality Data
Ernie Chang, Matteo Paltenghi, Yang Li +7
Scaling laws in language modeling traditionally quantify training loss as a function of dataset size and model parameters, providing compute-optimal estimates but often neglecting…
Target-Aware Language Modeling via Granular Data Sampling
Ernie Chang, Pin-Jie Lin, Yang Li +6
Language model pretraining generally targets a broad range of use cases and incorporates data from diverse sources. However, there are instances where we desire a model that excels…
Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applications
Yang Li, Daniel Agyei Asante, Changsheng Zhao +3
Large language models (LLMs) significantly enhance the performance of various applications, but they are computationally intensive and energy-demanding. This makes it challenging t…
Breaking Down Power Barriers in On-Device Streaming ASR: Insights and Solutions
Yang Li, Yuan Shangguan, Yuhao Wang +5
Power consumption plays a crucial role in on-device streaming speech recognition, significantly influencing the user experience. This study explores how the configuration of weight…