107 citations · 304 across the 20 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2023★ 5 cited
LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression
Ayush Kaushal, Tejas Vaidhya, Irina Rish
Low Rank Decomposition of matrix - splitting a large matrix into a product of two smaller matrix offers a means for compression that reduces the parameters of a model without spars…
cs.CL2023★ 12 cited
Continual Pre-Training of Large Language Models: How to (re)warm your model?
Kshitij Gupta, Benjamin Thérien, Adam Ibrahim +5
Large language models (LLMs) are routinely pre-trained on billions of tokens, only to restart the process over again once new data becomes available. A much cheaper and more effici…