961 citations · 961 across the 4 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.CL2024
Inference-Friendly Models With MixAttention
Shashank Rajput, Ying Sheng, Sean Owen +1
The size of the key-value (KV) cache plays a critical role in determining both the maximum context length and the number of concurrent requests supported during inference in modern…
cs.LG2024
Does your data spark joy? Performance gains from domain upsampling at the end of training
Cody Blakeney, Mansheej Paul, Brett W. Larsen +2
Pretraining datasets for large language models (LLMs) have grown to trillions of tokens composed of large amounts of CommonCrawl (CC) web scrape along with smaller, domain-specific…