2 papers
cs.LG2026
These Are Not All the Features You Are Looking For: A Fundamental Bottleneck in Supervised Pretraining
Xingyu Alice Yang, Jianyu Zhang, Léon Bottou
Transfer learning is widely used to adapt large pretrained models to new tasks with only a small amount of new data. However, a challenge persists -- the features from the original…
cs.CL2024
MagicPIG: LSH Sampling for Efficient LLM Generation
Zhuoming Chen, Ranajoy Sadhukhan, Zihao Ye +8
Large language models (LLMs) with long context windows have gained significant attention. However, the KV cache, stored to avoid re-computation, becomes a bottleneck. Various dynam…