5 citations · 7 across the 6 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.CL2024★ 2 cited
MagicPIG: LSH Sampling for Efficient LLM Generation
Zhuoming Chen, Ranajoy Sadhukhan, Zihao Ye +8
Large language models (LLMs) with long context windows have gained significant attention. However, the KV cache, stored to avoid re-computation, becomes a bottleneck. Various dynam…
cs.LG2024
Memory Mosaics
Jianyu Zhang, Niklas Nolte, Ranajoy Sadhukhan +2
Memory Mosaics are networks of associative memories working in concert to achieve a prediction task of interest. Like transformers, memory mosaics possess compositional capabilitie…
cs.LG2024
Fine-tuning with Very Large Dropout
Jianyu Zhang, Léon Bottou
It is impossible today to pretend that the practice of machine learning is always compatible with the idea that training and testing data follow the same distribution. Several auth…