2 citations · 2 across the 7 of their papers we have counts for
5 papers · 1 filter
SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering
Maryam Haghifam, Jason Cong, Yizhou Sun
Long-context inference with large language models (LLMs) is costly: self-attention during prefill scales quadratically with sequence length, and the key-value (KV) cache grows with…
HMT: Semantic Hierarchy-Aware Hierarchical Memory Transformer
Maryam Haghifam, Zifan He, Jason Cong +1
Transformer-based LLMs achieve strong results on many language tasks; however, long inputs remain challenging because context windows are finite, and prefill latency and memory gro…
Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference
Zongyue Qin, Ziniu Hu, Zifan He +3
Large language models (LLMs) have achieved remarkable success across diverse tasks, yet their inference processes are hindered by substantial time and energy demands due to single-…
HMT: Hierarchical Memory Transformer for Efficient Long Context Language Processing
Zifan He, Yingqi Cao, Zongyue Qin +3
Transformer-based large language models (LLM) have been widely used in language processing applications. However, due to the memory constraints of the devices, most of them restric…
Accelerating Large Language Model Pretraining via LFR Pedagogy: Learn, Focus, and Review
Neha Prakriya, Jui-Nan Yen, Cho-Jui Hsieh +1
Traditional Large Language Model (LLM) pretraining relies on autoregressive language modeling with randomly sampled data from web-scale datasets. Inspired by human learning techniq…