4 citations · 8 across the 9 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.CL2024★ 1 cited
Compressing Lengthy Context With UltraGist
Peitian Zhang, Zheng Liu, Shitao Xiao +3
Compressing lengthy context is a critical but technically challenging problem. In this paper, we propose a new method called UltraGist, which is distinguished for its high-quality…
cs.CL2024★ 4 cited
Extending Llama-3's Context Ten-Fold Overnight
Peitian Zhang, Ninglu Shao, Zheng Liu +4
We extend the context length of Llama-3-8B-Instruct from 8K to 80K via QLoRA fine-tuning. The entire training cycle is super efficient, which takes 8 hours on one 8xA800 (80G) GPU…
cs.CL2024★ 2 cited
Long Context Compression with Activation Beacon
Peitian Zhang, Zheng Liu, Shitao Xiao +3
Long context compression is a critical research problem due to its significance in reducing the high computational and memory costs associated with LLMs. In this paper, we propose…