3 citations · 3 across the 10 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.CL2024
SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation
Jialong Wu, Zhenglin Wang, Linhai Zhang +3
Key-Value (KV) cache has become a bottleneck of LLMs for long-context generation. Despite the numerous efforts in this area, the optimization for the decoding phase is generally ig…
cs.CL2024
SEED: Accelerating Reasoning Tree Construction via Scheduled Speculative Decoding
Zhenglin Wang, Jialong Wu, Yilong Lai +2
Large Language Models (LLMs) demonstrate remarkable emergent abilities across various tasks, yet fall short of complex reasoning and planning tasks. The tree-search-based reasoning…