3 papers
cs.CL2025
TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation
Tong Wu, Junzhe Shen, Zixia Jia +2
Generating ultra-long sequences with large language models (LLMs) has become increasingly crucial but remains a highly time-intensive task, particularly for sequences up to 100K to…
cs.CV2025
OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contexts
Yuxuan Wang, Yueqian Wang, Bo Chen +3
The rapid advancement of multi-modal language models (MLLMs) like GPT-4o has propelled the development of Omni language models, designed to process and proactively respond to conti…
cs.CL2024
An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding
Tong Wu, Yanpeng Zhao, Zilong Zheng
Recently, many methods have been developed to extend the context length of pre-trained large language models (LLMs), but they often require fine-tuning at the target length ($\gg4K…