5 papers
Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection
Yang Zhao, Li Du, Xiao Ding +10
Large language models (LLMs) have shown great potential across various industries due to their remarkable ability to generalize through instruction tuning. However, the limited ava…
Advancing Large Language Model Attribution through Self-Improving
Lei Huang, Xiaocheng Feng, Weitao Ma +7
Teaching large language models (LLMs) to generate text with citations to evidence sources can mitigate hallucinations and enhance verifiability in information-seeking systems. Howe…
GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization
Yangfan Ye, Xiachong Feng, Xiaocheng Feng +6
News summarization in today's global scene can be daunting with its flood of multilingual content and varied viewpoints from different sources. However, current studies often negle…
Extending Context Window of Large Language Models from a Distributional Perspective
Yingsheng Wu, Yuxuan Gu, Xiaocheng Feng +5
Scaling the rotary position embedding (RoPE) has become a common method for extending the context window of RoPE-based large language models (LLMs). However, existing scaling metho…
CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Information
Yuxin Wang, Minghua Ma, Zekun Wang +7
The colossal parameters and computational overhead of Large Language Models (LLMs) challenge their real-world applications. Network pruning, which targets unstructured or structure…