3 citations · 3 across the 8 of their papers we have counts for
9 papers
PruneTIR: Inference-Time Tool Call Pruning for Effective yet Efficient Tool-Integrated Reasoning
Luan Zhang, Dandan Song, Zhijing Wu +8
Tool-integrated reasoning (TIR) enables large language models (LLMs) to enhance their capabilities by interacting with external tools, such as code interpreters (CI). Most recent s…
Beyond Position Bias: Shifting Context Compression from Position-Driven to Semantic-Driven
Jiwei Tang, Zhijing Huang, Xinyu Zhang +5
Large Language Models (LLMs) have demonstrated exceptional performance across diverse tasks. However, their deployment in long-context scenarios faces high computational overhead a…
ZigzagAttention: Efficient Long-Context Inference with Exclusive Retrieval and Streaming Heads
Zhuorui Liu, Chen Zhang, Dawei Song
With the rapid development of large language models (LLMs), handling long context has become one of the vital abilities in LLMs. Such long-context ability is accompanied by difficu…
Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge
Yi Sui, Chaozhuo Li, Chen Zhang +2
Retrieval-augmented generation (RAG) aims to mitigate the hallucination of Large Language Models (LLMs) by retrieving and incorporating relevant external knowledge into the generat…
WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference
Youhui Zuo, Sibo Wei, Chen Zhang +3
With the advancements in long-context inference capabilities of large language models (LLMs), the KV cache has become one of the foundational components. However, its substantial G…
Beyond the Speculative Game: A Survey of Speculative Execution in Large Language Models
Chen Zhang, Zhuorui Liu, Dawei Song
With the increasingly giant scales of (causal) large language models (LLMs), the inference efficiency comes as one of the core concerns along the improved performance. In contrast…
On Elastic Language Models
Chen Zhang, Benyou Wang, Dawei Song
Large-scale pretrained language models have achieved compelling performance in a wide range of language understanding and information retrieval tasks. Knowledge distillation offers…
Sparse Contrastive Learning of Sentence Embeddings
Ruize An, Chen Zhang, Dawei Song
Recently, SimCSE has shown the feasibility of contrastive learning in training sentence embeddings and illustrates its expressiveness in spanning an aligned and uniform embedding s…
Towards the Law of Capacity Gap in Distilling Language Models
Chen Zhang, Qiuchi Li, Dawei Song +3
Language model (LM) distillation aims at distilling the knowledge in a large teacher LM to a small student one. As a critical issue facing LM distillation, a superior student often…