8 papers
LLMCodec: Adapting Video Codecs for Efficient Weight Compression of Large Language Models
Rui Wang, Yan Zhao, Li Song +1
The rapid development of large language models(LLMs) has led to remarkable advances in natural language processing. However, the increasing scale of these models introduces substan…
Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search
Shing Yin Wong, Shaocheng Liu, Linqi Song +2
Proving Shannon-type entropy inequalities is a fundamental task in information theory that often requires constructing non-trivial linear combinations of known constraints, which i…
Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Models
Shuqi Liu, Bowei He, Chen Ma +1
Large language models (LLMs) typically enhance their performance through either the retrieval of semantically similar information or the improvement of their reasoning capabilities…
Activation-Guided Consensus Merging for Large Language Models
Yuxuan Yao, Shuqi Liu, Zehua Liu +6
Recent research has increasingly focused on reconciling the reasoning capabilities of System 2 with the efficiency of System 1. While existing training-based and prompt-based appro…
1bit-Merging: Dynamic Quantized Merging for Large Language Models
Shuqi Liu, Yuxuan Yao, Bowei He +5
Recent advances in large language models have led to specialized models excelling in specific domains, creating a need for efficient model merging techniques. While traditional mer…
Beyond One-Size-Fits-All Pruning via Evolutionary Metric Search for Large Language Models
Shuqi Liu, Bowei He, Han Wu +1
Post-training pruning has emerged as a crucial optimization technique as large language models (LLMs) continue to grow rapidly. However, the significant variations in weight distri…