2 papers
cs.AI2026
Entropy-KL Divergence-based Token Masking: A Novel Approach for Selective Fine-tuning of Large Language Models
Qi Liu, Mingdi Sun, Yongyi He +5
Supervised fine-tuning (SFT) followed by reinforcement learning (RL) has become a standard post-training paradigm for large language models. This paradigm provides a cold-start for…
cs.CL2024
In-Context Former: Lightning-fast Compressing Context for Large Language Model
Xiangfeng Wang, Zaiyi Chen, Zheyong Xie +3
With the rising popularity of Transformer-based large language models (LLMs), reducing their high inference costs has become a significant research focus. One effective approach is…