8 papers
Thinking in Depth: Retrospective Inference for Tabular Foundation Models
Hao-Run Cai, Si-Yang Liu, Zi-Jian Cheng +8
Tabular foundation models (TFMs) are pretrained across diverse tabular tasks and make predictions on a new table at inference time using its labeled examples as context. Most recen…
CAPF: Guiding Search-Agent Rollouts with Credit-Attenuated Privileged Feedback
Bin Chen, Xinye Liao, Yiming Liu +2
Recent LLM search agents use reinforcement learning with verifiable rewards (RLVR) to learn search-augmented reasoning from outcome rewards. On hard problems, these agents rarely s…
Rethinking LLM Ensembling from the Perspective of Mixture Models
Jiale Fu, Yuchu Jiang, Peijun Wu +3
Model ensembling is a well-established technique for improving the performance of machine learning models. Conventionally, this involves averaging the output distributions of multi…
VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models
Chonghan Liu, Yimin Du, Qi An +8
Large language models frequently exhibit suboptimal performance on low resource languages, primarily due to inefficient subword segmentation and systemic training data imbalances.…
Flatter Tokens are More Valuable for Speculative Draft Model Training
Jiaming Fan, Daming Cao, Xiangzhong Luo +3
Speculative Decoding (SD) is a key technique for accelerating Large Language Model (LLM) inference, but it typically requires training a draft model on a large dataset. We approach…
Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
Xiaomin Yu, Yi Xin, Yuhui Zhang +12
Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of d…
dCache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
Yuchu Jiang, Yue Cai, Xiangzhong Luo +4
Diffusion-based large language models (dLLMs), despite their promising performance, still suffer from inferior inference efficiency. This is because dLLMs rely on bidirectional att…
AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence
Yuliang Liu, Junjie Lu, Zhaoling Chen +10
Current approaches for training Process Reward Models (PRMs) often involve breaking down responses into multiple reasoning steps using rule-based techniques, such as using predefin…