8 papers · 1 filter
Se-DPO: Self-Evolving Token Credit for Direct Preference Optimization
Wenxiao Zhao, Shu Wang, Ying Nian Wu
Direct Preference Optimization (DPO) aggregates token-level log-probability ratios via uniform summation, implicitly treating all tokens as contributing equally to the preference s…
A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination
Wenxiao Zhao, Dong Liu, Kaiyi Xu +9
Symbolic regression aims to discover closed-form equations from data, but existing LLM-guided methods often rely on a unified proposal loop that compresses heterogeneous search fai…
CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference
Dong Liu, Shu Wang, Yanxuan Yu +2
Large language models face significant computational bottlenecks during inference due to the expensive output layer computation over large vocabularies. We present CSV-Decode, a no…
TS-PEFT: Unveiling Token-Level Redundancy in Parameter-Efficient Fine-Tuning
Dabiao Ma, Ziming Dai, Zhimin Xin +3
Current Parameter-Efficient Fine-Tuning (PEFT) methods typically operate under an implicit assumption: Once a target module is selected, every token passing through it contributes…
Latent Thought Models with Variational Bayes Inference-Time Computation
Deqian Kong, Minglu Zhao, Dehong Xu +8
We propose a novel class of language models, Latent Thought Models (LTMs), which incorporate explicit latent thought vectors that follow an explicit prior model in latent space. Th…
Explore the Reasoning Capability of LLMs in the Chess Testbed
Shu Wang, Lei Ji, Renxi Wang +4
Reasoning is a central capability of human intelligence. In recent years, with the advent of large-scale datasets, pretrained large language models have emerged with new capabiliti…