7 papers
Nanbeige4-3B Technical Report: Exploring the Frontier of Small Language Models
Chen Yang, Guangyue Peng, Jiaying Zhu +16
We present Nanbeige4-3B, a family of small-scale but high-performing language models. Pretrained on 23T high-quality tokens and finetuned on over 30 million diverse instructions, w…
Enhancing Cross-task Transfer of Large Language Models via Activation Steering
Xinyu Tang, Zhihao Lv, Xiaoxue Cheng +5
Large language models (LLMs) have shown impressive abilities in leveraging pretrained knowledge through prompting, but they often struggle with unseen tasks, particularly in data-s…
ProtInvTree: Deliberate Protein Inverse Folding with Reward-guided Tree Search
Mengdi Liu, Xiaoxue Cheng, Zhangyang Gao +4
Designing protein sequences that fold into a target 3D structure, known as protein inverse folding, is a fundamental challenge in protein engineering. While recent deep learning me…
Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning
Xiaoxue Cheng, Junyi Li, Zhenduo Zhang +4
Large reasoning models (LRMs) have demonstrated strong performance on complex reasoning tasks, but often suffer from overthinking, generating redundant content regardless of task d…
Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking
Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao +1
Large language models (LLMs) demonstrate exceptional capabilities, yet still face the hallucination issue. Typical text generation approaches adopt an auto-regressive generation wi…
Enhancing LLM Reasoning with Reward-guided Tree Search
Jinhao Jiang, Zhipeng Chen, Yingqian Min +12
Recently, test-time scaling has garnered significant attention from the research community, largely due to the substantial advancements of the o1 model released by OpenAI. By alloc…