7 papers
Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures
SiYuan Ma, Yiqin Luo, Zhangji +8
The paper introduces a technique called forked futures that samples future operations after a prefix state to compare hidden states of language models, enabling the discovery of re…
DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training
Can Jin, Hongwu Peng, Mingcan Xiang +7
Sparse Mixture-of-Experts architectures are essential for scaling model capacity efficiently, yet the standard Top- routing imposes a rigid sparsity pattern that ignores the int…
GraphOmni: A Comprehensive and Extensible Benchmark Framework for Large Language Models on Graph-theoretic Tasks
Hao Xu, Xiangru Jian, Xinjian Zhao +9
This paper introduces GraphOmni, a comprehensive benchmark designed to evaluate the reasoning capabilities of LLMs on graph-theoretic tasks articulated in natural language. GraphOm…
Your Reward Function for RL is Your Best PRM for Search: Unifying RL and Search-Based TTS
Can Jin, Yang Zhou, Qixin Zhang +8
Test-time scaling (TTS) for large language models (LLMs) has thus far fallen into two largely separate paradigms: (1) reinforcement learning (RL) methods that optimize sparse outco…
Reasoning over Precedents Alongside Statutes: Case-Augmented Deliberative Alignment for LLM Safety
Can Jin, Rui Wu, Tong Che +10
Ensuring that Large Language Models (LLMs) adhere to safety principles without refusing benign requests remains a significant challenge. While OpenAI introduces deliberative alignm…
Two Heads are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoning
Can Jin, Hongwu Peng, Qixin Zhang +3
Multi-agent systems (MAS) built on large language models (LLMs) offer a promising path toward solving complex, real-world tasks that single-agent systems often struggle to manage.…