5 papers
DASH: Fast Differentiable Architecture Search for Hybrid Attention in Minutes on a Single GPU
Weizhe Chen, Miao Zhang, Junpeng Jiang +3
Hybrid attention architectures are becoming an increasingly important paradigm for improving LLM inference efficiency while preserving model quality, making hybrid architecture des…
LSPO: Length-aware Dynamic Sampling for Policy Optimization in LLM Reasoning
Weizhe Chen, Sven Koenig, Bistra Dilkina
Since the release of Deepseek-R1, reinforcement learning with verifiable rewards (RLVR) has become a central approach for training large language models (LLMs) on reasoning tasks.…
Iterative Deepening Sampling as Efficient Test-Time Scaling
Weizhe Chen, Sven Koenig, Bistra Dilkina
Recent reasoning models, such as OpenAI's O1 series, have demonstrated exceptional performance on complex reasoning tasks and revealed new test-time scaling laws. Inspired by this,…
Flaming-hot Initiation with Regular Execution Sampling for Large Language Models
Weizhe Chen, Zhicheng Zhang, Guanlin Liu +6
Since the release of ChatGPT, large language models (LLMs) have demonstrated remarkable capabilities across various domains. A key challenge in developing these general capabilitie…
RePrompt: Planning by Automatic Prompt Engineering for Large Language Models Agents
Weizhe Chen, Sven Koenig, Bistra Dilkina
In the past year, large language models (LLMs) have had remarkable success in domains outside the traditional natural language processing, and their capacity is further expanded in…