420 citations · 572 across the 28 of their papers we have counts for
10 papers · 1 filter
Gradually Compacting Large Language Models for Reasoning Like a Boiling Frog
Yiran Zhao, Shengyang Zhou, Zijian Wu +7
Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, but their substantial size often demands significant computational resources. To reduce resource c…
Scalable Chain of Thoughts via Elastic Reasoning
Yuhui Xu, Hanze Dong, Lei Wang +3
Large reasoning models (LRMs) have achieved remarkable progress on complex tasks by generating extended chains of thought (CoT). However, their uncontrolled output lengths pose sig…
Fractured Chain-of-Thought Reasoning
Baohao Liao, Hanze Dong, Yuhui Xu +4
Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference…
A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
Wei Xiong, Jiarui Yao, Yuhui Xu +8
Reinforcement learning (RL) has become a prevailing approach for fine-tuning large language models (LLMs) on complex reasoning tasks. Among recent methods, GRPO stands out for its…
Reward Models Identify Consistency, Not Causality
Yuhui Xu, Hanze Dong, Lei Wang +2
Reward models (RMs) play a crucial role in aligning large language models (LLMs) with human preferences and enhancing reasoning quality. Traditionally, RMs are trained to rank cand…
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models
Yuhui Xu, Lingxi Xie, Xiaotao Gu +6
Recently years have witnessed a rapid development of large language models (LLMs). Despite the strong ability in many language-understanding tasks, the heavy computational burden l…
TRP: Trained Rank Pruning for Efficient Deep Neural Networks
Yuhui Xu, Yuxi Li, Shuai Zhang +6
To enable DNNs on edge devices like mobile phones, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations. Several…
Fitting the Search Space of Weight-sharing NAS with Graph Convolutional Networks
Xin Chen, Lingxi Xie, Jun Wu +3
Neural architecture search has attracted wide attentions in both academia and industry. To accelerate it, researchers proposed weight-sharing methods which first train a super-netw…
DNQ: Dynamic Network Quantization
Yuhui Xu, Shuai Zhang, Yingyong Qi +3
Network quantization is an effective method for the deployment of neural networks on memory and energy constrained mobile devices. In this paper, we propose a Dynamic Network Quant…
Deep Neural Network Compression with Single and Multiple Level Quantization
Yuhui Xu, Yongzhuang Wang, Aojun Zhou +2
Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth info…