14 citations · 23 across the 14 of their papers we have counts for
14 papers
ChainPrune: Evaluating and Reducing Redundancy in Long Chain-of-Thought Reasoning
Weihang Pan, Zhengxu Yu, Yuxiang Zhang +5
Chain-of-Thought (CoT) reasoning has significantly enhanced the multi-step problem-solving capabilities of large language models (LLMs) by introducing explicit intermediate reasoni…
From Self-Attention to Connection Laplacian: A Unified Operator View of Transformers
Binbin Lin, Wei Chen, Yalun Li +3
Self-attention is a ubiquitous primitive in modern sequence models, yet its operator-level geometry is only partially understood. We view a token sequence as a vector field over th…
InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
Ke Li, Dong An, Xiaoling Zang +6
Low-bit activation quantization remains a major bottleneck in efficient large language model (LLM) deployment. The difficulty is not only that activations contain outliers, but tha…
HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference
Zhiyuan Shi, Qibo Qiu, Feng Xue +5
The linear memory growth of the KV cache poses a significant bottleneck for LLM inference in long-context tasks. Existing static compression methods often fail to preserve globally…
Improving Complex Reasoning with Dynamic Prompt Corruption: A soft prompt Optimization Approach
Sinan Fan, Liang Xie, Chen Shen +7
Prompt-tuning (PT) for large language models (LLMs) can facilitate the performance on various conventional NLP tasks with significantly fewer trainable parameters. However, our inv…
Enhancing Multiple Dimensions of Trustworthiness in LLMs via Sparse Activation Control
Yuxin Xiao, Chaoqun Wan, Yonggang Zhang +5
As the development and application of Large Language Models (LLMs) continue to advance rapidly, enhancing their trustworthiness and aligning them with human preferences has become…
SciPIP: An LLM-based Scientific Paper Idea Proposer
Wenxiao Wang, Lihui Gu, Liye Zhang +7
The rapid advancement of large language models (LLMs) has opened new possibilities for automating the proposal of innovative scientific ideas. This process involves two key phases:…
Depth Any Video with Scalable Synthetic Data
Honghui Yang, Di Huang, Wei Yin +6
Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introd…
Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts
Yanting Yang, Minghao Chen, Qibo Qiu +5
For a general-purpose robot to operate in reality, executing a broad range of instructions across various environments is imperative. Central to the reinforcement learning and plan…
AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learning
Minghao Chen, Yihang Li, Yanting Yang +3
Large Language Models (LLM) based agents have shown promise in autonomously completing tasks across various domains, e.g., robotics, games, and web navigation. However, these agent…
Model Compression and Efficient Inference for Large Language Models: A Survey
Wenxiao Wang, Wei Chen, Yicong Luo +6
Transformer based large language models have achieved tremendous success. However, the significant memory and computational costs incurred during the inference process make it chal…
UniPAD: A Universal Pre-training Paradigm for Autonomous Driving
Honghui Yang, Sha Zhang, Di Huang +9
In the context of autonomous driving, the significance of effective feature learning is widely acknowledged. While conventional 3D self-supervised pre-training methods have shown w…
MCS: Multi-Target Masked Point Modeling with Learnable Codebook and Siamese Decoders
Qibo Qiu, Honghui Yang, Wenxiao Wang +5
Masked point modeling has become a promising scheme of self-supervised pre-training for point clouds. Existing methods reconstruct either the original points or related features as…
CrossFormer++: A Versatile Vision Transformer Hinging on Cross-scale Attention
Wenxiao Wang, Wei Chen, Qibo Qiu +5
While features of different scales are perceptually important to visual inputs, existing vision transformers do not yet take advantage of them explicitly. To this end, we first pro…