5 citations · 9 across the 20 of their papers we have counts for
21 papers
Training-Free Hashing-Based Attention via Binary Principal Components
Daohai Yu, Zhanpeng Zeng, Keyu Chen +6
Long-context large language models (LLMs) are increasingly deployed in real-world applications, yet self-attention remains a major efficiency bottleneck -- especially during decodi…
Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation
Luxi Lin, Zhihang Lin, Zhanpeng Zeng +5
Speculative decoding accelerates LLM inference but suffers from performance degradation when target models are fine-tuned for specific domains. A naive solution is to retrain draft…
Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts
Wenhao Li, Daohai Yu, Gen Luo +7
Training Large Language Models (LLMs) on long contexts is severely constrained by prohibitive GPU memory overhead, not training time. The primary culprits are the activations, whos…
Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression
Yuntian Tang, Bohan Jia, Wenxuan Huang +7
Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs), yet it incurs substantial computational overhead for inference. E…
Multi-label Classification with Panoptic Context Aggregation Networks
Mingyuan Jiu, Hailong Zhu, Wenchuan Wei +3
Context modeling is crucial for visual recognition, enabling highly discriminative image representations by integrating both intrinsic and extrinsic relationships between objects a…
Spotlight Attention: Towards Efficient LLM Generation via Non-linear Hashing-based KV Cache Retrieval
Wenhao Li, Yuxin Zhang, Gen Luo +4
Reducing the key-value (KV) cache burden in Large Language Models (LLMs) significantly accelerates inference. Dynamically selecting critical KV caches during decoding helps maintai…
GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
Zhaohong Huang, Yuxin Zhang, Jingjing Xie +2
Recent advances in test-time adaptation (TTA) for Vision-Language Models (VLMs) have garnered increasing attention, particularly through the use of multiple augmented views of a si…
Training Long-Context LLMs Efficiently via Chunk-wise Optimization
Wenhao Li, Yuxin Zhang, Gen Luo +2
While long-context large language models (LLMs) exhibit remarkable document processing capabilities, their prohibitively high training costs often hinder customized applications. T…
Dynamic Low-Rank Sparse Adaptation for Large Language Models
Weizhong Huang, Yuxin Zhang, Xiawu Zheng +4
Despite the efficacy of network sparsity in alleviating the deployment strain of Large Language Models (LLMs), it endures significant performance degradation. Applying Low-Rank Ada…
Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective
Weizhong Huang, Yuxin Zhang, Xiawu Zheng +2
In this paper, we address the challenge of determining the layer-wise sparsity rates of large language models (LLMs) through a theoretical perspective. Specifically, we identify a…
Towards Efficient Automatic Self-Pruning of Large Language Models
Weizhong Huang, Yuxin Zhang, Xiawu Zheng +2
Despite exceptional capabilities, Large Language Models (LLMs) still face deployment challenges due to their enormous size. Post-training structured pruning is a promising solution…
Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers
Yunshan Zhong, Yuyao Zhou, Yuxin Zhang +5
Data-free quantization (DFQ) enables model quantization without accessing real data, addressing concerns regarding data security and privacy. With the growing adoption of Vision Tr…
Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model
Lirui Zhao, Tianshuo Yang, Wenqi Shao +5
This paper addresses an important problem of object addition for images with only text guidance. It is challenging because the new object must be integrated seamlessly into the ima…
Towards Accurate Post-Training Quantization of Vision Transformers via Error Reduction
Yunshan Zhong, You Huang, Jiawei Hu +2
Post-training quantization (PTQ) for vision transformers (ViTs) has received increasing attention from both academic and industrial communities due to its minimal data needs and hi…
DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model
Lirui Zhao, Yue Yang, Kaipeng Zhang +5
Text-to-image (T2I) generative models have attracted significant attention and found extensive applications within and beyond academic research. For example, the Civitai community,…
Semi-supervised Counting via Pixel-by-pixel Density Distribution Modelling
Hui Lin, Zhiheng Ma, Rongrong Ji +4
This paper focuses on semi-supervised crowd counting, where only a small portion of the training data are labeled. We formulate the pixel-wise density value to regress as a probabi…
Unified-Width Adaptive Dynamic Network for All-In-One Image Restoration
Yimin Xu, Nanxi Gao, Zhongyun Shan +2
In contrast to traditional image restoration methods, all-in-one image restoration techniques are gaining increased attention for their ability to restore images affected by divers…
Learning Image Demoireing from Unpaired Real Data
Yunshan Zhong, Yuyao Zhou, Yuxin Zhang +2
This paper focuses on addressing the issue of image demoireing. Unlike the large volume of existing studies that rely on learning from paired real data, we attempt to learn a demoi…
Boosting the Cross-Architecture Generalization of Dataset Distillation through an Empirical Study
Lirui Zhao, Yuxin Zhang, Fei Chao +1
The poor cross-architecture generalization of dataset distillation greatly weakens its practical significance. This paper attempts to mitigate this issue through an empirical study…
I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization
Yunshan Zhong, Jiawei Hu, Mingbao lin +2
Albeit the scalable performance of vision transformers (ViTs), the dense computational costs (training & inference) undermine their position in industrial applications. Post-traini…
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs
Yuxin Zhang, Lirui Zhao, Mingbao Lin +6
The ever-increasing large language models (LLMs), though opening a potential path for the upcoming artificial general intelligence, sadly drops a daunting obstacle on the way towar…