14 citations · 26 across the 8 of their papers we have counts for
9 papers
Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
Zhijun Tu, Jian Li, Yuanyuan Xi +5
1-bit LLM quantization offers significant advantages in reducing storage and computational costs. However, existing methods typically train 1-bit LLMs from scratch, failing to full…
Transferable text data distillation by trajectory matching
Rong Yao, Hailin Hu, Yifei Fu +5
In the realm of large language model (LLM), as the size of large models increases, it also brings higher training costs. There is a urgent need to minimize the data size in LLM tra…
Multi-Granularity Semantic Revision for Large Language Model Distillation
Xiaoyu Liu, Yun Zhang, Wei Li +7
Knowledge distillation plays a key role in compressing the Large Language Models (LLMs), which boosts a small-size student model under large teacher models' guidance. However, exis…
Distilling Semantic Priors from SAM to Efficient Image Restoration Models
Quan Zhang, Xiaoyu Liu, Wei Li +6
In image restoration (IR), leveraging semantic priors from segmentation models has been a common approach to improve performance. The recent segment anything model (SAM) has emerge…
IPT-V2: Efficient Image Processing Transformer using Hierarchical Attentions
Zhijun Tu, Kunpeng Du, Hanting Chen +4
Recent advances have demonstrated the powerful capability of transformer architecture in image restoration. However, our analysis indicates that existing transformerbased methods c…
SAM-DiffSR: Structure-Modulated Diffusion Model for Image Super-Resolution
Chengcheng Wang, Zhiwei Hao, Yehui Tang +4
Diffusion-based super-resolution (SR) models have recently garnered significant attention due to their potent restoration capabilities. But conventional diffusion models perform no…
A Survey on Transformer Compression
Yehui Tang, Yunhe Wang, Jianyuan Guo +4
Transformer plays a vital role in the realms of natural language processing (NLP) and computer vision (CV), specially for constructing large language models (LLM) and large vision…
GenDet: Towards Good Generalizations for AI-Generated Image Detection
Mingjian Zhu, Hanting Chen, Mouxiao Huang +4
The misuse of AI imagery can have harmful societal effects, prompting the creation of detectors to combat issues like the spread of fake news. Existing methods can effectively dete…
CBQ: Cross-Block Quantization for Large Language Models
Xin Ding, Xiaoyu Liu, Zhijun Tu +8
Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs. However, existing PTQ methods only focus on handling the ou…