5 papers
EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation
Shu-Hao Zhang, Le-Tong Huang, Xiang-Sheng Deng +5
Quantization has emerged as a mainstream approach for deploying Large Language Models (LLMs) on resource-constrained devices, yet compressing precision below 4-bit typically causes…
SimDiff: Depth Pruning via Similarity and Difference
Yuli Chen, Shuhao Zhang, Fanshen Meng +4
Depth pruning improves the deployment efficiency of large language models (LLMs) by identifying and removing redundant layers. A widely accepted standard for this identification pr…
Beyond A Fixed Seal: Adaptive Stealing Watermark in Large Language Models
Shuhao Zhang, Yuli Chen, Jiale Han +2
Watermarking provides a critical safeguard for large language model (LLM) services by facilitating the detection of LLM-generated text. Correspondingly, stealing watermark algorith…
DLP: Dynamic Layerwise Pruning in Large Language Models
Yuli Chen, Bo Cheng, Jiale Han +3
Pruning has recently been widely adopted to reduce the parameter scale and improve the inference efficiency of Large Language Models (LLMs). Mainstream pruning techniques often rel…
CEFW: A Comprehensive Evaluation Framework for Watermark in Large Language Models
Shuhao Zhang, Bo Cheng, Jiale Han +4
Text watermarking provides an effective solution for identifying synthetic text generated by large language models. However, existing techniques often focus on satisfying specific…