1 citations · 6 across the 24 of their papers we have counts for
5 papers · 2 filters
AWP: Activation-Aware Weight Pruning and Quantization with Projected Gradient Descent
Jing Liu, Toshiaki Koike-Akino, Ye Wang +2
To address the enormous size of Large Language Models (LLMs), model compression methods, such as quantization and pruning, are often deployed, especially on edge devices. In this w…
TuneComp: Joint Fine-tuning and Compression for Large Foundation Models
Xiangyu Chen, Jing Liu, Ye Wang +4
To reduce model size during post-training, compression methods, including knowledge distillation, low-rank approximation, and pruning, are often applied after fine-tuning the model…
-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
Toshiaki Koike-Akino, Jing Liu, Ye Wang
To tackle the huge computational demand of large foundation models, activation-aware compression techniques without retraining have been introduced. However, since these rely on ca…
LatentLLM: Attention-Aware Joint Tensor Compression
Toshiaki Koike-Akino, Xiangyu Chen, Jing Liu +4
Modern foundation models such as large language models (LLMs) and large multi-modal models (LMMs) require a massive amount of computational and memory resources. We propose a new f…
Smoothed Embeddings for Robust Language Models
Ryo Hase, Md Rafi Ur Rashid, Ashley Lewis +4
Improving the safety and reliability of large language models (LLMs) is a crucial aspect of realizing trustworthy AI systems. Although alignment methods aim to suppress harmful con…