7 papers
DLR: Zero-Inference-Cost Latent Residuals for Low-Rank Pre-Training
Dong Wang, Wenwu Tang, Yun Cheng +1
Large language models have driven recent progress in language and multimodal AI, yet pre-training them at scale is prohibitively expensive. Low-rank pre-training, which factorizes…
GRAIL: Post-hoc Compensation by Linear Reconstruction for Compressed Networks
Wenwu Tang, Dong Wang, Lothar Thiele +1
Structured deep model compression methods are hardware-friendly and substantially reduce memory and inference costs. However, under aggressive compression, the resulting accuracy d…
Cut Less, Fold More: Model Compression through the Lens of Projection Geometry
Olga Saukh, Dong Wang, Haris Å ikiÄ +2
Compressing neural networks without retraining is vital for deployment at scale. We study calibration-free compression through the lens of projection geometry: structured pruning i…
From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devices
Georg Slamanig, Francesco Corti, Olga Saukh
Parameter-efficient fine-tuning (PEFT) methods reduce the computational costs of updating deep learning models by minimizing the number of additional parameters used to adapt a mod…
APEX: Automated Parameter Exploration for Low-Power Wireless Protocols
Mohamed Hassaan M. Hydher, Markus Schuss, Olga Saukh +2
Careful parametrization of networking protocols is crucial to maximize the performance of low-power wireless systems and ensure that stringent application requirements can be met.…
FocusDD: Real-World Scene Infusion for Robust Dataset Distillation
Youbing Hu, Yun Cheng, Olga Saukh +4
Dataset distillation has emerged as a strategy to compress real-world datasets for efficient training. However, it struggles with large-scale and high-resolution datasets, limiting…