#model compression
23 resultsWIDE: Boosting Adaptive LLM Inference via Token-level Dynamic Width Pruning
Haozhe Hu, Hao Wu, Peiran Yin +3
WIDE introduces a token-level dynamic width pruning framework for large language model inference, allowing each token to selectively activate attention heads and feed‑forward chann…
MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers
Md. Mehrab Hossain Opi, Robiul Islam Ryad, Md. Umar Faruk
MixFrag is a framework that estimates how sensitive each part of a Vision Transformer is to quantization and then assigns mixed-precision bit-widths using a knapsack optimization t…
Memory Efficient Tabular Foundation Models
Shuting Luo, Monika Mikhail Kanaan, Cameron Gordon +2
The paper studies how to reduce the memory footprint of tabular foundation models like TabPFN using compression techniques, achieving up to 7.6× memory savings with little performa…
Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold
Songshuo Lu, Zhi Chen, Yaohua Tang
The paper proposes an expand‑then‑compress framework that builds a diverse set of RL‑trained teacher models and then distills them into a single student model, improving reasoning,…
Prox: Training-Free FFN Activation Sparsity via Approximate Intermediate-Channel Salience in LLMs
Jinyi Liu, Wei Chen, Pengyu Chen +4
The paper introduces Prox, a training-free framework that sparsifies feed‑forward network activations in large language models by approximating intermediate‑channel salience, enabl…
OrthKD: Extracting Generalized Clinical Knowledge from Heterogeneous Teachers for Lightweight Deployment
Yi Xu, Cheng Chen, Mufan Cao
The paper introduces OrthKD, a knowledge distillation framework that selectively trusts a strong CNN and a weaker transformer teacher to train a lightweight MobileNetV3 model for d…
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model
Mahendra Singh Rathor, Anagheem Azzam
The paper investigates how LoRA rank, adapted modules, and low‑bit quantization trade off accuracy and resource usage when fine‑tuning a 60 M‑parameter T5‑small model for the WikiS…
Gibbs randomness-compression proposition
M. Süzen, M Süzen
The paper proposes a theorem linking Gibbs entropy (a measure of randomness) to lossy model compression, showing that the entropy of remaining network weights correlates with learn…
Zero-Shot Quantization for Object Detectors using Off-the-Shelf Generative Models
Hyunho Lee, Kyomin Hwang, Hyeonjin Kim +3
The paper proposes GoodQ, a method that uses off-the-shelf generative models to create synthetic training data for zero-shot quantization of object detectors, enabling low-bit quan…
Non-vacuous Generalization Bounds for Reinforcement Learning with Verifiable Rewards
Yuxuan Zhu, Rohan Alur, Daniel Kang
The paper derives the first non‑vacuous PAC‑Bayes generalization bounds for parameter‑efficient reinforcement learning with verifiable rewards applied to billion‑parameter language…
Gold-Guided Programmatic Distillation for Financial Reasoning over Hybrid Tables and Text
Yun Dong, Erica Zhao, Elana Chen
The paper introduces a method that distills reliable numerical reasoning from a large language model into a smaller model by using execution-verified Python programs instead of tex…
DarwinLM: Evolutionary Structured Pruning of Large Language Models
Shengkun Tang, Oliver Sieberling, Eldar Kurtic +2
The paper introduces DarwinLM, an evolutionary search method for training-aware structured pruning of large language models that integrates lightweight post‑pruning training to fin…
ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level
Chethan Reddy G. P
The paper presents ExTernD, a post‑training factorization that expands the rank of ternary matrix decompositions to correct quantization errors, enabling large language models to a…
Constraint-Driven Model Optimization: An Industry Framework for Selecting Compression and Acceleration Techniques in Modern Machine Learning Systems
Dhruv Shivkant, Saket Mohanty, Utkarsh Wadhwa +1
The paper introduces a constraint-driven, multi-objective framework for selecting model compression and acceleration techniques based on deployment constraints such as latency, mem…
Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate
Joshua Hill
The paper demonstrates that for 4‑bit mixed‑precision quantization, the loss from quantizing a set of layers is largely additive across individual layers, and introduces a simple c…
MobileSAM2: Lightweight Segment Anything for Spatial Intelligence
Kai Jiang, Jiaxing Huang, Jingyi Zhang +5
The paper introduces MobileSAM2, a lightweight version of the SAM2 segmentation model designed for mobile devices, using hypergraph-based knowledge distillation to transfer tempora…
dMX: Differentiable Mixed-Precision Assignment for Low-Precision Floating-Point Formats
Giuseppe Franco, Ian Colbert, Pablo Monteagudo-Lago +2
The paper presents dMX, a differentiable framework that learns per-layer floating‑point bit‑widths for large language models, enabling mixed‑precision quantization that balances ac…
Score-Only Distillation for Compact Dense Retrieval
Kirill Dubovikov, Martin Takac, Salem Lahlou
The paper proposes a method to compress large dense retrieval models into much smaller ones by distilling only the teacher's score vectors, using a row‑centered score‑vector object…
Towards Efficient Convolutional Neural Network for Embedded Hardware via Multi-Dimensional Pruning
Hao Kong, Di Liu, Xiangzhong Luo +5
The paper introduces TECO, a framework that jointly prunes depth, width, and input resolution of convolutional neural networks to improve speed and resource usage on embedded devic…
Prune, Update and Trim: Robust Structured Pruning for Large Language Models
Diego Coello de Portugal Mecke, Tom Hanika, Lars Schmidt-Thieme
The paper introduces Putri, a post‑training pruning method for large language models that updates remaining weights, prunes feed‑forward layers sequentially, and removes individual…
Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Data
Shikai Qiu, Marc Finzi, Yujia Zheng +2
The paper proposes requential coding, a method where a teacher model selects training samples from the student’s own distribution so that only disagreements need to be encoded, yie…
Sparse Inter-Layer Dependencies of Transformer FFN Neurons
Johannes Knittel, Hanspeter Pfister
The paper introduces a training‑free method to attribute the activation of individual feed‑forward network neurons in Transformers to a small set of upstream neuron activations and…
Securing LLMs in the Wild: Privacy and Security Challenges at the Edge
Ren-Yi Huang, Mingchen Li, Dumindu Samaraweera +1
The paper examines how efficiency‑focused optimizations for deploying large language models on edge devices create new security and privacy vulnerabilities, and it introduces a tax…