4 papers · 1 filter
When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation
Mojtaba Soltanalian
When can additional low-bit residual computation replace missing numerical precision for a fixed input-output map? We model a quantized residual system over a fixed horizon as a pu…
Random Parameter Noise Does Not Make Exact ReLU Verification Easy
Mojtaba Soltanalian
We study exact verification of ReLU networks in an adversarial smoothed model. Every network weight and bias is independently perturbed by Gaussian noise, clipped to , and…
Trust, but Verify: Peeling Low-Bit Transformer Networks for Training Monitoring
Arian Eamaz, Farhang Yeganegi, Mojtaba Soltanalian
Understanding whether deep neural networks are effectively optimized remains challenging, as training occurs in highly nonconvex landscapes and standard metrics provide limited vis…
Linearization Explains Fine-Tuning in Large Language Models
Zahra Rahimi Afzal, Tara Esmaeilbeig, Mojtaba Soltanalian +1
Parameter-Efficient Fine-Tuning (PEFT) is a popular class of techniques that strive to adapt large models in a scalable and resource-efficient manner. Yet, the mechanisms underlyin…