From the 1 of 6 linked papers with an AI index.
6 papers
When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation
Mojtaba Soltanalian
When can additional low-bit residual computation replace missing numerical precision for a fixed input-output map? We model a quantized residual system over a fixed horizon as a pu…
Random Parameter Noise Does Not Make Exact ReLU Verification Easy
Mojtaba Soltanalian
The paper proves that exact verification of ReLU neural networks remains computationally hard even when all weights and biases are independently perturbed by Gaussian noise and rou…
Trust, but Verify: Peeling Low-Bit Transformer Networks for Training Monitoring
Arian Eamaz, Farhang Yeganegi, Mojtaba Soltanalian
Understanding whether deep neural networks are effectively optimized remains challenging, as training occurs in highly nonconvex landscapes and standard metrics provide limited vis…
Linearization Explains Fine-Tuning in Large Language Models
Zahra Rahimi Afzal, Tara Esmaeilbeig, Mojtaba Soltanalian +1
Parameter-Efficient Fine-Tuning (PEFT) is a popular class of techniques that strive to adapt large models in a scalable and resource-efficient manner. Yet, the mechanisms underlyin…
RoCoFT: Efficient Finetuning of Large Language Models with Row-Column Updates
Md Kowsher, Tara Esmaeilbeig, Chun-Nam Yu +3
We propose RoCoFT, a parameter-efficient fine-tuning method for large-scale language models (LMs) based on updating only a few rows and columns of the weight matrices in transforme…
Predicting Through Generation: Why Generation Is Better for Prediction
Md Kowsher, Nusrat Jahan Prottasha, Prakash Bhat +6
This paper argues that generating output tokens is more effective than using pooled representations for prediction tasks because token-level generation retains more mutual informat…