Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
GRIT -- Geometry-Aware PEFT with K-FACPreconditioning, Fisher-Guided Reprojection, andDynamic Rank Adaptation
Pritish Saha, Chandrav Rajbangshi, Rudra Goyal +6
Parameter-efficient fine-tuning (PEFT) is the default way to adapt LLMs, but widely used LoRA and QLoRA are largely geometry-agnostic: they optimize in fixed, randomly oriented low…
cs.LG2019
DZip: improved general-purpose lossless compression based on novel neural network modeling
Mohit Goyal, Kedar Tatwawadi, Shubham Chandak +1
We consider lossless compression based on statistical data modeling followed by prediction-based encoding, where an accurate statistical model for the input data leads to substanti…
cs.LG2019
Learning Activation Functions: A new paradigm for understanding Neural Networks
Mohit Goyal, Rajan Goyal, Brejesh Lall
The scope of research in the domain of activation functions remains limited and centered around improving the ease of optimization or generalization quality of neural networks (NNs…