13 citations · 15 across the 7 of their papers we have counts for
5 papers · 1 filter
DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models
Wenlong Deng, Yize Zhao, Vala Vakilian +3
Storing open-source fine-tuned models separately introduces redundancy and increases response times in applications utilizing multiple models. Delta-parameter pruning (DPP), partic…
Engineering the Neural Collapse Geometry of Supervised-Contrastive Loss
Jaidev Gill, Vala Vakilian, Christos Thrampoulidis
Supervised-contrastive loss (SCL) is an alternative to cross-entropy (CE) for classification tasks that makes use of similarities in the embedding space to allow for richer represe…
Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching
Ganesh Ramachandra Kini, Vala Vakilian, Tina Behnia +2
Supervised contrastive loss (SCL) is a competitive and often superior alternative to the cross-entropy loss for classification. While prior studies have demonstrated that both loss…
On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data
Tina Behnia, Ganesh Ramachandra Kini, Vala Vakilian +1
Various logit-adjusted parameterizations of the cross-entropy (CE) loss have been proposed as alternatives to weighted CE for training large models on label-imbalanced data far bey…
Imbalance Trouble: Revisiting Neural-Collapse Geometry
Christos Thrampoulidis, Ganesh R. Kini, Vala Vakilian +1
Neural Collapse refers to the remarkable structural properties characterizing the geometry of class embeddings and classifier weights, found by deep nets when trained beyond zero t…