2 citations · 6 across the 8 of their papers we have counts for
9 papers
From Attention Sensitivity to Layer Role: Revisiting Mixed-Precision Quantization of Transformers
Nafiseh HosseinpourFardi, Negar Alihadi, Mahmoudreza Babaei +2
Most post-training quantization pipelines fit each weight matrix to its pretrained counterpart, one matrix at a time. Whether that proxy tracks what an attention block actually com…
POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
Zeju Qiu, Lixin Liu, Adrian Weller +2
Efficient and stable training of large language models (LLMs) remains a core challenge in modern machine learning systems. To address this challenge, Reparameterized Orthogonal Equ…
Orthogonal Finetuning Made Scalable
Zeju Qiu, Weiyang Liu, Adrian Weller +1
Orthogonal finetuning (OFT) offers highly parameter-efficient adaptation while preventing catastrophic forgetting, but its high runtime and memory demands limit practical deploymen…
Can Large Language Models Understand Symbolic Graphics Programs?
Zeju Qiu, Weiyang Liu, Haiwen Feng +7
Against the backdrop of enthusiasm for large language models (LLMs), there is a growing need to scientifically assess their capabilities and shortcomings. This is nontrivial in par…
Certification for Differentially Private Prediction in Gradient-Based Training
Matthew Wicker, Philip Sosnin, Igor Shilov +5
We study private prediction where differential privacy is achieved by adding noise to the outputs of a non-private model. Existing methods rely on noise proportional to the global…
Estimation of Concept Explanations Should be Uncertainty Aware
Vihari Piratla, Juyeon Heo, Katherine M. Collins +2
Model explanations can be valuable for interpreting and debugging predictive models. We study a specific kind called Concept Explanations, where the goal is to interpret a model us…
Certification of Distributional Individual Fairness
Matthew Wicker, Vihari Piratia, Adrian Weller
Providing formal guarantees of algorithmic fairness is of paramount importance to socially responsible deployment of machine learning algorithms. In this work, we study formal guar…
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
Weiyang Liu, Zeju Qiu, Yao Feng +11
Large foundation models are becoming ubiquitous, but training them from scratch is prohibitively expensive. Thus, efficiently adapting these powerful models to downstream tasks is…
Pairwise Similarity Learning is SimPLE
Yandong Wen, Weiyang Liu, Yao Feng +5
In this paper, we focus on a general yet important learning problem, pairwise similarity learning (PSL). PSL subsumes a wide range of important applications, such as open-set face…