1 citations · 1 across the 4 of their papers we have counts for
7 papers · 1 filter
POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
Zeju Qiu, Lixin Liu, Adrian Weller +2
Efficient and stable training of large language models (LLMs) remains a core challenge in modern machine learning systems. To address this challenge, Reparameterized Orthogonal Equ…
Orthogonal Finetuning Made Scalable
Zeju Qiu, Weiyang Liu, Adrian Weller +1
Orthogonal finetuning (OFT) offers highly parameter-efficient adaptation while preventing catastrophic forgetting, but its high runtime and memory demands limit practical deploymen…
Can Large Language Models Understand Symbolic Graphics Programs?
Zeju Qiu, Weiyang Liu, Haiwen Feng +7
Against the backdrop of enthusiasm for large language models (LLMs), there is a growing need to scientifically assess their capabilities and shortcomings. This is nontrivial in par…
Certification for Differentially Private Prediction in Gradient-Based Training
Matthew Wicker, Philip Sosnin, Igor Shilov +5
We study private prediction where differential privacy is achieved by adding noise to the outputs of a non-private model. Existing methods rely on noise proportional to the global…
Estimation of Concept Explanations Should be Uncertainty Aware
Vihari Piratla, Juyeon Heo, Katherine M. Collins +2
Model explanations can be valuable for interpreting and debugging predictive models. We study a specific kind called Concept Explanations, where the goal is to interpret a model us…
Certification of Distributional Individual Fairness
Matthew Wicker, Vihari Piratia, Adrian Weller
Providing formal guarantees of algorithmic fairness is of paramount importance to socially responsible deployment of machine learning algorithms. In this work, we study formal guar…