13 citations · 43 across the 18 of their papers we have counts for
5 papers · 1 filter
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
Han Guo, Philip Greengard, Eric P. Xing +1
We propose a simple approach for memory-efficient adaptation of pretrained language models. Our approach uses an iterative algorithm to decompose each pretrained matrix into a high…
Uniform approximation of common Gaussian process kernels using equispaced Fourier grids
Alex Barnett, Philip Greengard, Manas Rachh
The high efficiency of a recently proposed method for computing with Gaussian processes relies on expanding a (translationally invariant) covariance kernel into complex exponential…
A calibrated BISG for inferring race from surname and geolocation
Philip Greengard, Andrew Gelman
Bayesian Improved Surname Geocoding (BISG) is a ubiquitous tool for predicting race and ethnicity using an individual's geolocation and surname. Here we demonstrate that statistica…
Learning to Grow Pretrained Models for Efficient Transformer Training
Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen +6
Scaling transformers has led to significant breakthroughs in many domains, leading to a paradigm in which larger versions of existing models are trained and released on a periodic…
Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach
Han Guo, Philip Greengard, Hongyi Wang +3
The canonical formulation of federated learning treats it as a distributed optimization problem where the model parameters are optimized against a global loss function that decompo…