4 papers
Scalable Thermodynamic Second-order Optimization
Kaelan Donatella, Samuel Duffield, Denis Melanson +7
Many hardware proposals have aimed to accelerate inference in AI workloads. Less attention has been paid to hardware acceleration of training, despite the enormous societal impact…
Thermodynamic Bayesian Inference
Maxwell Aifer, Samuel Duffield, Kaelan Donatella +6
A fully Bayesian treatment of complicated predictive models (such as deep neural networks) would enable rigorous uncertainty quantification and the automation of higher-level tasks…
Extended Mind Transformers
Phoebe Klett, Thomas Ahle
Pre-trained language models demonstrate general intelligence and common sense, but long inputs quickly become a bottleneck for memorizing information at inference time. We resurfac…
Scalable Bayesian Learning with posteriors
Samuel Duffield, Kaelan Donatella, Johnathan Chiu +2
Although theoretically compelling, Bayesian learning with modern machine learning models is computationally challenging since it requires approximating a high dimensional posterior…