3 citations · 5 across the 5 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024
Modular Duality in Deep Learning
Jeremy Bernstein, Laker Newhouse
An old idea in optimization theory says that since the gradient is a dual vector it may not be subtracted from the weights without first being mapped to the primal space where the…
cs.LG2024
Old Optimizer, New Norm: An Anthology
Jeremy Bernstein, Laker Newhouse
Deep learning optimizers are often motivated through a mix of convex and approximate second-order theory. We select three such methods -- Adam, Shampoo and Prodigy -- and argue tha…
cs.CL2024★ 1 cited
An Assessment of Model-On-Model Deception
Julius Heitkoetter, Michael Gerovitch, Laker Newhouse
The trustworthiness of highly capable language models is put at risk when they are able to produce deceptive outputs. Moreover, when models are vulnerable to deception it undermine…