49 citations · 93 across the 11 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.CL2022★ 1 cited
Circling Back to Recurrent Models of Language
Gábor Melis
Just because some purely recurrent models suffer from being hard to optimize and inefficient on today's hardware, they are not necessarily bad models of language. We demonstrate th…
stat.ML2022
Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight Averaging for Better Generalization
Gábor Melis
Tail Averaging improves on Polyak averaging's non-asymptotic behaviour by excluding a number of leading iterates of stochastic optimization from its calculations. In practice, with…