128 citations · 131 across the 4 of their papers we have counts for
4 papers · 1 filter
Parcae: Scaling Laws For Stable Looped Language Models
Hayden Prairie, Zachary Novack, Taylor Berg-Kirkpatrick +1
Traditional fixed-depth architectures scale quality by increasing training FLOPs, typically through increased parameterization, at the expense of a higher memory footprint, or data…
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text
Bohan Li, Junxian He, Graham Neubig +2
When trained effectively, the Variational Autoencoder (VAE) is both a powerful language model and an effective representation learning framework. In practice, however, VAEs are tra…
Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
Junxian He, Daniel Spokoyny, Graham Neubig +1
The variational autoencoder (VAE) is a popular combination of deep latent variable model and accompanying variational learning technique. By using a neural inference network to app…
Efficient Correlated Topic Modeling with Topic Embedding
Junxian He, Zhiting Hu, Taylor Berg-Kirkpatrick +2
Correlated topic modeling has been limited to small model and problem sizes due to their high computational cost and poor scaling. In this paper, we propose a new model which learn…