24 citations · 80 across the 7 of their papers we have counts for
7 papers
Inverse Scaling: When Bigger Isn't Better
Ian R. McKenzie, Alexander Lyzhov, Michael Pieler +24
Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale (model size, training data, and compute). Here, w…
Improving Mutual Information Estimation with Annealed and Energy-Based Bounds
Rob Brekelmans, Sicong Huang, Marzyeh Ghassemi +3
Mutual information (MI) is a fundamental quantity in information theory and machine learning. However, direct estimation of MI is intractable, even if the true joint probability de…
Efficient Parametric Approximations of Neural Network Function Space Distance
Nikita Dhawan, Sicong Huang, Juhan Bae +1
It is often useful to compactly summarize important properties of model parameters and training data so that they can be used later without storing and/or iterating over the entire…
ED-FAITH: Evaluating Dialogue Summarization on Faithfulness
Sicong Huang, Asli Celikyilmaz, Haoran Li
Abstractive summarization models typically generate content unfaithful to the input, thus highlighting the significance of evaluating the faithfulness of generated summaries. Most…
Evaluating Lossy Compression Rates of Deep Generative Models
Sicong Huang, Alireza Makhzani, Yanshuai Cao +1
The field of deep generative modeling has succeeded in producing astonishingly realistic-seeming images and audio, but quantitative evaluation remains a challenge. Log-likelihood i…
TimbreTron: A WaveNet(CycleGAN(CQT(Audio))) Pipeline for Musical Timbre Transfer
Sicong Huang, Qiyang Li, Cem Anil +3
In this work, we address the problem of musical timbre transfer, where the goal is to manipulate the timbre of a sound sample from one instrument to match another instrument while…
Unsupervised Cipher Cracking Using Discrete GANs
Aidan N. Gomez, Sicong Huang, Ivan Zhang +3
This work details CipherGAN, an architecture inspired by CycleGAN used for inferring the underlying cipher mapping given banks of unpaired ciphertext and plaintext. We demonstrate…