5 citations · 12 across the 14 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.CL2023
On Retrieval Augmentation and the Limitations of Language Model Training
Ting-Rui Chiang, Xinyan Velocity Yu, Joshua Robinson +3
Augmenting a language model (LM) with -nearest neighbors (NN) retrieval on its training data alone can decrease its perplexity, though the underlying reasons for this remain…
cs.CL2023
The Distributional Hypothesis Does Not Fully Explain the Benefits of Masked Language Model Pretraining
Ting-Rui Chiang, Dani Yogatama
We analyze the masked language modeling pretraining objective function from the perspective of the distributional hypothesis. We investigate whether better sample efficiency and th…
cs.LG2023★ 3 cited
Interpretable Diffusion via Information Decomposition
Xianghao Kong, Ollie Liu, Han Li +2
Denoising diffusion models enable conditional generation and density modeling of complex relationships like images and text. However, the nature of the learned relationships is opa…