45 citations · 90 across the 30 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Discrete Diffusion Models Exploit Asymmetry to Solve Lookahead Planning Tasks
Itamar Trainin, Shauli Ravfogel, Omri Abend +1
While Autoregressive (AR) Transformer-based Generative Language Models are frequently employed for lookahead tasks, recent research suggests a potential discrepancy in their abilit…
cs.LG2020★ 14 cited
PMI-Masking: Principled masking of correlated spans
Yoav Levine, Barak Lenz, Opher Lieber +4
Masking tokens uniformly at random constitutes a common flaw in the pretraining of Masked Language Models (MLMs) such as BERT. We show that such uniform masking allows an MLM to mi…