13 citations · 28 across the 11 of their papers we have counts for
12 papers
Unlocking Lossless Speedups in LLMs via Discrete Diffusion
Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham +14
Large Language Models (LLMs) owe much of their success to next-token prediction (NTP), but their autoregressive (AR) structure requires slow, sequential token generation. To overco…
Scaling Beyond Masked Diffusion Language Models
Subham Sekhar Sahoo, Jean-Marie Lemercier, Zhihan Yang +4
Diffusion language models are a promising alternative to autoregressive models due to their potential for faster generation. Among discrete diffusion approaches, Masked diffusion c…
Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens
Pin-Jui Ku, He Huang, Jean-Marie Lemercier +3
This paper introduces a discrete diffusion model (DDM) framework for text-aligned speech tokenization and reconstruction. By replacing the auto-regressive speech decoder with a dis…
The Diffusion Duality
Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan +3
Uniform-state discrete diffusion models hold the promise of fast text generation due to their inherent ability to self-correct. However, they are typically outperformed by autoregr…
Esoteric Language Models: A Family of Any-Order Diffusion LLMs
Subham Sekhar Sahoo, Zhihan Yang, Yash Akhauri +7
Diffusion-based language models offer a compelling alternative to autoregressive (AR) models by enabling parallel and controllable generation. Within this family, Masked Diffusion…
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
Marianne Arriola, Aaron Gokaslan, Justin T. Chiu +5
Diffusion language models offer unique benefits over autoregressive models due to their potential for parallelized generation and controllability, yet they lag in likelihood modeli…
Remasking Discrete Diffusion Models with Inference-Time Scaling
Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo +1
Part of the success of diffusion models stems from their ability to perform iterative refinement, i.e., repeatedly correcting outputs during generation. However, modern masked disc…
Simple Guidance Mechanisms for Discrete Diffusion Models
Yair Schiff, Subham Sekhar Sahoo, Hao Phung +7
Diffusion models for continuous data gained widespread adoption owing to their high quality generation and control mechanisms. However, controllable diffusion on discrete data face…
Spike Talk: Genesis and Neural Coding Scheme Translations
Subham Sahoo
Although digitalization of future power grids offer several coordination incentives, the reliability and security of information and communication technologies (ICT) hinders its ov…
Simple and Effective Masked Diffusion Language Models
Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff +5
While diffusion models excel at generating high-quality images, prior work reports a significant performance gap between diffusion and autoregressive (AR) methods in language model…
Diffusion Models With Learned Adaptive Noise
Subham Sekhar Sahoo, Aaron Gokaslan, Chris De Sa +1
Diffusion models have gained traction as powerful algorithms for synthesizing high-quality images. Central to these algorithms is the diffusion process, a set of equations which ma…
Chasing the Intruder: A Reinforcement Learning Approach for Tracking Intruder Drones
Shivam Kainth, Subham Sahoo, Rajtilak Pal +1
Drones are becoming versatile in a myriad of applications. This has led to the use of drones for spying and intruding into the restricted or private air spaces. Such foul use of dr…