6 papers
MARS: Enabling Autoregressive Models Multi-Token Generation
Ziqi Jin, Lei Wang, Ziwei Luo +1
Autoregressive (AR) language models generate text one token at a time, even when consecutive tokens are highly predictable given earlier context. We introduce MARS (Mask AutoRegreS…
Advancing Machine Learning Applications in Quantum Few-Body Systems
Jin Ziqi, Paolo Recchia, Mario Gattobigio
This paper presents a general neural network framework for solving quantum few-body systems, extending prior methods to handle diverse particle masses, interaction types, and syste…
Document Reconstruction Unlocks Scalable Long-Context RLVR
Yao Xiao, Lei Wang, Yue Deng +6
Reinforcement Learning with Verifiable Rewards~(RLVR) has become a prominent paradigm to enhance the capabilities (i.e.\ long-context) of Large Language Models~(LLMs). However, it…
Self-Rewarding Sequential Monte Carlo for Masked Diffusion Language Models
Ziwei Luo, Ziqi Jin, Lei Wang +2
This work presents self-rewarding sequential Monte Carlo (SMC), an inference-time scaling algorithm enabling effective sampling of masked diffusion language models (MDLMs). Our alg…
On the Role of Discreteness in Diffusion LLMs
Ziqi Jin, Bin Wang, Xiang Lin +2
Diffusion models offer appealing properties for language generation, such as parallel decoding and iterative refinement, but the discrete and highly structured nature of text chall…
As Simple as Fine-tuning: LLM Alignment via Bidirectional Negative Feedback Loss
Xin Mao, Feng-Lin Li, Huimin Xu +3
Direct Preference Optimization (DPO) has emerged as a more computationally efficient alternative to Reinforcement Learning from Human Feedback (RLHF) with Proximal Policy Optimizat…