8 papers
AsyncLane: Decoupling Refinement from Advancement in Diffusion Language Model Decoding
Yingxuan Ren, Yuxuan Lou, Yong Liu +4
Block-wise semi-autoregressive decoding is the standard inference paradigm for diffusion large language models (DLMs), but it imposes a strict dependency between blocks: the next b…
On-Policy Adversarial Flow Distillation for Autoregressive Video Generation
Yang Luo, Shengju Qian, Xiaohang Tang +4
Autoregressive video generators are attractive for streaming, long-horizon, and interactive applications, but distilling strong black-box teachers into causal students remains diff…
CAMEL: Confidence-Gated Reflection for Reward Modeling
Zirui Zhu, Hailun Xu, Yang Luo +4
Reward models play a fundamental role in aligning large language models with human preferences. Existing methods predominantly follow two paradigms: scalar discriminative preferenc…
Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
Yong Liu, Zirui Zhu, Chaoyu Gong +3
While fine-tuning large language models (LLMs) for specific tasks often yields impressive results, it comes at the cost of memory inefficiency due to back-propagation in gradient-b…
FOCUS: Efficient Keyframe Selection for Long Video Understanding
Zirui Zhu, Hailun Xu, Yang Luo +4
Multimodal large language models (MLLMs) represent images and video frames as visual tokens. Scaling from single images to hour-long videos, however, inflates the token budget far…
Unbiased Gradient Low-Rank Projection
Rui Pan, Yang Luo, Yuxing Liu +2
Memory-efficient optimization is critical for training increasingly large language models (LLMs). A popular strategy involves gradient low-rank projection, storing only the project…