8 papers
Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMs
Tanzila Rahman, Mehran Taghian Jazi, Yunke Peng +10
Low-bit quantization offers a promising avenue for reducing the computational and memory demands of Multimodal Large Language Models (MLLMs). Recent hardware support for low-precis…
HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models
Hei Yi Mak, Shadan Golestan, Hoang Le +10
We present, to our knowledge, the first end-to-end FP4 RL post-training, in which both the rollout and training policies, including their forward and backward passes, operate at 4-…
Right Answer, Wrong Tokens: Tail-Aware Credit Calibration for RLVR
Xiuyi Lou, Zicheng Xu, Yu-Neng Chuang +4
Although reinforcement learning (RL) improves the reasoning capabilities of large language models (LLMs), GRPO-style methods can suffer from a positive-credit contamination failure…
Forget Without Compromise: Nexus Sampling for Streaming KV-Cache Eviction Under Fixed Budgets
Duc Duong, Hoang Anh Duy Le, Jianwen Xie +2
Long-context and agentic LLM workloads push the KV cache past any fixed memory budget, forcing the inference stack to permanently evict tokens at every step of a continuous-inferen…
Learning at the Right Pace: Adaptive Data Scheduling Improves LLM Reinforcement Learning
Zicheng Xu, Ruixuan Zhang, Yu-Neng Chuang +7
Large Language Models (LLMs) achieve remarkable reasoning capabilities through reinforcement learning (RL) post-training. However, existing RL post-training commonly relies on unif…
Scout Before You Attend: Sketch-and-Walk Sparse Attention for Efficient LLM Inference
Hoang Anh Duy Le, Sahil Joshi, Zeyu Yang +2
Self-attention dominates the computational and memory cost of long-context LLM inference across both prefill and decode phases. To address this challenge, we introduce Sketch&Walk…
SOCKET: SOft Collision Kernel EsTimator for Sparse Attention
Sahil Joshi, Agniva Chowdhury, Wyatt Bellinger +5
Exploiting sparsity is key to efficient long-context inference, as attention dominates the cost of autoregressive decoding. Sparse attention reduces this cost by restricting comput…
A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
Kaituo Zhang, Mingzhi Hu, Hoang Anh Duy Le +9
Large Language Models (LLMs) have emerged as powerful tools for generating data across various modalities. By transforming data from a scarce resource into a controllable asset, LL…