7 papers
SparseSAM: Structured Sparsification of Activations in Segment Anything Models
Hoai-Chau Tran, Chi H. Nguyen, Duy M. H. Nguyen +3
The Segment Anything Model (SAM) achieves strong open-vocabulary segmentation, but its ViT-based image encoders dominate inference latency and memory. Existing activation compressi…
OPPO: Accelerating PPO-based RLHF via Pipeline Overlap
Kaizhuo Yan, Yingjie Yu, Yifan Yu +2
Proximal Policy Optimization (PPO)-based reinforcement learning from human feedback (RLHF) is a widely adopted paradigm for aligning large language models (LLMs) with human prefere…
tLoRA: Efficient Multi-LoRA Training with Elastic Shared Super-Models
Kevin Li, Dibyadeep Saha, Avni Kanodia +1
As Low-Rank Adaptation (LoRA) becomes the standard approach for efficiently fine-tuning large language models (LLMs), shared clusters increasingly execute many concurrent LoRA trai…
PackInfer: Compute- and I/O-Efficient Attention for Batched LLM Inference
Rui Ning, Wei Zhang, Fan Lai
Attention efficiency is critical to large language model (LLM) inference. While prior advances optimize attention execution for individual requests (e.g., FlashAttention), producti…
Inv-Entropy: A Fully Probabilistic Framework for Uncertainty Quantification in Language Models
Haoyi Song, Ruihan Ji, Naichen Shi +2
Large language models (LLMs) have transformed natural language processing, but their reliable deployment requires effective uncertainty quantification (UQ). Existing UQ methods are…
Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
Haizhong Zheng, Yang Zhou, Brian R. Bartoldson +4
Reinforcement learning, such as PPO and GRPO, has powered recent breakthroughs in LLM reasoning. Scaling rollout to sample more prompts enables models to selectively use higher-qua…