8 papers
DIP: Efficient Large Multimodal Model Training with Dynamic Interleaved Pipeline
Zhenliang Xue, Hanpeng Hu, Xing Chen +7
Large multimodal models (LMMs) have demonstrated excellent capabilities in both understanding and generation tasks with various modalities. While these models can accept flexible c…
ProfInfer: An eBPF-based Fine-Grained LLM Inference Profiler
Bohua Zou, Debayan Roy, Dhimankumar Yogesh Airao +4
As large language models (LLMs) move from research to production, understanding how inference engines behave in real time has become both essential and elusive. Unlike general-purp…
RMLer: Synthesizing Novel Objects across Diverse Categories via Reinforcement Mixing Learning
Jun Li, Zikun Chen, Haibo Chen +2
Novel object synthesis by integrating distinct textual concepts from diverse categories remains a significant challenge in Text-to-Image (T2I) generation. Existing methods often su…
FairBatching: Fairness-Aware Batch Formation for LLM Inference
Hongtao Lyu, Boyue Liu, Mingyu Wu +1
Large language model (LLM) inference systems face a fundamental tension between minimizing Time-to-First-Token (TTFT) latency for new requests and maintaining a high, steady token…
SmallThinker: A Family of Efficient Large Language Models Natively Trained for Local Deployment
Yixin Song, Zhenliang Xue, Dongliang Wei +11
While frontier large language models (LLMs) continue to push capability boundaries, their deployment remains confined to GPU-powered cloud infrastructure. We challenge this paradig…
PowerInfer-2: Fast Large Language Model Inference on a Smartphone
Zhenliang Xue, Yixin Song, Zeyu Mi +3
Large language models (LLMs) on smartphones enable real-time AI assistance and privacy-preserving, offline operation. However, resource constraints of smartphones limit current dep…