1 citations · 1 across the 7 of their papers we have counts for
4 papers · 1 filter
SmallThinker: A Family of Efficient Large Language Models Natively Trained for Local Deployment
Yixin Song, Zhenliang Xue, Dongliang Wei +11
While frontier large language models (LLMs) continue to push capability boundaries, their deployment remains confined to GPU-powered cloud infrastructure. We challenge this paradig…
Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters
Yixin Song, Haotong Xie, Zhengyan Zhang +4
Exploiting activation sparsity is a promising approach to significantly accelerating the inference process of large language models (LLMs) without compromising performance. However…
PowerInfer-2: Fast Large Language Model Inference on a Smartphone
Zhenliang Xue, Yixin Song, Zeyu Mi +3
Large language models (LLMs) on smartphones enable real-time AI assistance and privacy-preserving, offline operation. However, resource constraints of smartphones limit current dep…
PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU
Yixin Song, Zeyu Mi, Haotong Xie +1
This paper introduces PowerInfer, a high-speed Large Language Model (LLM) inference engine on a personal computer (PC) equipped with a single consumer-grade GPU. The key principle…