5 papers
PipeMax: Enhancing Offline LLM Inference on Commodity GPU Servers
Hongbin Zhang, Taosheng Wei, Jiazhi Jiang +3
Offline LLM inference seeks to maximize request processing under fixed budgets, making commodity GPU servers a promising choice. However, prior work typically considers offloading…
PolyKAN: Efficient Fused GPU Operators for Polynomial Kolmogorov-Arnold Network Variants
Mingkun Yu, Heming Zhong, Dan Huang +2
Kolmogorov-Arnold Networks (KANs) promise higher expressive capability and stronger interpretability than Multi-Layer Perceptron, particularly in the domain of AI for Science. Howe…
STEM: Efficient Relative Capability Evaluation of LLMs through Structured Transition Samples
Haiquan Hu, Jiazhi Jiang, Shiyou Xu +2
Evaluating large language models (LLMs) has become increasingly challenging as model capabilities advance rapidly. While recent models often achieve higher scores on standard bench…
Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism
Jinhui Wei, Ye Huang, Yuhui Zhou +3
In-situ LLM inference on end-user devices has gained significant interest due to its privacy benefits and reduced dependency on external infrastructure. However, as the decoding pr…
SAIH: A Scalable Evaluation Methodology for Understanding AI Performance Trend on HPC Systems
Jiangsu Du, Dongsheng Li, Yingpeng Wen +4
Novel artificial intelligence (AI) technology has expedited various scientific research, e.g., cosmology, physics and bioinformatics, inevitably becoming a significant category of…