6 papers
KernelGenBench: A Multi-Source and Multi-Chip Benchmark for LLM-based Kernel Generation
Peiyu Zang, Jian Tao, Jialing Zhang +4
Large language models (LLMs) have significantly increased the demand for efficient accelerator kernels, but kernel development remains a highly specialized and labor-intensive task…
Towards Automated Kernel Generation in the Era of LLMs
Yang Yu, Peiyu Zang, Chi Hsu Tsai +11
The performance of modern AI systems is fundamentally constrained by the quality of their underlying GPU kernels, which translate high-level algorithmic semantics into low-level ha…
HetCCL: Enabling Collective Communication For Mixed-Vendor Heterogeneous Clusters
Yuejie Wang, Tao Chang, Yuanyuan Zhao +10
Training Large Language Models (LLMs) on heterogeneous clusters presents significant challenges for collective communication, as hardware from multiple vendors introduces diverse n…
FreqCache: Accelerating Embodied VLN Models with Adaptive Frequency-Guided Token Caching
Zihao Zheng, Xingyue Zhou, Zhihao Mao +7
Vision-Language-Navigation (VLN) models exhibit excellent navigation accuracy but incur high computational overhead. Token caching has emerged as a promising training-free strategy…
Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Models
Jijie Li, Li Du, Hanyu Zhao +5
Large Language Models (LLMs) demonstrate strong performance in real-world applications, yet existing open-source instruction datasets often concentrate on narrow domains, such as m…
CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large Language Models
Guang Liu, Liangdong Wang, Jijie Li +6
We introduce CCI4.0, a large-scale bilingual pre-training dataset engineered for superior data quality and diverse human-like reasoning trajectory. CCI4.0 occupies roughly TB…