4 papers
From Tokens to Regions: CUDA-Sensitive Instruction Tuning for GPU Kernel Generation
Wentao Chen, Jiace Zhu, Xing Zhe Chai +4
High-performance CUDA kernels are essential for scalable AI systems, while Large Language Models (LLMs) still struggle to generate correct kernels due to strict and implicit execut…
Memorize or Generalize? Evaluating LLM Code Generation with Code Rewriting
Lizhe Zhang, Wentao Chen, Li Zhong +3
Large language models (LLMs) have recently demonstrated exceptional code generation capabilities. However, there is a growing debate whether LLMs are mostly doing memorization (i.e…
CUDA-LLM: LLMs Can Write Efficient CUDA Kernels
Wentao Chen, Jiace Zhu, Qi Fan +2
Large Language Models (LLMs) have demonstrated strong capabilities in general-purpose code generation. However, generating the code which is deeply hardware-specific, architecture-…
The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Model
Jiawei Chen, Wentao Chen, Jing Su +6
Large language models (LLMs) have shown significant multilingual capabilities. However, the mechanisms underlying the development of these capabilities during pre-training are not…