LLM-HPC++: Evaluating LLM-Generated Modern C++ and MPI+OpenMP Codes for Scalable Mandelbrot Set Computation
arXiv:2512.17023 · doi:10.1109/IPDPSW71298.2026.00075
Abstract
Parallel programming remains one of the most challenging aspects of High-Performance Computing (HPC), requiring deep knowledge of synchronization, communication, and memory models. While modern C++ standards and frameworks like OpenMP and MPI have simplified parallelism, mastering these paradigms is still complex. Recently, Large Language Models (LLMs) have shown promise in automating code generation, but their effectiveness in producing correct and efficient HPC code is not well understood. In this work, we systematically evaluate leading LLMs including ChatGPT 4 and 5, Claude, and LLaMA on the task of generating C++ implementations of the Mandelbrot set using shared-memory, directive-based, and distributed-memory paradigms. Each generated program is compiled and executed with GCC 11.5.0 to assess its correctness, robustness, and scalability. Results show that ChatGPT-4 and ChatGPT-5 achieve strong syntactic precision and scalable performance.
References in corpus (10)
- HPC-GPT: Integrating Large Language Model for High-Performance Computing
- HPC-Coder: Modeling Parallel Programs using Large Language Models
- Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation
- LM4HPC: Towards Effective Language Model Application in High-Performance Computing
- OMPGPT: A Generative Pre-trained Transformer Model for OpenMP
- Benchmarking the Parallel 1D Heat Equation Solver in Chapel, Charm++, C++, HPX, Go, Julia, Python, Rust, Swift, and Java
- Shared memory parallelism in Modern C++ and HPX
- LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languages
- LLM & HPC:Benchmarking DeepSeek's Performance in High-Performance Computing Tasks
- LLM-HPC++: Evaluating LLM-Generated Modern C++ and MPI+OpenMP Codes for Scalable Mandelbrot Set Computation