Mutation-based Consistency Testing for Evaluating the Code Understanding Capability of LLMs

Ziyu Li, Donghwan Shin

Large Language Models (LLMs) have shown remarkable capabilities in processing both natural and programming languages, which have enabled various applications in software engineerin…