MUCOCO: Automated Consistency Testing of Code LLMs

Chua Jin Chou, Khant That Lwin, Ezekiel Soremekun

Code LLMs often portray inconsistent program behaviors. Developers typically employ benchmarks to assess Code LLMs, but most benchmarks are hand-crafted, static and do not target c…