6 papers
Memorization Diagnostics for Code LLMs Should be Scale-Aware
Prateek Kumar Rajput, Abdoul Aziz Bonkoungou, Alberick Euraste Djiré +6
The extent to which large language models for code rely on memorization over genuine understanding remains highly debated. While current literature frequently reports widespread me…
Empirical Study for Structured Output Control in LLMs for Software Engineering
Yewei Song, Prateek Rajput, Tiezhu Sun +3
LLM-generated outputs in software engineering rarely exist in isolation. They must plug into toolchains, APIs, and data pipelines that impose strict, often organization-specific st…
Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?
Prateek Rajput, Yewei Song, Iyiola E. Olatunji +2
Can large language models reliably express a human-like personality, or are they merely mimicking surface cues without a stable underlying profile? To investigate this, we induce p…
Correctness isnt Efficiency: Runtime Memory Divergence in LLM-Generated Code
Prateek Rajput, Yewei Song, Abdoul Aziz Bonkoungou +4
Large language models (LLMs) can generate programs that pass unit tests, but passing tests does not guarantee reliable runtime behavior. We find that different correct solutions to…
Dynamic Stability of LLM-Generated Code
Prateek Rajput, Abdoul Aziz Bonkoungou, Yewei Song +4
Current evaluations of LLMs for code generation emphasize functional correctness, overlooking the fact that functionally correct solutions can differ significantly in algorithmic c…
Measuring LLM Code Generation Stability via Structural Entropy
Yewei Song, Tiezhu Sun, Xunzhu Tang +3
Assessing the stability of code generation from large language models (LLMs) is essential for judging their reliability in real-world development. We extend prior "structural-entro…