15 citations · 28 across the 22 of their papers we have counts for
8 papers · 1 filter
Memorization Diagnostics for Code LLMs Should be Scale-Aware
Prateek Kumar Rajput, Abdoul Aziz Bonkoungou, Alberick Euraste Djiré +6
The extent to which large language models for code rely on memorization over genuine understanding remains highly debated. While current literature frequently reports widespread me…
Empirical Study for Structured Output Control in LLMs for Software Engineering
Yewei Song, Prateek Rajput, Tiezhu Sun +3
LLM-generated outputs in software engineering rarely exist in isolation. They must plug into toolchains, APIs, and data pipelines that impose strict, often organization-specific st…
The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
Lujun Li, Lama Sleem, Niccolo Gentile +4
Recent vision-language models (VLMs) excel at multimodal understanding and reasoning, yet their fine-grained visual perception remains underexplored. A natural extension of ``How m…
Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?
Prateek Rajput, Yewei Song, Iyiola E. Olatunji +2
Can large language models reliably express a human-like personality, or are they merely mimicking surface cues without a stable underlying profile? To investigate this, we induce p…
Empirical Evaluation of PDF Parsing and Chunking for Financial Question Answering with RAG
Omar El Bachyr, Yewei Song, Saad Ezzini +5
PDF files are primarily intended for human reading rather than automated processing. In addition, the heterogeneous content of PDFs, such as text, tables, and images, poses signifi…
The Necessity of Setting Temperature in LLM-as-a-Judge
Lujun Li, Lama Sleem, Yangjie Xu +4
Large Language Models (LLMs) are increasingly used as automated judges, yet decoding temperature is often set by intuition rather than evidence. We run over one million large scale…
Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environments
Yangjie Xu, Lujun Li, Lama Sleem +7
Skills are getting increasingly used in recent agentic frameworks, performing robustly with proprietary LLMs. Yet, their effectiveness for small and medium-sized open-source models…
Correctness isnt Efficiency: Runtime Memory Divergence in LLM-Generated Code
Prateek Rajput, Yewei Song, Abdoul Aziz Bonkoungou +4
Large language models (LLMs) can generate programs that pass unit tests, but passing tests does not guarantee reliable runtime behavior. We find that different correct solutions to…