15 citations · 28 across the 22 of their papers we have counts for
22 papers
Memorization Diagnostics for Code LLMs Should be Scale-Aware
Prateek Kumar Rajput, Abdoul Aziz Bonkoungou, Alberick Euraste Djiré +6
The extent to which large language models for code rely on memorization over genuine understanding remains highly debated. While current literature frequently reports widespread me…
Empirical Study for Structured Output Control in LLMs for Software Engineering
Yewei Song, Prateek Rajput, Tiezhu Sun +3
LLM-generated outputs in software engineering rarely exist in isolation. They must plug into toolchains, APIs, and data pipelines that impose strict, often organization-specific st…
The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models
Lujun Li, Lama Sleem, Niccolo Gentile +4
Recent vision-language models (VLMs) excel at multimodal understanding and reasoning, yet their fine-grained visual perception remains underexplored. A natural extension of ``How m…
Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?
Prateek Rajput, Yewei Song, Iyiola E. Olatunji +2
Can large language models reliably express a human-like personality, or are they merely mimicking surface cues without a stable underlying profile? To investigate this, we induce p…
Empirical Evaluation of PDF Parsing and Chunking for Financial Question Answering with RAG
Omar El Bachyr, Yewei Song, Saad Ezzini +5
PDF files are primarily intended for human reading rather than automated processing. In addition, the heterogeneous content of PDFs, such as text, tables, and images, poses signifi…
The Necessity of Setting Temperature in LLM-as-a-Judge
Lujun Li, Lama Sleem, Yangjie Xu +4
Large Language Models (LLMs) are increasingly used as automated judges, yet decoding temperature is often set by intuition rather than evidence. We run over one million large scale…
Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environments
Yangjie Xu, Lujun Li, Lama Sleem +7
Skills are getting increasingly used in recent agentic frameworks, performing robustly with proprietary LLMs. Yet, their effectiveness for small and medium-sized open-source models…
Correctness isnt Efficiency: Runtime Memory Divergence in LLM-Generated Code
Prateek Rajput, Yewei Song, Abdoul Aziz Bonkoungou +4
Large language models (LLMs) can generate programs that pass unit tests, but passing tests does not guarantee reliable runtime behavior. We find that different correct solutions to…
Dynamic Stability of LLM-Generated Code
Prateek Rajput, Abdoul Aziz Bonkoungou, Yewei Song +4
Current evaluations of LLMs for code generation emphasize functional correctness, overlooking the fact that functionally correct solutions can differ significantly in algorithmic c…
Do Large Language Models Grasp The Grammar? Evidence from Grammar-Book-Guided Probing in Luxembourgish
Lujun Li, Yewei Song, Lama Sleem +7
Grammar refers to the system of rules that governs the structural organization and the semantic relations among linguistic units such as sentences, phrases, and words within a give…
MalLoc: Toward Fine-grained Android Malicious Payload Localization via LLMs
Tiezhu Sun, Marco Alecci, Aleksandr Pilgun +5
The rapid evolution of Android malware poses significant challenges to the maintenance and security of mobile applications (apps). Traditional detection techniques often struggle t…
Measuring LLM Code Generation Stability via Structural Entropy
Yewei Song, Tiezhu Sun, Xunzhu Tang +3
Assessing the stability of code generation from large language models (LLMs) is essential for judging their reliability in real-world development. We extend prior "structural-entro…
Is Small Language Model the Silver Bullet to Low-Resource Languages Machine Translation?
Yewei Song, Lujun Li, Cedric Lothritz +6
Low-resource languages (LRLs) lack sufficient linguistic resources and are underrepresented in benchmark datasets, resulting in persistently lower translation quality than high-res…
CallNavi, A Challenge and Empirical Study on LLM Function Calling and Routing
Yewei Song, Xunzhu Tang, Cedric Lothritz +6
API-driven chatbot systems are increasingly integral to software engineering applications, yet their effectiveness hinges on accurately generating and executing API calls. This is…
Large-scale, Independent and Comprehensive study of the power of LLMs for test case generation
Wendkûuni C. Ouédraogo, Kader Kaboré, Yinghua Li +5
Unit testing is essential for software reliability, yet manual test creation is time-consuming and often neglected. Search-based software testing improves efficiency but produces t…
Revisiting Code Similarity Evaluation with Abstract Syntax Tree Edit Distance
Yewei Song, Cedric Lothritz, Daniel Tang +2
This paper revisits recent code similarity evaluation metrics, particularly focusing on the application of Abstract Syntax Tree (AST) editing distance in diverse programming langua…
CodeAgent: Autonomous Communicative Agents for Code Review
Xunzhu Tang, Kisub Kim, Yewei Song +6
Code review, which aims at ensuring the overall quality and reliability of software, is a cornerstone of software development. Unfortunately, while crucial, Code review is a labor-…
Enhancing Text-to-SQL Translation for Financial System Design
Yewei Song, Saad Ezzini, Xunzhu Tang +6
Text-to-SQL, the task of translating natural language questions into SQL queries, is part of various business processes. Its automation, which is an emerging challenge, will empowe…
Just-in-Time Detection of Silent Security Patches
Xunzhu Tang, Zhenghan Chen, Kisub Kim +3
Open-source code is pervasive. In this setting, embedded vulnerabilities are spreading to downstream software at an alarming rate. While such vulnerabilities are generally identifi…
Hyperbolic Code Retrieval: A Novel Approach for Efficient Code Search Using Hyperbolic Space Embeddings
Xunzhu Tang, zhenghan Chen, Saad Ezzini +4
Within the realm of advanced code retrieval, existing methods have primarily relied on intricate matching and attention-based mechanisms. However, these methods often lead to compu…
Multilevel Semantic Embedding of Software Patches: A Fine-to-Coarse Grained Approach Towards Security Patch Detection
Xunzhu Tang, zhenghan Chen, Saad Ezzini +4
The growth of open-source software has increased the risk of hidden vulnerabilities that can affect downstream software applications. This concern is further exacerbated by softwar…
Letz Translate: Low-Resource Machine Translation for Luxembourgish
Yewei Song, Saad Ezzini, Jacques Klein +3
Natural language processing of Low-Resource Languages (LRL) is often challenged by the lack of data. Therefore, achieving accurate machine translation (MT) in a low-resource enviro…