11 papers
Exploration-Preserving Policy Optimization
Hangzhan jin, Mohammad Hamdaqa, Doina Precup
Reinforcement learning with verifiable rewards improves reasoning, while the allocation of learning signal shapes which solutions remain accessible under repeated sampling. Group-r…
The Hidden Environmental Cost of Poor Coding Practices in TensorFlow and Keras Applications: A Study on Resource Leaks and Carbon Emissions
Bashar Abdallah, Gustavo Santos, Rola Al Bataineh +2
Efficiency and sustainability are critical considerations in the development and deployment of machine learning (ML) applications. Among the factors influencing sustainability, res…
Rotation-Preserving Supervised Fine-Tuning
Hangzhan Jin, Tianwei Ni, Lu Li +3
Supervised fine-tuning (SFT) improves in-domain performance but can degrade out-of-domain (OOD) generalization. Prior work suggests that this degradation is related to changes in d…
PLMGH: What Matters in PLM-GNN Hybrids for Code Classification and Vulnerability Detection
Mohamed Taoufik Kaouthar El Idrissi, Edward Zulkoski, Mohammad Hamdaqa
Code understanding models increasingly rely on pretrained language models (PLMs) and graph neural networks (GNNs), which capture complementary semantic and structural information.…
Mining the YARA Ecosystem: From Ad-Hoc Sharing to Data-Driven Threat Intelligence
Dectot--Le Monnier de Gouville Esteban, Mohammad Hamdaqa, Moataz Chouchen
YARA has established itself as the de facto standard for "Detection as Code," enabling analysts and DevSecOps practitioners to define signatures for malware identification across t…
From Code Smells to Best Practices: Tackling Resource Leaks in PyTorch, TensorFlow, and Keras
Bashar Abdallah, Martyna E. Wojciechowska, Gustavo Santos +4
Much of the existing ML research focuses on model performance metrics, leaving limited attention to the long-term sustainability and resource efficiency of ML applications. While h…
RL Fine-Tuning Heals OOD Forgetting in SFT
Hangzhan Jin, Sitao Luan, Tianwei Ni +5
Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) is a standard post-training recipe for improving Large Language Models (LLM) reasoning, but why it works remain…
RL Is Neither a Panacea Nor a Mirage: Understanding Supervised vs. Reinforcement Learning Fine-Tuning for LLMs
Hangzhan Jin, Sicheng Lv, Sifan Wu +1
Training large language models (LLMs) from scratch is increasingly impractical, making post-training methods such as supervised fine-tuning (SFT) and reinforcement-learning fine-tu…
CCCI: Code Completion with Contextual Information for Complex Data Transfer Tasks Using Large Language Models
Hangzhan Jin, Mohammad Hamdaqa
Unlike code generation, which involves creating code from scratch, code completion focuses on integrating new lines or blocks of code into an existing codebase. This process requir…
Smells-sus: Sustainability Smells in IaC
Seif Kosbar, Mohammad Hamdaqa
Practitioners use Infrastructure as Code (IaC) scripts to efficiently configure IT infrastructures through machine-readable definition files. However, during the development of the…
Preventing Out-of-Gas Exceptions by Typing
Luca Aceto, Daniele Gorla, Stian Lybech +1
We continue the development of TinySol, a minimal object-oriented language based on Solidity, the standard smart-contract language used for the Ethereum platform. We first extend T…