4 papers · 1 filter
SHERLOC: Structured Diagnostic Localization for Code Repair Agents
Hovhannes Tamoyan, Sean Narenthiran, Erik Arakelyan +2
LLM agents solve repository-level coding tasks through multi-turn tool use, but utilize half their budget on locating faults before editing. Dedicated localization frameworks have…
More Yap Less Meaning: Uncovering Self-Improvement Behavior in SLMs
Marina Igitkhanian, Erik Arakelyan
Recently, language models have made rapid progress across various domains and applications. However, their capability for self-improvement, i.e., whether they are adept at recognis…
SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages
Gayane Ghazaryan, Erik Arakelyan, Pasquale Minervini +1
Question Answering (QA) datasets have been instrumental in developing and evaluating Large Language Model (LLM) capabilities. However, such datasets are scarce for languages other…
Semantic Sensitivities and Inconsistent Predictions: Measuring the Fragility of NLI Models
Erik Arakelyan, Zhaoqi Liu, Isabelle Augenstein
Recent studies of the emergent capabilities of transformer-based Natural Language Understanding (NLU) models have indicated that they have an understanding of lexical and compositi…