47 citations · 85 across the 19 of their papers we have counts for
20 papers
An Emergent Mirage: Is Emergent Misalignment and Realignment Indeed a Robust Phenomenon?
Abhinav Rao, Liancheng Gong, Bin Hu +1
Recent work has reported Emergent Misalignment (EM), where language models fine-tuned on narrow, domain-specific misaligned datasets abruptly acquire broadly misaligned behavior, a…
Steer, Don't Solve: Training Small Critic Models for Large Code Agents
Shubham Gandhi, Yiqing Xie, Atharva Naik +2
Coding tasks are typically complicated and require multiple capabilities, ranging from high-level planning to low-level implementation. While coding agents are optimized for the jo…
ReaComp: Compiling LLM Reasoning into Symbolic Solvers for Efficient Program Synthesis
Atharva Naik, Yash Mathur, Prakam +2
LLMs can solve program synthesis tasks but remain inefficient and unreliable on hard instances requiring large combinatorial search. Given a small set of reasoning traces, we use c…
ChartEditBench: Evaluating Grounded Multi-Turn Chart Editing in Multimodal Language Models
Manav Nitin Kapadnis, Lawanya Baghel, Atharva Naik +1
While Multimodal Large Language Models (MLLMs) perform strongly on single-turn chart generation, their ability to support real-world exploratory data analysis remains underexplored…
MetaLint: Easy-to-Hard Generalization for Code Linting
Atharva Naik, Lawanya Baghel, Dhakshin Govindarajan +4
Large language models excel at code generation but struggle with code linting, particularly in generalizing to unseen or evolving best practices beyond those observed during traini…
CRScore++: Reinforcement Learning with Verifiable Tool and AI Feedback for Code Review
Manav Nitin Kapadnis, Atharva Naik, Carolyn Rose
Reinforcement learning (RL) to improve code review comment generation requires handling unstructured outputs, making reinforcement learning (RL) feedback challenging. The two main…
An Empirical Study on Strong-Weak Model Collaboration for Repo-level Code Generation
Shubham Gandhi, Atharva Naik, Yiqing Xie +1
We study cost-efficient collaboration between strong and weak language models for repository-level code generation, where the weak model handles simpler tasks at lower cost, and th…
PBEBench: A Multi-Step Programming by Examples Reasoning Benchmark inspired by Historical Linguistics
Atharva Naik, Prakam, Yash Mathur +6
Although many benchmarks evaluate the reasoning abilities of Large Language Models (LLMs) within domains such as mathematics, coding, or data wrangling, few abstract away from doma…
Programming by Examples Meets Historical Linguistics: A Large Language Model Based Approach to Sound Law Induction
Atharva Naik, Darsh Agrawal, Hong Sng +8
Historical linguists have long written "programs" that convert reconstructed words in an ancestor language into their attested descendants via ordered string rewrite functions (cal…
CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells
Atharva Naik, Marcus Alenius, Daniel Fried +1
The task of automated code review has recently gained a lot of attention from the machine learning community. However, current review comment evaluation metrics rely on comparisons…
Can Large Language Models Code Like a Linguist?: A Case Study in Low Resource Sound Law Induction
Atharva Naik, Kexun Zhang, Nathaniel Robinson +7
Historical linguists have long written a kind of incompletely formalized ''program'' that converts reconstructed words in an ancestor language into words in one of its attested des…
Generating Situated Reflection Triggers about Alternative Solution Paths: A Case Study of Generative AI for Computer-Supported Collaborative Learning
Atharva Naik, Jessica Ruhan Yin, Anusha Kamath +6
An advantage of Large Language Models (LLMs) is their contextualization capability - providing different responses based on student inputs like solution strategy or prior discussio…
On the Limitations of Embedding Based Methods for Measuring Functional Correctness for Code Generation
Atharva Naik
The task of code generation from natural language (NL2Code) has become extremely popular, especially with the advent of Large Language Models (LLMs). However, efforts to quantify a…
Data Augmentation for Code Translation with Comparable Corpora and Multiple References
Yiqing Xie, Atharva Naik, Daniel Fried +1
One major challenge of translating code between programming languages is that parallel training data is often limited. To overcome this challenge, we present two data augmentation…
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
Abhinav Rao, Sachin Vashistha, Atharva Naik +2
Recent explorations with commercial Large Language Models (LLMs) have shown that non-expert users can jailbreak LLMs by simply manipulating their prompts; resulting in degenerate o…
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi +37
How well can NLP models generalize to a variety of unseen tasks when provided with task instructions? To address this question, we first introduce Super-NaturalInstructions, a benc…
Representation Learning for Conversational Data using Discourse Mutual Information Maximization
Bishal Santra, Sumegh Roychowdhury, Aishik Mandal +4
Although many pretrained models exist for text or images, there have been relatively fewer attempts to train representations specifically for dialog understanding. Prior works usua…
Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy Logic
Zijun Wu, Zi Xuan Zhang, Atharva Naik +3
Natural language inference (NLI) aims to determine the logical relationship between two sentences, such as Entailment, Contradiction, and Neutral. In recent years, deep learning mo…
Understanding the Role of Affect Dimensions in Detecting Emotions from Tweets: A Multi-task Approach
Rajdeep Mukherjee, Atharva Naik, Sriyash Poddar +2
We propose VADEC, a multi-task framework that exploits the correlation between the categorical and dimensional models of emotion representation for better subjectivity analysis. Fo…
How Have We Reacted To The COVID-19 Pandemic? Analyzing Changing Indian Emotions Through The Lens of Twitter
Rajdeep Mukherjee, Sriyash Poddar, Atharva Naik +1
Since its outbreak, the ongoing COVID-19 pandemic has caused unprecedented losses to human lives and economies around the world. As of 18th July 2020, the World Health Organization…