Publications (47)
Validating a virtual human and automated feedback system for training doctor-patient communication skills
Kurtis Haut, Caleb Wohn, Benjamin Kane +6
Effective communication between a clinician and their patient is critical for delivering healthcare maximizing outcomes. Unfortunately, traditional communication training approache…
Visible Emission Line Coronagraph (VELC) onboard ADITYA-L1
Jagdev Singh, R. Ramesh, B. Raghavendra Prasad +21
Aditya-L1, India's first dedicated mission to study the Sun and its atmosphere from the Sun-Earth Lagrangian L1 location was successfully launched on 2023 September 2. It carries s…
CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance
Myeongsoo Kim, Shweta Garg, Baishakhi Ray +2
Programming assistants powered by large language models have improved dramatically, yet existing benchmarks still evaluate them in narrow code-generation settings. Recent efforts s…
Real-Time Prediction of Gas Flow Dynamics in Diesel Engines using a Deep Neural Operator Framework
Varun Kumar, Somdatta Goswami, Daniel J. Smith +1
We develop a data-driven deep neural operator framework to approximate multiple output states for a diesel engine and generate real-time predictions with reasonable accuracy. As em…
SEER: Sustainable E-commerce with Environmental-impact Rating
Md Saiful Islam, Adiba Mahbub, Caleb Wohn +5
With online shopping gaining massive popularity over the past few years, e-commerce platforms can play a significant role in tackling climate change and other environmental problem…
Training Language Model Agents to Find Vulnerabilities with CTF-Dojo
Terry Yue Zhuo, Dingmin Wang, Hantian Ding +2
Large language models (LLMs) have demonstrated exceptional capabilities when trained within executable runtime environments, notably excelling at software engineering tasks through…
Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning
Scott Cyphers, Arjun K. Bansal, Anahita Bhiwandiwalla +18
The Deep Learning (DL) community sees many novel topologies published each year. Achieving high performance on each new topology remains challenging, as each requires some level of…
Efficient Semi-Supervised Learning for Natural Language Understanding by Optimizing Diversity
Eunah Cho, He Xie, John P. Lalor +2
Expanding new functionalities efficiently is an ongoing challenge for single-turn task-oriented dialogue systems. In this work, we explore functionality-specific semi-supervised le…
A Closer Look At Feature Space Data Augmentation For Few-Shot Intent Classification
Varun Kumar, Hadrien Glaude, Cyprien de Lichy +1
New conversation topics and functionalities are constantly being added to conversational AI agents like Amazon Alexa and Apple Siri. As data collection and annotation is not scalab…
Cyber-Zero: Training Cybersecurity Agents without Runtime
Terry Yue Zhuo, Dingmin Wang, Hantian Ding +2
Large Language Models (LLMs) have achieved remarkable success in software engineering tasks when trained with executable runtime environments, particularly in resolving GitHub issu…
Learning Nonlinear Responses in PET Bottle Buckling with a Hybrid DeepONet-Transolver Framework
Varun Kumar, Jing Bi, Cyril Ngo Ngoc +2
Neural surrogates and operator networks for solving partial differential equation (PDE) problems have attracted significant research interest in recent years. However, most existin…
Multi-lingual Evaluation of Code Generation Models
Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang +22
We present new benchmarks on evaluation code generation models: MBXP and Multilingual HumanEval, and MathQA-X. These datasets cover over 10 programming languages and are generated…
Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code
Myeongsoo Kim, Dingmin Wang, Siwei Cui +6
Code agents resolve 65-70% of SWE-bench Verified issues, but Pass@1 cannot tell us why the rest fail, and, as we show, capable-model failures are systematically misdiagnosed withou…
Spectropolarimeter on-board the Aditya-L1: Polarization Modulation and Demodulation
K. Nagaraju, B. Raghavendra Prasad, Bhavana S. Hegde +5
One of the major science goals of the Visible Emission Line Coronagraph (VELC) payload aboard the Aditya-L1 mission is to map the coronal magnetic field topology and the quantitati…
A Digital Twin for Diesel Engines: Operator-infused Physics-Informed Neural Networks with Transfer Learning for Engine Health Monitoring
Kamaljyoti Nath, Varun Kumar, Daniel J. Smith +1
Improving diesel engine efficiency, reducing emissions, and enabling robust health monitoring have been critical research topics in engine modelling. While recent advancements in t…
Empowering Multi-Turn Tool-Integrated Agentic Reasoning with Group Turn Policy Optimization
Yifeng Ding, Hung Le, Songyang Han +5
Training Large Language Models (LLMs) for multi-turn Tool-Integrated Reasoning (TIR) - where models iteratively reason, generate code, and verify through execution - remains challe…
Agentic Risk-Aware Set-Based Engineering Design
Varun Kumar, George Em Karniadakis
This paper introduces a multi-agent framework guided by Large Language Models (LLMs) to assist in the early stages of engineering design, a phase often characterized by vast parame…
CODESTRUCT: Code Agents over Structured Action Spaces
Myeongsoo Kim, Joe Hsu, Dingmin Wang +3
LLM-based code agents treat repositories as unstructured text, applying edits through brittle string matching that frequently fails due to formatting drift or ambiguous patterns. W…
Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal
Umang Gupta, Jwala Dhamala, Varun Kumar +7
Language models excel at generating coherent text, and model compression techniques such as knowledge distillation have enabled their use in resource-constrained settings. However,…
CodeScout: Contextual Problem Statement Enhancement for Software Agents
Manan Suri, Xiangci Li, Mehdi Shojaie +5
Current AI-powered code assistance tools often struggle with poorly-defined problem statements that lack sufficient task context and requirements specification. Recent analysis of…
Data processing of Visible Emission Line Coronagraph Onboard ADITYA L1
Muthu Priyal, Jagdev Singh, B. Raghavendra Prasad +28
ADITYA-L1 is India's first dedicated mission to observe the sun and its atmosphere from a halo orbit around L1 point. Visible emission line coronagraph (VELC) is the prime payload…
Learning Code Preference via Synthetic Evolution
Jiawei Liu, Thanh Nguyen, Mingyue Shang +5
Large Language Models (LLMs) have recently demonstrated remarkable coding capabilities. However, assessing code generation based on well-formed properties and aligning it with deve…
Is the Elephant Flying? Resolving Ambiguities in Text-to-Image Generative Models
Ninareh Mehrabi, Palash Goyal, Apurv Verma +7
Natural language often contains ambiguities that can lead to misinterpretation and miscommunication. While humans can handle ambiguities effectively by asking clarifying questions…
An Analysis of the Effects of Decoding Algorithms on Fairness in Open-Ended Language Generation
Jwala Dhamala, Varun Kumar, Rahul Gupta +2
Several prior works have shown that language models (LMs) can generate text containing harmful social biases and stereotypes. While decoding algorithms play a central role in deter…
Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies
Junlin Wang, Siddhartha Jain, Dejiao Zhang +3
A diverse array of reasoning strategies has been proposed to elicit the capabilities of large language models. However, in this paper, we point out that traditional evaluations whi…
ReCode: Robustness Evaluation of Code Generation Models
Shiqi Wang, Zheng Li, Haifeng Qian +11
Code generation models have achieved impressive performance. However, they tend to be brittle as slight edits to a prompt could lead to very different generations; these robustness…
BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation
Jwala Dhamala, Tony Sun, Varun Kumar +4
Recent advances in deep learning techniques have enabled machines to generate cohesive open-ended text when prompted with a sequence of words as context. While these models now emp…
Why Didn't You Listen to Me? Comparing User Control of Human-in-the-Loop Topic Models
Varun Kumar, Alison Smith-Renner, Leah Findlater +2
To address the lack of comparative evaluation of Human-in-the-Loop Topic Modeling (HLTM) systems, we implement and evaluate three contrasting HLTM modeling approaches using simulat…
Data Augmentation using Pre-trained Transformer Models
Varun Kumar, Ashutosh Choudhary, Eunah Cho
Language model based pre-trained models such as BERT have provided significant gains across different NLP tasks. In this paper, we study different types of transformer based pre-tr…
LeDex: Training LLMs to Better Self-Debug and Explain Code
Nan Jiang, Xiaopeng Li, Shiqi Wang +6
In the domain of code generation, self-debugging is crucial. It allows LLMs to refine their generated code based on execution feedback. This is particularly important because gener…
On Mitigating Code LLM Hallucinations with API Documentation
Nihal Jain, Robert Kwiatkowski, Baishakhi Ray +2
In this study, we address the issue of API hallucinations in various software engineering contexts. We introduce CloudAPIBench, a new benchmark designed to measure API hallucinatio…
The Amazon Nova Family of Models: Technical Report and Model Card
Amazon AGI, Aaron Langford, Aayush Shah +783
We present Amazon Nova, a new generation of state-of-the-art foundation models that deliver frontier intelligence and industry-leading price performance. Amazon Nova Pro is a highl…
Synergistic Learning with Multi-Task DeepONet for Efficient PDE Problem Solving
Varun Kumar, Somdatta Goswami, Katiana Kontolati +2
Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-ta…
Fusion-DeepONet: A Data-Efficient Neural Operator for Geometry-Dependent Hypersonic and Supersonic Flows
Ahmad Peyvan, Varun Kumar, George Em Karniadakis
Shape optimization is essential in aerospace vehicle design, including reentry systems, and propulsion system components, as it directly influences aerodynamic efficiency, structur…
Data Pipeline Architecture and Development for VELC onboard Space Solar Mission AdityaL1
Jagdev Singh, B. Raghavendra Prasad, Chavali Sumana +4
ADITYA L-1 is India's first dedicated mission to study Sun and its atmosphere with Visible Emission Line Coronagraph (VELC), a major payload on ADITYA-L1. VELC has provision to mak…
Greener yet Powerful: Taming Large Code Generation Models with Quantization
Xiaokai Wei, Sujan Gonugondla, Wasi Ahmad +13
ML-powered code generation aims to assist developers to write code in a more productive manner, by intelligently generating code blocks based on natural language prompts. Recently,…
Toward Autonomous Engineering Design: A Knowledge-Guided Multi-Agent Framework
Varun Kumar, George Em Karniadakis
The engineering design process often demands expertise from multiple domains, leading to complex collaborations and iterative refinements. Traditional methods can be resource-inten…
On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations
Yang Trista Cao, Yada Pruksachatkun, Kai-Wei Chang +4
Multiple metrics have been introduced to measure fairness in various natural language processing tasks. These metrics can be roughly categorized into two categories: 1) \emph{extri…
MyCrunchGPT: A chatGPT assisted framework for scientific machine learning
Varun Kumar, Leonard Gleyzer, Adar Kahana +2
Scientific Machine Learning (SciML) has advanced recently across many different areas in computational science and engineering. The objective is to integrate data and physics seaml…
Industry Scale Semi-Supervised Learning for Natural Language Understanding
Luoxin Chen, Francisco Garcia, Varun Kumar +2
This paper presents a production Semi-Supervised Learning (SSL) pipeline based on the student-teacher framework, which leverages millions of unlabeled examples to improve Natural L…
SpecAgent: A Speculative Retrieval and Forecasting Agent for Code Completion
George Ma, Anurag Koul, Qi Chen +6
Large Language Models (LLMs) excel at code-related tasks but often struggle in realistic software repositories, where project-specific APIs and cross-file dependencies are crucial.…
Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks
Shoumik Saha, Jifan Chen, Sam Mayers +3
Code-capable large language model (LLM) agents are embedded in software engineering workflows where they can read, write, and execute code, raising "jailbreak" stakes beyond text-o…
SpaceNet MVOI: a Multi-View Overhead Imagery Dataset
Nicholas Weir, David Lindenbaum, Alexei Bastidas +5
Detection and segmentation of objects in overheard imagery is a challenging task. The variable density, random orientation, small size, and instance-to-instance heterogeneity of ob…
Planning-Aware Code Infilling via Horizon-Length Prediction
Yifeng Ding, Hantian Ding, Shiqi Wang +3
Fill-in-the-Middle (FIM), or infilling, has become integral to code language models, enabling generation of missing code given both left and right contexts. However, the current FI…
ProtoDA: Efficient Transfer Learning for Few-Shot Intent Classification
Manoj Kumar, Varun Kumar, Hadrien Glaude +3
Practical sequence classification tasks in natural language processing often suffer from low training data availability for target classes. Recent works towards mitigating this pro…
A Static Evaluation of Code Completion by Large Language Models
Hantian Ding, Varun Kumar, Yuchen Tian +9
Large language models trained on code have shown great potential to increase productivity of software developers. Several execution-based benchmarks have been proposed to evaluate…
Fewer Truncations Improve Language Modeling
Hantian Ding, Zijian Wang, Giovanni Paolini +4
In large language model training, input documents are typically concatenated together and then split into sequences of equal length to avoid padding tokens. Despite its efficiency,…