565 citations · 730 across the 94 of their papers we have counts for
23 papers · 1 filter
The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness
Neeraj Varshney, Pavel Dolin, Agastya Seth +1
As Large Language Models (LLMs) play an increasingly pivotal role in natural language processing applications, their safety concerns become critical areas of NLP research. This pap…
ECLIPSE: A Resource-Efficient Text-to-Image Prior for Image Generations
Maitreya Patel, Changhoon Kim, Sheng Cheng +2
Text-to-image (T2I) diffusion models, notably the unCLIP models (e.g., DALL-E-2), achieve state-of-the-art (SOTA) performance on various compositional T2I benchmarks, at the cost o…
LongBoX: Evaluating Transformers on Long-Sequence Clinical Tasks
Mihir Parmar, Aakanksha Naik, Himanshu Gupta +2
Many large language models (LLMs) for medicine have largely been evaluated on short texts, and their ability to handle longer sequences such as a complete electronic health record…
Accelerating LLaMA Inference by Enabling Intermediate Layer Decoding via Instruction Tuning with LITE
Neeraj Varshney, Agneet Chatterjee, Mihir Parmar +1
Large Language Models (LLMs) have achieved remarkable performance across a wide variety of natural language tasks; however, their large size makes their inference slow and computat…
InstructExcel: A Benchmark for Natural Language Instruction in Excel
Justin Payan, Swaroop Mishra, Mukul Singh +7
With the evolution of Large Language Models (LLMs) we can solve increasingly more complex NLP tasks across various domains, including spreadsheets. This work investigates whether L…
TarGEN: Targeted Data Generation with Large Language Models
Himanshu Gupta, Kevin Scaria, Ujjwala Anantheswaran +5
The rapid advancement of large language models (LLMs) has sparked interest in data synthesis techniques, aiming to generate diverse and high-quality synthetic datasets. However, th…
Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models
Man Luo, Shrinidhi Kumbhar, Ming shen +5
Logical reasoning is fundamental for humans yet presents a substantial challenge in the domain of Artificial Intelligence. Initially, researchers used Knowledge Representation and…
Can NLP Models 'Identify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer?
Ayushi Agarwal, Nisarg Patel, Neeraj Varshney +7
Though state-of-the-art (SOTA) NLP systems have achieved remarkable performance on a variety of language understanding tasks, they primarily focus on questions that have a correct…
MDDial: A Multi-turn Differential Diagnosis Dialogue Dataset with Reliability Evaluation
Srija Macherla, Man Luo, Mihir Parmar +1
Dialogue systems for Automatic Differential Diagnosis (ADD) have a wide range of real-life applications. These dialogue systems are promising for providing easy access and reducing…
ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models
Maitreya Patel, Tejas Gokhale, Chitta Baral +1
The ability to understand visual concepts and replicate and compose these concepts from images is a central goal for computer vision. Recent advances in text-to-image (T2I) models…
End-to-end Knowledge Retrieval with Multi-modal Queries
Man Luo, Zhiyuan Fang, Tejas Gokhale +2
We investigate knowledge retrieval with multi-modal queries, i.e. queries containing information split across image and text inputs, a challenging task that differs from previous w…
EDM3: Event Detection as Multi-task Text Generation
Ujjwala Anantheswaran, Himanshu Gupta, Mihir Parmar +2
Event detection refers to identifying event occurrences in a text and comprises of two subtasks; event identification and classification. We present EDM3, a novel approach for Even…
Dr.ICL: Demonstration-Retrieved In-context Learning
Man Luo, Xin Xu, Zhuyun Dai +5
In-context learning (ICL), teaching a large language model (LLM) to perform a task with few-shot demonstrations rather than adjusting the model parameters, has emerged as a strong…
Can NLP Models Correctly Reason Over Contexts that Break the Common Assumptions?
Neeraj Varshney, Mihir Parmar, Nisarg Patel +4
Pre-training on large corpora of text enables the language models to acquire a vast amount of factual and commonsense knowledge which allows them to achieve remarkable performance…
Instruction Tuned Models are Quick Learners
Himanshu Gupta, Saurabh Arjun Sawant, Swaroop Mishra +4
Instruction tuning of language models has demonstrated the ability to enhance model generalization to unseen tasks via in-context learning using a few examples. However, typical su…
A Unified Evaluation Framework for Novelty Detection and Accommodation in NLP with an Instantiation in Authorship Attribution
Neeraj Varshney, Himanshu Gupta, Eric Robertson +2
State-of-the-art natural language processing models have been shown to achieve remarkable performance in 'closed-world' settings where all the labels in the evaluation set are know…
Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA
Neeraj Varshney, Chitta Baral
Despite remarkable progress made in natural language processing, even the state-of-the-art models often make incorrect predictions. Such predictions hamper the reliability of syste…
Methods and Mechanisms for Interactive Novelty Handling in Adversarial Environments
Tung Thai, Ming Shen, Mayank Garg +10
Learning to detect, characterize and accommodate novelties is a challenge that agents operating in open-world domains need to address to be able to guarantee satisfactory task perf…
Prompt-Based Learning for Thread Structure Prediction in Cybersecurity Forums
Kazuaki Kashihara, Kuntal Kumar Pal, Chitta Baral +1
With recent trends indicating cyber crimes increasing in both frequency and cost, it is imperative to develop new methods that leverage data-rich hacker forums to assist in combati…
Exploring the Limits of Transfer Learning with Unified Model in the Cybersecurity Domain
Kuntal Kumar Pal, Kazuaki Kashihara, Ujjwala Anantheswaran +3
With the increase in cybersecurity vulnerabilities of software systems, the ways to exploit them are also increasing. Besides these, malware threats, irregular network interactions…
Real-Time Visual Feedback to Guide Benchmark Creation: A Human-and-Metric-in-the-Loop Workflow
Anjana Arunkumar, Swaroop Mishra, Bhavdeep Sachdeva +2
Recent research has shown that language models exploit `artifacts' in benchmarks to solve tasks, rather than truly learning them, leading to inflated model performance. In pursuit…
InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis
Kevin Scaria, Himanshu Gupta, Siddharth Goyal +3
We introduce InstructABSA, an instruction learning paradigm for Aspect-Based Sentiment Analysis (ABSA) subtasks. Our method introduces positive, negative, and neutral examples to e…
Lexi: Self-Supervised Learning of the UI Language
Pratyay Banerjee, Shweti Mahajan, Kushal Arora +2
Humans can learn to operate the user interface (UI) of an application by reading an instruction manual or how-to guide. Along with text, these resources include visual content such…