Publications (56)
Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negation
Neeraj Varshney, Satyam Raj, Venkatesh Mishra +4
Large Language Models (LLMs) have achieved remarkable performance across a wide variety of natural language tasks. However, they have been shown to suffer from a critical limitatio…
Unlocking Latent Value: Taxonomy-Guided Recovery of High-Performing Data from Low-Tier Web Corpora
Neeraj Varshney, Sanket Lokegaonkar, Nasser Zalmout +3
Dominant web data curation pipelines for pretraining collapse document quality into a single composite score, systematically missing high-value content along dimensions the scorer…
NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks
Swaroop Mishra, Arindam Mitra, Neeraj Varshney +4
Given the ubiquitous nature of numbers in text, reasoning with numbers to perform simple calculations is an important skill of AI systems. While many datasets and models have been…
Hybrid Transceiver Design for Tera-Hertz MIMO Systems Relying on Bayesian Learning Aided Sparse Channel Estimation
Suraj Srivastava, Ajeet Tripathi, Neeraj Varshney +2
Hybrid transceiver design in multiple-input multiple-output (MIMO) Tera-Hertz (THz) systems relying on sparse channel state information (CSI) estimation techniques is conceived. To…
On Flow-Induced Diffusive Mobile Molecular Communication: First Hitting Time and Performance Analysis
Neeraj Varshney, Werner Haselmayr, Weisi Guo
This work considers the problem of flow-induced diffusive molecular communication under various mobility conditions such as (i) both transmitter (TX) and receiver (RX) nanomachines…
Can NLP Models Correctly Reason Over Contexts that Break the Common Assumptions?
Neeraj Varshney, Mihir Parmar, Nisarg Patel +4
Pre-training on large corpora of text enables the language models to acquire a vast amount of factual and commonsense knowledge which allows them to achieve remarkable performance…
A Unified Evaluation Framework for Novelty Detection and Accommodation in NLP with an Instantiation in Authorship Attribution
Neeraj Varshney, Himanshu Gupta, Eric Robertson +2
State-of-the-art natural language processing models have been shown to achieve remarkable performance in 'closed-world' settings where all the labels in the evaluation set are know…
Beamformed Energy Detection in the Presence of an Interferer for Cognitive mmWave Network
Madhuri Latha Mannedu, Sai Krishna Charan Dara, Sachin Chaudhari +1
In this paper, we propose beamformed energy detection (BFED) spectrum sensing schemes for a single secondary user (SU) or a cognitive radio to detect a primary user (PU) transmissi…
Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models
Man Luo, Shrinidhi Kumbhar, Ming shen +5
Logical reasoning is fundamental for humans yet presents a substantial challenge in the domain of Artificial Intelligence. Initially, researchers used Knowledge Representation and…
The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness
Neeraj Varshney, Pavel Dolin, Agastya Seth +1
As Large Language Models (LLMs) play an increasingly pivotal role in natural language processing applications, their safety concerns become critical areas of NLP research. This pap…
On the Impact of Transposition Errors in Diffusion-Based Channels
Werner Haselmayr, Neeraj Varshney, A. Taufiq Asyhari +2
In this work, we consider diffusion-based molecular communication with and without drift between two static nano-machines. We employ type-based information encoding, releasing a si…
Impact of Multiple Fully-Absorbing Receivers in Molecular Communications
Nithin V. Sabu, Abhishek K. Gupta, Neeraj Varshney +1
Molecular communication is a promising solution to enable intra-body communications among nanomachines. However, malicious and non-cooperative receivers can degrade the performance…
Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Models
Nisarg Patel, Mohith Kulkarni, Mihir Parmar +4
As Large Language Models (LLMs) continue to exhibit remarkable performance in natural language understanding tasks, there is a crucial need to measure their ability for human-like…
Let the Model Decide its Curriculum for Multitask Learning
Neeraj Varshney, Swaroop Mishra, Chitta Baral
Curriculum learning strategies in prior multi-task learning approaches arrange datasets in a difficulty hierarchy either based on human perception or by exhaustively searching the…
Simplified Ray Tracing for the Millimeter Wave Channel: A Performance Evaluation
Mattia Lecci, Paolo Testolina, Marco Giordani +6
Millimeter-wave (mmWave) communication is one of the cornerstone innovations of fifth-generation (5G) wireless networks, thanks to the massive bandwidth available in these frequenc…
Towards Question Format Independent Numerical Reasoning: A Set of Prerequisite Tasks
Swaroop Mishra, Arindam Mitra, Neeraj Varshney +2
Numerical reasoning is often important to accurately understand the world. Recently, several format-specific datasets have been proposed, such as numerical reasoning in the setting…
Opportunistic Scheduling in Underlay Cognitive Radio based MIMO-RF/FSO Networks
Neeraj Varshney, Prabhat K. Sharma, Mohamed-Slim Alouini
This work proposes an optimal metric for opportunistic scheduling of secondary user transmitters (SU-TXs) in underlay cognitive radio based multiple-input multiple-output radio fre…
Opportunistic Scheduling in Underlay Cognitive Radio based Systems: User Selection Probability Analysis
Neeraj Varshney, Prabhat K. Sharma, Mohamed Slim Alouini
In this paper, an underlay cognitive radio (CR) system is considered with multiple cognitive or secondary users contending to transmit their information to the cognitive destinatio…
Diffusion Based Cooperative Molecular Communication in Nano-Networks
Neeraj Varshney, Adarsh Patel, Aditya K. Jagannatham
This work presents a novel diffusion based dual-phase molecular communication system where the source leverages multiple cooperating nanomachines to improve the end-to-end reliabil…
Multi-TRP Assisted UAV Detection in 3GPP 5G-Advanced ISAC Network
Neeraj Varshney, Steve Blandino, Jian Wang +3
ISAC is currently being standardized within the 3GPP New Radio (NR) to enable cellular infrastructure to perform sensing using existing communication waveforms. While standardizati…
On Hybrid MoSK-CSK Modulation based Molecular Communication: Error Rate Performance Analysis using Stochastic Geometry
Nithin V. Sabu, Neeraj Varshney, Abhishek K. Gupta
Data transmission rate in molecular communication systems can be improved by using multiple transmitters and receivers. In molecular multiple-input multiple-output (MIMO) systems w…
Interviewer-Candidate Role Play: Towards Developing Real-World NLP Systems
Neeraj Varshney, Swaroop Mishra, Chitta Baral
Standard NLP tasks do not incorporate several common real-world scenarios such as seeking clarifications about the question, taking advantage of clues, abstaining in order to avoid…
Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning
Pratik Jayarao, Chaitanya Dwivedi, Himanshu Gupta +5
The performance gap across languages in LLMs is well documented, and closing it natively requires pretraining or fine-tuning on corpora that, for most languages, are quite limited.…
Evaluation of gNB Monostatic Sensing for UAV Use Case
Steve Blandino, Neeraj Varshney, Jian Wang +3
3GPP Release 19 has initiated the standardization of integrated sensing and communications (ISAC), including a channel model for monostatic sensing, evaluation scenarios, and perfo…
Unsupervised Natural Language Inference Using PHL Triplet Generation
Neeraj Varshney, Pratyay Banerjee, Tejas Gokhale +1
Transformer-based models achieve impressive performance on numerous Natural Language Inference (NLI) benchmarks when trained on respective training datasets. However, in certain ca…
Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustness
Pratik Jayarao, Himanshu Gupta, Neeraj Varshney +1
As Large Language Models (LLMs) are increasingly adopted as automated judges in benchmarking and reward modeling, ensuring their reliability, efficiency, and robustness has become…
Can NLP Models 'Identify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer?
Ayushi Agarwal, Nisarg Patel, Neeraj Varshney +7
Though state-of-the-art (SOTA) NLP systems have achieved remarkable performance on a variety of language understanding tasks, they primarily focus on questions that have a correct…
Efficient Transmission Scheme for LEO Satellite-Based NB-IoT: A Data-Driven Perspective
Ayush Kumar Dwivedi, Houcine Chougrani, Sachin Chaudhari +2
This study analyses the medium access control (MAC) layer aspects of a low-Earth-orbit (LEO) satellite-based Internet of Things (IoT) network. A transmission scheme based on change…
LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
Mihir Parmar, Nisarg Patel, Neeraj Varshney +5
Recently developed large language models (LLMs) have been shown to perform remarkably well on a wide range of language understanding tasks. But, can they really "reason" over the n…
Impact of Cooperation in Flow-Induced Diffusive Mobile Molecular Communication
Neeraj Varshney, Adarsh Patel, Werner Haselmayr +3
Motivated by the numerous healthcare applications of molecular communication (MC) inside blood vessels, this work considers relay/cooperative nanomachine (CN)-assisted mobile MC be…
Performance Analysis of LEO Satellite-Based IoT Networks in the Presence of Interference
Ayush Kumar Dwivedi, Sachin Chaudhari, Neeraj Varshney +1
This paper presents a star-of-star topology for internet-of-things (IoT) networks using mega low-Earth-orbit constellations. The proposed topology enables IoT users to broadcast th…
Channel Characterization and Performance of a 3-D Molecular Communication System with Multiple Fully-Absorbing Receivers
Nithin V. Sabu, Abhishek K. Gupta, Neeraj Varshney +1
Molecular communication (MC) can enable the transfer of information between nanomachines using molecules as the information carrier. In MC systems, multiple receiver nanomachines o…
Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings
Neeraj Varshney, Swaroop Mishra, Chitta Baral
In order to equip NLP systems with selective prediction capability, several task-specific approaches have been proposed. However, which approaches work best across tasks or even if…
On the Performance of the Primary and Secondary Links in a 3-D Underlay Cognitive Molecular Communication
Nithin V. Sabu, Neeraj Varshney, Abhishek K. Gupta
Molecular communication often involves coexisting links where certain links may have priority over others. In this work, we consider a system in three-dimensional (3-D) space with…
Diffusive Molecular Communication with Nanomachine Mobility
Neeraj Varshney, Aditya K. Jagannatham, Pramod K. Varshney
This work presents a performance analysis for diffusive molecular communication with mobile transmit and receive nanomachines. To begin with, the optimal test is obtained for symbo…
A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation
Neeraj Varshney, Wenlin Yao, Hongming Zhang +2
Recently developed large language models have achieved remarkable success in generating fluent and coherent text. However, these models often tend to 'hallucinate' which critically…
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi +37
How well can NLP models generalize to a variety of unseen tasks when provided with task instructions? To address this question, we first introduce Super-NaturalInstructions, a benc…
Cognitive MIMO-RF/FSO Cooperative Relay Communication with Mobile Nodes and Imperfect Channel State Information
Neeraj Varshney, Aditya K. Jagannatham, Pramod K. Varshney
This work analyzes the performance of an underlay cognitive radio based decode-and-forward mixed multiple-input multiple-output (MIMO) radio frequency/free space optical (RF/FSO) c…
Towards Improving Selective Prediction Ability of NLP Systems
Neeraj Varshney, Swaroop Mishra, Chitta Baral
It's better to say "I can't answer" than to answer incorrectly. This selective prediction ability is crucial for NLP systems to be reliably deployed in real-world applications. Pri…
Deep Learning-based Human Gesture Channel Modeling for Integrated Sensing and Communication Scenarios
Zhengyu Zhang, Neeraj Varshney, Jelena Senic +6
With the development of Integrated Sensing and Communication (ISAC) for Sixth-Generation (6G) wireless systems, contactless human recognition has emerged as one of the key applicat…
ILDAE: Instance-Level Difficulty Analysis of Evaluation Data
Neeraj Varshney, Swaroop Mishra, Chitta Baral
Knowledge of questions' difficulty level helps a teacher in several ways, such as estimating students' potential quickly by asking carefully selected questions and improving qualit…
Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
Chaitanya Dwivedi, Binxuan Huang, Himanshu Gupta +3
Mixture-of-Experts (MoE) has become the dominant architecture for scaling large language models: frontier models routinely decouple total parameters from per-token computation thro…
Performance Analysis of Novel Direct Access Schemes for LEO Satellites Based IoT Network
Ayush Kumar Dwivedi, Sai Praneeth Chokkarapu, Sachin Chaudhari +1
This paper analyzes the performance of low earth orbit (LEO) satellites based internet-of-things (IoT) network where each IoT node makes use of multiple satellites to communicate w…
Design and Performance Analysis of Dual and Multi-hop Diffusive Molecular Communication Systems
Neeraj Varshney, Adarsh Patel, Aditya K. Jagannatham +1
This work presents a comprehensive performance analysis of diffusion based direct, dual-hop, and multi-hop molecular communication systems with Brownian motion and drift in the pre…
Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA
Neeraj Varshney, Chitta Baral
Despite remarkable progress made in natural language processing, even the state-of-the-art models often make incorrect predictions. Such predictions hamper the reliability of syste…
Methods and Mechanisms for Interactive Novelty Handling in Adversarial Environments
Tung Thai, Ming Shen, Mayank Garg +10
Learning to detect, characterize and accommodate novelties is a challenge that agents operating in open-world domains need to address to be able to guarantee satisfactory task perf…
Toward Honest Language Models for Deductive Reasoning
Jiarui Liu, Kaustubh Dhole, Yingheng Wang +7
Deductive reasoning is the process of deriving conclusions strictly from the given premises, without relying on external knowledge. We define honesty in this setting as a model's a…
Code Mixologist : A Practitioner's Guide to Building Code-Mixed LLMs
Himanshu Gupta, Pratik Jayarao, Chaitanya Dwivedi +1
Code-mixing and code-switching (CSW) remain challenging phenomena for large language models (LLMs). Despite recent advances in multilingual modeling, LLMs often struggle in mixed-l…
Abnormality Detection inside Blood Vessels with Mobile Nanomachines
Neeraj Varshney, Adarsh Patel, Yansha Deng +3
Motivated by the numerous healthcare applications of molecular communication within Internet of Bio-Nano Things (IoBNT), this work addresses the problem of abnormality detection in…
Can Transformers Reason About Effects of Actions?
Pratyay Banerjee, Chitta Baral, Man Luo +4
A recent work has shown that transformers are able to "reason" with facts and rules in a limited setting where the rules are natural language expressions of conjunctions of conditi…
Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systems
Neeraj Varshney, Chitta Baral
Do all instances need inference through the big models for a correct prediction? Perhaps not; some instances are easy and can be answered correctly by even small capacity models. T…
"John is 50 years old, can his son be 65?" Evaluating NLP Models' Understanding of Feasibility
Himanshu Gupta, Neeraj Varshney, Swaroop Mishra +5
In current NLP research, large-scale language models and their abilities are widely being discussed. Some recent works have also found notable failures of these models. Often these…
Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans?
Neeraj Varshney, Man Luo, Chitta Baral
Recent state-of-the-art open-domain QA models are typically based on a two stage retriever-reader approach in which the retriever first finds the relevant knowledge/passages and th…
3-D Diffusive Molecular Communication with Two Fully-Absorbing Receivers: Hitting Probability and Performance Analysis
Nithin V. Sabu, Neeraj Varshney, Abhishek K. Gupta
Exact analytical channel models for molecular communication via diffusion (MCvD) systems involving multiple fully absorbing receivers (FARs) in a three-dimensional (3- D) medium ar…
Chaos with Keywords: Exposing Large Language Models Sycophantic Hallucination to Misleading Keywords and Evaluating Defense Strategies
Aswin RRV, Nemika Tyagi, Md Nayem Uddin +2
This study explores the sycophantic tendencies of Large Language Models (LLMs), where these models tend to provide answers that match what users want to hear, even if they are not…
Accelerating LLaMA Inference by Enabling Intermediate Layer Decoding via Instruction Tuning with LITE
Neeraj Varshney, Agneet Chatterjee, Mihir Parmar +1
Large Language Models (LLMs) have achieved remarkable performance across a wide variety of natural language tasks; however, their large size makes their inference slow and computat…