7 citations · 19 across the 19 of their papers we have counts for
20 papers
ARE: Scaling Up Agent Environments and Evaluations
Romain Froger, Pierre Andrews, Matteo Bettini +21
We introduce Meta Agents Research Environments (ARE), a research platform for scalable creation of environments, integration of synthetic or real applications, and execution of age…
When is the consistent prediction likely to be a correct prediction?
Alex Nguyen, Dheeraj Mekala, Chengyu Dong +1
Self-consistency (Wang et al., 2023) suggests that the most consistent answer obtained through large language models (LLMs) is more likely to be correct. In this paper, we challeng…
DOCMASTER: A Unified Platform for Annotation, Training, & Inference in Document Question-Answering
Alex Nguyen, Zilong Wang, Jingbo Shang +1
The application of natural language processing models to PDF documents is pivotal for various business applications yet the challenge of training models for this purpose persists i…
TOOLVERIFIER: Generalization to New Tools via Self-Verification
Dheeraj Mekala, Jason Weston, Jack Lanchantin +4
Teaching language models to use tools is an important milestone towards building general assistants, but remains an open problem. While there has been significant progress on learn…
Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models
Dheeraj Mekala, Alex Nguyen, Jingbo Shang
Instruction-tuning language models has become a crucial step in aligning them for general use. Typically, this process involves extensive training on large datasets, incurring high…
MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimization
Yasaman Jafari, Dheeraj Mekala, Rose Yu +1
RL-based techniques can be employed to search for prompts that, when fed into a target language model, maximize a set of user-specified reward functions. However, in many target ap…
DAIL: Data Augmentation for In-Context Learning via Self-Paraphrase
Dawei Li, Yaxuan Li, Dheeraj Mekala +5
In-Context Learning (ICL) combined with pre-trained large language models has achieved promising results on various NLP tasks. However, ICL requires high-quality annotated demonstr…
SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank
Dheeraj Mekala, Adithya Samavedhi, Chengyu Dong +1
Deep neural classifiers trained with cross-entropy loss (CE loss) often suffer from poor calibration, necessitating the task of out-of-distribution (OOD) detection. Traditional sup…
A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approaches
Zihan Wang, Tianle Wang, Dheeraj Mekala +1
Etremely Weakly Supervised Text Classification (XWS-TC) refers to text classification based on minimal high-level human guidance, such as a few label-indicative seed words or class…
ZEROTOP: Zero-Shot Task-Oriented Semantic Parsing using Large Language Models
Dheeraj Mekala, Jason Wolfe, Subhro Roy
We explore the use of large language models (LLMs) for zero-shot semantic parsing. Semantic parsing involves mapping natural language utterances to task-specific meaning representa…
Progressive Sentiment Analysis for Code-Switched Text Data
Sudhanshu Ranjan, Dheeraj Mekala, Jingbo Shang
Multilingual transformer language models have recently attracted much attention from researchers and are used in cross-lingual transfer learning for many NLP tasks such as text cla…
Leveraging QA Datasets to Improve Generative Data Augmentation
Dheeraj Mekala, Tu Vu, Timo Schick +1
The ability of generative language models (GLMs) to generate text has improved considerably in the last few years, enabling their use for generative data augmentation. In this work…
LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification
Dheeraj Mekala, Chengyu Dong, Jingbo Shang
Weakly supervised text classification methods typically train a deep neural classifier based on pseudo-labels. The quality of pseudo-labels is crucial to final performance but they…
Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data
Dheeraj Mekala, Varun Gangal, Jingbo Shang
Existing text classification methods mainly focus on a fixed label set, whereas many real-world applications require extending to new fine-grained classes as the number of samples…
BFClass: A Backdoor-free Text Classification Framework
Zichao Li, Dheeraj Mekala, Chengyu Dong +1
Backdoor attack introduces artificial vulnerabilities into the model by poisoning a subset of the training data via injecting triggers and modifying labels. Various trigger design…
News Meets Microblog: Hashtag Annotation via Retriever-Generator
Xiuwen Zheng, Dheeraj Mekala, Amarnath Gupta +1
Hashtag annotation for microblog posts has been recently formulated as a sequence generation problem to handle emerging hashtags that are unseen in the training set. The state-of-t…
X-Class: Text Classification with Extremely Weak Supervision
Zihan Wang, Dheeraj Mekala, Jingbo Shang
In this paper, we explore text classification with extremely weak supervision, i.e., only relying on the surface text of class names. This is a more challenging setting than the se…
Bayes-optimal Hierarchical Classification over Asymmetric Tree-Distance Loss
Dheeraj Mekala, Vivek Gupta, Purushottam Kar +1
Hierarchical classification is supervised multi-class classification problem over the set of class labels organized according to a hierarchy. In this report, we study the work by R…
User Bias Removal in Review Score Prediction
Rahul Wadbude, Vivek Gupta, Dheeraj Mekala +1
Review score prediction of text reviews has recently gained a lot of attention in recommendation systems. A major problem in models for review score prediction is the presence of n…
SCDV : Sparse Composite Document Vectors using soft clustering over distributional representations
Dheeraj Mekala, Vivek Gupta, Bhargavi Paranjape +1
We present a feature vector formation technique for documents - Sparse Composite Document Vector (SCDV) - which overcomes several shortcomings of the current distributional paragra…