activity
20202026
most citedBeyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

565 citations · 730 across the 94 of their papers we have counts for

collaborators
Showing 2021Show all

12 papers · 1 filter

cs.CL2021

Unsupervised Natural Language Inference Using PHL Triplet Generation

Neeraj Varshney, Pratyay Banerjee, Tejas Gokhale +1

Transformer-based models achieve impressive performance on numerous Natural Language Inference (NLI) benchmarks when trained on respective training datasets. However, in certain ca…

cs.CV2021

Semantically Distributed Robust Optimization for Vision-and-Language Inference

Tejas Gokhale, Abhishek Chaudhary, Pratyay Banerjee +2

Analysis of vision-and-language models has revealed their brittleness under linguistic phenomena such as paraphrasing, negation, textual entailment, and word substitutions with syn…

cs.CL2021

Investigating Numeracy Learning Ability of a Text-to-Text Transfer Model

Kuntal Kumar Pal, Chitta Baral

The transformer-based pre-trained language models have been tremendously successful in most of the conventional NLP tasks. But they often struggle in those tasks where numerical un…

cs.CL2021

Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering

Man Luo, Yankai Zeng, Pratyay Banerjee +1

Knowledge-based visual question answering (VQA) requires answering questions with external knowledge in addition to the content of images. One dataset that is mostly used in evalua…

cs.CV2021★ 1 cited

Weakly Supervised Relative Spatial Reasoning for Visual Question Answering

Pratyay Banerjee, Tejas Gokhale, Yezhou Yang +1

Vision-and-language (V\&L) reasoning necessitates perception of visual concepts such as objects and actions, understanding semantics and language grounding, and reasoning about the…

cs.CL2021

A Simple Approach to Jointly Rank Passages and Select Relevant Sentences in the OBQA Context

Man Luo, Shuguang Chen, Chitta Baral

In the open book question answering (OBQA) task, selecting the relevant passages and sentences from distracting information is crucial to reason the answer to a question. HotpotQA…

cs.CL2021★ 3 cited

Reframing Instructional Prompts to GPTk's Language

Swaroop Mishra, Daniel Khashabi, Chitta Baral +2

What kinds of instructional prompts are easier to follow for Language Models (LMs)? We study this question by conducting extensive empirical analysis that shed light on important f…

cs.CL2021

Interviewer-Candidate Role Play: Towards Developing Real-World NLP Systems

Neeraj Varshney, Swaroop Mishra, Chitta Baral

Standard NLP tasks do not incorporate several common real-world scenarios such as seeking clarifications about the question, taking advantage of clues, abstaining in order to avoid…

cs.CL2021

Constructing Flow Graphs from Procedural Cybersecurity Texts

Kuntal Kumar Pal, Kazuaki Kashihara, Pratyay Banerjee +3

Following procedural texts written in natural languages is challenging. We must read the whole text to identify the relevant information or identify the instruction flows to comple…

cs.CL2021

Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction

Ming Shen, Pratyay Banerjee, Chitta Baral

In this work, we propose Masked Noun-Phrase Prediction (MNPP), a pre-training strategy to tackle pronoun resolution in a fully unsupervised setting. Firstly, We evaluate our pre-tr…

cs.CV2021

CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images

Shailaja Keyur Sampat, Akshay Kumar, Yezhou Yang +1

Most existing research on visual question answering (VQA) is limited to information explicitly present in an image or a video. In this paper, we take visual understanding to a high…

cs.CL2021★ 5 cited

Cross-Task Generalization via Natural Language Crowdsourcing Instructions

Swaroop Mishra, Daniel Khashabi, Chitta Baral +1

Humans (e.g., crowdworkers) have a remarkable ability in solving different tasks, by simply reading textual instructions that define them and looking at a few examples. Despite the…