98 citations · 372 across the 23 of their papers we have counts for
27 papers
Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey +4
Large Language Models (LLMs) have demonstrated significant performance improvements across various cognitive tasks. An emerging application is using LLMs to enhance retrieval-augme…
PostMark: A Robust Blackbox Watermark for Large Language Models
Yapei Chang, Kalpesh Krishna, Amir Houmansadr +2
The most effective techniques to detect LLM-generated text rely on inserting a detectable signature -- or watermark -- during the model's decoding process. Most existing watermarki…
GEE! Grammar Error Explanation with Large Language Models
Yixiao Song, Kalpesh Krishna, Rajesh Bhatt +2
Grammatical error correction tools are effective at correcting grammatical errors in users' input sentences but do not provide users with \textit{natural language} explanations abo…
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Sewon Min, Kalpesh Krishna, Xinxi Lyu +6
Evaluating the factuality of long-form text generated by large language models (LMs) is non-trivial because (1) generations often contain a mixture of supported and unsupported pie…
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Kalpesh Krishna, Yixiao Song, Marzena Karpinska +2
The rise in malicious usage of large language models, such as fake content creation and academic plagiarism, has motivated the development of approaches that identify AI-generated…
Stealing the Decoding Algorithms of Language Models
Ali Naseh, Kalpesh Krishna, Mohit Iyyer +1
A key component of generating text from modern language models (LM) is the selection and tuning of decoding algorithms. These algorithms determine how to generate text from the int…
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Kalpesh Krishna, Erin Bransom, Bailey Kuehl +4
While human evaluation remains best practice for accurately judging the faithfulness of automatically-generated summaries, few solutions exist to address the increased difficulty a…
Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature
Katherine Thai, Marzena Karpinska, Kalpesh Krishna +4
Literary translation is a culturally significant task, but it is bottlenecked by the small number of qualified literary translators relative to the many untranslated works publishe…
SLING: Sino Linguistic Evaluation of Large Language Models
Yixiao Song, Kalpesh Krishna, Rajesh Bhatt +1
To understand what kinds of linguistic knowledge are encoded by pretrained Chinese language models (LMs), we introduce the benchmark of Sino LINGuistics (SLING), which consists of…
ezCoref: Towards Unifying Annotation Guidelines for Coreference Resolution
Ankita Gupta, Marzena Karpinska, Wenlong Zhao +5
Large-scale, high-quality corpora are critical for advancing research in coreference resolution. However, existing datasets vary in their definition of coreferences and have been c…
RankGen: Improving Text Generation with Large Ranking Models
Kalpesh Krishna, Yapei Chang, John Wieting +1
Given an input sequence (or prefix), modern language models often assign high probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix; as suc…
RELIC: Retrieving Evidence for Literary Claims
Katherine Thai, Yapei Chang, Kalpesh Krishna +1
Humanities scholars commonly provide evidence for claims that they make about a work of literature (e.g., a novel) in the form of quotations from the work. We collect a large-scale…
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann +122
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained…
Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings
Kalpesh Krishna, Deepak Nathani, Xavier Garcia +2
Style transfer is the task of rewriting a sentence into a target style while approximately preserving content. While most prior literature assumes access to a large style-labelled…
Do Long-Range Language Models Actually Use Long-Range Context?
Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke +1
Language models are generally trained on short, truncated input sequences, which limits their ability to use discourse-level information present in long-range context to improve th…
Weakly-Supervised Open-Retrieval Conversational Question Answering
Chen Qu, Liu Yang, Cen Chen +3
Recent studies on Question Answering (QA) and Conversational QA (ConvQA) emphasize the role of retrieval: a system first retrieves evidence from a large collection and then extract…
Hurdles to Progress in Long-form Question Answering
Kalpesh Krishna, Aurko Roy, Mohit Iyyer
The task of long-form question answering (LFQA) involves retrieving documents relevant to a given question and using them to generate a paragraph-length answer. While many models h…
Long Document Summarization in a Low Resource Setting using Pretrained Language Models
Ahsaas Bajaj, Pavitra Dangati, Kalpesh Krishna +7
Abstractive summarization is the task of compressing a long document into a coherent short document while retaining salient information. Modern abstractive summarization methods ar…
An Analysis of Frame-skipping in Reinforcement Learning
Shivaram Kalyanakrishnan, Siddharth Aravindan, Vishwajeet Bagdawat +5
In the practice of sequential decision making, agents are often designed to sense state at regular intervals of time steps, , ignoring state information in between sensi…
Reformulating Unsupervised Style Transfer as Paraphrase Generation
Kalpesh Krishna, John Wieting, Mohit Iyyer
Modern NLP defines the task of style transfer as modifying the style of a given sentence without appreciably changing its semantics, which implies that the outputs of style transfe…
Thieves on Sesame Street! Model Extraction of BERT-based APIs
Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh +2
We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that…
Generating Question-Answer Hierarchies
Kalpesh Krishna, Mohit Iyyer
The process of knowledge acquisition can be viewed as a question-answer game between a student and a teacher in which the student typically starts by asking broad, open-ended quest…
Syntactically Supervised Transformers for Faster Neural Machine Translation
Nader Akoury, Kalpesh Krishna, Mohit Iyyer
Standard decoders for neural machine translation autoregressively generate a single target token per time step, which slows inference especially for long outputs. While architectur…
Trick or TReAT: Thematic Reinforcement for Artistic Typography
Purva Tendulkar, Kalpesh Krishna, Ramprasaath R. Selvaraju +1
An approach to make text visually appealing and memorable is semantic reinforcement - the use of visual cues alluding to the context or theme in which the word is being used to rei…