132 citations · 270 across the 24 of their papers we have counts for
25 papers
The Amazing Agent Race: Strong Tool Users, Weak Navigators
Zae Myung Kim, Dongseok Lee, Jaehyung Kim +2
Existing tool-use benchmarks for LLM agents are overwhelmingly linear: our analysis of six benchmarks shows 55 to 100% of instances are simple chains of 2 to 5 steps. We introduce…
BAID: A Benchmark for Bias Assessment of AI Detectors
Priyam Basu, Yunfeng Zhang, Vipul Raheja
AI-generated text detectors have recently gained adoption in educational and professional contexts. Prior research has uncovered isolated cases of bias, particularly against Englis…
Are LLM Agents Behaviorally Coherent? A Latent Profiling Approach for Social Simulation
James Mooney, Josef Woldense, Zheng Robert Jia +4
LLM agents are increasingly proposed as substitutes for human participants in social research, usually on the strength of survey responses that match those of the humans they repre…
APIO: Automatic Prompt Induction and Optimization for Grammatical Error Correction and Text Simplification
Artem Chernodub, Aman Saini, Yejin Huh +2
Recent advancements in large language models (LLMs) have enabled a wide range of natural language processing (NLP) tasks to be performed through simple prompt-based interactions. C…
Toward Evaluative Thinking: Meta Policy Optimization with Evolving Reward Models
Zae Myung Kim, Chanwoo Park, Vipul Raheja +2
Reward-based alignment methods for large language models (LLMs) face two key limitations: vulnerability to reward hacking, where models exploit flaws in the reward signal; and reli…
Learning Explainable Dense Reward Shapes via Bayesian Optimization
Ryan Koo, Ian Yang, Vipul Raheja +3
Current reinforcement learning from human feedback (RLHF) pipelines for large language model (LLM) alignment typically assign scalar rewards to sequences, using the final token as…
Spivavtor: An Instruction Tuned Ukrainian Text Editing Model
Aman Saini, Artem Chernodub, Vipul Raheja +1
We introduce Spivavtor, a dataset, and instruction-tuned models for text editing focused on the Ukrainian language. Spivavtor is the Ukrainian-focused adaptation of the English-onl…
A Design Space for Intelligent and Interactive Writing Assistants
Mina Lee, Katy Ilonka Gero, John Joon Young Chung +33
In our era of rapid technological advancement, the research landscape for writing assistants has become increasingly fragmented across various research communities. We seek to addr…
Personalized Text Generation with Fine-Grained Linguistic Control
Bashar Alhafni, Vivek Kulkarni, Dhruv Kumar +1
As the text generation capabilities of large language models become increasingly prominent, recent studies have focused on controlling particular aspects of the generated text to m…
mEdIT: Multilingual Text Editing via Instruction Tuning
Vipul Raheja, Dimitris Alikaniotis, Vivek Kulkarni +2
We introduce mEdIT, a multi-lingual extension to CoEdIT -- the recent state-of-the-art text editing models for writing assistance. mEdIT models are trained by fine-tuning multi-lin…
Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs
Zae Myung Kim, Kwang Hee Lee, Preston Zhu +2
With the advent of large language models (LLM), the line between human-crafted and machine-generated texts has become increasingly blurred. This paper delves into the inquiry of id…
ContraDoc: Understanding Self-Contradictions in Documents with Large Language Models
Jierui Li, Vipul Raheja, Dhruv Kumar
In recent times, large language models (LLMs) have shown impressive performance on various document-level tasks such as document classification, summarization, and question-answeri…
Speakerly: A Voice-based Writing Assistant for Text Composition
Dhruv Kumar, Vipul Raheja, Alice Kaiser-Schatzlein +5
We present Speakerly, a new real-time voice-based writing assistance system that helps users with text composition across various use cases such as emails, instant messages, and no…
Benchmarking Cognitive Biases in Large Language Models as Evaluators
Ryan Koo, Minhwa Lee, Vipul Raheja +3
Large Language Models are cognitively biased judges. Large Language Models (LLMs) have recently been shown to be effective as automatic evaluators with simple prompting and in-cont…
Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymization
Oleksandr Yermilov, Vipul Raheja, Artem Chernodub
This work investigates the effectiveness of different pseudonymization techniques, ranging from rule-based substitutions to using pre-trained Large Language Models (LLMs), on a var…
CoEdIT: Text Editing by Task-Specific Instruction Tuning
Vipul Raheja, Dhruv Kumar, Ryan Koo +1
We introduce CoEdIT, a state-of-the-art text editing system for writing assistance. CoEdIT takes instructions from the user specifying the attributes of the desired text, such as "…
Writing Assistants Should Model Social Factors of Language
Vivek Kulkarni, Vipul Raheja
Intelligent writing assistants powered by large language models (LLMs) are more popular today than ever before, but their further widespread adoption is precluded by sub-optimal pe…
Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks
Zae Myung Kim, Wanyu Du, Vipul Raheja +2
Iterative text revision improves text quality by fixing grammatical errors, rephrasing for better readability or contextual appropriateness, or reorganizing sentence structures thr…
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran +74
Evaluation in machine learning is usually informed by past choices, for example which datasets or metrics to use. This standardization enables the comparison on equal footing using…
Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revision
Wanyu Du, Zae Myung Kim, Vipul Raheja +2
Revision is an essential part of the human writing process. It tends to be strategic, adaptive, and, more importantly, iterative in nature. Despite the success of large language mo…
Understanding Iterative Revision from Human-Written Text
Wanyu Du, Vipul Raheja, Dhruv Kumar +3
Writing is, by nature, a strategic, adaptive, and more importantly, an iterative process. A crucial part of writing is editing and revising the text. Previous works on text revisio…
Text Simplification by Tagging
Kostiantyn Omelianchuk, Vipul Raheja, Oleksandr Skurzhanskyi
Edit-based approaches have recently shown promising results on multiple monolingual sequence transduction tasks. In contrast to conventional sequence-to-sequence (Seq2Seq) models,…
Adversarial Grammatical Error Correction
Vipul Raheja, Dimitrios Alikaniotis
Recent works in Grammatical Error Correction (GEC) have leveraged the progress in Neural Machine Translation (NMT), to learn rewrites from parallel corpora of grammatically incorre…
The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction
Dimitrios Alikaniotis, Vipul Raheja
Recent work on Grammatical Error Correction (GEC) has highlighted the importance of language modeling in that it is certainly possible to achieve good performance by comparing the…