158 citations · 593 across the 50 of their papers we have counts for
51 papers
Lexical Repetitions Lead to Rote Learning: Unveiling the Impact of Lexical Overlap in Train and Test Reference Summaries
Prafulla Kumar Choubey, Alexander R. Fabbri, Caiming Xiong +1
Ideal summarization models should generalize to novel summary-worthy content without remembering reference training summaries by rote. However, a single average performance score o…
Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment
Philippe Laban, Lidiya Murakhovs'ka, Caiming Xiong +1
The interactive nature of Large Language Models (LLMs) theoretically allows models to refine and improve their answers, yet systematic analysis of the multi-turn behavior of LLMs r…
Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems
Lidiya Murakhovs'ka, Philippe Laban, Tian Xie +2
Making big purchases requires consumers to research or consult a salesperson to gain domain expertise. However, existing conversational recommender systems (CRS) often overlook use…
Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
Philippe Laban, Jesse Vig, Marti A. Hearst +2
Conversational interfaces powered by Large Language Models (LLMs) have recently become a popular way to obtain feedback during document editing. However, standard chat-based conver…
XGen-7B Technical Report
Erik Nijkamp, Tian Xie, Hiroaki Hayashi +22
Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LL…
Art or Artifice? Large Language Models and the False Promise of Creativity
Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal +2
Researchers have argued that large language models (LLMs) exhibit high-quality writing capabilities from blogs to stories. However, evaluating objectively the creativity of a piece…
Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles
Kung-Hsiang Huang, Philippe Laban, Alexander R. Fabbri +4
Previous research in multi-document news summarization has typically concentrated on collating information that all sources agree upon. However, the summarization of diverse inform…
Next Steps for Human-Centered Generative AI: A Technical Perspective
Xiang 'Anthony' Chen, Jeff Burke, Ruofei Du +8
Through iterative, cross-disciplinary discussions, we define and propose next-steps for Human-centered Generative AI (HGAI). We contribute a comprehensive research agenda that lays…
Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning
Fan Yin, Jesse Vig, Philippe Laban +3
Large language models (LLMs) have shown impressive performance in following natural language instructions to solve unseen tasks. However, it remains unclear whether models truly un…
SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages
Philippe Laban, Jesse Vig, Wojciech Kryscinski +3
Text simplification research has mostly focused on sentence-level simplification, even though many desirable edits - such as adding relevant background information or reordering co…
LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
Philippe Laban, Wojciech Kryściński, Divyansh Agarwal +4
With the recent appearance of LLMs in practical settings, having methods that can effectively detect factual inconsistencies is crucial to reduce the propagation of misinformation…
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation
Yixin Liu, Alexander R. Fabbri, Yilun Zhao +5
Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics. In this work, we develop strong-performing automatic metrics for refe…
Designing and Evaluating Interfaces that Highlight News Coverage Diversity Using Discord Questions
Philippe Laban, Chien-Sheng Wu, Lidiya Murakhovs'ka +2
Modern news aggregators do the hard work of organizing a large news stream, creating collections for a given news story with tens of source options. This paper shows that navigatin…
Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization
Artidoro Pagnoni, Alexander R. Fabbri, Wojciech Kryściński +1
In long document controllable summarization, where labeled data is scarce, pretrained models struggle to adapt to the task and effectively respond to user queries. In this paper, w…
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
Yixin Liu, Alexander R. Fabbri, Pengfei Liu +8
Human evaluation is the foundation upon which the evaluation of both summarization systems and automatic metrics rests. However, existing human evaluation studies for summarization…
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Alexander R. Fabbri, Prafulla Kumar Choubey, Jesse Vig +2
State-of-the-art summarization models still struggle to be factually consistent with the input text. A model-agnostic way to address this problem is post-editing the generated summ…
Discord Questions: A Computational Approach To Diversity Analysis in News Coverage
Philippe Laban, Chien-Sheng Wu, Lidiya Murakhovs'ka +2
There are many potential benefits to news readers accessing diverse sources. Modern news aggregators do the hard work of organizing the news, offering readers a plethora of source…
Conformal Predictor for Improving Zero-shot Text Classification Efficiency
Prafulla Kumar Choubey, Yu Bai, Chien-Sheng Wu +2
Pre-trained language models (PLMs) have been shown effective for zero-shot (0shot) text classification. 0shot models based on natural language inference (NLI) and next sentence pre…
Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning
Xiangyu Peng, Chen Xing, Prafulla Kumar Choubey +2
Prompt tuning approaches, which learn task-specific soft prompts for a downstream task conditioning on frozen pre-trained models, have attracted growing interest due to its paramet…
Marvista: Exploring the Design of a Human-AI Collaborative News Reading Tool
Xiang 'Anthony' Chen, Chien-Sheng Wu, Lidiya Murakhovs'ka +4
We explore the design of Marvista -- a human-AI collaborative tool that employs a suite of natural language processing models to provide end-to-end support for reading online news…
Quiz Design Task: Helping Teachers Create Quizzes with Automated Question Generation
Philippe Laban, Chien-Sheng Wu, Lidiya Murakhovs'ka +2
Question generation (QGen) models are often evaluated with standardized NLG metrics that are based on n-gram overlap. In this paper, we measure whether these metric improvements tr…
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets
Philippe Laban, Chien-Sheng Wu, Wenhao Liu +1
Precisely assessing the progress in natural language generation (NLG) tasks is challenging, and human evaluation to establish a preference in a model's output over another is often…
Structure Extraction in Task-Oriented Dialogues with Slot Clustering
Liang Qiu, Chien-Sheng Wu, Wenhao Liu +1
Extracting structure information from dialogue data can help us better understand user and system behaviors. In task-oriented dialogues, dialogue structure has often been considere…
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Tianbao Xie, Chen Henry Wu, Peng Shi +20
Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases.…