27 citations · 161 across the 39 of their papers we have counts for
40 papers
Flipping the Dialogue: Training and Evaluating User Language Models
Tarek Naous, Philippe Laban, Wei Xu +1
Conversations with LMs involve two participants: a human user leading the conversation, and an LM assistant responding to the user's request. To satisfy this specific role, LMs are…
Attribution Gradients: Incrementally Unfolding Citations for Critical Examination of Attributed AI Answers
Hita Kambhamettu, Alyssa Hwang, Philippe Laban +1
AI answer engines are a relatively new kind of information search tool: rather than returning a ranked list of documents, they generate an answer to a search question with inline c…
DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence
Pranav Narayanan Venkit, Philippe Laban, Yilun Zhou +3
Generative search engines and deep research LLM agents promise trustworthy, source-grounded synthesis, yet users regularly encounter overconfidence, weak sourcing, and confusing ci…
LLMs Get Lost In Multi-Turn Conversation
Philippe Laban, Hiroaki Hayashi, Yingbo Zhou +1
Large Language Models (LLMs) are conversational interfaces. As such, LLMs have the potential to assist their users not only when they can fully specify the task at hand, but also t…
AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computation
Tuhin Chakrabarty, Philippe Laban, Chien-Sheng Wu
AI-generated text is proliferating across domains, from creative writing and journalism to marketing content and scientific articles. Models can follow user-provided instructions t…
BingoGuard: LLM Content Moderation Tools with Risk Levels
Fan Yin, Philippe Laban, Xiangyu Peng +7
Malicious content generated by large language models (LLMs) can pose varying degrees of harm. Although existing LLM-based moderators can detect harmful content, they struggle to as…
Why Vision Language Models Struggle with Visual Arithmetic? Towards Enhanced Chart and Geometry Understanding
Kung-Hsiang Huang, Can Qin, Haoyi Qiu +4
Vision Language Models (VLMs) have achieved remarkable progress in multimodal tasks, yet they often struggle with visual arithmetic, seemingly simple capabilities like object count…
SummExecEdit: A Factual Consistency Benchmark in Summarization with Executable Edits
Onkar Thorat, Philippe Laban, Chien-Sheng Wu
Detecting factual inconsistencies in summarization is critical, yet existing benchmarks lack the necessary challenge and interpretability for robust evaluation. In this paper, we i…
CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments
Kung-Hsiang Huang, Akshara Prabhakar, Sidharth Dhawan +6
Customer Relationship Management (CRM) systems are vital for modern enterprises, providing a foundation for managing customer interactions and data. Integrating AI agents into CRM…
Do RAG Systems Cover What Matters? Evaluating and Optimizing Responses with Sub-Question Coverage
Kaige Xie, Philippe Laban, Prafulla Kumar Choubey +2
Evaluating retrieval-augmented generation (RAG) systems remains challenging, particularly for open-ended questions that lack definitive answers and require coverage of multiple sub…
Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses
Pranav Narayanan Venkit, Philippe Laban, Yilun Zhou +2
Large Language Model (LLM)-based applications are graduating from research prototypes to products serving millions of users, influencing how people write and consume information. A…
Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits
Tuhin Chakrabarty, Philippe Laban, Chien-Sheng Wu
LLM-based applications are helping people write, and LLM-generated text is making its way into social media, journalism, and our classrooms. However, the differences between LLM-ge…
Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems
Philippe Laban, Alexander R. Fabbri, Caiming Xiong +1
LLMs and RAG systems are now capable of handling millions of input tokens or more. However, evaluating the output quality of such systems on long-context tasks remains challenging,…
Prompt Leakage effect and defense strategies for multi-turn LLM interactions
Divyansh Agarwal, Alexander R. Fabbri, Ben Risher +3
Prompt leakage poses a compelling security and privacy threat in LLM applications. Leakage of system prompts may compromise intellectual property, and act as adversarial reconnaiss…
MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents
Liyan Tang, Philippe Laban, Greg Durrett
Recognizing if LLM output can be grounded in evidence is central to many tasks in NLP: retrieval-augmented generation, summarization, document-grounded dialogue, and more. Current…
Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment
Philippe Laban, Lidiya Murakhovs'ka, Caiming Xiong +1
The interactive nature of Large Language Models (LLMs) theoretically allows models to refine and improve their answers, yet systematic analysis of the multi-turn behavior of LLMs r…
Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems
Lidiya Murakhovs'ka, Philippe Laban, Tian Xie +2
Making big purchases requires consumers to research or consult a salesperson to gain domain expertise. However, existing conversational recommender systems (CRS) often overlook use…
Automatic and Human-AI Interactive Text Generation
Yao Dou, Philippe Laban, Claire Gardent +1
In this tutorial, we focus on text-to-text generation, a class of natural language generation (NLG) tasks, that takes a piece of text as input and then generates a revision that is…
Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
Philippe Laban, Jesse Vig, Marti A. Hearst +2
Conversational interfaces powered by Large Language Models (LLMs) have recently become a popular way to obtain feedback during document editing. However, standard chat-based conver…
XGen-7B Technical Report
Erik Nijkamp, Tian Xie, Hiroaki Hayashi +22
Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LL…
Art or Artifice? Large Language Models and the False Promise of Creativity
Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal +2
Researchers have argued that large language models (LLMs) exhibit high-quality writing capabilities from blogs to stories. However, evaluating objectively the creativity of a piece…
Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles
Kung-Hsiang Huang, Philippe Laban, Alexander R. Fabbri +4
Previous research in multi-document news summarization has typically concentrated on collating information that all sources agree upon. However, the summarization of diverse inform…
Next Steps for Human-Centered Generative AI: A Technical Perspective
Xiang 'Anthony' Chen, Jeff Burke, Ruofei Du +8
Through iterative, cross-disciplinary discussions, we define and propose next-steps for Human-centered Generative AI (HGAI). We contribute a comprehensive research agenda that lays…
Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning
Fan Yin, Jesse Vig, Philippe Laban +3
Large language models (LLMs) have shown impressive performance in following natural language instructions to solve unseen tasks. However, it remains unclear whether models truly un…