318 citations · 442 across the 22 of their papers we have counts for
5 papers · 1 filter
Beyond a Single Extractor: Re-thinking HTML-to-Text Extraction for LLM Pretraining
Jeffrey Li, Josh Gardner, Doug Kang +10
One of the first pre-processing steps for constructing web-scale LLM pretraining datasets involves extracting text from HTML. Despite the immense diversity of web content, existing…
Learning from Self Critique and Refinement for Faithful LLM Summarization
Ting-Yao Hu, Hema Swetha Koppula, Hadi Pouransari +3
Large Language Models (LLMs) often suffer from hallucinations: output content that is not grounded in the input context, when performing long-form text generation tasks such as sum…
Learning to Reason for Hallucination Span Detection
Hsuan Su, Ting-Yao Hu, Hema Swetha Koppula +7
Large language models (LLMs) often generate hallucinations -- unsupported content that undermines reliability. While most prior works frame hallucination detection as a binary task…
Pretraining with hierarchical memories: separating long-tail and common knowledge
Hadi Pouransari, David Grangier, C Thomas +2
The impressive performance gains of modern language models currently rely on scaling parameters: larger models store more world knowledge and reason better. Yet compressing all wor…
Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarization
Yen-Ju Lu, Ting-Yao Hu, Hema Swetha Koppula +8
In this work, we propose Mutual Reinforcing Data Synthesis (MRDS) within LLMs to improve few-shot dialogue summarization task. Unlike prior methods that require external knowledge,…