5 papers · 1 filter
Text Corpora as Concept Fields: Black-Box Hallucination and Novelty Measurement
Nicholas S. Kersting, Vittorio Castelli, Chieh Ting Yeh +3
We introduce the \textbf{Concept Field} of a text corpus: a local drift field with pointwise uncertainty, estimated in sentence-embedding space from the deltas between consecutive…
CiteEval: Principle-Driven Citation Evaluation for Source Attribution
Yumo Xu, Peng Qi, Jifan Chen +7
Citation quality is crucial in information-seeking systems, directly influencing trust and the effectiveness of information access. Current evaluation frameworks, both human and au…
RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering
Rujun Han, Yuhao Zhang, Peng Qi +6
Question answering based on retrieval augmented generation (RAG-QA) is an important research topic in NLP and has a wide range of real-world applications. However, most existing da…
NewsQs: Multi-Source Question Generation for the Inquiring Mind
Alyssa Hwang, Kalpit Dixit, Miguel Ballesteros +5
We present NewsQs (news-cues), a dataset that provides question-answer pairs for multiple news documents. To create NewsQs, we augment a traditional multi-document summarization da…
From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification
Fei Wang, Chao Shang, Sarthak Jain +6
User alignment is crucial for adapting general-purpose language models (LMs) to downstream tasks, but human annotations are often not available for all types of instructions, espec…