5 papers
FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG
Maxime Dassen, Rebecca Kotula, Kenton Murray +5
Retrieval-Augmented Generation (RAG) models are critically undermined by citation hallucinations, a deceptive failure where a model cites a source that fails to support its claim.…
DOTResize: Reducing LLM Width via Discrete Optimal Transport-based Neuron Merging
Neha Verma, Kenton Murray, Kevin Duh
Structured pruning methods designed for Large Language Models (LLMs) generally focus on identifying and removing the least important components to optimize model size. However, in…
Query Decomposition for RAG: Balancing Exploration-Exploitation
Roxana Petcu, Kenton Murray, Daniel Khashabi +4
Retrieval-augmented generation (RAG) systems address complex user requests by decomposing them into subqueries, retrieving potentially relevant documents for each, and then aggrega…
Whisper-UT: A Unified Translation Framework for Speech and Text
Cihan Xiao, Matthew Wiesner, Debashish Chakraborty +7
Encoder-decoder models have achieved remarkable success in speech and text tasks, yet efficiently adapting these models to diverse uni/multi-modal scenarios remains an open challen…
Merging Feed-Forward Sublayers for Compressed Transformers
Neha Verma, Kenton Murray, Kevin Duh
With the rise and ubiquity of larger deep learning models, the need for high-quality compression techniques is growing in order to deploy these models widely. The sheer parameter c…