8 papers
A Unified Definition of Hallucination: It's The World Model, Stupid!
Emmy Liu, Varun Gangal, Chelsea Zou +7
Despite numerous attempts at mitigation since the inception of language models, hallucinations remain a persistent problem even in today's frontier LLMs. Why is this? We review exi…
VFUSE: Virulent Feature Understanding with Sparse autoEncoders
Michael Yu, Matthew L. Olson
Generative models have shown remarkable progress in a variety of domains such as protein design, but such power enables the opaque generation of hazardous proteins. In this work, w…
HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models
Emmy Liu, Varun Gangal, Michael Yu +4
Hallucination remains a central failure mode of large language models, but existing benchmarks operationalize it inconsistently across summarization, question answering, retrieval-…
Leveraging Large Language Models for Sentiment Analysis: Multi-Modal Analysis of Decentraland's MANA Token
Xintong Wu, Peiting Tsai, Jing Yuan +3
Decentraland, a decentralized virtual reality platform operating within the expanding Metaverse ecosystem, utilizes its native MANA token to facilitate virtual asset transactions a…
To Memorize or to Retrieve: Scaling the Interaction Between Pretraining and Retrieval
Karan Singh, Michael Yu, Varun Gangal +4
Retrieval-augmented generation (RAG) improves language model (LM) performance by providing relevant context at test time for knowledge-intensive situations. In this work, we system…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1144
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…