7 papers
A Unified Definition of Hallucination: It's The World Model, Stupid!
Emmy Liu, Varun Gangal, Chelsea Zou +7
Despite numerous attempts at mitigation since the inception of language models, hallucinations remain a persistent problem even in today's frontier LLMs. Why is this? We review exi…
Defenses & Enablers For Skill Injection Attacks on Terminal Based Agents
Yoshinari Fujinuma, Varun Gangal, Traian Rebedea +4
Large language model (LLM) agents increasingly rely on reusable skills i.e. documents describing task-specific procedures. However, this introduces a new attack surface for agents…
HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models
Emmy Liu, Varun Gangal, Michael Yu +4
Hallucination remains a central failure mode of large language models, but existing benchmarks operationalize it inconsistently across summarization, question answering, retrieval-…
To Memorize or to Retrieve: Scaling the Interaction Between Pretraining and Retrieval
Karan Singh, Michael Yu, Varun Gangal +4
Retrieval-augmented generation (RAG) improves language model (LM) performance by providing relevant context at test time for knowledge-intensive situations. In this work, we system…
MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments
Darshan Deshpande, Varun Gangal, Hersh Mehta +3
Recent works on context and memory benchmarking have primarily focused on conversational instances but the need for evaluating memory in dynamic enterprise environments is crucial…
Sparse Rewards Can Self-Train Dialogue Agents
Barrett Martin Lattimer, Varun Gangal, Ryan McDonald +1
Recent advancements in state-of-the-art (SOTA) Large Language Model (LLM) agents, especially in multi-turn dialogue tasks, have been primarily driven by supervised fine-tuning and…