4 papers
Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmetic
Yuval Reif, Guy Kaplan, Roy Schwartz
Large language models (LLMs) often encode word-form variation (e.g., walk vs. walked) as linear directions in the embedding space. However, standard tokenization algorithms treat s…
Why Fine-Tuning Encourages Hallucinations and How to Fix It
Guy Kaplan, Zorik Gekhman, Zhen Zhu +5
Large language models are prone to hallucinating factually incorrect statements. A key source of these errors is exposure to new factual information through supervised fine-tuning…
Follow the Flow: On Information Flow Across Textual Tokens in Text-to-Image Models
Guy Kaplan, Michael Toker, Yuval Reif +2
Text-to-image generation models suffer from alignment problems, where generated images fail to accurately capture the objects and relations in the text prompt. Prior work has focus…
From Tokens to Words: On the Inner Lexicon of LLMs
Guy Kaplan, Matanel Oren, Yuval Reif +1
Natural language is composed of words, but modern large language models (LLMs) process sub-words as input. A natural question raised by this discrepancy is whether LLMs encode word…