8 papers · 1 filter
Will it Merge? On The Causes of Model Mergeability
Adir Rahamim, Asaf Yehudai, Boaz Carmeli +3
Model merging has emerged as a promising technique for combining multiple fine-tuned models into a single multitask model without retraining. However, the factors that determine wh…
CRISP: Complex Reasoning with Interpretable Step-based Plans
Matan Vetzler, Koren Lazar, Guy Uziel +3
Recent advancements in large language models (LLMs) underscore the need for stronger reasoning capabilities to solve complex problems effectively. While Chain-of-Thought (CoT) reas…
LLM Hypnosis: Exploiting User Feedback for Unauthorized Knowledge Injection to All Users
Almog Hilel, Riddhi Bhagwat, Idan Shenfeld +2
We describe a vulnerability in language models (LMs) trained with user feedback, whereby a single user can persistently alter LM knowledge and behavior given only the ability to pr…
Can Gradient Descent Simulate Prompting?
Eric Zhang, Leshem Choshen, Jacob Andreas
There are two primary ways of incorporating new information into a language model (LM): changing its prompt or changing its parameters, e.g. via fine-tuning. Parameter updates incu…
Pretraining Language Models for Diachronic Linguistic Change Discovery
Elisabeth Fittschen, Sabrina Li, Tom Lippincott +2
Large language models (LLMs) have shown potential as tools for scientific discovery. This has engendered growing interest in their use in humanistic disciplines, such as historical…
NeurIPS 2023 LLM Efficiency Fine-tuning Competition
Mark Saroufim, Yotam Perlitz, Leshem Choshen +11
Our analysis of the NeurIPS 2023 large language model (LLM) fine-tuning competition revealed the following trend: top-performing models exhibit significant overfitting on benchmark…