Can Gradient Descent Simulate Prompting?

Eric Zhang, Leshem Choshen, Jacob Andreas

There are two primary ways of incorporating new information into a language model (LM): changing its prompt or changing its parameters, e.g. via fine-tuning. Parameter updates incu…