How Do Language Models Choose Between Context and Memory?

Benjamin Shih, John Winnicki, Arianna Cao

When contextual information conflicts with the knowledge stored in model parameters, activation directions can be used to decode and steer which source the model follows. However,…