Neologism Learning as a Parameter-Efficient Alternative to Fine-Tuning for Model Steering

Sungjoon Park, Varun Ramamurthi, Owen Terry

In language modeling, neologisms are new tokens trained to represent a concept not already included in a given model's vocabulary. Neologisms can be used to encourage specific beha…