Low-Rank Ternary Adaptation for Fine-Tuning Transformers

Alexandru-Dragos Manolache, Yunqiang Li, Jan van Gemert

Ternary transformers offer extreme memory and compute efficiency, but existing low-bit LoRA-based methods cannot directly fine-tune ternary weights. Current approaches either requi…