LARA: Lightweight Adapters in the Residual Stream for Composable Adaptation and Alignment

Pascal Ekin, Hyosun Choi, Wei Jie

We present LARA (Lightweight Additive Residual Adaptation), a method for efficient adaptation that operates in the residual stream of a frozen model rather than in its weights. Whe…