2 citations · 7 across the 11 of their papers we have counts for
6 papers · 1 filter
Priming: Hybrid State Space Models From Pre-trained Transformers
Aditya Chattopadhyay, Elvis Nunez, Prannay Kaul +6
Hybrid State-Space models combine Attention with recurrent State-Space Model (SSM) layers, balancing eidetic memory from Attention with compressed fading memory from SSMs. This yie…
B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory
Luca Zancato, Arjun Seshadri, Yonatan Dukler +6
We describe a family of architectures to support transductive inference by allowing memory to grow to a finite but a-priori unknown bound while making efficient use of finite resou…
SAFE: Machine Unlearning With Shard Graphs
Yonatan Dukler, Benjamin Bowman, Alessandro Achille +3
We present Synergy Aware Forgetting Ensemble (SAFE), a method to adapt large models on a diverse collection of data while minimizing the expected cost to remove the influence of tr…
Your representations are in the network: composable and parallel adaptation for large scale models
Yonatan Dukler, Alessandro Achille, Hao Yang +7
We propose InCA, a lightweight method for transfer learning that cross-attends to any activation layer of a pre-trained model. During training, InCA uses a single forward pass to e…
À-la-carte Prompt Tuning (APT): Combining Distinct Data Via Composable Prompting
Benjamin Bowman, Alessandro Achille, Luca Zancato +4
We introduce À-la-carte Prompt Tuning (APT), a transformer-based scheme to tune prompts on distinct data so that they can be arbitrarily composed at inference time. The individual…
Characterizing the Spectrum of the NTK via a Power Series Expansion
Michael Murray, Hui Jin, Benjamin Bowman +1
Under mild conditions on the network initialization we derive a power series expansion for the Neural Tangent Kernel (NTK) of arbitrarily deep feedforward networks in the infinite…