2 papers
cs.LG2024
ROSA: Random Subspace Adaptation for Efficient Fine-Tuning
Marawan Gamal Abdel Hameed, Aristides Milios, Siva Reddy +1
Model training requires significantly more memory, compared with inference. Parameter efficient fine-tuning (PEFT) methods provide a means of adapting large models to downstream ta…
cs.LG2024
A Tensor Decomposition Perspective on Second-order RNNs
Maude Lizaire, Michael Rizvi-Martel, Marawan Gamal Abdel Hameed +1
Second-order Recurrent Neural Networks (2RNNs) extend RNNs by leveraging second-order interactions for sequence modelling. These models are provably more expressive than their firs…