6 papers
ROSA: Random Subspace Adaptation for Efficient Fine-Tuning
Marawan Gamal Abdel Hameed, Aristides Milios, Siva Reddy +1
Model training requires significantly more memory, compared with inference. Parameter efficient fine-tuning (PEFT) methods provide a means of adapting large models to downstream ta…
A Tensor Decomposition Perspective on Second-order RNNs
Maude Lizaire, Michael Rizvi-Martel, Marawan Gamal Abdel Hameed +1
Second-order Recurrent Neural Networks (2RNNs) extend RNNs by leveraging second-order interactions for sequence modelling. These models are provably more expressive than their firs…
Efficient Leverage Score Sampling for Tensor Train Decomposition
Vivek Bharadwaj, Beheshteh T. Rakhshan, Osman Asif Malik +1
Tensor Train~(TT) decomposition is widely used in the machine learning and quantum physics communities as a popular tool to efficiently compress high-dimensional tensor data. In th…
Simulating Weighted Automata over Sequences and Trees with Transformers
Michael Rizvi, Maude Lizaire, Clara Lacroce +1
Transformers are ubiquitous models in the natural language processing (NLP) community and have shown impressive empirical successes in the past few years. However, little is unders…
Fast and Attributed Change Detection on Dynamic Graphs with Density of States
Shenyang Huang, Jacob Danovitch, Guillaume Rabusseau +1
How can we detect traffic disturbances from international flight transportation logs or changes to collaboration dynamics in academic networks? These problems can be formulated as…
Laplacian Change Point Detection for Single and Multi-view Dynamic Graphs
Shenyang Huang, Samy Coulombe, Yasmeen Hitti +2
Dynamic graphs are rich data structures that are used to model complex relationships between entities over time. In particular, anomaly detection in temporal graphs is crucial for…