4 papers
Spatial Priors via Space Filling Curves for Small and Limited Data Vision Transformers
Leyla Naz Candogan, Arshia Afzal, Pol Puigdemont +1
Though Vision Transformers (ViTs) have become the dominant backbone in many computer vision tasks, due to permutation equivariance, their attention mechanism lacks explicit spatial…
Linear Attention for Efficient Bidirectional Sequence Modeling
Arshia Afzal, Elias Abad Rocamora, Leyla Naz Candogan +5
Linear Transformers and State Space Models have emerged as efficient alternatives to softmax Transformers for causal sequence modeling, enabling parallel training via matrix multip…
Quantum-PEFT: Ultra parameter-efficient fine-tuning
Toshiaki Koike-Akino, Francesco Tonin, Yongtao Wu +3
This paper introduces Quantum-PEFT that leverages quantum computations for parameter-efficient fine-tuning (PEFT). Unlike other additive PEFT methods, such as low-rank adaptation (…
Single-pass Detection of Jailbreaking Input in Large Language Models
Leyla Naz Candogan, Yongtao Wu, Elias Abad Rocamora +2
Defending aligned Large Language Models (LLMs) against jailbreaking attacks is a challenging problem, with existing approaches requiring multiple requests or even queries to auxili…