4 papers
Can Spectral-Clipping Enable Better Learning While Forgetting Less for Low-Rank Adaptation?
Hyowon Wi, Noseong Park
In recent years, low-rank adaptation (LoRA) has emerged as a significant paradigm that freezes pre-trained weights and introduces small, learnable adapters instead of fine-tuning t…
Learning Advanced Self-Attention for Linear Transformers in the Singular Value Domain
Hyowon Wi, Jeongwhan Choi, Noseong Park
Transformers have demonstrated remarkable performance across diverse domains. The key component of Transformers is self-attention, which learns the relationship between any two tok…
SCONE: A Novel Stochastic Sampling to Generate Contrastive Views and Hard Negative Samples for Recommendation
Chaejeong Lee, Jeongwhan Choi, Hyowon Wi +2
Graph-based collaborative filtering (CF) has emerged as a promising approach in recommender systems. Despite its achievements, graph-based CF models face challenges due to data spa…
Graph Convolutions Enrich the Self-Attention in Transformers!
Jeongwhan Choi, Hyowon Wi, Jayoung Kim +4
Transformers, renowned for their self-attention mechanism, have achieved state-of-the-art performance across various tasks in natural language processing, computer vision, time-ser…