1 citations · 2 across the 6 of their papers we have counts for
6 papers
FOGO: Forgetting-aware Orthogonalization Optimizer
Toan Nguyen, Yang Liu, Trung Le +2
We argue that forgetting is not confined to continual learning but is a general optimization phenomenon: during standard training, dominant mini-batch gradients suppress rare but u…
RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts
Tuan Truong, Chau Nguyen, Huy Nguyen +3
Low-rank Adaptation (LoRA) has emerged as a powerful method for fine-tuning large-scale foundation models. Despite its popularity, the theoretical understanding of LoRA has remaine…
Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning
Quyen Tran, Hoang Phan, Minh Le +6
Humans perceive the world as a series of sequential events, which can be hierarchically organized with different levels of abstraction based on conceptual knowledge. Drawing inspir…
Improving Generalization with Flat Hilbert Bayesian Inference
Tuan Truong, Quyen Tran, Quan Pham-Ngoc +3
We introduce Flat Hilbert Bayesian Inference (FHBI), an algorithm designed to enhance generalization in Bayesian inference. Our approach involves an iterative two-step procedure wi…
Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts
Minh Le, Chau Nguyen, Huy Nguyen +3
Prompt-based techniques, such as prompt-tuning and prefix-tuning, have gained prominence for their efficiency in fine-tuning large pre-trained models. Despite their widespread adop…
Preserving Generalization of Language models in Few-shot Continual Relation Extraction
Quyen Tran, Nguyen Xuan Thanh, Nguyen Hoang Anh +4
Few-shot Continual Relations Extraction (FCRE) is an emerging and dynamic area of study where models can sequentially integrate knowledge from new relations with limited labeled da…