4 papers
Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models
Duc Anh Nguyen, Tien Ngoc Luu, Tung Pham +1
State-based fine-tuning has emerged as a compelling alternative to weight-based adaptation for transformers, updating lightweight controls into states rather than model weights, of…
Selective Sinkhorn Routing for Improved Sparse Mixture of Experts
Duc Anh Nguyen, Huu Binh Ta, Nhuan Le Duc +2
Sparse Mixture-of-Experts (SMoE) models are scalable and computationally efficient, enabling large increases in model capacity with limited inference overhead. Existing SMoE method…
Generalization Bounds for Robust Contrastive Learning: From Theory to Practice
Ngoc N. Tran, Lam Tran, Hoang Phan +5
Contrastive Learning first extracts features from unlabeled data, followed by linear probing with labeled data. Adversarial Contrastive Learning (ACL) integrates Adversarial Traini…
KOPPA: Improving Prompt-based Continual Learning with Key-Query Orthogonal Projection and Prototype-based One-Versus-All
Quyen Tran, Hoang Phan, Lam Tran +4
Drawing inspiration from prompt tuning techniques applied to Large Language Models, recent methods based on pre-trained ViT networks have achieved remarkable results in the field o…