11 papers
DWA-KD: Dual-Space Weighting and Time-Warped Alignment for Cross-Tokenizer Knowledge Distillation
Duc Trung Vu, Pham Khanh Chi, Dat Phi Van +3
Knowledge Distillation (KD) has emerged as a crucial technique for compressing Large Language Models (LLMs). Although existing cross-tokenizer KD methods have made notable progress…
Efficient Temporal-aware Matryoshka Adaptation for Temporal Information Retrieval
Tuan-Luc Huynh, Weiqing Wang, Trung Le +4
Retrievers are a key bottleneck in temporal Retrieval-Augmented Generation (RAG) systems: failing to retrieve temporally relevant context can degrade downstream generation, regardl…
Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models
Dung Anh Hoang, Cuong Pham, Cuong Nguyen +3
Large Language Models (LLMs) deliver strong performance across a wide range of NLP tasks, but their massive sizes hinder deployment on resource-constrained devices. To reduce their…
Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models
Cuong Pham, Hoang Anh Dung, Cuong C. Nguyen +4
Large language models require significant computational resources for deployment, making quantization essential for practical applications. However, the main obstacle to effective…
Layer-Wise High-Impact Parameter Ratio Optimization in Post-Training Quantization for Large Language Models
Cuong Pham, Hoang Anh Dung, Cuong C. Nguyen +3
Large language models (LLMs) have significantly advanced natural language processing, but their massive parameter counts create substantial computational and memory challenges duri…
On the Mechanisms of Collaborative Learning in VAE Recommenders
Tung-Long Vuong, Julien Monteil, Hien Dang +3
Variational Autoencoders (VAEs) are a powerful alternative to matrix factorization for recommendation. A common technique in VAE-based collaborative filtering (CF) consists in appl…
MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
Tuan-Luc Huynh, Thuy-Trang Vu, Weiqing Wang +4
Continually updating model-based indexes in generative retrieval with new documents remains challenging, as full retraining is computationally expensive and impractical under resou…
Mitigating Semantic Collapse in Generative Personalization with Test-Time Embedding Adjustment
Anh Bui, Trang Vu, Trung Le +5
In this paper, we investigate the semantic collapsing problem in generative personalization, an under-explored topic where the learned visual concept () gradually shifts from it…
Optimizing Specific and Shared Parameters for Efficient Parameter Tuning
Van-Anh Nguyen, Thanh-Toan Do, Mehrtash Harandi +2
Foundation models, with a vast number of parameters and pretraining on massive datasets, achieve state-of-the-art performance across various applications. However, efficiently adap…
Enhancing Dataset Distillation via Non-Critical Region Refinement
Minh-Tuan Tran, Trung Le, Xuan-May Le +2
Dataset distillation has become a popular method for compressing large datasets into smaller, more efficient representations while preserving critical information for model trainin…
PromptDSI: Prompt-based Rehearsal-free Continual Learning for Document Retrieval
Tuan-Luc Huynh, Thuy-Trang Vu, Weiqing Wang +5
Differentiable Search Index (DSI) utilizes pre-trained language models to perform indexing and document retrieval via end-to-end learning without relying on external indexes. Howev…