15 papers · 1 filter
TALAS: Teacher-Anchored Layer Alignment with Adaptive Sharpness-Aware Minimization for Embedding Distillation
Quoc Phong Dao, Hoang Son Nguyen, Pham Khanh Chi +4
Knowledge Distillation (KD) has established itself as a pivotal technique for compressing large pre-trained language models. However, existing methods that force a student to stric…
TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching
Truong Nguyen, Tien-Phat Nguyen, Linh Ngo Van +3
Direct Preference Optimization (DPO) is a widely used RL-free method for aligning language models from pairwise preferences, but it models preferences over full sequences even thou…
LLM-XTM: Enhancing Cross-Lingual Topic Models with Large Language Models
Minh Chu Xuan, Tien-Phat Nguyen, Linh Ngo Van +3
Cross-lingual topic modeling aims to discover shared semantic structures across languages, yet existing models depend on sparse bilingual resources and often yield incoherent or we…
MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining
Phung Gia Huy, Hai An Vu, Minh-Phuc Truong +4
Representation learning is fundamental to NLP, but building embeddings that work well at different computational budgets is challenging. Matryoshka Representation Learning (MRL) of…
WAVE++: Capturing Within-Task Variance for Continual Relation Extraction with Adaptive Prompting
Bao-Ngoc Dao, Minh Le, Quang Nguyen +3
Memory-based approaches have shown strong performance in Continual Relation Extraction (CRE). However, storing examples from previous tasks increases memory usage and raises privac…
CTPD: Cross Tokenizer Preference Distillation
Truong Nguyen, Phi Van Dat, Ngan Nguyen +3
While knowledge distillation has seen widespread use in pre-training and instruction tuning, its application to aligning language models with human preferences remains underexplore…