1 citations · 2 across the 14 of their papers we have counts for
7 papers · 1 filter
Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
Yuchen Zhang, Hanyue Du, Chun Cao +1
Low-Rank Adaptation (LoRA) has become a widely adopted parameter-efficient fine-tuning (PEFT) technique for adapting large language models (LLMs) to downstream tasks. While prior w…
Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
Siyuan Yan, Guo-Qing Jiang, Yuchen Zhang +4
Large language models (LLMs) now support context windows of hundreds of thousands to millions of tokens, enabling applications such as long-document summarization, large-scale code…
Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learning
Wannan, Yang, Xinchi Qiu +6
Large Language Models (LLMs) exhibit impressive capabilities but often hallucinate, confidently providing incorrect answers instead of admitting ignorance. Prior work has shown tha…
Toward Efficient Influence Function: Dropout as a Compression Tool
Yuchen Zhang, Mohammad Mohammadi Amiri
Assessing the impact the training data on machine learning models is crucial for understanding the behavior of the model, enhancing the transparency, and selecting training data. I…
Echo: Decoupling Inference and Training for Large-Scale RL Alignment on Heterogeneous Swarms
Jie Xiao, Changyuan Fan, Qingnan Ren +6
Modern RL-based post-training for large language models (LLMs) co-locate trajectory sampling and policy optimisation on the same GPU cluster, forcing the system to switch between i…
The Eloquence team submission for task 1 of MLC-SLM challenge
Lorenzo Concina, Jordi Luque, Alessio Brutti +2
In this paper, we present our studies and experiments carried out for the task 1 of the Challenge and Workshop on Multilingual Conversational Speech Language Model (MLC-SLM), which…
Adversarial Curriculum Graph-Free Knowledge Distillation for Graph Neural Networks
Yuang Jia, Xiaojuan Shan, Jun Xia +5
Data-free Knowledge Distillation (DFKD) is a method that constructs pseudo-samples using a generator without real data, and transfers knowledge from a teacher model to a student by…