6 papers
\k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating
Jianghui Wang, Silong Yong, Francesco Orabona +3
Low-Rank Adaptation (LoRA) has become a widely adopted technique for efficient neural network fine-tuning, decomposing model updates into low-rank matrices. However, LoRA remains c…
Nonparametric Teaching of Attention Learners
Chen Zhang, Jianghui Wang, Bingyang Cheng +7
Attention learners, neural networks built on the attention mechanism, e.g., transformers, excel at learning the implicit relationships that relate sequences to their corresponding…
OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions
Cheng Luo, Jianghui Wang, Bing Li +2
In this paper, we introduce Online Multimodal Conversational Response Generation (OMCRG), a novel task designed to produce synchronized verbal and non-verbal listener feedback onli…
Task-Robust Pre-Training for Worst-Case Downstream Adaptation
Jianghui Wang, Yang Chen, Xingyu Xie +2
Pre-training has achieved remarkable success when transferred to downstream tasks. In machine learning, we care about not only the good performance of a model but also its behavior…
MoviePuzzle: Visual Narrative Reasoning through Multimodal Order Learning
Jianghui Wang, Yuxuan Wang, Dongyan Zhao +1
We introduce MoviePuzzle, a novel challenge that targets visual narrative reasoning and holistic movie understanding. Despite the notable progress that has been witnessed in the re…
Shuo Wen Jie Zi: Rethinking Dictionaries and Glyphs for Chinese Language Pre-training
Yuxuan Wang, Jianghui Wang, Dongyan Zhao +1
We introduce CDBERT, a new learning paradigm that enhances the semantics understanding ability of the Chinese PLMs with dictionary knowledge and structure of Chinese characters. We…