4 papers
Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
Yu-Ang Lee, Ching-Yun Ko, Pin-Yu Chen +1
Low-Rank Adaptation (LoRA) is the prevailing approach for efficient large language model (LLM) fine-tuning. Building on this paradigm, recent studies have proposed alternative init…
Beyond Facts: Benchmarking Distributional Reading Comprehension in Large Language Models
Pei-Fu Guo, Ya-An Tsai, Chun-Chia Hsu +6
While most reading comprehension benchmarks for LLMs focus on factual information that can be answered by localizing specific textual evidence, many real-world tasks require unders…
LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs
Pei-Fu Guo, Yun-Da Tsai, Chun-Chia Hsu +6
Evaluating cross-lingual knowledge transfer in large language models is challenging, as correct answers in a target language may arise either from genuine transfer or from prior ex…
STAR: Spectral Truncation and Rescale for Model Merging
Yu-Ang Lee, Ching-Yun Ko, Tejaswini Pedapati +3
Model merging is an efficient way of obtaining a multi-task model from several pretrained models without further fine-tuning, and it has gained attention in various domains, includ…