7 papers
Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
Yu-Ang Lee, Ching-Yun Ko, Pin-Yu Chen +1
Low-Rank Adaptation (LoRA) is the prevailing approach for efficient large language model (LLM) fine-tuning. Building on this paradigm, recent studies have proposed alternative init…
Sparse Bayesian Message Passing under Structural Uncertainty
Yoonhyuk Choi, Jiho Choi, Chanran Kim +5
Semi-supervised learning on real-world graphs is frequently challenged by heterophily, where the observed graph is unreliable or label-disassortative. Many existing graph neural ne…
Can Large Language Models Develop Gambling Addiction?
Seungpil Lee, Donghyeon Shin, Yunjeong Lee +1
This study identifies the specific conditions under which large language models exhibit human-like gambling addiction patterns, providing critical insights into their decision-maki…
Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directions
Yu-Ang Lee, Guan-Ting Yi, Mei-Yi Liu +3
Recent advancements in large language models (LLMs) and AI systems have led to a paradigm shift in the design and optimization of complex AI workflows. By integrating multiple comp…
Rethinking the shape convention of an MLP
Meng-Hsi Chen, Yu-Ang Lee, Feng-Ting Liao +1
Multi-layer perceptrons (MLPs) conventionally follow a narrow-wide-narrow design where skip connections operate at the input/output dimensions while processing occurs in expanded h…
Riemannian Geometric-based Meta Learning
JuneYoung Park, YuMi Lee, Tae-Joon Kim +1
Meta-learning, or "learning to learn," aims to enable models to quickly adapt to new tasks with minimal data. While traditional methods like Model-Agnostic Meta-Learning (MAML) opt…