3 papers
cs.LG2026
FourierMoE: Fourier Mixture-of-Experts Adaptation of Large Language Models
Juyong Jiang, Fan Wang, Hong Qi +2
Parameter-efficient fine-tuning (PEFT) has emerged as a crucial paradigm for adapting large language models (LLMs) under constrained computational budgets. However, standard PEFT m…
cs.CL2024
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models
Fan Wang, Juyong Jiang, Chansung Park +2
The increasing sizes of large language models (LLMs) result in significant computational overhead and memory usage when adapting these models to specific tasks or domains. Various…
cs.LG2024
A Survey on Mixture of Experts in Large Language Models
Weilin Cai, Juyong Jiang, Fan Wang +3
Large language models (LLMs) have garnered unprecedented advancements across diverse fields, ranging from natural language processing to computer vision and beyond. The prowess of…