5 citations · 8 across the 7 of their papers we have counts for
1 paper · 1 filter
Wenbing Li, Zikai Song, Hang Zhou +3
Recent attempts to combine low-rank adaptation (LoRA) with mixture-of-experts (MoE) for multi-task adaptation of Large Language Models (LLMs) often replace whole attention/FFN laye…