3 papers
cs.LG2025
Rethinking the shape convention of an MLP
Meng-Hsi Chen, Yu-Ang Lee, Feng-Ting Liao +1
Multi-layer perceptrons (MLPs) conventionally follow a narrow-wide-narrow design where skip connections operate at the input/output dimensions while processing occurs in expanded h…
cs.CL2025
Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directions
Yu-Ang Lee, Guan-Ting Yi, Mei-Yi Liu +3
Recent advancements in large language models (LLMs) and AI systems have led to a paradigm shift in the design and optimization of complex AI workflows. By integrating multiple comp…
cs.CL2025
STAR: Spectral Truncation and Rescale for Model Merging
Yu-Ang Lee, Ching-Yun Ko, Tejaswini Pedapati +3
Model merging is an efficient way of obtaining a multi-task model from several pretrained models without further fine-tuning, and it has gained attention in various domains, includ…