Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
SUPN: Shallow Universal Polynomial Networks
Zachary Morrow, Michael Penwarden, Brian Chen +3
Deep neural networks (DNNs) and Kolmogorov-Arnold networks (KANs) are popular methods for function approximation due to their flexibility and expressivity. However, they typically…
cs.LG2025
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients
Yezhen Wang, Zhouhao Yang, Brian K Chen +4
Building upon the success of low-rank adapter (LoRA), low-rank gradient projection (LoRP) has emerged as a promising solution for memory-efficient fine-tuning. However, existing Lo…
cs.LG2024
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
Brian K Chen, Tianyang Hu, Hui Jin +2
In-Context Learning (ICL) has been a powerful emergent property of large language models that has attracted increasing attention in recent years. In contrast to regular gradient-ba…