8 papers
TEMPER: Tensorized Efficient Manifold-constrained Parameterization for Expressive Residual Routing
Yuxuan Gu, Wuyang Zhou, Huijun Xing +1
Residual connections rely on a static residual pathway, and are essential for training deep neural networks. Hyper-connections (HC) increase the expressivity of residual routing by…
KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matrices
Wuyang Zhou, Yuxuan Gu, Giorgos Iacovides +1
The success of Hyper-Connections (HC) in neural networks (NN) has also highlighted issues related to training instability and restricted scalability. The Manifold-Constrained Hyper…
TeRA: Vector-based Random Tensor Network for High-Rank Adaptation of Large Language Models
Yuxuan Gu, Wuyang Zhou, Giorgos Iacovides +1
Parameter-Efficient Fine-Tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA), have significantly reduced the number of trainable parameters needed in fine-tuning large langua…
Masked Auto-Regressive Variational Acceleration: Fast Inference Makes Practical Reinforcement Learning
Yuxuan Gu, Weimin Bai, Yifei Wang +2
Masked auto-regressive diffusion models (MAR) benefit from the expressive modeling ability of diffusion models and the flexibility of masked auto-regressive ordering. However, vani…
Learning Pore-scale Multiphase Flow from 4D Velocimetry
Chunyang Wang, Linqi Zhu, Yuxuan Gu +9
Multiphase flow in porous media underpins subsurface energy and environmental technologies, including geological CO storage and underground hydrogen storage, yet pore-scale dyn…
PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra
Xiachong Feng, Liang Zhao, Weihong Zhong +5
Current methods for personality control in Large Language Models rely on static prompting or expensive fine-tuning, failing to capture the dynamic and compositional nature of human…