8 papers
TEMPER: Tensorized Efficient Manifold-constrained Parameterization for Expressive Residual Routing
Yuxuan Gu, Wuyang Zhou, Huijun Xing +1
Residual connections rely on a static residual pathway, and are essential for training deep neural networks. Hyper-connections (HC) increase the expressivity of residual routing by…
Learning Pore-scale Multiphase Flow from 4D Velocimetry
Chunyang Wang, Linqi Zhu, Yuxuan Gu +9
Multiphase flow in porous media underpins subsurface energy and environmental technologies, including geological CO storage and underground hydrogen storage, yet pore-scale dyn…
PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra
Xiachong Feng, Liang Zhao, Weihong Zhong +5
Current methods for personality control in Large Language Models rely on static prompting or expensive fine-tuning, failing to capture the dynamic and compositional nature of human…
KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matrices
Wuyang Zhou, Yuxuan Gu, Giorgos Iacovides +1
The success of Hyper-Connections (HC) in neural networks (NN) has also highlighted issues related to training instability and restricted scalability. The Manifold-Constrained Hyper…
Masked Auto-Regressive Variational Acceleration: Fast Inference Makes Practical Reinforcement Learning
Yuxuan Gu, Weimin Bai, Yifei Wang +2
Masked auto-regressive diffusion models (MAR) benefit from the expressive modeling ability of diffusion models and the flexibility of masked auto-regressive ordering. However, vani…
TeRA: Vector-based Random Tensor Network for High-Rank Adaptation of Large Language Models
Yuxuan Gu, Wuyang Zhou, Giorgos Iacovides +1
Parameter-Efficient Fine-Tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA), have significantly reduced the number of trainable parameters needed in fine-tuning large langua…
TensorLLM: Tensorising Multi-Head Attention for Enhanced Reasoning and Compression in LLMs
Yuxuan Gu, Wuyang Zhou, Giorgos Iacovides +1
The reasoning abilities of Large Language Models (LLMs) can be improved by structurally denoising their weights, yet existing techniques primarily focus on denoising the feed-forwa…
Learning Pore-scale Multi-phase Flow from Experimental Data with Graph Neural Network
Yuxuan Gu, Catherine Spurin, Gege Wen
Understanding the process of multiphase fluid flow through porous media is crucial for many climate change mitigation technologies, including CO geological storage, hydrogen st…