4 papers
Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding
Duy-Tung Pham, An The Nguyen, Viet-Hoang Tran +4
This paper investigates the dynamical properties of tokens in pre-trained Transformer models and explores their application to improving Transformers. To this end, we analyze the d…
Equivariant Polynomial Functional Networks
Thieu N. Vo, Viet-Hoang Tran, Tho Tran Huu +5
Neural Functional Networks (NFNs) have gained increasing interest due to their wide range of applications, including extracting information from implicit representations of data, e…
Equivariant Neural Functional Networks for Transformers
Viet-Hoang Tran, Thieu N. Vo, An Nguyen The +5
This paper systematically explores neural functional networks (NFN) for transformer architectures. NFN are specialized neural networks that treat the weights, gradients, or sparsit…
NeuroMax: Enhancing Neural Topic Modeling via Maximizing Mutual Information and Group Topic Regularization
Duy-Tung Pham, Thien Trang Nguyen Vu, Tung Nguyen +3
Recent advances in neural topic models have concentrated on two primary directions: the integration of the inference network (encoder) with a pre-trained language model (PLM) and t…