25 citations · 58 across the 25 of their papers we have counts for
8 papers · 1 filter
Accelerating Toeplitz Neural Network with Constant-time Inference Complexity
Zhen Qin, Yiran Zhong
Toeplitz Neural Networks (TNNs) have exhibited outstanding performance in various sequence modeling tasks. They outperform commonly used Transformer-based models while benefiting f…
Hierarchically Gated Recurrent Neural Network for Sequence Modeling
Zhen Qin, Songlin Yang, Yiran Zhong
Transformers have surpassed RNNs in popularity due to their superior abilities in parallel training and long-term dependency modeling. Recently, there has been a renewed interest i…
All-pairs Consistency Learning for Weakly Supervised Semantic Segmentation
Weixuan Sun, Yanhao Zhang, Zhen Qin +5
In this work, we propose a new transformer-based regularization to better localize objects for Weakly supervised semantic segmentation (WSSS). In image-level WSSS, Class Activation…
Linearized Relative Positional Encoding
Zhen Qin, Weixuan Sun, Kaiyue Lu +6
Relative positional encoding is widely used in vanilla and linear transformers to represent positional information. However, existing encoding methods of a vanilla transformer are…
TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer
Zhen Qin, Dong Li, Weigao Sun +8
We present TransNormerLLM, the first linear attention-based Large Language Model (LLM) that outperforms conventional softmax attention-based models in terms of both accuracy and ef…
Exploring Transformer Extrapolation
Zhen Qin, Yiran Zhong, Hui Deng
Length extrapolation has attracted considerable attention recently since it allows transformers to be tested on longer sequences than those used in training. Previous research has…
Toeplitz Neural Network for Sequence Modeling
Zhen Qin, Xiaodong Han, Weixuan Sun +6
Sequence modeling has important applications in natural language processing and computer vision. Recently, the transformer-based models have shown strong performance on various seq…
Fine-grained Audible Video Description
Xuyang Shen, Dong Li, Jinxing Zhou +9
We explore a new task for audio-visual-language modeling called fine-grained audible video description (FAVD). It aims to provide detailed textual descriptions for the given audibl…