7 citations · 14 across the 11 of their papers we have counts for
11 papers
Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functions
Haotian Jiang, Zeyu Bao, Shida Wang +1
The evolution of sequence modeling architectures, from recurrent neural networks and convolutional models to Transformers and structured state-space models, reflects ongoing effort…
Learning Relational Tabular Data without Shared Features
Zhaomin Wu, Shida Wang, Ziyang Wang +1
Learning relational tabular data has gained significant attention recently, but most studies focus on single tables, overlooking the potential of cross-table learning. Cross-table…
LongSSM: On the Length Extension of State-space Models in Language Modelling
Shida Wang
In this paper, we investigate the length-extension of state-space models (SSMs) in language modeling. Length extension involves training models on short sequences and testing them…
Integrating Deep Learning and Synthetic Biology: A Co-Design Approach for Enhancing Gene Expression via N-terminal Coding Sequences
Zhanglu Yan, Weiran Chu, Yuhua Sheng +4
N-terminal coding sequence (NCS) influences gene expression by impacting the translation initiation rate. The NCS optimization problem is to find an NCS that maximizes gene express…
StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization
Shida Wang, Qianxiao Li
In this paper, we investigate the long-term memory learning capabilities of state-space models (SSMs) from the perspective of parameterization. We prove that state-space models wit…
State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory
Shida Wang, Beichen Xue
State-space models have gained popularity in sequence modelling due to their simple and efficient network structures. However, the absence of nonlinear activation along the tempora…
HyperSNN: A new efficient and robust deep learning model for resource constrained control applications
Zhanglu Yan, Shida Wang, Kaiwen Tang +1
In light of the increasing adoption of edge computing in areas such as intelligent furniture, robotics, and smart homes, this paper introduces HyperSNN, an innovative method for co…
Improve Long-term Memory Learning Through Rescaling the Error Temporally
Shida Wang, Zhanglu Yan
This paper studies the error metric selection for long-term memory learning in sequence modelling. We examine the bias towards short-term memory in commonly used errors, including…
Inverse Approximation Theory for Nonlinear Recurrent Neural Networks
Shida Wang, Zhong Li, Qianxiao Li
We prove an inverse approximation theorem for the approximation of nonlinear sequence-to-sequence relationships using recurrent neural networks (RNNs). This is a so-called Bernstei…
A Brief Survey on the Approximation Theory for Sequence Modelling
Haotian Jiang, Qianxiao Li, Zhong Li +1
We survey current developments in the approximation theory of sequence modelling in machine learning. Particular emphasis is placed on classifying existing results for various mode…
Efficient Hyperdimensional Computing
Zhanglu Yan, Shida Wang, Kaiwen Tang +1
Hyperdimensional computing (HDC) is a method to perform classification that uses binary vectors with high dimensions and the majority rule. This approach has the potential to be en…