3 citations · 7 across the 24 of their papers we have counts for
3 papers · 1 filter
Learned Queries and Keys Are All You Need: Replacing the Value Projection with Structured Transforms
Ene Meco, Emadeldeen Hamdan, A. Enis Cetin
To reduce the number of parameters and cache memory requirements of transformers we introduce dual-headed transformers instead of three heads. We studied Walsh-Hadamard Transform (…
Sparse Mamba: Introducing Controllability, Observability, And Stability To Structural State Space Models
Emadeldeen Hamdan, Hongyi Pan, Ahmet Enis Cetin
Structured state space models' (SSMs) development in recent studies, such as Mamba and Mamba2, outperformed and solved the computational inefficiency of transformers and large lang…
A novel asymmetrical autoencoder with a sparsifying discrete cosine Stockwell transform layer for gearbox sensor data compression
Xin Zhu, Daoguang Yang, Hongyi Pan +3
The lack of an efficient compression model remains a challenge for the wireless transmission of gearbox data in non-contact gear fault diagnosis problems. In this paper, we present…