2 citations · 3 across the 7 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
Wonderful Matrices: Combining for a More Efficient and Effective Foundation Model Architecture
Jingze Shi, Bingheng Wu
In order to make the foundation model more efficient and effective, our idea is combining sequence transformation and state transformation. First, we prove the availability of rota…
cs.LG2024
Wonderful Matrices: More Efficient and Effective Architecture for Language Modeling Tasks
Jingze Shi, Bingheng Wu, Lu He +1
We prove the availability of inner product form position encoding in the state space dual algorithm and study the effectiveness of different position embeddings in the hybrid quadr…