Linear-Time and Constant-Memory Text Embeddings Based on Recurrent Language Models

Tobias Grantner, Emanuel Sallinger, Martin Flechl

Transformer-based embedding models suffer from quadratic computational and linear memory complexity, limiting their utility for long sequences. We propose recurrent architectures a…