6 papers
Don't Read Everything: A Curvature-Conditioned Query for Linear Attention
Dong Le, Thong Nguyen, Cong-Duy Nguyen +1
Linear attention reduces the quadratic cost of softmax attention by maintaining a recurrent fast-weight state, but it consistently lags on in-context retrieval and long-context tas…
Sparton: Fast and Memory-Efficient Triton Kernel for Learned Sparse Retrieval
Thong Nguyen, Cosimo Rulli, Franco Maria Nardini +2
State-of-the-art Learned Sparse Retrieval (LSR) models, such as Splade, typically employ a Language Modeling (LM) head to project latent hidden states into a lexically-anchored log…
Milco: Learned Sparse Retrieval Across Languages via a Multilingual Connector
Thong Nguyen, Yibin Lei, Jia-Huei Ju +2
Learned Sparse Retrieval (LSR) combines the efficiency of bi-encoders with the transparency of lexical matching, but existing approaches struggle to scale beyond English. We introd…
SERVAL: Surprisingly Effective Zero-Shot Visual Document Retrieval Powered by Large Vision and Language Models
Thong Nguyen, Yibin Lei, Jia-Huei Ju +1
Visual Document Retrieval (VDR) typically operates as text-to-image retrieval using specialized bi-encoders trained to directly embed document images. We revisit a zero-shot genera…
Effective Inference-Free Retrieval for Learned Sparse Representations
Franco Maria Nardini, Thong Nguyen, Cosimo Rulli +2
Learned Sparse Retrieval (LSR) is an effective IR approach that exploits pre-trained language models for encoding text into a learned bag of words. Several efforts in the literatur…
Leveraging Decoder Architectures for Learned Sparse Retrieval
Jingfen Qiao, Thong Nguyen, Evangelos Kanoulas +1
Learned Sparse Retrieval (LSR) has traditionally focused on small-scale encoder-only transformer architectures. With the advent of large-scale pre-trained language models, their ca…