7 citations · 8 across the 11 of their papers we have counts for
3 papers · 1 filter
State Space Models are Strong Text Rerankers
Zhichao Xu, Jinghua Yan, Ashim Gupta +1
Transformers dominate NLP and IR; but their inference inefficiencies and challenges in extrapolating to longer contexts have sparked interest in alternative model architectures. Am…
A Controlled Study on Long Context Extension and Generalization in LLMs
Yi Lu, Jing Nathan Yan, Songlin Yang +6
Broad textual understanding and in-context learning require language models that utilize full document contexts. Due to the implementation challenges associated with directly train…
MambaByte: Token-free Selective State Space Model
Junxiong Wang, Tushaar Gangavarapu, Jing Nathan Yan +1
Token-free language models learn directly from raw bytes and remove the inductive bias of subword tokenization. Operating on bytes, however, results in significantly longer sequenc…