5 papers
The Pre-Training Study of Expanded-SPLADE Models on Web Document Titles
Hiun Kim, Tae Kwan Lee, Taeryun Won
Masked Language Modeling (MLM) pre-training is one of the primary ways to initialize Neural Information Retrieval (IR) models prior to retrieval fine-tuning. However, studies show…
Efficiency and Effectiveness of SPLADE Models on Billion-Scale Web Document Title
Taeryun Won, Tae Kwan Lee, Hiun Kim +1
This paper presents a comprehensive comparison of BM25, SPLADE, and Expanded-SPLADE models in the context of large-scale web document retrieval. We evaluate the effectiveness and e…
The Role of Vocabularies in Learning Sparse Representations for Ranking
Hiun Kim, Tae Kwan Lee, Taeryun Won
Learned Sparse Retrieval (LSR) such as SPLADE has growing interest for effective semantic 1st stage matching while enjoying the efficiency of inverted indices. A recent work on lea…
KoBigBird-large: Transformation of Transformer for Korean Language Understanding
Kisu Yang, Yoonna Jang, Taewoo Lee +4
This work presents KoBigBird-large, a large size of Korean BigBird that achieves state-of-the-art performance and allows long sequence processing for Korean language understanding.…
Adaptable Multi-Domain Language Model for Transformer ASR
Taewoo Lee, Min-Joong Lee, Tae Gyoon Kang +10
We propose an adapter based multi-domain Transformer based language model (LM) for Transformer ASR. The model consists of a big size common LM and small size adapters. The model ca…