8 citations · 17 across the 13 of their papers we have counts for
13 papers
LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference
Guangyuan Ma, Yongliang Ma, Xuanrui Gou +3
Large Language Models (LLMs)-based text retrieval retrieves documents relevant to search queries based on vector similarities. Documents are pre-encoded offline, while queries arri…
CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts
Zhenpeng Su, Xing Wu, Zijia Lin +6
Large language models (LLM) have been attracting much attention from the community recently, due to their remarkable performance in all kinds of downstream tasks. According to the…
MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
Zhenpeng Su, Zijia Lin, Xue Bai +8
Scaling the size of a model enhances its capabilities but significantly increases computation complexity. Mixture-of-Experts models (MoE) address the issue by allowing model size t…
Task-level Distributionally Robust Optimization for Large Language Model-based Dense Retrieval
Guangyuan Ma, Yongliang Ma, Xing Wu +3
Large Language Model-based Dense Retrieval (LLM-DR) optimizes over numerous heterogeneous fine-tuning collections from different domains. However, the discussion about its training…
Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval
Guangyuan Ma, Xing Wu, Zijia Lin +1
Masked auto-encoder pre-training has emerged as a prevalent technique for initializing and enhancing dense retrieval systems. It generally utilizes additional Transformer decoder b…
HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus
Zhenpeng Su, Xing Wu, Wei Zhou +2
ChatGPT has garnered significant interest due to its impressive performance; however, there is growing concern about its potential risks, particularly in the detection of AI-genera…
Pre-training with Large Language Model-based Document Expansion for Dense Passage Retrieval
Guangyuan Ma, Xing Wu, Peng Wang +2
In this paper, we systematically study the potential of pre-training with Large Language Model(LLM)-based document expansion for dense passage retrieval. Concretely, we leverage th…
Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems
Zhenpeng Su, Xing Wu, Wei Zhou +2
Dialogue response selection aims to select an appropriate response from several candidates based on a given user and system utterance history. Most existing works primarily focus o…
CoT-MoTE: Exploring ConTextual Masked Auto-Encoder Pre-training with Mixture-of-Textual-Experts for Passage Retrieval
Guangyuan Ma, Xing Wu, Peng Wang +1
Passage retrieval aims to retrieve relevant passages from large collections of the open-domain corpus. Contextual Masked Auto-Encoding has been proven effective in representation b…
CoT-MAE v2: Contextual Masked Auto-Encoder with Multi-view Modeling for Passage Retrieval
Xing Wu, Guangyuan Ma, Peng Wang +4
Growing techniques have been emerging to improve the performance of passage retrieval. As an effective representation bottleneck pretraining technique, the contextual masked auto-e…
PUNR: Pre-training with User Behavior Modeling for News Recommendation
Guangyuan Ma, Hongtao Liu, Xing Wu +4
News recommendation aims to predict click behaviors based on user behaviors. How to effectively model the user representations is the key to recommending preferred news. Existing w…
Query-as-context Pre-training for Dense Passage Retrieval
Xing Wu, Guangyuan Ma, Wanhui Qian +2
Recently, methods have been developed to improve the performance of dense passage retrieval by using context-supervised pre-training. These methods simply consider two passages fro…
ConTextual Masked Auto-Encoder for Dense Passage Retrieval
Xing Wu, Guangyuan Ma, Meng Lin +3
Dense passage retrieval aims to retrieve the relevant passages of a query from a large corpus based on dense representations (i.e., vectors) of the query and the passages. Recent s…