Publications (49)
CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection
Yihan Chen, Jiawei Chen, Guozhao Mo +4
The growing integration of large language models (LLMs) into the peer review process presents potential risks to the fairness and reliability of scholarly evaluation. While LLMs of…
TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking
Yongqi Fan, Xiaoyang Chen, Dezhi Ye +6
Reasoning-intensive ranking models built on Large Language Models (LLMs) have made notable progress. However, existing approaches often rely on large-scale LLMs and explicit Chain-…
CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution
Ruiyang Xu, Jialun Cao, Yaojie Lu +6
Code benchmarks such as HumanEval are widely adopted to evaluate Large Language Models' (LLMs) coding capabilities. However, there is an unignorable programming language bias in ex…
Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofitting
Xinyan Guan, Yanjiang Liu, Hongyu Lin +4
Incorporating factual knowledge in knowledge graph is regarded as a promising approach for mitigating the hallucination of large language models (LLMs). Existing methods usually on…
Influence of External Information on Large Language Models Mirrors Social Cognitive Patterns
Ning Bian, Hongyu Lin, Peilin Liu +5
Social cognitive theory explains how people learn and acquire knowledge through observing others. Recent years have witnessed the rapid development of large language models (LLMs),…
Understanding Differential Search Index for Text Retrieval
Xiaoyang Chen, Yanjiang Liu, Ben He +2
The Differentiable Search Index (DSI) is a novel information retrieval (IR) framework that utilizes a differentiable function to generate a sorted list of document identifiers in r…
Cheems: A Practical Guidance for Building and Evaluating Chinese Reward Models from Scratch
Xueru Wen, Jie Lou, Zichao Li +9
Reward models (RMs) are crucial for aligning large language models (LLMs) with human preferences. However, most RM research is centered on English and relies heavily on synthetic r…
PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides
Hao Zheng, Xinyan Guan, Hao Kong +7
Automatically generating presentations from documents is a challenging task that requires accommodating content quality, visual appeal, and structural coherence. Existing methods p…
LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?
Guozhao Mo, Wenliang Zhong, Jiawei Chen +7
Model Context Protocol (MCP) has become a key infrastructure for connecting LLMs with external tools, scaling to 10,000+ MCP servers with diverse tools. Unfortunately, there is sti…
Image Captioning based on Deep Learning Methods: A Survey
Yiyu Wang, Jungang Xu, Yingfei Sun +1
Image captioning is a challenging task and attracting more and more attention in the field of Artificial Intelligence, and which can be applied to efficient image retrieval, intell…
ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models
Ning Bian, Xianpei Han, Le Sun +5
Large language models (LLMs) have made significant progress in NLP. However, their ability to memorize, represent, and leverage commonsense knowledge has been a well-known pain poi…
PARADE: Passage Representation Aggregation for Document Reranking
Canjia Li, Andrew Yates, Sean MacAvaney +2
Pretrained transformer models, such as BERT and T5, have shown to be highly effective at ad-hoc passage and document ranking. Due to inherent sequence length limits of these models…
Towards Scalable Automated Alignment of LLMs: A Survey
Boxi Cao, Keming Lu, Xinyu Lu +10
Alignment is the most critical step in building large language models (LLMs) that meet human needs. With the rapid development of LLMs gradually surpassing human capabilities, trad…
Memorizing is Not Enough: Deep Knowledge Injection Through Reasoning
Ruoxi Xu, Yunjie Ji, Boxi Cao +7
Although large language models (LLMs) excel in knowledge recall and reasoning, their static nature leads to outdated information as the real world evolves or when adapting to domai…
Rule or Story, Which is a Better Commonsense Expression for Talking with Large Language Models?
Ning Bian, Xianpei Han, Hongyu Lin +3
Building machines with commonsense has been a longstanding challenge in NLP due to the reporting bias of commonsense rules and the exposure bias of rule-based commonsense reasoning…
Beyond Local Edits: Embedding-Virtualized Knowledge for Broader Evaluation and Preservation of Model Editing
Shuainan Liu, Xuanang Chen, Ben He +1
Knowledge editing methods for large language models are commonly evaluated using predefined benchmarks that assess edited facts together with a limited set of related or neighborin…
ConsistentChat: Building Skeleton-Guided Consistent Multi-Turn Dialogues for Large Language Models from Scratch
Jiawei Chen, Xinyan Guan, Qianhao Yuan +7
Current instruction data synthesis methods primarily focus on single-turn instructions and often neglect cross-turn coherence, resulting in context drift and reduced task completio…
Co-BERT: A Context-Aware BERT Retrieval Model Incorporating Local and Query-specific Context
Xiaoyang Chen, Kai Hui, Ben He +3
BERT-based text ranking models have dramatically advanced the state-of-the-art in ad-hoc retrieval, wherein most models tend to consider individual query-document pairs independent…
Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection
Xinlin Peng, Ying Zhou, Ben He +2
Large language models (LLMs) have exhibited remarkable capabilities in text generation tasks. However, the utilization of these models carries inherent risks, including but not lim…
Utilizing Embeddings for Ad-hoc Retrieval by Document-to-document Similarity
Chenhao Yang, Ben He, Yanhua Ran
Latent semantic representations of words or paragraphs, namely the embeddings, have been widely applied to information retrieval (IR). One of the common approaches of utilizing emb…
Navigating the Shadows: Unveiling Effective Disturbances for Modern AI Content Detectors
Ying Zhou, Ben He, Le Sun
With the launch of ChatGPT, large language models (LLMs) have attracted global attention. In the realm of article writing, LLMs have witnessed extensive utilization, giving rise to…
Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision
Ning Bian, Xianpei Han, Bo Chen +3
Despite recent success in machine reading comprehension (MRC), learning high-quality MRC models still requires large-scale labeled training data, even using strong pre-trained lang…
Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering
Xinyan Guan, Yanjiang Liu, Xinyu Lu +8
The evolution of machine learning has increasingly prioritized the development of powerful models and more scalable supervision signals. However, the emergence of foundation models…
EmbedAgent: Benchmarking Large Language Models in Embedded System Development
Ruiyang Xu, Jialun Cao, Mingyuan Wu +6
Large Language Models (LLMs) have shown promise in various tasks, yet few benchmarks assess their capabilities in embedded system development. In this paper, we introduce EmbedAgen…
Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attack
Ying Zhou, Ben He, Le Sun
With the development of large language models (LLMs), detecting whether text is generated by a machine becomes increasingly challenging in the face of malicious use cases like the…
Offline Pseudo Relevance Feedback for Efficient and Effective Single-pass Dense Retrieval
Xueru Wen, Xiaoyang Chen, Xuanang Chen +2
Dense retrieval has made significant advancements in information retrieval (IR) by achieving high levels of effectiveness while maintaining online efficiency during a single-pass r…
Defense of Adversarial Ranking Attack in Text Retrieval: Benchmark and Baseline via Detection
Xuanang Chen, Ben He, Le Sun +1
Neural ranking models (NRMs) have undergone significant development and have become integral components of information retrieval (IR) systems. Unfortunately, recent research has un…
Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalization
Hao Xiang, Qiaoyu Tang, Le Yu +8
Reinforcement Learning (RL) with verifiable environments has emerged as a powerful approach for enhancing the reasoning capabilities of Large Language Models (LLMs). While prior re…
Towards Imperceptible Document Manipulations against Neural Ranking Models
Xuanang Chen, Ben He, Zheng Ye +2
Adversarial attacks have gained traction in order to identify potential vulnerabilities in neural ranking models (NRMs), but current attack methods often introduce grammatical erro…
SAISA: Towards Multimodal Large Language Models with Both Training and Inference Efficiency
Qianhao Yuan, Yanjiang Liu, Yaojie Lu +4
Multimodal Large Language Models (MLLMs) mainly fall into two architectures, each involving a trade-off between training and inference efficiency: embedding space alignment (e.g.,…
Simplified TinyBERT: Knowledge Distillation for Document Retrieval
Xuanang Chen, Ben He, Kai Hui +2
Despite the effectiveness of utilizing the BERT model for document ranking, the high computational cost of such approaches limits their uses. To this end, this paper first empirica…
Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval Perspective
Ying Zhou, Xuanang Chen, Ben He +2
Knowledge graph completion (KGC) aims to infer missing knowledge triples based on known facts in a knowledge graph. Current KGC research mostly follows an entity ranking protocol,…
Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction
Tianyun Zhong, Guozhao Mo, Yanjiang Liu +9
With the emergence of large language models (LLMs), there is an expectation that LLMs can effectively extract explicit information from complex real-world documents (e.g., papers,…
NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval
Canjia Li, Yingfei Sun, Ben He +5
Pseudo-relevance feedback (PRF) is commonly used to boost the performance of traditional information retrieval (IR) models by using top-ranked documents to identify and weight new…
Self-Retrieval: End-to-End Information Retrieval with One Large Language Model
Qiaoyu Tang, Jiawei Chen, Zhuoqun Li +10
The rise of large language models (LLMs) has significantly transformed both the construction and application of information retrieval (IR) systems. However, current interactions be…
Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steering
Fangan Dong, Zuming Yan, Xuri Ge +7
Despite the strong reasoning capabilities of recent large language models (LLMs), achieving reliable performance on challenging tasks often requires post-training or computationall…
On-Policy Self-Alignment with Fine-grained Knowledge Feedback for Hallucination Mitigation
Xueru Wen, Jie Lou, Xinyu Lu +8
Hallucination occurs when large language models exhibit behavior that deviates from the boundaries of their knowledge during response generation. To address this critical issue, pr…
BERT-QE: Contextualized Query Expansion for Document Re-ranking
Zhi Zheng, Kai Hui, Ben He +3
Query expansion aims to mitigate the mismatch between the language used in a query and in a document. However, query expansion methods can suffer from introducing non-relevant info…
Rank4Gen: RAG-Preference-Aligned Document Set Selection and Ranking
Yongqi Fan, Yuxiang Chu, Zhentao Xia +9
In the RAG paradigm, document ranking determines the evidence available to downstream generators. Through controlled analysis, we identify two phenomena underexplored by existing r…
All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG
Dan Wang, Guozhao Mo, Yafei Shi +9
Multilingual Retrieval-Augmented Generation (mRAG) leverages cross-lingual evidence to ground Large Language Models (LLMs) in global knowledge. However, we show that current mRAG s…
DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
Ying Zhou, Xinyao Wang, Yulei Niu +6
Recent advancements in large language models (LLMs) have significantly enhanced their knowledge and generative capabilities, leading to a surge of interest in leveraging LLMs for h…
Groupwise Query Performance Prediction with BERT
Xiaoyang Chen, Ben He, Le Sun
While large-scale pre-trained language models like BERT have advanced the state-of-the-art in IR, its application in query performance prediction (QPP) is so far based on pointwise…
Coupled Variational Reinforcement Learning for Language Model General Reasoning
Xueru Wen, Jie Lou, Yanjiang Liu +6
While reinforcement learning has achieved impressive progress in language model reasoning, it is constrained by the requirement for verifiable rewards. Recent verifier-free RL meth…
Spiral of Silence: How is Large Language Model Killing Information Retrieval? -- A Case Study on Open Domain Question Answering
Xiaoyang Chen, Ben He, Hongyu Lin +5
The practice of Retrieval-Augmented Generation (RAG), which integrates Large Language Models (LLMs) with retrieval systems, has become increasingly prevalent. However, the repercus…
Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models
Hao Xiang, Bowen Yu, Hongyu Lin +7
The key to effective alignment lies in high-quality preference data. Recent research has focused on automated alignment, which involves developing alignment systems with minimal hu…
Auto-RT: Automatic Jailbreak Strategy Exploration for Red-Teaming Large Language Models
Yanjiang Liu, Shuhen Zhou, Yaojie Lu +6
Automated red-teaming has become a crucial approach for uncovering vulnerabilities in large language models (LLMs). However, most existing methods focus on isolated safety flaws, l…
Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models
Qiao Liang, Yanjiang Liu, Weixiang Zhou +7
Does the prior knowledge of the vision encoder constrain the capability boundary of Multi-modal Large Language Models (MLLMs)? While most existing research treats MLLMs as unified…
Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
Yanjiang Liu, Jie Lou, Xinyan Guan +7
On-policy distillation transfers reasoning capabilities by training a student model on its own generated trajectories using token-level feedback from a teacher. However, we identif…
Rethinking Reward Model Evaluation: Are We Barking up the Wrong Tree?
Xueru Wen, Jie Lou, Yaojie Lu +7
Reward Models (RMs) are crucial for aligning language models with human preferences. Currently, the evaluation of RMs depends on measuring accuracy against a validation set of manu…