9 papers
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
Lu Dai, Ziyang Rao, Yili Wang +3
Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this fai…
LLM-Oriented Information Retrieval: A Denoising-First Perspective
Lu Dai, Liang Sun, Fanpu Cao +4
Modern information retrieval (IR) is no longer consumed primarily by humans but increasingly by large language models (LLMs) via retrieval-augmented generation (RAG) and agentic se…
TableVision: A Large-Scale Benchmark for Spatially Grounded Reasoning over Complex Hierarchical Tables
Xiaoyu Chen, Lu Dai, Hanqing Wang +6
Structured tables are essential for conveying high-density information in professional domains such as finance, healthcare, and scientific research. Despite the progress in Multimo…
Enhancing Multivariate Time Series Forecasting with Global Temporal Retrieval
Fanpu Cao, Lu Dai, Jindong Han +1
Multivariate time series forecasting (MTSF) plays a vital role in numerous real-world applications, yet existing models remain constrained by their reliance on a limited historical…
VideoAfford: Grounding 3D Affordance from Human-Object-Interaction Videos via Multimodal Large Language Model
Hanqing Wang, Mingyu Liu, Xiaoyu Chen +9
3D affordance grounding aims to highlight the actionable regions on 3D objects, which is crucial for robotic manipulation. Previous research primarily focused on learning affordanc…
ScIRGen: Synthesize Realistic and Large-Scale RAG Dataset for Scientific Research
Junyong Lin, Lu Dai, Ruiqian Han +7
Scientific researchers need intensive information about datasets to effectively evaluate and develop theories and methodologies. The information needs regarding datasets are implic…
SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction
Lu Dai, Yijie Xu, Jinhui Ye +2
Large Language Models (LLMs) have demonstrated improved generation performance by incorporating externally retrieved knowledge, a process known as retrieval-augmented generation (R…
Improve Dense Passage Retrieval with Entailment Tuning
Lu Dai, Hao Liu, Hui Xiong
Retrieval module can be plugged into many downstream NLP tasks to improve their performance, such as open-domain question answering and retrieval-augmented generation. The key to a…
Cloth2Body: Generating 3D Human Body Mesh from 2D Clothing
Lu Dai, Liqian Ma, Shenhan Qian +3
In this paper, we define and study a new Cloth2Body problem which has a goal of generating 3D human body meshes from a 2D clothing image. Unlike the existing human mesh recovery pr…