14 papers
IdeaScientist: Orchestrating Agents for Grounded Scientific Ideation
Jiarui Liu, Renjie Tao, Yiwei Liao +15
Despite rapid progress in automating scientific research, generating promising and well grounded research solutions remains a central challenge. We isolate research ideation as a s…
MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories
Guangzhi Xiong, Xinyuan Zhang, Xiao Yang +14
Long-term egocentric video enables personalized AI assistants to reason about daily life. However, as video histories grow to hundreds of hours spanning months or years, reprocessi…
CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection
Wen Dong, Zhao Wang, Shuangqing Zhang +5
Multimodal Large Language Models (MLLMs) excel in diverse vision tasks, but full-parameter retraining is computationally expensive as real-world knowledge evolves. Existing continu…
PixSearch: Region-Grounded Retrieval for Knowledgeable Large Multimodal Models
Jeonghwan Kim, Renjie Tao, Sanat Sharma +8
Multimodal Retrieval-Augmented Generation (MM-RAG) augments visual question answering (VQA) with external knowledge, but many systems rely on modular pipelines that separate visual…
WearVox: An Egocentric Multichannel Voice Assistant Benchmark for Wearables
Zhaojiang Lin, Yong Xu, Kai Sun +17
Wearable devices such as AI glasses are transforming voice assistants into always-available, hands-free collaborators that integrate seamlessly with daily life, but they also intro…
CRAG-MM: Multi-modal Multi-turn Comprehensive RAG Benchmark
Jiaqi Wang, Xiao Yang, Kai Sun +38
Wearable devices such as smart glasses are transforming the way people interact with their surroundings, enabling users to seek information regarding entities in their view. Multi-…
AssoMem: Scalable Memory QA with Multi-Signal Associative Retrieval
Kai Zhang, Xinyuan Zhang, Ejaz Ahmed +11
Accurate recall from large scale memories remains a core challenge for memory augmented AI assistants performing question answering (QA), especially in similarity dense scenarios w…
Stream RAG: Instant and Accurate Spoken Dialogue Systems with Streaming Tool Usage
Siddhant Arora, Haidar Khan, Kai Sun +14
End-to-end speech-in speech-out dialogue systems are emerging as a powerful alternative to traditional ASR-LLM-TTS pipelines, generating more natural, expressive responses with sig…
SCRIBES: Web-Scale Script-Based Semi-Structured Data Extraction with Reinforcement Learning
Shicheng Liu, Kai Sun, Lisheng Fu +8
Semi-structured content in HTML tables, lists, and infoboxes accounts for a substantial share of factual data on the web, yet the formatting complicates usage, and reliably extract…
Knowledge Extraction on Semi-Structured Content: Does It Remain Relevant for Question Answering in the Era of LLMs?
Kai Sun, Yin Huang, Srishti Mehra +11
The advent of Large Language Models (LLMs) has significantly advanced web-based Question Answering (QA) systems over semi-structured content, raising questions about the continued…
TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning
Zhepei Wei, Xiao Yang, Kai Sun +12
While large language models (LLMs) have demonstrated strong performance on factoid question answering, they are still prone to hallucination and untruthful responses, particularly…
Refine-n-Judge: Curating High-Quality Preference Chains for LLM-Fine-Tuning
Derin Cayir, Renjie Tao, Rashi Rungta +6
Large Language Models (LLMs) have demonstrated remarkable progress through preference-based fine-tuning, which critically depends on the quality of the underlying training data. Wh…
PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoning
Mohammad Kachuee, Teja Gollapudi, Minseok Kim +10
Retrieval-augmented generation (RAG) often falls short when retrieved context includes confusing semi-relevant passages, or when answering questions require deep contextual underst…
ConfRAG: Confidence-Guided Retrieval-Augmenting Generation
Yin Huang, Yifan Ethan Xu, Kai Sun +12
Can Large Language Models (LLMs) be trained to avoid hallucinating factual statements, and can Retrieval-Augmented Generation (RAG) be triggered only when necessary to reduce retri…