4 papers
Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptation
Kening Zheng, Wei-Chieh Huang, Jiahao Huo +9
Mixture-of-Experts (MoE) models exhibit striking performance disparities across languages, yet the internal mechanisms driving these gaps remain poorly understood. In this work, we…
TreeHop: Efficient Embedding-Level Query Rewriter
Zhonghao Li, Kunpeng Zhang, Jinghuai Ou +2
Retrieval-augmented generation (RAG) systems face significant challenges in multi-hop question answering (MHQA), where complex queries require synthesizing information across multi…
Reasoning Factual Knowledge in Structured Data with Large Language Models
Sirui Huang, Yanggan Gu, Xuming Hu +3
Large language models (LLMs) have made remarkable progress in various natural language processing tasks as a benefit of their capability to comprehend and reason with factual knowl…
Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilities
Zhonghao Li, Xuming Hu, Aiwei Liu +3
Large Language Models (LLMs) are limited by their parametric knowledge, leading to hallucinations in knowledge-extensive tasks. To address this, Retrieval-Augmented Generation (RAG…