6 citations · 24 across the 55 of their papers we have counts for
24 papers · 1 filter
HGMEM: Hypergraph-based Working Memory to Improve Multi-step RAG for Long-Context Complex Relational Modeling
Chulun Zhou, Chunkang Zhang, Guoxin Yu +4
Multi-step retrieval-augmented generation (RAG) has become a widely adopted strategy for enhancing large language models (LLMs) on tasks that demand global comprehension and intens…
Efficient Covariance Estimation for Sparsified Functional Data
Sijie Zheng, Fandong Meng, Jie Zhou
Motivated by recent work involving the analysis of leveraging spatial correlations in sparsified mean estimation, we present a novel procedure for constructing covariance estimator…
UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings
Zhibin Lan, Liqiang Niu, Fandong Meng +2
The remarkable success of multimodal large language models (MLLMs) has driven advances in multimodal embeddings, yet existing models remain inherently discriminative, limiting thei…
Continuous Autoregressive Language Models
Chenze Shao, Darren Li, Fandong Meng +1
The efficiency of large language models (LLMs) is fundamentally limited by their sequential, token-by-token generation process. We argue that overcoming this bottleneck requires a…
Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning
Xue Zhang, Yunlong Liang, Fandong Meng +5
Large Reasoning Models (LRMs) have achieved remarkable performance on complex reasoning tasks by adopting the ``think-then-answer'' paradigm, which enhances both accuracy and inter…
CM-Align: Consistency-based Multilingual Alignment for Large Language Models
Xue Zhang, Yunlong Liang, Fandong Meng +4
Current large language models (LLMs) generally show a significant performance gap in alignment between English and other languages. To bridge this gap, existing research typically…
LaCo: Efficient Layer-wise Compression of Visual Tokens for Multimodal Large Language Models
Juntao Liu, Liqiang Niu, Wenchao Chen +2
Existing visual token compression methods for Multimodal Large Language Models (MLLMs) predominantly operate as post-encoder modules, limiting their potential for efficiency gains.…
Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts
Xue Zhang, Yunlong Liang, Fandong Meng +4
Continually expanding new languages for existing large language models (LLMs) is a promising yet challenging approach to building powerful multilingual LLMs. The biggest challenge…
SlangDIT: Benchmarking LLMs in Interpretative Slang Translation
Yunlong Liang, Fandong Meng, Jiaan Wang +1
The challenge of slang translation lies in capturing context-dependent semantic extensions, as slang terms often convey meanings beyond their literal interpretation. While slang de…
THOR-MoE: Hierarchical Task-Guided and Context-Responsive Routing for Neural Machine Translation
Yunlong Liang, Fandong Meng, Jie Zhou
The sparse Mixture-of-Experts (MoE) has achieved significant progress for neural machine translation (NMT). However, there exist two limitations in current MoE solutions which may…
ExTrans: Multilingual Deep Reasoning Translation via Exemplar-Enhanced Reinforcement Learning
Jiaan Wang, Fandong Meng, Jie Zhou
In recent years, the emergence of large reasoning models (LRMs), such as OpenAI-o1 and DeepSeek-R1, has shown impressive capabilities in complex problems, e.g., mathematics and cod…
An Empirical Study of Many-to-Many Summarization with Large Language Models
Jiaan Wang, Fandong Meng, Zengkui Sun +5
Many-to-many summarization (M2MS) aims to process documents in any language and generate the corresponding summaries also in any language. Recently, large language models (LLMs) ha…
Continuous Visual Autoregressive Generation via Score Maximization
Chenze Shao, Fandong Meng, Jie Zhou
Conventional wisdom suggests that autoregressive models are used to process discrete data. When applied to continuous modalities such as visual data, Visual AutoRegressive modeling…
Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space
Zhengrui Ma, Yang Feng, Chenze Shao +3
We introduce SLED, an alternative approach to speech language modeling by encoding speech waveforms into sequences of continuous latent representations and modeling them autoregres…
A Dual-Space Framework for General Knowledge Distillation of Large Language Models
Xue Zhang, Songming Zhang, Yunlong Liang +4
Knowledge distillation (KD) is a promising solution to compress large language models (LLMs) by transferring their knowledge to smaller models. During this process, white-box KD me…
DeepTrans: Deep Reasoning Translation via Reinforcement Learning
Jiaan Wang, Fandong Meng, Jie Zhou
Recently, deep reasoning LLMs (e.g., OpenAI o1 and DeepSeek-R1) have shown promising performance in various downstream tasks. Free translation is an important and interesting task…
D2C: Unlocking the Potential of Continuous Autoregressive Image Generation with Discrete Tokens
Panpan Wang, Liqiang Niu, Fandong Meng +3
In the domain of image generation, latent-based generative models occupy a dominant status; however, these models rely heavily on image tokenizer. To meet modeling requirements, au…
A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiences
Jiaxin Shen, Jinan Xu, Huiqi Hu +6
While progress has been made in legal applications, law reasoning, crucial for fair adjudication, remains unexplored. We propose a transparent law reasoning schema enriched with hi…
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning
Zhibin Lan, Liqiang Niu, Fandong Meng +2
Universal multimodal embedding models play a critical role in tasks such as interleaved image-text retrieval, multimodal RAG, and multimodal clustering. However, our empirical resu…
Warmup-Distill: Bridge the Distribution Mismatch between Teacher and Student before Knowledge Distillation
Zengkui Sun, Yijin Liu, Fandong Meng +3
The widespread deployment of Large Language Models (LLMs) is hindered by the high computational demands, making knowledge distillation (KD) crucial for developing compact smaller o…
Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping
Yijie Chen, Yijin Liu, Fandong Meng +3
Knowledge Distillation (KD) has emerged as a prominent technique for model compression. However, conventional KD approaches primarily focus on homogeneous architectures with identi…
LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information
Bowen Ping, Jiali Zeng, Fandong Meng +3
Long-form generation is crucial for academic writing papers and repo-level code generation. Despite this, current models, including GPT-4o, still exhibit unsatisfactory performance…
DeepRAG: Thinking to Retrieve Step by Step for Large Language Models
Xinyan Guan, Jiali Zeng, Fandong Meng +6
Large Language Models (LLMs) have shown remarkable reasoning capabilities, while their practical applications are limited by severe factual hallucinations due to limitations in the…
Personalized Language Model Learning on Text Data Without User Identifiers
Yucheng Ding, Yangwenjian Tan, Xiangyu Liu +6
In many practical natural language applications, user data are highly sensitive, requiring anonymous uploads of text data from mobile devices to the cloud without user identifiers.…