6 citations · 15 across the 17 of their papers we have counts for
17 papers
NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction
NCP Team, Jiaqi Cao, Chiyu Chen +25
We introduce NCP-ArchPreview, a latent-space language model that pushes autoregressive pretraining beyond standard next-token prediction (NTP). Alongside NTP, the model learns thro…
Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory
Han Zhang, Zihao Tang, Xin Yu +8
In existing memory benchmarks for Large Language Models (LLMs), the evaluated dialogue sessions often lack long-term semantic consistency, and the underlying personas tend to be fl…
Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory
Zihao Tang, Xin Yu, Ziyu Xiao +9
AI Memory, specifically how models organizes and retrieves historical messages, becomes increasingly valuable to Large Language Models (LLMs), yet existing methods (RAG and Graph-R…
GeAR: Generation Augmented Retrieval
Haoyu Liu, Shaohan Huang, Jianfeng Liu +6
Document retrieval techniques are essential for developing large-scale information systems. The common approach involves using a bi-encoder to compute the semantic similarity betwe…
Context-DPO: Aligning Language Models for Context-Faithfulness
Baolong Bi, Shaohan Huang, Yiwei Wang +11
Reliable responses from large language models (LLMs) require adherence to user instructions and retrieved information. While alignment techniques help LLMs align with human intenti…
E5-V: Universal Embeddings with Multimodal Large Language Models
Ting Jiang, Minghui Song, Zihan Zhang +6
Multimodal large language models (MLLMs) have shown promising advancements in general visual and language understanding. However, the representation of multimodal information using…
ASI++: Towards Distributionally Balanced End-to-End Generative Retrieval
Yuxuan Liu, Tianchi Yang, Zihan Zhang +5
Generative retrieval, a promising new paradigm in information retrieval, employs a seq2seq model to encode document features into parameters and decode relevant document identifier…
MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
Ting Jiang, Shaohan Huang, Shengyue Luo +8
Low-rank adaptation is a popular parameter-efficient fine-tuning method for large language models. In this paper, we analyze the impact of low-rank updating, as implemented in LoRA…
ResLoRA: Identity Residual Mapping in Low-Rank Adaption
Shuhua Shi, Shaohan Huang, Minghui Song +7
As one of the most popular parameter-efficient fine-tuning (PEFT) methods, low-rank adaptation (LoRA) is commonly applied to fine-tune large language models (LLMs). However, updati…
HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition
Yuxuan Liu, Tianchi Yang, Shaohan Huang +6
Large language models (LLMs) have emerged as a promising alternative to expensive human evaluations. However, the alignment and coverage of LLM-based evaluations are often limited…
Text Diffusion with Reinforced Conditioning
Yuxuan Liu, Tianchi Yang, Shaohan Huang +6
Diffusion models have demonstrated exceptional capability in generating high-quality images, videos, and audio. Due to their adaptiveness in iterative refinement, they provide a st…
: Sequential Example Selection for In-Context Learning
Haoyu Liu, Jianfeng Liu, Shaohan Huang +5
The remarkable capability of large language models (LLMs) for in-context learning (ICL) needs to be activated by demonstration examples. Prior work has extensively explored the sel…
Improving Domain Adaptation through Extended-Text Reading Comprehension
Ting Jiang, Shaohan Huang, Shengyue Luo +8
To enhance the domain-specific capabilities of large language models, continued pre-training on a domain-specific corpus is a prevalent method. Recent work demonstrates that adapti…
Auto Search Indexer for End-to-End Document Retrieval
Tianchi Yang, Minghui Song, Zihan Zhang +4
Generative retrieval, which is a new advanced paradigm for document retrieval, has recently attracted research interests, since it encodes all documents into the model and directly…
Democratizing Reasoning Ability: Tailored Learning from Large Language Model
Zhaoyang Wang, Shaohan Huang, Yuxuan Liu +8
Large language models (LLMs) exhibit impressive emergent abilities in natural language processing, but their democratization is hindered due to huge computation requirements and cl…
Calibrating LLM-Based Evaluator
Yuxuan Liu, Tianchi Yang, Shaohan Huang +6
Recent advancements in large language models (LLMs) on language modeling and emergent capabilities make them a promising reference-free evaluator of natural language generation qua…
Model-enhanced Vector Index
Hailin Zhang, Yujing Wang, Qi Chen +16
Embedding-based retrieval methods construct vector indices to search for document representations that are most similar to the query representations. They are widely used in docume…