7 papers · 1 filter
Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer-Review Process
Zhijun Chen, Zeyu Ji, Qianren Mao +12
We propose LLM-PeerReview, an unsupervised LLM Ensemble method that selects the most ideal response from multiple LLM-generated candidates for each query, harnessing the collective…
Pretraining Large Language Models with NVFP4
NVIDIA, Felix Abecassis, Anjulie Agrusa +87
Large Language Models (LLMs) today are powerful problem solvers across many domains, and they continue to get stronger as they scale in model size, training set size, and training…
MemOS: A Memory OS for AI System
Zhiyu Li, Chenyang Xi, Chunyu Li +36
Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI), yet their lack of well-defined memory management systems hinders the…
Making Language Model a Hierarchical Classifier
Yihong Wang, Zhonglin Jiang, Ningyuan Xi +8
Decoder-only language models, such as GPT and LLaMA, generally decode on the last layer. Motivated by human's hierarchical thinking capability, we propose that a hierarchical decod…
CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models
Yuanjie Lyu, Zhiyu Li, Simin Niu +7
Retrieval-Augmented Generation (RAG) is a technique that enhances the capabilities of large language models (LLMs) by incorporating external knowledge sources. This method addresse…
: Language Modeling with Explicit Memory
Hongkang Yang, Zehao Lin, Wenjin Wang +13
The training and inference of large language models (LLMs) are together a costly process that transports knowledge from raw data to meaningful computation. Inspired by the memory h…