7 citations · 12 across the 11 of their papers we have counts for
11 papers
SOAP, Muon, and Beyond: Pushing LLM Pretraining Scales
Mikail Khona, Aditya Vavre, Boxiang Wang +11
Higher-order optimizers such as Muon and SOAP offer faster convergence than AdamW, but their computational cost and numerical stability challenges have limited adoption at scale. I…
Do Models Hear Like Us? Probing the Representational Alignment of Audio LLMs and Naturalistic EEG
Haoyun Yang, Xin Xiao, Jiang Zhong +5
Audio Large Language Models (Audio LLMs) have demonstrated strong capabilities in integrating speech perception with language understanding. However, whether their internal represe…
Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer-Review Process
Zhijun Chen, Zeyu Ji, Qianren Mao +13
We propose LLM-PeerReview, an unsupervised LLM Ensemble method that selects the most ideal response from multiple LLM-generated candidates for each query, harnessing the collective…
Epistemic Bias Injection: Manipulating LLM Opinion via Selective Context Retrieval
Hao Wu, Prateek Saxena
When answering user queries, LLMs often retrieve knowledge from external sources stored in retrieval-augmented generation (RAG) databases. These are often populated from unvetted s…
Pretraining Large Language Models with NVFP4
NVIDIA, Felix Abecassis, Anjulie Agrusa +87
Large Language Models (LLMs) today are powerful problem solvers across many domains, and they continue to get stronger as they scale in model size, training set size, and training…
Role-Augmented Intent-Driven Generative Search Engine Optimization
Xiaolu Chen, Haojie Wu, Jie Bao +3
Generative Search Engines (GSEs), powered by Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG), are reshaping information retrieval. While commercial systems (e…
Making Language Model a Hierarchical Classifier
Yihong Wang, Zhonglin Jiang, Ningyuan Xi +8
Decoder-only language models, such as GPT and LLaMA, generally decode on the last layer. Motivated by human's hierarchical thinking capability, we propose that a hierarchical decod…
MemOS: A Memory OS for AI System
Zhiyu Li, Chenyang Xi, Chunyu Li +36
Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI), yet their lack of well-defined memory management systems hinders the…
: Language Modeling with Explicit Memory
Hongkang Yang, Zehao Lin, Wenjin Wang +13
The training and inference of large language models (LLMs) are together a costly process that transports knowledge from raw data to meaningful computation. Inspired by the memory h…
emrQA-msquad: A Medical Dataset Structured with the SQuAD V2.0 Framework, Enriched with emrQA Medical Information
Jimenez Eladio, Hao Wu
Machine Reading Comprehension (MRC) holds a pivotal role in shaping Medical Question Answering Systems (QAS) and transforming the landscape of accessing and applying medical inform…
CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models
Yuanjie Lyu, Zhiyu Li, Simin Niu +7
Retrieval-Augmented Generation (RAG) is a technique that enhances the capabilities of large language models (LLMs) by incorporating external knowledge sources. This method addresse…