4 citations · 4 across the 12 of their papers we have counts for
16 papers
MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning
Yixuan Xie, Xueqiang Xu, Xinyi Fan +8
Agent memory enables large language model (LLM) agents to reuse past experiences without updating their parameters. However, semantic similarity alone does not guarantee memory use…
SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
Shaowen Wang, Ge Zhang, Kairong Luo +6
Looped Transformers increase effective depth by iterating a shared block of layers, but most evaluations compare at fixed model size, conflating architectural advantage with extra…
AdaRoPE: Not All Attention Heads Should Rotate and Scale Equally
Shaowen Wang, Yuke Zheng, Tansheng Zhu +4
Rotary Position Embedding (RoPE) is widely adopted in Transformers to encode positional information, yet standard implementations enforce a uniform frequency schedule and scaling a…
Loop-P: Residual Scaling and Hyperparameter Transfer in Looped Transformers
Shaowen Wang, Bingrui Li, Ge Zhang +4
Looped Transformers have drawn growing attention because looping improves multi-step reasoning and increases the executed depth of a model without adding parameters. They achieve t…
Retrieval is Cheap, Show Me the Code: Executable Multi-Hop Reasoning for Retrieval-Augmented Generation
Jiashuo Sun, Jimeng Shi, Yixuan Xie +10
Retrieval-Augmented Generation (RAG) has become a standard approach for knowledge-intensive question answering, but existing systems remain brittle on multi-hop questions, where so…
ORBIT: Preserving Foundational Language Capabilities in GenRetrieval via Origin-Regulated Merging
Neha Verma, Nikhil Mehta, Shao-Chuan Wang +7
Despite the rapid advancements in large language model (LLM) development, fine-tuning them for specific tasks often results in the catastrophic forgetting of their general, languag…
SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection
Kexian Tang, Jiani Wang, Shaowen Wang +1
While large language models (LLMs) are pretrained on massive amounts of data, their knowledge coverage remains incomplete in specialized, data-scarce domains, motivating extensive…
Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space
Xingwei Qu, Shaowen Wang, Zihao Huang +16
Large Language Models (LLMs) apply uniform computation to all tokens, despite language exhibiting highly non-uniform information density. This token-uniform regime wastes capacity…
CEMG: Collaborative-Enhanced Multimodal Generative Recommendation
Yuzhen Lin, Hongyi Chen, Xuanjing Chen +3
Generative recommendation models often struggle with two key challenges: (1) the superficial integration of collaborative signals, and (2) the decoupled fusion of multimodal featur…
Adaptive Cache Pollution Control for Large Language Model Inference Workloads Using Temporal CNN-Based Prediction and Priority-Aware Replacement
Songze Liu, Hongkun Du, Shaowen Wang
Large Language Models (LLMs), such as GPT and LLaMA, introduce unique memory access characteristics during inference due to frequent token sequence lookups and embedding vector ret…
When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs
Shaowen Wang, Yiqi Dong, Ruinian Chang +4
Despite substantial advances, large language models (LLMs) continue to exhibit hallucinations, generating plausible yet incorrect responses. In this paper, we highlight a critical…
PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations
Ruining He, Lukasz Heldt, Lichan Hong +20
Large Language Models (LLMs) pose a new paradigm of modeling and computation for information tasks. Recommendation systems are a critical application domain poised to benefit signi…
Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws
Zhixuan Pan, Shaowen Wang, Jian Li
Large Language Models (LLMs) have demonstrated remarkable capabilities across numerous tasks, yet principled explanations for their underlying mechanisms and several phenomena, suc…
CAdam: Confidence-Based Optimization for Online Learning
Shaowen Wang, Anan Liu, Jian Xiao +9
Modern recommendation systems frequently employ online learning to dynamically update their models with freshly collected data. The most commonly used optimizer for updating neural…
LoRA-GA: Low-Rank Adaptation with Gradient Approximation
Shaowen Wang, Linxi Yu, Jian Li
Fine-tuning large-scale pretrained models is prohibitively expensive in terms of computational and memory costs. LoRA, as one of the most popular Parameter-Efficient Fine-Tuning (P…
Generative Table Pre-training Empowers Models for Tabular Prediction
Tianping Zhang, Shaowen Wang, Shuicheng Yan +2
Recently, the topic of table pre-training has attracted considerable research interest. However, how to employ table pre-training to boost the performance of tabular prediction rem…