1 citations · 1 across the 13 of their papers we have counts for
14 papers
Protecting Shared Agent Workflows from Invalidation Attacks with Signed Trajectories and Runtime Verification
Ruixiao Li, Fahao Chen, Peng Li
Large language model (LLM)-based agents increasingly rely on shared workflow repositories to transfer experience across tasks and agents. By abstracting past executions into reusab…
Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding
Zhigeng Liu, Zhiyuan Ning, Ruixiao Li +5
The development of long-context Large Language Models (LLMs) is constrained by the memory bandwidth bottleneck and quadratic complexity of the attention mechanism during decoding.…
MOSS-VL Technical Report
Pengyu Wang, Chenkun Tan, Shaojun Zhou +29
We present MOSS-VL, an open vision-language model family that treats real-time interaction -- perceiving while it speaks -- as a first-class capability. It is co-designed across th…
MOSS-Audio Technical Report
Chen Yang, Chufan Yu, Hanfu Chen +27
MOSS-Audio is a unified audio-language model for speech, environmental sound, and music understanding, supporting audio captioning, time-aware question answering, timestamped trans…
MOSS-TTS Technical Report
Yitian Gong, Botian Jiang, Yiwei Zhao +23
This technical report presents MOSS-TTS, a speech generation foundation model built on a scalable recipe: discrete audio tokens, autoregressive modeling, and large-scale pretrainin…
What Makes Position Zero Special? A Mechanistic Study of Position Zero Attention Sinks in LLMs
Runyu Peng, Ruixiao Li, Mingshu Chen +5
Transformers frequently allocate disproportionate attention to specific tokens, a phenomenon known as attention sinks. Causal large language models reliably form one at position ze…
MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models
Yitian Gong, Kuangwei Chen, Zhaoye Fei +9
Discrete audio tokenizers are fundamental to empowering large language models with native audio processing and generation capabilities. Despite recent progress, existing approaches…
MOVA: Towards Scalable and Synchronized Video-Audio Generation
OpenMOSS Team, Donghua Yu, Mingshu Chen +38
Audio is indispensable for real-world video, yet generation models have largely overlooked audio components. Current approaches to producing audio-visual content often rely on casc…
Count Counts: Motivating Exploration in LLM Reasoning with Count-based Intrinsic Rewards
Xuan Zhang, Ruixiao Li, Zhijian Zhou +7
Reinforcement Learning (RL) has become a compelling way to strengthen the multi step reasoning ability of Large Language Models (LLMs). However, prevalent RL paradigms still lean o…
MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance
Xingjian Zhao, Zhe Xu, Qinyuan Cheng +20
Spoken dialogue systems often rely on cascaded pipelines that transcribe, process, and resynthesize speech. While effective, this design discards paralinguistic cues and limits exp…
Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
Yuerong Song, Xiaoran Liu, Ruixiao Li +5
Diffusion Large Language Models (dLLMs) enable breakthroughs in reasoning and parallel decoding but suffer from prohibitive quadratic computational complexity and memory overhead d…
Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache
Xiaoran Liu, Siyang He, Qiqi Wang +9
Large Language Models struggle with memory demands from the growing Key-Value (KV) cache as context lengths increase. Existing compression methods homogenize head dimensions or rel…
Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
Ruixiao Li, Fahao Chen, Peng Li
Speculative decoding accelerates Large Language Model (LLM) inference by employing a small speculative model (SSM) to generate multiple candidate tokens and verify them using the L…
Thus Spake Long-Context Large Language Model
Xiaoran Liu, Ruixiao Li, Mianqiu Huang +11
Long context is an important topic in Natural Language Processing (NLP), running through the development of NLP architectures, and offers immense opportunities for Large Language M…