49 citations · 54 across the 11 of their papers we have counts for
11 papers
Opinion Leader Dynamics: How Sparse Attention Shapes Token Clustering
Jingkun Liu, Yue Song
Sparse attention reduces the quadratic cost of global self-attention while retaining strong empirical performance, but how its restricted interactions shape the evolution of token…
Spontaneous symmetry breaking and Goldstone modes for deep information propagation
Nabil Iqbal, T. Anderson Keller, Yue Song +2
In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation o…
Higher-Order Equilibrium Tracking for EM-Compressible Online Estimation
ZhiMing Li, Yue Song
We study online estimation in latent-variable models by recasting the problem as tracking a moving empirical equilibrium. Standard online EM and stochastic approximation analyses p…
Anisotropic Modality Align
Xiaomin Yu, Yijiang Li, Yuhui Zhang +8
Training multimodal large language models has long been limited by the scarcity of high-quality paired multimodal data. Recent studies show that the shared representation space of…
MOVA: Towards Scalable and Synchronized Video-Audio Generation
OpenMOSS Team, Donghua Yu, Mingshu Chen +38
Audio is indispensable for real-world video, yet generation models have largely overlooked audio components. Current approaches to producing audio-visual content often rely on casc…
Krause Synchronization Transformers
Jingkun Liu, Yisong Yue, Max Welling +1
Self-attention in Transformers relies on globally normalized softmax weights, causing all tokens to compete for influence at every layer. When composed across depth, this interacti…
Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks
Hoagy Cunningham, Jerry Wei, Zihan Wang +26
We introduce enhanced Constitutional Classifiers that deliver production-grade jailbreak robustness with dramatically reduced computational costs and refusal rates compared to prev…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
Aligning Large Language Models with Representation Editing: A Control Perspective
Lingkai Kong, Haorui Wang, Wenhao Mu +7
Aligning large language models (LLMs) with human objectives is crucial for real-world applications. However, fine-tuning LLMs for alignment often suffers from unstable training and…
Navigating Chemical Space with Latent Flows
Guanghao Wei, Yining Huang, Chenru Duan +2
Recent progress of deep generative models in the vision and language domain has stimulated significant interest in more structured data generation such as molecules. However, beyon…
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
Zhihang Yuan, Yuzhang Shang, Yue Song +4
In this paper, we introduce a new post-training compression paradigm for Large Language Models (LLMs) to facilitate their wider adoption. We delve into LLM weight low-rank decompos…