5 papers · 1 filter
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Yiwen Chen, Joshua Ainslie, Krzysztof Choromanski +4
Rotary Position Embedding (RoPE) has been widely adopted in transformer-based large language models. However, its log-linear frequency schedule, originally designed to produce long…
Mixture of Geodesic Factor Analyzers on Riemannian Homogeneous Spaces
Hengchao Chen, Yuanyao Tan, Chao Huang +2
This paper introduces Mixtures of Geodesic Factor Analyzers (MGFA) on Riemannian homogeneous spaces. MGFA uses a geodesic factor model within each mixture component, providing grea…
Distributional Matching for Vector Quantization: A Unified Theoretical and Empirical Framework
Xianghong Fang, Litao Guo, Hengchao Chen +8
The effectiveness of modern visual representation learning and autoregressive models critically depends on vector quantization (VQ), which discretizes continuous feature representa…
CORTEG: Foundation Models Enable Cross-Modality Representation Transfer from Scalp to Intracranial Brain Recordings
Liuyin Yang, Qiang Sun, Bob Van Dyck +2
Intracranial electrocorticography (ECoG) offers high signal-to-noise access to cortical activity for brain-computer interfaces, yet limited per-patient data has led most prior work…
dFlowGRPO: Rate-Aware Policy Optimization for Discrete Flow Models
Zhengyan Wan, Yidong Ouyang, Panwen Hu +1
Discrete flow models (DFMs) are a class of flexible generative models for generating discrete data, and diffusion large language models (dLLMs) can be viewed as a special case with…