3 papers
cs.LG2026
The Effect of Layer Ordering in Hybrid Architectures
John Cooper, Mingchen Ma, Jiaqi Mo +4
Hybrid attention is rapidly becoming the dominant architectural choice for frontier models. Despite their successes, such architectures complicate model design by introducing addit…
cs.AR2026
FORGE: Procedural Weight Generation for Sparse MoE Inference
Chanyoung Moon, Travis Pence, Daisuke Yamada +4
We introduce FORGE (Fast On-chip Rotor Generation for Experts), a procedural weight generation system for Sparse Mixture-of-Experts (MoE) inference. Rather than storing and loading…
cs.CV2025
MamFusion: Multi-Mamba with Temporal Fusion for Partially Relevant Video Retrieval
Xinru Ying, Jiaqi Mo, Jingyang Lin +3
Partially Relevant Video Retrieval (PRVR) is a challenging task in the domain of multimedia retrieval. It is designed to identify and retrieve untrimmed videos that are partially r…