13 papers
Long-Context Drafting Does Not Require Long-Context Drafters
Aref Jafari, Vansh Bhatia, Matt Qin +5
Speculative decoding pairs a cheap drafter with parallel verification against the target. At ultra-long context it stops paying, because a drafter that attends over the full prefix…
Zebra3D: Efficient Hybrid Foundation Models For Scalable 3D Reconstruction
Utkarsh Saxena, Ruijing Yang, Mehdi Rezagholizadeh +3
Feed-forward 3D foundation models reconstruct scenes from unposed images, but the quadratic cost of global attention limits their scalability to large view sets. We present Zebra3D…
CORAL-KV: Compact Cross-Layer Prediction and Residual Coding for Autoregressive Video KV Caches
Albert Ge, Akash Haridas, Utkarsh Saxena +5
Autoregressive video diffusion models maintain a growing attention key-value (KV) cache, making memory a key bottleneck for long video generation. Recently, clustering-based quanti…
Instella-MoE Technical Report
Jiang Liu, Sudhanshu Ranjan, Prakamya Mishra +10
In this work, we introduce Instella-MoE, a fully open Mixture-of-Experts (MoE) language model with 16 billion total parameters and 2.8 billion active parameters per token, trained…
Rendering-Aware Bayesian 3D Gaussian Splatting with Native Uncertainty and Adaptive Complexity Control
Gaoxiang Jia, Vikram Appia, Junzhou Huang +1
3D Gaussian splatting (3DGS) is a strong representation for real-time novel-view synthesis, but its standard training pipeline relies on point estimates and hand-tuned heuristics,…
AgentKernelArena: Generalization-Aware Benchmarking of GPU Kernel Optimization Agents
Sharareh Younesian, Wenwen Ouyang, Sina Rafati +11
GPU kernel optimization is increasingly critical for efficient deep learning systems, but writing high-performance kernels still requires substantial low-level expertise. Recent AI…
CASCADE: Context-Aware Relaxation for Speculative Image Decoding
Selin Yildirim, Subhajit Dutta Chowdhury, Mohammad Mahdi Kamani +2
Autoregressive generation is a powerful approach for high-fidelity image synthesis, but it remains computationally demanding and slow even on the most advanced accelerators. While…
Long-Context Aware Upcycling: A New Frontier for Hybrid LLM Scaling
Parsa Ashrafi Fashi, Utkarsh Saxena, Mehdi Rezagholizadeh +7
Hybrid sequence models that combine efficient Transformer components with linear sequence modeling blocks are a promising alternative to pure Transformers, but most are still pretr…
DC-DiT: Adaptive Compute and Elastic Inference for Visual Generation via Dynamic Chunking
Akash Haridas, Utkarsh Saxena, Parsa Ashrafi Fashi +3
Diffusion Transformers rely on static patchify tokenization, assigning the same token budget to smooth backgrounds, detailed object regions, noisy early timesteps, and late-stage r…
Verifier Threshold: An Efficient Test-Time Scaling Approach for Image Generation
Vignesh Sundaresha, Akash Haridas, Vikram Appia +1
Image generation has emerged as a mainstream application of large generative models. Just as test-time compute and reasoning have improved language model capabilities, similar bene…
FarSkip-Collective: Unhobbling Blocking Communication in Mixture of Experts Models
Yonatan Dukler, Guihong Li, Deval Shah +3
Blocking communication presents a major hurdle in running MoEs efficiently in distributed settings. To address this, we present FarSkip-Collective which modifies the architecture o…
Zebra-Llama: Towards Extremely Efficient Hybrid Models
Mingyu Yang, Mehdi Rezagholizadeh, Guihong Li +2
With the growing demand for deploying large language models (LLMs) across diverse applications, improving their inference efficiency is crucial for sustainable and democratized acc…
X-EcoMLA: Upcycling Pre-Trained Attention into MLA for Efficient and Extreme KV Compression
Guihong Li, Mehdi Rezagholizadeh, Mingyu Yang +2
Multi-head latent attention (MLA) is designed to optimize KV cache memory through low-rank key-value joint compression. Rather than caching keys and values separately, MLA stores t…