5 papers
StreamIndex: Memory-Bounded Compressed Sparse Attention via Streaming Top-k
Jaber Jaber, Osama Jaber
DeepSeek-V3.2 and V4 introduce Compressed Sparse Attention (CSA): a lightning indexer (a learned scoring projection over compressed keys) scores them, the top-k are selected per qu…
Ouroboros: Dynamic Weight Generation for Recursive Transformers via Input-Conditioned LoRA Modulation
Jaber Jaber, Osama Jaber
Recursive transformers reuse a shared weight block across multiple depth steps, trading parameters for compute. A core limitation: every step applies the same transformation, preve…
HCLSM: Hierarchical Causal Latent State Machines for Object-Centric World Modeling
Jaber Jaber, Osama Jaber
World models that predict future states from video remain limited by flat latent representations that entangle objects, ignore causal structure, and collapse temporal dynamics into…
TIDE: Token-Informed Depth Execution for Per-Token Early Exit in LLM Inference
Jaber Jaber, Osama Jaber
Large language models run every token through every layer, regardless of difficulty. We present TIDE, a post-training system that attaches tiny learned routers at periodic checkpoi…
AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search
Jaber Jaber, Osama Jaber
Writing high-performance GPU kernels is among the most labor-intensive tasks in machine learning systems engineering. We present AutoKernel, an open-source framework that applies a…