108 citations · 323 across the 11 of their papers we have counts for
10 papers · 1 filter
Set Block Decoding is a Language Model Inference Accelerator
Itai Gat, Heli Ben-Hamu, Marton Havasi +6
Autoregressive next token prediction language models offer powerful capabilities but face significant challenges in practical deployment due to the high computational and memory co…
Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking
Heli Ben-Hamu, Itai Gat, Daniel Severo +2
Recent masked diffusion models (MDMs) have shown competitive performance compared to autoregressive models (ARMs) for language modeling. While most literature has focused on perfor…
Flow Matching Guide and Code
Yaron Lipman, Marton Havasi, Peter Holderrieth +7
Flow Matching (FM) is a recent framework for generative modeling that has achieved state-of-the-art performance across various domains, including image, video, audio, speech, and b…
D-Flow: Differentiating through Flows for Controlled Generation
Heli Ben-Hamu, Omri Puny, Itai Gat +3
Taming the generation outcome of state of the art Diffusion and Flow-Matching (FM) models without having to re-train a task-specific model unlocks a powerful tool for solving inver…
Multisample Flow Matching: Straightening Flows with Minibatch Couplings
Aram-Alexandre Pooladian, Heli Ben-Hamu, Carles Domingo-Enrich +3
Simulation-free methods for training continuous-time generative models construct probability paths that go between noise distributions and individual data samples. Recent works, su…
Flow Matching for Generative Modeling
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu +2
We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the not…
Frame Averaging for Invariant and Equivariant Network Design
Omri Puny, Matan Atzmon, Heli Ben-Hamu +4
Many machine learning tasks involve learning functions that are known to be invariant or equivariant to certain symmetries of the input data. However, it is often challenging to de…
Global Attention Improves Graph Networks Generalization
Omri Puny, Heli Ben-Hamu, Yaron Lipman
This paper advocates incorporating a Low-Rank Global Attention (LRGA) module, a computation and memory efficient variant of the dot-product attention (Vaswani et al., 2017), to Gra…
Provably Powerful Graph Networks
Haggai Maron, Heli Ben-Hamu, Hadar Serviansky +1
Recently, the Weisfeiler-Lehman (WL) graph isomorphism test was used to measure the expressive power of graph neural networks (GNN). It was shown that the popular message passing G…
Invariant and Equivariant Graph Networks
Haggai Maron, Heli Ben-Hamu, Nadav Shamir +1
Invariant and equivariant networks have been successfully used for learning images, sets, point clouds, and graphs. A basic challenge in developing such networks is finding the max…