6 papers
The Query Knows What to Forget: A Second Erase Direction for Linear Attention
Dhruman Gupta, Aritra Das, Debayan Gupta
Linear attention keeps a state of fixed size. At long context, many stored items share this state, and interference between them degrades retrieval. Gated DeltaNet-2 (GDN-2), like…
Linearized 2-Simplicial Attention
Aritra Das, Dhruman Gupta, Debayan Gupta
We present a linearized form of 2-simplicial attention by rewriting the trilinear score as an inner product between a composite query and a key, so that the sum over one token axis…
AdaWeather: Adaptively Mixing Probabilistic Weather Forecasts with Logarithmic Regret
Saptarishi Dhanuka, Sarvesh Iyer, Manmeet Singh +5
Recent advances in machine learning have produced probabilistic weather forecasting models comparable to state-of-the-art numerical weather predictors. But no model consistently do…
Generating DDPM-based Samples from Tilted Distributions
Himadri Mandal, Dhruman Gupta, Rushil Gupta +5
Given independent samples from a -dimensional probability distribution, our aim is to generate diffusion-based samples from a distribution obtained by tilting the original,…
Joint 3D Gravity and Magnetic Inversion via Rectified Flow and Ginzburg-Landau Guidance
Dhruman Gupta, Yashas Shende, Aritra Das +2
Subsurface ore detection is of paramount importance given the rising depletion of shallow mineral resources in recent years. It is crucial to explore approaches that go beyond the…
Fundamental limits for weighted empirical approximations of tilted distributions
Sarvesh Ravichandran Iyer, Himadri Mandal, Dhruman Gupta +5
Consider the task of generating samples from a tilted distribution of a random vector whose underlying distribution is unknown, but samples from it are available. This finds applic…