5 papers · 1 filter
Executing Causal Structure Learning with Linear-Attention Transformers
Amartya Roy, Sayar Karmakar
Transformers can execute algorithms on data given in their input. We ask whether they can do the same for causal discovery. We study a standard continuous method that repeatedly up…
Training-Loss Guarantees for Muon with Finite-Step Newton--Schulz Orthogonalization
Amartya Roy, Souvik Chakraborty
Existing convergence analyses of Muon either assume exact orthogonalization or analyze classical Newton--Schulz polynomials, and guarantee only stationarity, so it is unresolved wh…
See, Hypothesize, Validate: Multimodal Agentic Framework for Discovering Governing PDEs
Sarang Manoj Pekhale, Amartya Roy, Rajat Sarkar +1
Discovering governing partial differential equations (PDEs) from observational data remains a core challenge across the sciences. Existing sparse-regression, symbolic-regression, a…
Why LLMs Fail at Causal Discovery and How Interventional Agents Escape
Amartya Roy, Sonali Parbhoo
Causal discovery is a cornerstone of scientific reasoning, yet whether large language models can perform it reliably remains an open question. Recent benchmarks show that even fine…
The -Combinator for LLMs: Solving Long-Context Rot with -Calculus
Amartya Roy, Rasul Tutunov, Xiaotong Ji +2
LLMs are increasingly used as general-purpose reasoners, but long inputs remain bottlenecked by a fixed context window. Recursive Language Models (RLMs) address this by externalisi…