10 papers
Executing Causal Structure Learning with Linear-Attention Transformers
Amartya Roy, Sayar Karmakar
Transformers can execute algorithms on data given in their input. We ask whether they can do the same for causal discovery. We study a standard continuous method that repeatedly up…
Training-Loss Guarantees for Muon with Finite-Step Newton--Schulz Orthogonalization
Amartya Roy, Souvik Chakraborty
Existing convergence analyses of Muon either assume exact orthogonalization or analyze classical Newton--Schulz polynomials, and guarantee only stationarity, so it is unresolved wh…
See, Hypothesize, Validate: Multimodal Agentic Framework for Discovering Governing PDEs
Sarang Manoj Pekhale, Amartya Roy, Rajat Sarkar +1
Discovering governing partial differential equations (PDEs) from observational data remains a core challenge across the sciences. Existing sparse-regression, symbolic-regression, a…
Why LLMs Fail at Causal Discovery and How Interventional Agents Escape
Amartya Roy, Sonali Parbhoo
Causal discovery is a cornerstone of scientific reasoning, yet whether large language models can perform it reliably remains an open question. Recent benchmarks show that even fine…
The -Combinator for LLMs: Solving Long-Context Rot with -Calculus
Amartya Roy, Rasul Tutunov, Xiaotong Ji +2
LLMs are increasingly used as general-purpose reasoners, but long inputs remain bottlenecked by a fixed context window. Recursive Language Models (RLMs) address this by externalisi…
Causal Reasoning Favors Encoders: On The Limits of Decoder-Only Models
Amartya Roy, Elamparithy M, Kripabandhu Ghosh +2
In context learning (ICL) underpins recent advances in large language models (LLMs), although its role and performance in causal reasoning remains unclear. Causal reasoning demands…
Competition is the key: A Game Theoretic Causal Discovery Approach
Amartya Roy, Souvik Chakraborty
Causal discovery remains a central challenge in machine learning, yet existing methods face a fundamental gap: algorithms like GES and GraN-DAG achieve strong empirical performance…
Guide: Generalized-Prior and Data Encoders for DAG Estimation
Amartya Roy, Devharish N, Shreya Ganguly +1
Modern causal discovery methods face critical limitations in scalability, computational efficiency, and adaptability to mixed data types, as evidenced by benchmarks on node scalabi…
On the effective transfer of knowledge from English to Hindi Wikipedia
Paramita Das, Amartya Roy, Ritabrata Chakraborty +1
Although Wikipedia is the largest multilingual encyclopedia, it remains inherently incomplete. There is a significant disparity in the quality of content between high-resource lang…
Do the Right Thing, Just Debias! Multi-Category Bias Mitigation Using LLMs
Amartya Roy, Danush Khanna, Devanshu Mahapatra +3
This paper tackles the challenge of building robust and generalizable bias mitigation models for language. Recognizing the limitations of existing datasets, we introduce ANUBIS, a…