2 citations · 2 across the 11 of their papers we have counts for
4 papers · 1 filter
Transformers are Universal Predictors
Sourya Basu, Moulik Choraria, Lav R. Varshney
We find limits to the Transformer architecture for language modeling and show it has a universal prediction property in an information-theoretic sense. We further analyze performan…
Learning Optimal Features via Partial Invariance
Moulik Choraria, Ibtihal Ferwana, Ankur Mani +1
Learning models that are robust to distribution shifts is a key concern in the context of their real-life applicability. Invariant Risk Minimization (IRM) is a popular framework th…
The Spectral Bias of Polynomial Neural Networks
Moulik Choraria, Leello Tadesse Dadi, Grigorios Chrysos +2
Polynomial neural networks (PNNs) have been recently shown to be particularly effective at image generation and face recognition, where high-frequency information is critical. Prev…
Balancing Fairness and Robustness via Partial Invariance
Moulik Choraria, Ibtihal Ferwana, Ankur Mani +1
The Invariant Risk Minimization (IRM) framework aims to learn invariant features from a set of environments for solving the out-of-distribution (OOD) generalization problem. The un…