13 citations · 46 across the 21 of their papers we have counts for
5 papers · 1 filter
Low-Rank Correction for Quantized LLMs
Meyer Scetbon, James Hensman
We consider the problem of model compression for Large Language Models (LLMs) at post-training time, where the task is to compress a well-trained model using only a small set of ca…
SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training
Chao Ma, Wenbo Gong, Meyer Scetbon +1
Adaptive optimizers such as Adam (Kingma & Ba, 2015) have been central to the success of large language models. However, they often require to maintain optimizer states throughout…
Amortized Inference of Causal Models via Conditional Fixed-Point Iterations
Divyat Mahajan, Jannes Gladrow, Agrin Hilmkil +2
Structural Causal Models (SCMs) offer a principled framework to reason about interventions and support out-of-distribution generalization, which are key goals in scientific discove…
A Fixed-Point Approach for Causal Generative Modeling
Meyer Scetbon, Joel Jennings, Agrin Hilmkil +2
We propose a novel formalism for describing Structural Causal Models (SCMs) as fixed-point problems on causally ordered variables, eliminating the need for Directed Acyclic Graphs…
The Essential Role of Causality in Foundation World Models for Embodied AI
Tarun Gupta, Wenbo Gong, Chao Ma +11
Recent advances in foundation models, especially in large multi-modal models and conversational agents, have ignited interest in the potential of generally capable embodied agents.…