14 citations · 43 across the 11 of their papers we have counts for
4 papers · 1 filter
A Causal World Model Underlying Next Token Prediction: Exploring GPT in a Controlled Environment
Raanan Y. Rohekar, Yaniv Gurwicz, Sungduk Yu +2
Are generative pre-trained transformer (GPT) models, trained only to predict the next token, implicitly learning a world model from which sequences are generated one token at a tim…
Causal Interpretation of Self-Attention in Pre-Trained Transformers
Raanan Y. Rohekar, Yaniv Gurwicz, Shami Nisimov
We propose a causal interpretation of self-attention in the Transformer neural network architecture. We interpret self-attention as a mechanism that estimates a structural equation…
From Temporal to Contemporaneous Iterative Causal Discovery in the Presence of Latent Confounders
Raanan Y. Rohekar, Shami Nisimov, Yaniv Gurwicz +1
We present a constraint-based algorithm for learning causal structures from observational time-series data, in the presence of latent confounders. We assume a discrete-time, statio…
A Single Iterative Step for Anytime Causal Discovery
Raanan Y. Rohekar, Yaniv Gurwicz, Shami Nisimov +1
We present a sound and complete algorithm for recovering causal graphs from observed, non-interventional data, in the possible presence of latent confounders and selection bias. We…