6 citations · 13 across the 4 of their papers we have counts for
5 papers
Pretraining Latent Information Feedback Transformers with Teacher Supervision
Dor Tirosh, Ido Amos, Mor Geva
Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for information to flow downward across…
Latent Reasoning with Supervised Thinking States
Ido Amos, Avi Caciularu, Mor Geva +4
Reasoning with a chain-of-thought (CoT) enables Large Language Models (LLMs) to solve complex tasks but incurs significant inference costs due to the generation of long rationales.…
MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language
Yoel Shoshan, Moshiko Raboh, Michal Ozery-Flato +18
Large language models applied to vast biological datasets have the potential to transform biology by uncovering disease mechanisms and accelerating drug development. However, curre…
Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors
Ido Amos, Jonathan Berant, Ankit Gupta
Modeling long-range dependencies across sequences is a longstanding goal in machine learning and has led to architectures, such as state space models, that dramatically outperform…
Graph Neural Networks Use Graphs When They Shouldn't
Maya Bechler-Speicher, Ido Amos, Ran Gilad-Bachrach +1
Predictions over graphs play a crucial role in various domains, including social networks and medicine. Graph Neural Networks (GNNs) have emerged as the dominant approach for learn…