M. Rabe

activity

20102023

most citedThe promising future of a robust cosmological neutrino mass measurement

118 citations · 344 across the 22 of their papers we have counts for

collaborators

Showing 2021Show all

3 papers · 1 filter

cs.LG2021★ 18 cited

Self-attention Does Not Need $O (n^{2})$ Memory

Markus N. Rabe, Charles Staats

We present a very simple algorithm for attention that requires $O (1)$ memory with respect to sequence length and an extension to self-attention that requires $O (lo g n)$ memory. Th…

cs.LG2021★ 1 cited

Neural Circuit Synthesis from Specification Patterns

Frederik Schmitt, Christopher Hahn, Markus N. Rabe +1

We train hierarchical Transformers on the task of synthesizing hardware circuits directly out of high-level logical specifications in linear-time temporal logic (LTL). The LTL synt…

cs.LG2021★ 9 cited

LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning

Yuhuai Wu, Markus Rabe, Wenda Li +3

While designing inductive bias in neural architectures has been widely studied, we hypothesize that transformer networks are flexible enough to learn inductive bias from suitable g…

Self-attention Does Not Need O(n2) Memory

Neural Circuit Synthesis from Specification Patterns

LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning

Self-attention Does Not Need $O (n^{2})$ Memory