57 citations · 143 across the 11 of their papers we have counts for
12 papers
TinyGSM: achieving >80% on GSM8k with small language models
Bingbin Liu, Sebastien Bubeck, Ronen Eldan +5
Small-scale models offer various computational advantages, and yet to which extent size is critical for problem-solving abilities remains an open question. Specifically for solving…
Positional Description Matters for Transformers Arithmetic
Ruoqi Shen, Sébastien Bubeck, Ronen Eldan +3
Transformers, central to the successes in modern Natural Language Processing, often falter on arithmetic tasks despite their vast capabilities --which paradoxically include remarka…
A law of robustness for two-layers neural networks
Sébastien Bubeck, Yuanzhi Li, Dheeraj Nagaraj
We initiate the study of the inherent tradeoffs between the size of a neural network and its robustness, as measured by its Lipschitz constant. We make a precise conjecture that, f…
Complexity of Highly Parallel Non-Smooth Convex Optimization
Sébastien Bubeck, Qijia Jiang, Yin Tat Lee +2
A landmark result of non-smooth convex optimization is that gradient descent is an optimal algorithm whenever the number of computed gradients is smaller than the dimension . In…
Non-Stochastic Multi-Player Multi-Armed Bandits: Optimal Rate With Collision Information, Sublinear Without
Sébastien Bubeck, Yuanzhi Li, Yuval Peres +1
We consider the non-stochastic version of the (cooperative) multi-player multi-armed bandit problem. The model assumes no communication at all between the players, and furthermore…
Improved Path-length Regret Bounds for Bandits
Sébastien Bubeck, Yuanzhi Li, Haipeng Luo +1
We study adaptive regret bounds in terms of the variation of the losses (the so-called path-length bounds) for both multi-armed bandit and more generally linear bandit. We first sh…
Near-optimal method for highly smooth convex optimization
Sébastien Bubeck, Qijia Jiang, Yin Tat Lee +2
We propose a near-optimal method for highly smooth convex optimization. More precisely, in the oracle model where one obtains the order Taylor expansion of a function at t…
Competitively Chasing Convex Bodies
Sébastien Bubeck, Yin Tat Lee, Yuanzhi Li +1
Let be a family of sets in some metric space. In the -chasing problem, an online algorithm observes a request sequence of sets in and respo…
Chasing Nested Convex Bodies Nearly Optimally
Sébastien Bubeck, Bo'az Klartag, Yin Tat Lee +2
The convex body chasing problem, introduced by Friedman and Linial, is a competitive analysis problem on any normed vector space. In convex body chasing, for each timestep $t\in\ma…
Make the Minority Great Again: First-Order Regret Bound for Contextual Bandits
Zeyuan Allen-Zhu, Sébastien Bubeck, Yuanzhi Li
Regret bounds in online learning compare the player's performance to , the optimal performance in hindsight with a fixed strategy. Typically such bounds scale with the square…
Sparsity, variance and curvature in multi-armed bandits
Sébastien Bubeck, Michael B. Cohen, Yuanzhi Li
In (online) learning theory the concepts of sparsity, variance and curvature are well-understood and are routinely used to obtain refined regret and generalization bounds. In this…
An homotopy method for regression provably beyond self-concordance and in input-sparsity time
Sébastien Bubeck, Michael B. Cohen, Yin Tat Lee +1
We consider the problem of linear regression where the norm loss (i.e., the usual least squares loss) is replaced by the norm. We show how to solve such probl…