3 citations · 10 across the 7 of their papers we have counts for
3 papers · 1 filter
Attention Is All You Need But You Don't Need All Of It For Inference of Large Language Models
Georgy Tyukin, Gbetondji J-S Dovonon, Jean Kaddour +1
The inference demand for LLMs has skyrocketed in recent months, and serving models with low latencies remains challenging due to the quadratic input length complexity of the attent…
Setting the Record Straight on Transformer Oversmoothing
Gbètondji J-S Dovonon, Michael M. Bronstein, Matt J. Kusner
Transformer-based models have recently become wildly successful across a diverse set of domains. At the same time, recent work has shown empirically and theoretically that Transfor…
Long-run Behaviour of Multi-fidelity Bayesian Optimisation
Gbetondji J-S Dovonon, Jakob Zeitler
Multi-fidelity Bayesian Optimisation (MFBO) has been shown to generally converge faster than single-fidelity Bayesian Optimisation (SFBO) (Poloczek et al. (2017)). Inspired by rece…