Closing the Curvature Gap: Full Transformer Hessians

Egor Petrov, Nikita Kiselev, Vladislav Meshkov +1

The optimization landscape of Transformer models remains poorly understood despite their widespread adoption. While recent studies have derived curvature properties for isolated se…