33 citations · 72 across the 4 of their papers we have counts for
4 papers
Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems
David T. Hoffmann, Simon Schrodi, Jelena Bratulić +3
In this work, we study rapid improvements of the training loss in transformers when being confronted with multi-step decision tasks. We found that transformers struggle to learn th…
Hybrid Learning of Optical Flow and Next Frame Prediction to Boost Optical Flow in the Wild
Nima Sedaghat, Mohammadreza Zolfaghari, Thomas Brox
CNN-based optical flow estimation has attracted attention recently, mainly due to its impressively high frame rates. These networks perform well on synthetic datasets, but they are…
Learning to Generate Chairs, Tables and Cars with Convolutional Networks
Alexey Dosovitskiy, Jost Tobias Springenberg, Maxim Tatarchenko +1
We train generative 'up-convolutional' neural networks which are able to generate images of objects given object style, viewpoint, and color. We train the networks on rendered 3D m…
Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks
Alexey Dosovitskiy, Philipp Fischer, Jost Tobias Springenberg +2
Deep convolutional networks have proven to be very successful in learning task specific features that allow for unprecedented performance on various computer vision tasks. Training…