60 citations · 60 across the 11 of their papers we have counts for
11 papers
Drifting Fields are not Conservative
Leonard T. Franz, Sebastian Hoffmann, Tim Weiland +2
Drifting models have recently gained attention for generating high-quality samples in a single forward pass. During training, they learn a push-forward map by following a vector-va…
Hyperbolic Busemann Neural Networks
Ziheng Chen, Bernhard Schölkopf, Nicu Sebe
Hyperbolic spaces provide a natural geometry for representing hierarchical and tree-structured data due to their exponential volume growth. To leverage these benefits, neural netwo…
Scaling Behavior of Discrete Diffusion Language Models
Dimitri von Rütte, Janis Fluri, Omead Pooladzandi +3
Modern LLM pre-training consumes vast amounts of compute and training data, making the scaling behavior, or scaling laws, of different models a key distinguishing factor. Discrete…
Intrinsically Interpretable Attention via Sparse Post-Training
Florent Draye, Anson Lei, Hsiao-Ru Pan +2
We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance. Applying a flexible sparsity regularisation under a constrained-…
Riemannian Batch Normalization: A Gyro Approach
Ziheng Chen, Xiao-Jun Wu, Bernhard Schölkopf +1
Normalization layers are crucial for deep learning, but their Euclidean formulations are inadequate for data on manifolds. On the other hand, many Riemannian manifolds in machine l…
Generalized Interpolating Discrete Diffusion
Dimitri von Rütte, Janis Fluri, Yuhui Ding +3
While state-of-the-art language models achieve impressive results through next-token prediction, they have inherent limitations such as the inability to revise already generated to…
DiffRatio: Training One-Step Diffusion Models Without Teacher Supervision
Wenlin Chen, Mingtian Zhang, Jiajun He +4
Score-based distillation methods (e.g., variational score distillation) train one-step diffusion models by first pre-training a teacher score model and then distilling it into a on…
Landscaping Linear Mode Connectivity
Sidak Pal Singh, Linara Adilova, Michael Kamp +3
The presence of linear paths in parameter space between two different network solutions in certain cases, i.e., linear mode connectivity (LMC), has garnered interest from both theo…
Hallmarks of Optimization Trajectories in Neural Networks: Directional Exploration and Redundancy
Sidak Pal Singh, Bobby He, Thomas Hofmann +1
We propose a fresh take on understanding the mechanisms of neural networks by analyzing the rich directional structure of optimization trajectories, represented by their pointwise…
RAVEN: Rethinking Adversarial Video Generation with Efficient Tri-plane Networks
Partha Ghosh, Soubhik Sanyal, Cordelia Schmid +1
We present a novel unconditional video generative model designed to address long-term spatial and temporal dependencies, with attention to computational and dataset efficiency. To…
First-order Adversarial Vulnerability of Neural Networks and Input Dimension
Carl-Johann Simon-Gabriel, Yann Ollivier, Léon Bottou +2
Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We…