14 citations · 32 across the 21 of their papers we have counts for
22 papers
Orbis 2: A Hierarchical World Model for Driving
Sudhanshu Mittal, Arian Mousakhan, Silvio Galesso +4
Current world models operate at a single level of abstraction, with most prioritizing perceptual fidelity while lacking the spatial reasoning and semantic understanding required fo…
The Surprising Effectiveness of Canonical Knowledge Distillation for Semantic Segmentation
Muhammad Ali, Kevin Alexander Laube, Madan Ravi Ganesh +3
Recent knowledge distillation (KD) methods for semantic segmentation introduce increasingly complex hand-crafted objectives, yet are typically evaluated under fixed iteration sched…
Simple LLM Baselines are Competitive for Model Diffing
Elias Kempf, Simon Schrodi, Bartosz Cywiński +3
Standard LLM evaluations only test capabilities or dispositions that evaluators designed them for, missing unexpected differences such as behavioral shifts between model revisions…
What Drives Compositional Generalization in Visual Generative Models? The Importance of Continuous Training Objectives
Karim Farid, Rajat Sahay, Yumna Ali Alnaggar +4
Compositional generalization, the ability to generate novel combinations of known concepts, is a key ingredient for visual generative models. Yet, not all mechanisms that enable or…
Towards Understanding Subliminal Learning: When and How Hidden Biases Transfer
Simon Schrodi, Elias Kempf, Fazl Barez +1
Language models can transfer hidden biases during distillation. For example, a teacher that "likes owls" can make its student "like owls" too, even when the training data consists…
Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Models
Arian Mousakhan, Sudhanshu Mittal, Silvio Galesso +2
Existing world models for autonomous driving struggle with long-horizon generation and generalization to challenging scenarios. In this work, we develop a model using simple design…
cVLA: Towards Efficient Camera-Space VLAs
Max Argus, Jelena Bratulic, Houman Masnavi +4
Vision-Language-Action (VLA) models offer a compelling framework for tackling complex robotic manipulation tasks, but they are often expensive to train. In this paper, we propose a…
When and How Does CLIP Enable Domain and Compositional Generalization?
Elias Kempf, Simon Schrodi, Max Argus +1
The remarkable generalization performance of contrastive vision-language models like CLIP is often attributed to the diversity of their training distributions. However, key questio…
Learning Robotic Manipulation Policies from Point Clouds with Conditional Flow Matching
Eugenio Chisari, Nick Heppert, Max Argus +3
Learning from expert demonstrations is a promising approach for training robotic manipulation policies from limited data. However, imitation learning algorithms require a number of…
Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning
Soumajyoti Sarkar, Leonard Lausen, Volkan Cevher +3
Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. These models use conditionally activated feedforward subnetworks…
Concept Bottleneck Models Without Predefined Concepts
Simon Schrodi, Julian Schur, Max Argus +1
There has been considerable recent interest in interpretable concept-based models such as Concept Bottleneck Models (CBMs), which first predict human-interpretable concepts and the…
Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models
Simon Schrodi, David T. Hoffmann, Max Argus +2
Contrastive vision-language models (VLMs), like CLIP, have gained popularity for their versatile applicability to various downstream tasks. Despite their successes in some tasks, l…
Constrained Reinforcement Learning with Smoothed Log Barrier Function
Baohe Zhang, Yuan Zhang, Lilli Frison +2
Reinforcement Learning (RL) has been widely applied to many control tasks and substantially improved the performances compared to conventional control methods in many domains where…
DITTO: Demonstration Imitation by Trajectory Transformation
Nick Heppert, Max Argus, Tim Welschehold +2
Teaching robots new skills quickly and conveniently is crucial for the broader adoption of robotic systems. In this work, we address the problem of one-shot imitation from a single…
Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy
Simon Ging, María A. Bravo, Thomas Brox
The evaluation of text-generative vision-language models is a challenging yet crucial endeavor. By addressing the limitations of existing Visual Question Answering (VQA) benchmarks…
Is Mamba Capable of In-Context Learning?
Riccardo Grazzi, Julien Siems, Simon Schrodi +2
State of the art foundation models such as GPT-4 perform surprisingly well at in-context learning (ICL), a variant of meta-learning concerning the learned ability to solve tasks du…
Amodal Optical Flow
Maximilian Luz, Rohit Mohan, Ahmed Rida Sekkat +4
Optical flow estimation is very challenging in situations with transparent or occluded objects. In this work, we address these challenges at the task level by introducing Amodal Op…
Latent Diffusion Counterfactual Explanations
Karim Farid, Simon Schrodi, Max Argus +1
Counterfactual explanations have emerged as a promising method for elucidating the behavior of opaque black-box models. Recently, several works leveraged pixel-space diffusion mode…
Climate-sensitive Urban Planning through Optimization of Tree Placements
Simon Schrodi, Ferdinand Briegel, Max Argus +2
Climate change is increasing the intensity and frequency of many extreme weather events, including heatwaves, which results in increased thermal discomfort and mortality rates. Whi…
Compositional Servoing by Recombining Demonstrations
Max Argus, Abhijeet Nayak, Martin Büchner +3
Learning-based manipulation policies from image inputs often show weak task transfer capabilities. In contrast, visual servoing methods allow efficient task transfer in high-precis…
Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems
David T. Hoffmann, Simon Schrodi, Jelena Bratulić +3
In this work, we study rapid improvements of the training loss in transformers when being confronted with multi-step decision tasks. We found that transformers struggle to learn th…
Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detection
Silvio Galesso, Max Argus, Thomas Brox
The key to out-of-distribution detection is density estimation of the in-distribution data or of its feature representations. This is particularly challenging for dense anomaly det…