6 papers
Modality-Aware Out-of-Distribution Detection for Multi-Modal Action Recognition
Lars Doorenbos, Duc Manh Vu, Serdar Ozsoy +1
The incorporation of additional modalities into action recognition models increases their performance across a wide range of settings. However, how this additional information can…
The Unreasonable Effectiveness of VLMs for Zero-shot Procedural Mistake Detection
Serdar Ozsoy, Lars Doorenbos, Federico Spurio +2
Procedural mistake detection is important for quality control and user assistance across many disciplines. Recent work in this field has achieved significant gains by using the rea…
Constrained hybrid modelling to predict microbial dynamics and organic matter turnover in soil systems
Paul Collart, Juergen Gall, Andrea Schnepf +2
Soil microorganisms control organic matter cycling and largely determine how soil systems can cope with and mitigate climate change and environmental threats. Representing microbia…
TRAFA: Anticipating User Actions to Reduce Errors in Procedural Tasks with Predictive Feedback
Sassan Mokhtar, Lars Doorenbos, Fatemeh Jabbari +3
Interactive assistance systems typically provide feedback after an action has been completed, supporting error recovery but not preventing the error itself. We present TRAFA, a rea…
Video Panels for Long Video Understanding
Lars Doorenbos, Federico Spurio, Juergen Gall
Recent Video-Language Models (VLMs) achieve promising results on long-video understanding, but their performance still lags behind that achieved on tasks involving images or short…
EgoControl: Controllable Egocentric Video Generation via 3D Full-Body Poses
Enrico Pallotta, Sina Mokhtarzadeh Azar, Lars Doorenbos +3
Egocentric video generation with fine-grained control through body motion is a key requirement towards embodied AI agents that can simulate, predict, and plan actions. In this work…