Tactile Modality Fusion for Vision-Language-Action Models
arXiv:2603.14604
The paper introduces TacFiLM, a lightweight method that fuses tactile data with visual features in vision‑language‑action models to improve robot manipulation tasks that involve contact. By conditioning visual features on pretrained tactile representations using FiLM, the approach boosts success rates and stability without heavy computational overhead.
Abstract
We propose TacFiLM, a lightweight modality-fusion approach that integrates visual-tactile signals into vision-language-action (VLA) models. While advances in VLAs have introduced robot policies that are both generalizable and semantically grounded, these models mainly rely on vision-based perception. Vision alone, however, cannot capture the complex interaction dynamics that occur during contact-rich manipulation, including contact forces, surface friction, compliance, and shear. While recent attempts to integrate tactile signals into VLA models often increase complexity through token concatenation or large-scale pretraining, the heavy computational demands of behaviour models necessitate lightweight fusion strategies. To address these challenges, TacFiLM outlines a post-training finetuning approach that conditions intermediate visual features on pretrained tactile representations using feature-wise linear modulation (FiLM). Experimental results on insertion and drawer opening tasks demonstrate consistent improvements in success rate, direct task performance, completion time, and force stability across both in-distribution and out-of-distribution tasks. Together, these results support our method as an effective approach to integrating tactile signals into VLA models, improving contact-rich manipulation behaviours. Project page: https://charliem7.github.io/projects/TacFilm/
Accepted to the European Conference on Computer Vision (ECCV), 2026, 20 pages, 5 figures