Reference-free Human-Object Interaction Editing

Abstract

This paper presents InteractEdit, a novel framework for reference-free Human-Object Interaction (HOI) editing that tackles the challenging task of transforming an existing interaction in an image into a new, desired interaction while preserving the identities of the subject and object. Unlike prior image editing tasks such as attribute manipulation, object replacement or style transfer, HOI editing involves complex spatial, contextual, and relational dependencies inherent in HOI. Existing methods often overfit to the source image structure, limiting adaptability to the substantial structural modifications demanded by the new interactions. To address this, InteractEdit disassembles each scene into subject, object, and background components to disentangle intricate HOI relationship, and introduces a selective inversion strategy combined with Selective-Rank Adaptation (SeRA) to leverage pretrained interaction priors while learning visual identity from the source image. This enables a balanced trade-off between interaction editing and identity preservation. We also introduce IEBench, a new benchmark for HOI editing, and a new metric that jointly evaluates the trade-off between successful interaction editing and identity preservation. Extensive experiments show that InteractEdit outperforms 23 existing methods, providing a strong baseline for future HOI editing research. Code and the dataset: https://jiuntian.github.io/InteractEdit/.

Website: https://jiuntian.github.io/interactedit, Paper: https://www.sciencedirect.com/science/article/abs/pii/S0925231226022678