5 papers · 1 filter
FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation
Guangyu Sun, Shlok Kumar Mishra, Wentao Bao +8
Traditional multimodal representation learning and generation are two stages: a contrastive or self-supervised visual encoder is trained first, followed by a separate downstream ge…
BLENDER: Blended Text Embeddings and Diffusion Residuals for Intra-Class Image Synthesis in Deep Metric Learning
Jan Niklas Kolf, Ozan Tezcan, Justin Theiss +7
The rise of Deep Generative Models (DGM) has enabled the generation of high-quality synthetic data. When used to augment authentic data in Deep Metric Learning (DML), these synthet…
Uni-Hand: Universal Hand Motion Forecasting in Egocentric Views
Junyi Ma, Wentao Bao, Jingyi Xu +5
Forecasting how human hands move in egocentric views is critical for applications like augmented reality and human-robot policy transfer. Recently, several hand trajectory predicti…
Novel Diffusion Models for Multimodal 3D Hand Trajectory Prediction
Junyi Ma, Wentao Bao, Jingyi Xu +3
Predicting hand motion is critical for understanding human intentions and bridging the action space between human movements and robot manipulations. Existing hand trajectory predic…
MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos
Junyi Ma, Xieyuanli Chen, Wentao Bao +2
Understanding human intentions and actions through egocentric videos is important on the path to embodied artificial intelligence. As a branch of egocentric vision techniques, hand…