251 citations · 280 across the 9 of their papers we have counts for
7 papers · 1 filter
Context Unrolling in Omni Models
Ceyuan Yang, Zhijie Lin, Yang Zhao +16
We present Omni, a unified multimodal model natively trained on diverse modalities, including text, images, videos, 3D geometry, and hidden representations. We find that such train…
Unsupervised Discovery of Interpretable Directions in h-space of Pre-trained Diffusion Models
Zijian Zhang, Luping Liu, Zhijie Lin +2
We propose the first unsupervised and learning-based method to identify interpretable directions in h-space of pre-trained diffusion models. Our method is derived from an existing…
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs
Yang Zhao, Zhijie Lin, Daquan Zhou +3
LLMs have demonstrated remarkable abilities at interacting with humans through language, especially with the usage of instruction-following data. Recent advancements in LLMs, such…
DATE: Domain Adaptive Product Seeker for E-commerce
Haoyuan Li, Hao Jiang, Tao Jin +5
Product Retrieval (PR) and Grounding (PG), aiming to seek image and object-level products respectively according to a textual query, have attracted great interest recently for bett…
SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memory
Zhijie Lin, Zhou Zhao, Haoyuan Li +4
Lip reading, aiming to recognize spoken sentences according to the given video of lip movements without relying on the audio stream, has attracted great interest due to its applica…
Localizing Unseen Activities in Video via Image Query
Zhu Zhang, Zhou Zhao, Zhijie Lin +2
Action localization in untrimmed videos is an important topic in the field of video understanding. However, existing action localization methods are restricted to a pre-defined set…
Open-Ended Long-Form Video Question Answering via Hierarchical Convolutional Self-Attention Networks
Zhu Zhang, Zhou Zhao, Zhijie Lin +2
Open-ended video question answering aims to automatically generate the natural-language answer from referenced video contents according to the given question. Currently, most exist…