4 citations · 14 across the 17 of their papers we have counts for
22 papers
3DCodeVerse: Exploring 3D Worlds as Code
Yipeng Gao, Ziyao Zeng, Jinfa Huang +16
We introduce 3DCodeVerse, a research ecosystem that enables coding agents to create editable and verifiable 3D content at scale. 3DCodeVerse comprises large-scale, high-quality dat…
SCOPE: Semantic Bounded Credit Optimization for Prompt Engineering in Text-to-image Generation
Yufei Jiao, Jinfa Huang, Zeyue Xue +5
Reinforcement learning (RL) enables language models to improve text-to-image (T2I) generation by rewriting prompts for a frozen image generator. Existing methods typically apply a…
VideoLoop: Looped Working Memory Against Semantic Thrashing in Long-Form Video Agents
Jinfa Huang, Jianming Xu, Jingyang Lin +2
Long-form video understanding requires multimodal agents to iteratively gather evidence over many reasoning steps. However, most existing agentic methods suffer from semantic thras…
One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding
Wang Chen, Yu Chen, Xiang Wang +3
Frame selection is essential for applying Large Multimodal Models (LMMs) to long videos due to severe frame redundancy and limited context windows. Since the appropriate frame budg…
WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation
Yuhui Zeng, Wang Chen, Jinfa Huang +5
Existing Large Vision-Language Models (LVLMs) struggle with long-form video understanding due to the quadratic computational cost of visual tokens. While recent efficient methods a…
TimePLE: Rethinking Temporal Representation for Video Temporal Grounding
Yuhui Zeng, Xinyu Mao, Xiaokun Liu +4
Video temporal grounding (VTG) aims to localize the continuous video interval described by a natural-language query. However, current VLM-based methods typically produce this inter…
SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning
Haoyu Huang, Jinfa Huang, Zhongwei Wan +3
Agentic multimodal large language models (MLLMs) (e.g., OpenAI o3 and Gemini Agentic Vision) achieve remarkable reasoning capabilities through iterative visual tool invocation. How…
SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
Tianyu Xie, Jinfa Huang, Yuexiao Ma +11
Evaluating omni-modal large language models (OLMs) in multi-party dialogue requires more than answer correctness on pre-segmented inputs. We introduce SocialOmni, an offline diagno…
A Survey on Latent Reasoning
Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng +30
Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate s…
LeanPO: Lean Preference Optimization for Likelihood Alignment in Video-LLMs
Xiaodong Wang, Jinfa Huang, Li Yuan +1
Most Video Large Language Models (Video-LLMs) adopt preference alignment techniques, e.g., DPO~\citep{rafailov2024dpo}, to optimize the reward margin between a winning response ($y…
OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation
Shenghai Yuan, Xianyi He, Yufan Deng +5
Subject-to-Video (S2V) generation aims to create videos that faithfully incorporate reference content, providing enhanced flexibility in the production of videos. To establish the…
TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configuration
Yanshu Li, Jianjiang Yang, Tian Yun +3
Multimodal in-context learning (ICL) has emerged as a key mechanism for harnessing the capabilities of large vision-language models (LVLMs). However, its effectiveness remains high…
QuoTA: Query-oriented Token Assignment via CoT Query Decouple for Long Video Comprehension
Yongdong Luo, Wang Chen, Xiawu Zheng +8
Recent advances in long video understanding typically mitigate visual redundancy through visual token pruning based on attention distribution. However, while existing methods emplo…
Identity-Preserving Text-to-Video Generation by Frequency Decomposition
Shenghai Yuan, Jinfa Huang, Xianyi He +5
Identity-preserving text-to-video (IPT2V) generation aims to create high-fidelity videos with consistent human identity. It is an important task in video generation but remains an…
Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension
Yongdong Luo, Xiawu Zheng, Guilin Li +8
Existing large video-language models (LVLMs) struggle to comprehend long videos correctly due to limited context. To address this problem, fine-tuning long-context LVLMs and employ…
Autoregressive Models in Vision: A Survey
Jing Xiong, Gongye Liu, Lun Huang +17
Autoregressive modeling has been a huge success in the field of natural language processing (NLP). Recently, autoregressive models have emerged as a significant area of focus in co…
Evolver: Chain-of-Evolution Prompting to Boost Large Multimodal Models for Hateful Meme Detection
Jinfa Huang, Jinsheng Pan, Zhongwei Wan +2
Recent advances show that two-stream approaches have achieved outstanding performance in hateful meme detection. However, hateful memes constantly evolve as new memes emerge by fus…
LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference
Zhongwei Wan, Ziang Wu, Che Liu +5
Long-context Multimodal Large Language Models (MLLMs) demand substantial computational resources for inference as the growth of their multimodal Key-Value (KV) cache, in response t…
ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation
Shenghai Yuan, Jinfa Huang, Yongqi Xu +7
We propose a novel text-to-video (T2V) generation benchmark, ChronoMagic-Bench, to evaluate the temporal and metamorphic capabilities of the T2V models (e.g. Sora and Lumiere) in t…
MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators
Shenghai Yuan, Jinfa Huang, Yujun Shi +6
Recent advances in Text-to-Video generation (T2V) have achieved remarkable success in synthesizing high-quality general videos from textual descriptions. A largely overlooked probl…
Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach
Shaofeng Zhang, Jinfa Huang, Qiang Zhou +4
Image outpainting aims to generate the content of an input sub-image beyond its original boundaries. It is an important task in content generation yet remains an open problem for g…
GPT-4V(ision) as A Social Media Analysis Engine
Hanjia Lyu, Jinfa Huang, Daoan Zhang +6
Recent research has offered insights into the extraordinary capabilities of Large Multimodal Models (LMMs) in various general vision and language tasks. There is growing interest i…