21 citations · 26 across the 15 of their papers we have counts for
18 papers
Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
Xiaomin Yu, Yi Xin, Yuhui Zhang +12
Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of d…
ReactionMamba: Generating Short & Long Human Reaction Sequences
Hajra Anwar Beg, Baptiste Chopin, Hao Tang +1
We present ReactionMamba, a novel framework for generating long 3D human reaction motions. Reaction-Mamba integrates a motion VAE for efficient motion encoding with Mamba-based sta…
An Evaluation of Interleaved Instruction Tuning on Semantic Reasoning Performance in an Audio MLLM
Jiawei Liu, Enis Berk Çoban, Zarina Schevchenko +4
Standard training for Multi-modal Large Language Models (MLLMs) involves concatenating non-textual information, like vision or audio, with a text prompt. This approach may not enco…
Enhancing Materials Discovery with Valence Constrained Design in Generative Modeling
Mouyang Cheng, Weiliang Luo, Hao Tang +6
Diffusion-based deep generative models have emerged as powerful tools for inverse materials design. Yet, many existing approaches overlook essential chemical constraints such as ox…
Mitigating Task Objective Conflicts for Continual Learning in Unified Multimodal Models
Jiaxing Zhang, Hao Tang
Recent advances in unified autoregressive (AR) models have enabled multimodal understanding (MMU) and text-to-image generation (T2I) within a single architecture. However, in conti…
Improved Algorithms for Differentially Private Language Model Alignment
Keyu Chen, Hao Tang, Qinglin Liu +1
Language model alignment is crucial for ensuring that large language models (LLMs) align with human preferences, yet it often involves sensitive user data, raising significant priv…
Multimodal Large Language Models for Medicine: A Comprehensive Survey
Jiarui Ye, Hao Tang
MLLMs have recently become a focal point in the field of artificial intelligence research. Building on the strong capabilities of LLMs, MLLMs are adept at addressing complex multi-…
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models
Xu Ma, Peize Sun, Haoyu Ma +22
Autoregressive (AR) models, long dominant in language generation, are increasingly applied to image synthesis but are often considered less competitive than Diffusion-based models.…
Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming
Jiaxuan Zhu, Hao Tang
Representing and rendering dynamic scenes from 2D images is a fundamental yet challenging problem in computer vision and graphics. This survey provides a comprehensive review of th…
When Continue Learning Meets Multimodal Large Language Model: A Survey
Yukang Huo, Hao Tang
Recent advancements in Artificial Intelligence have led to the development of Multimodal Large Language Models (MLLMs). However, adapting these pre-trained models to dynamic data d…
CustomVideoX: 3D Reference Attention Driven Dynamic Adaptation for Zero-Shot Customized Video Diffusion Transformers
D. She, Mushui Liu, Jingxuan Pang +9
Customized generation has achieved significant progress in image synthesis, yet personalized video generation remains challenging due to temporal inconsistencies and quality degrad…
UDiTQC: U-Net-Style Diffusion Transformer for Quantum Circuit Synthesis
Zhiwei Chen, Hao Tang
Quantum computing is a transformative technology with wide-ranging applications, and efficient quantum circuit generation is crucial for unlocking its full potential. Current diffu…
End-to-End Long Document Summarization using Gradient Caching
Rohit Saxena, Hao Tang, Frank Keller
Training transformer-based encoder-decoder models for long document summarization poses a significant challenge due to the quadratic memory consumption during training. Several app…
PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model
Yuqing Wang, Zhongling Huang, Shuxin Yang +4
PolSAR data presents unique challenges due to its rich and complex characteristics. Existing data representations, such as complex-valued data, polarimetric features, and amplitude…
Network Inversion and Its Applications
Pirzada Suhail, Hao Tang, Amit Sethi
Neural networks have emerged as powerful tools across various applications, yet their decision-making process often remains opaque, leading to them being perceived as "black boxes.…
Multimodal Alignment and Fusion: A Survey
Songtao Li, Hao Tang
This survey provides a comprehensive overview of recent advances in multimodal alignment and fusion within the field of machine learning, driven by the increasing availability and…
Hierarchical Indexing for Retrieval-Augmented Opinion Summarization
Tom Hosking, Hao Tang, Mirella Lapata
We propose a method for unsupervised abstractive opinion summarization, that combines the attributability and scalability of extractive approaches with the coherence and fluency of…
Distilling ODE Solvers of Diffusion Models into Smaller Steps
Sanghwan Kim, Hao Tang, Fisher Yu
Abstract Diffusion models have recently gained prominence as a novel category of generative models. Despite their success, these models face a notable drawback in terms of slow sam…