21 citations · 26 across the 15 of their papers we have counts for
11 papers · 1 filter
An Evaluation of Interleaved Instruction Tuning on Semantic Reasoning Performance in an Audio MLLM
Jiawei Liu, Enis Berk Çoban, Zarina Schevchenko +4
Standard training for Multi-modal Large Language Models (MLLMs) involves concatenating non-textual information, like vision or audio, with a text prompt. This approach may not enco…
Enhancing Materials Discovery with Valence Constrained Design in Generative Modeling
Mouyang Cheng, Weiliang Luo, Hao Tang +6
Diffusion-based deep generative models have emerged as powerful tools for inverse materials design. Yet, many existing approaches overlook essential chemical constraints such as ox…
Mitigating Task Objective Conflicts for Continual Learning in Unified Multimodal Models
Jiaxing Zhang, Hao Tang
Recent advances in unified autoregressive (AR) models have enabled multimodal understanding (MMU) and text-to-image generation (T2I) within a single architecture. However, in conti…
Improved Algorithms for Differentially Private Language Model Alignment
Keyu Chen, Hao Tang, Qinglin Liu +1
Language model alignment is crucial for ensuring that large language models (LLMs) align with human preferences, yet it often involves sensitive user data, raising significant priv…
Multimodal Large Language Models for Medicine: A Comprehensive Survey
Jiarui Ye, Hao Tang
MLLMs have recently become a focal point in the field of artificial intelligence research. Building on the strong capabilities of LLMs, MLLMs are adept at addressing complex multi-…
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models
Xu Ma, Peize Sun, Haoyu Ma +22
Autoregressive (AR) models, long dominant in language generation, are increasingly applied to image synthesis but are often considered less competitive than Diffusion-based models.…
Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming
Jiaxuan Zhu, Hao Tang
Representing and rendering dynamic scenes from 2D images is a fundamental yet challenging problem in computer vision and graphics. This survey provides a comprehensive review of th…
When Continue Learning Meets Multimodal Large Language Model: A Survey
Yukang Huo, Hao Tang
Recent advancements in Artificial Intelligence have led to the development of Multimodal Large Language Models (MLLMs). However, adapting these pre-trained models to dynamic data d…
CustomVideoX: 3D Reference Attention Driven Dynamic Adaptation for Zero-Shot Customized Video Diffusion Transformers
D. She, Mushui Liu, Jingxuan Pang +9
Customized generation has achieved significant progress in image synthesis, yet personalized video generation remains challenging due to temporal inconsistencies and quality degrad…
UDiTQC: U-Net-Style Diffusion Transformer for Quantum Circuit Synthesis
Zhiwei Chen, Hao Tang
Quantum computing is a transformative technology with wide-ranging applications, and efficient quantum circuit generation is crucial for unlocking its full potential. Current diffu…
End-to-End Long Document Summarization using Gradient Caching
Rohit Saxena, Hao Tang, Frank Keller
Training transformer-based encoder-decoder models for long document summarization poses a significant challenge due to the quadratic memory consumption during training. Several app…