27 citations · 103 across the 15 of their papers we have counts for
18 papers
A-SelecT: Automatic Timestep Selection for Diffusion Transformer Representation Learning
Changyu Liu, James Chenhao Liang, Wenhao Yang +6
Diffusion models have significantly reshaped the field of generative artificial intelligence and are now increasingly explored for their capacity in discriminative representation l…
On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning
Changyu Liu, Yiyang Liu, Taowen Wang +7
Vision-Language-Action models have recently emerged as a powerful paradigm for general-purpose robot learning, enabling agents to map visual observations and natural-language instr…
All You Need is One: Capsule Prompt Tuning with a Single Vector
Yiyang Liu, James C. Liang, Heng Fan +7
Prompt-based learning has emerged as a parameter-efficient finetuning (PEFT) approach to facilitate Large Language Model (LLM) adaptation to downstream tasks by conditioning genera…
Probabilistic Token Alignment for Large Language Model Fusion
Runjia Zeng, James Chenhao Liang, Cheng Han +8
Training large language models (LLMs) from scratch can yield models with unique functionalities and strengths, but it is costly and often leads to redundant capabilities. A more co…
Re-Imagining Multimodal Instruction Tuning: A Representation View
Yiyang Liu, James Chenhao Liang, Ruixiang Tang +8
Multimodal instruction tuning has proven to be an effective strategy for achieving zero-shot generalization by fine-tuning pre-trained Large Multimodal Models (LMMs) with instructi…
Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics
Taowen Wang, Cheng Han, James Chenhao Liang +6
Recently in robotics, Vision-Language-Action (VLA) models have emerged as a transformative approach, enabling robots to execute complex tasks by integrating visual and linguistic i…
MPT: Multimodal Prompt Tuning for Zero-shot Instruction Learning
Taowen Wang, Yiyang Liu, James Chenhao Liang +11
Multimodal Large Language Models (MLLMs) demonstrate remarkable performance across a wide range of domains, with increasing emphasis on enhancing their zero-shot generalization cap…
Diff-PIC: Revolutionizing Particle-In-Cell Nuclear Fusion Simulation with Diffusion Models
Chuan Liu, Chunshu Wu, Shihui Cao +8
The rapid development of AI highlights the pressing need for sustainable energy, a critical global challenge for decades. Nuclear fusion, generally seen as an ultimate solution, ha…
Inertial Confinement Fusion Forecasting via Large Language Models
Mingkai Chen, Taowen Wang, Shihui Cao +10
Controlled fusion energy is deemed pivotal for the advancement of human civilization. In this study, we introduce , a novel integration of Large Language Models (…
Self-supervised Adversarial Training of Monocular Depth Estimation against Physical-World Attacks
Zhiyuan Cheng, Cheng Han, James Liang +3
Monocular Depth Estimation (MDE) plays a vital role in applications such as autonomous driving. However, various attacks target MDE models, with physical attacks posing significant…
Prototypical Transformer as Unified Motion Learners
Cheng Han, Yawen Lu, Guohao Sun +9
In this work, we introduce the Prototypical Transformer (ProtoFormer), a general and unified framework that approaches various motion tasks from a prototype perspective. ProtoForme…
Image Translation as Diffusion Visual Programmers
Cheng Han, James C. Liang, Qifan Wang +5
We introduce the novel Diffusion Visual Programmer (DVP), a neuro-symbolic image translation framework. Our proposed DVP seamlessly embeds a condition-flexible diffusion model with…
ClusterFormer: Clustering As A Universal Visual Learner
James C. Liang, Yiming Cui, Qifan Wang +3
This paper presents CLUSTERFORMER, a universal vision model that is based on the CLUSTERing paradigm with TransFORMER. It comprises two novel designs: 1. recurrent cross-attention…
CLUSTSEG: Clustering for Universal Segmentation
James Liang, Tianfei Zhou, Dongfang Liu +1
We present CLUSTSEG, a general, transformer-based framework that tackles different image segmentation tasks (i.e., superpixel, semantic, instance, and panoptic) through a unified n…
Fusion is Not Enough: Single Modal Attacks on Fusion Models for 3D Object Detection
Zhiyuan Cheng, Hongjun Choi, James Liang +5
Multi-sensor fusion (MSF) is widely used in autonomous vehicles (AVs) for perception, particularly for 3D object detection with camera and LiDAR sensors. The purpose of fusion is t…
Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks
Zhiyuan Cheng, James Liang, Guanhong Tao +2
Monocular Depth Estimation (MDE) is a critical component in applications such as autonomous driving. There are various attacks against MDE networks. These attacks, especially the p…
Learning Equivariant Segmentation with Instance-Unique Querying
Wenguan Wang, James Liang, Dongfang Liu
Prevalent state-of-the-art instance segmentation methods fall into a query-based scheme, in which instance masks are derived by querying the image feature using a set of instance-a…
Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches
Zhiyuan Cheng, James Liang, Hongjun Choi +4
Deep learning has substantially boosted the performance of Monocular Depth Estimation (MDE), a critical component in fully vision-based autonomous driving (AD) systems (e.g., Tesla…