45 citations · 103 across the 17 of their papers we have counts for
21 papers
On the Token Value Inequality in Efficient Reasoning
Runjia Zeng, Hang Hua, Yiyang Liu +5
Chain-of-Thought reasoning has enabled large language models to achieve substantial performance gains on complex tasks. However, these gains come at the cost of dramatically increa…
A-SelecT: Automatic Timestep Selection for Diffusion Transformer Representation Learning
Changyu Liu, James Chenhao Liang, Wenhao Yang +6
Diffusion models have significantly reshaped the field of generative artificial intelligence and are now increasingly explored for their capacity in discriminative representation l…
TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching
Runjia Zeng, Qifan Wang, Qiang Guan +6
Fine tuning has been regarded as a de facto approach for adapting large language models (LLMs) to downstream tasks, but the high training memory consumption inherited from LLMs mak…
On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning
Changyu Liu, Yiyang Liu, Taowen Wang +7
Vision-Language-Action models have recently emerged as a powerful paradigm for general-purpose robot learning, enabling agents to map visual observations and natural-language instr…
All You Need is One: Capsule Prompt Tuning with a Single Vector
Yiyang Liu, James C. Liang, Heng Fan +7
Prompt-based learning has emerged as a parameter-efficient finetuning (PEFT) approach to facilitate Large Language Model (LLM) adaptation to downstream tasks by conditioning genera…
Probabilistic Token Alignment for Large Language Model Fusion
Runjia Zeng, James Chenhao Liang, Cheng Han +8
Training large language models (LLMs) from scratch can yield models with unique functionalities and strengths, but it is costly and often leads to redundant capabilities. A more co…
MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper
Runjia Zeng, Guangyan Sun, Qifan Wang +8
Considering deep neural networks as manifold mappers, the pretrain-then-fine-tune paradigm can be interpreted as a two-stage process: pretrain establishes a broad knowledge base, a…
Re-Imagining Multimodal Instruction Tuning: A Representation View
Yiyang Liu, James Chenhao Liang, Ruixiang Tang +8
Multimodal instruction tuning has proven to be an effective strategy for achieving zero-shot generalization by fine-tuning pre-trained Large Multimodal Models (LMMs) with instructi…
Visual Fourier Prompt Tuning
Runjia Zeng, Cheng Han, Qifan Wang +5
With the scale of vision Transformer-based models continuing to grow, finetuning these large-scale pretrained models for new tasks has become increasingly parameter-intensive. Visu…
Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics
Taowen Wang, Cheng Han, James Chenhao Liang +6
Recently in robotics, Vision-Language-Action (VLA) models have emerged as a transformative approach, enabling robots to execute complex tasks by integrating visual and linguistic i…
MPT: Multimodal Prompt Tuning for Zero-shot Instruction Learning
Taowen Wang, Yiyang Liu, James Chenhao Liang +11
Multimodal Large Language Models (MLLMs) demonstrate remarkable performance across a wide range of domains, with increasing emphasis on enhancing their zero-shot generalization cap…
AMD: Automatic Multi-step Distillation of Large-scale Vision Models
Cheng Han, Qifan Wang, Sohail A. Dianat +6
Transformer-based architectures have become the de-facto standard models for diverse vision tasks owing to their superior performance. As the size of the models continues to scale…
Self-supervised Adversarial Training of Monocular Depth Estimation against Physical-World Attacks
Zhiyuan Cheng, Cheng Han, James Liang +3
Monocular Depth Estimation (MDE) plays a vital role in applications such as autonomous driving. However, various attacks target MDE models, with physical attacks posing significant…
ProMotion: Prototypes As Motion Learners
Yawen Lu, Dongfang Liu, Qifan Wang +6
In this work, we introduce ProMotion, a unified prototypical framework engineered to model fundamental motion tasks. ProMotion offers a range of compelling attributes that set it a…
Prototypical Transformer as Unified Motion Learners
Cheng Han, Yawen Lu, Guohao Sun +9
In this work, we introduce the Prototypical Transformer (ProtoFormer), a general and unified framework that approaches various motion tasks from a prototype perspective. ProtoForme…
SSGA-Net: Stepwise Spatial Global-local Aggregation Networks for for Autonomous Driving
Yiming Cui, Cheng Han, Dongfang Liu
Visual-based perception is the key module for autonomous driving. Among those visual perception tasks, video object detection is a primary yet challenging one because of feature de…
Image Translation as Diffusion Visual Programmers
Cheng Han, James C. Liang, Qifan Wang +5
We introduce the novel Diffusion Visual Programmer (DVP), a neuro-symbolic image translation framework. Our proposed DVP seamlessly embeds a condition-flexible diffusion model with…
Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning?
Cheng Han, Qifan Wang, Yiming Cui +4
As the scale of vision models continues to grow, the emergence of Visual Prompt Tuning (VPT) as a parameter-efficient transfer learning technique has gained attention due to its su…
CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation
Yiming Cui, Cheng Han, Dongfang Liu
The advancement of computer vision has pushed visual analysis tasks from still images to the video domain. In recent years, video instance segmentation, which aims to track and seg…
E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning
Cheng Han, Qifan Wang, Yiming Cui +4
As the size of transformer-based models continues to grow, fine-tuning these large-scale pretrained vision models for new tasks has become increasingly parameter-intensive. Paramet…
Visual Recognition with Deep Nearest Centroids
Wenguan Wang, Cheng Han, Tianfei Zhou +1
We devise deep nearest centroids (DNC), a conceptually elegant yet surprisingly effective network for large-scale visual recognition, by revisiting Nearest Centroids, one of the mo…