32 citations · 46 across the 20 of their papers we have counts for
22 papers
Coordination-Free Workload Regularization for Variable-Cost Distributed Training
Tao Huang, Chen Zhu, Antonio Alonso +4
Synchronous data-parallel training usually assigns every rank the same number of records, even when record costs vary by orders of magnitude. For variable-length models, this conve…
LinMorph: Learning to Morph Weights for Efficient Video DiT Linearization
Lingyi Long, Congyue Zhao, Xiaoou Li +2
Post-hoc linearization is an attractive way to accelerate pretrained Video Diffusion Transformers (Video DiTs), yet existing methods largely treat the quality loss induced by atten…
StellaVLA: In-Context Structured Demonstration for Generalizable Vision-Language-Action Models
Siyu Xu, Yunke Wang, Zijian Wang +6
Vision-Language-Action (VLA) models can follow instructions and manipulate objects, but their performance often collapses out of distribution (OOD), when the scene, viewpoint, or o…
Revisiting Parameter Redundancy in Vision-Language-Action Models: Insights from VLM-to-VLA Adaptation
Fengnian Zhang, Tao Huang, Siyu Xu +2
Vision-Language-Action (VLA) models have made significant strides in embodied intelligence by integrating the powerful representations of pre-trained Vision-Language Models (VLMs).…
Differentiable Efficient Operator Search
Xiaohuan Pei, Jiyuan Zhang, Yuanfan Guo +4
Efficient multimodal foundation models often rely on manually designed token-reduction operators, such as pruning, merging, pooling, and adaptive reweighting. Although these operat…
Retrieval-Augmented Linguistic Calibration
Yi-Fan Yeh, Linwei Tao, Minjing Dong +4
Linguistic cues such as "I believe" and "probably" offer an intuitive interface for communicating confidence, yet a generalisable, principled calibration framework for linguistic c…
Adversarial Error Correction for Visual Autoregressive Generation
Ligong Bi, Tao Huang, Jianyuan Guo +1
Visual Autoregressive (VAR) models have emerged as a powerful paradigm for image synthesis by performing hierarchical next-scale prediction. However, VAR models are inherently pron…
Scalable Hyperparameter-Divergent Ensemble Training with Automatic Learning Rate Exploration for Large Models
Hailing Cheng, Tao Huang, Chen Zhu +1
Training large neural networks with data-parallel stochastic gradient descent allocates N GPU replicas to compute effectively identical updates -- a practice that leaves the rich s…
DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression
Bingzhou Li, Tao Huang
Omnimodal large language models (OmniLLMs) jointly process audio and visual streams, but the resulting long multimodal token sequences make inference prohibitively expensive. Exist…
Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulation
Siyu Xu, Zijian Wang, Yunke Wang +3
Vision-Language-Action (VLA) models have shown great performance in robotic manipulation by mapping visual observations and language instructions directly to actions. However, they…
Efficient Conditional Generation on Scale-based Visual Autoregressive Models
Jiaqi Liu, Tao Huang, Chang Xu
Recent advances in autoregressive (AR) models have demonstrated their potential to rival diffusion models in image synthesis. However, for complex spatially-conditioned generation,…
Can Large Language Models Express Uncertainty Like Human?
Linwei Tao, Yi-Fan Yeh, Bo Kai +6
Large language models (LLMs) are increasingly used in high-stakes settings, where overconfident responses can mislead users. Reliable confidence estimation has been shown to enhanc…
Revisiting Uncertainty Estimation and Calibration of Large Language Models
Linwei Tao, Yi-Fan Yeh, Minjing Dong +3
As large language models (LLMs) are increasingly deployed in high-stakes applications, robust uncertainty estimation is essential for ensuring the safe and trustworthy deployment o…
Rethinking Causal Mask Attention for Vision-Language Inference
Xiaohuan Pei, Tao Huang, YanXiang Ma +1
Causal attention has become a foundational mechanism in autoregressive vision-language models (VLMs), unifying textual and visual inputs under a single generative framework. Howeve…
Learning Mask Invariant Mutual Information for Masked Image Modeling
Tao Huang, Yanxiang Ma, Shan You +1
Masked autoencoders (MAEs) represent a prominent self-supervised learning paradigm in computer vision. Despite their empirical success, the underlying mechanisms of MAEs remain ins…
VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching
Siyu Xu, Yunke Wang, Chenghao Xia +3
Vision-Language-Action (VLA) models have demonstrated strong multi-modal reasoning capabilities, enabling direct action generation from visual perception and language instructions…
Cross-Self KV Cache Pruning for Efficient Vision-Language Inference
Xiaohuan Pei, Tao Huang, Chang Xu
KV cache pruning has emerged as a promising technique for reducing memory and computation costs in long-context auto-regressive generation. Existing methods for vision-language mod…
Training-free Composite Scene Generation for Layout-to-Image Synthesis
Jiaqi Liu, Tao Huang, Chang Xu
Recent breakthroughs in text-to-image diffusion models have significantly advanced the generation of high-fidelity, photo-realistic images from textual descriptions. Yet, these mod…
EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
Xiaohuan Pei, Tao Huang, Chang Xu
Prior efforts in light-weight model development mainly centered on CNN and Transformer-based designs yet faced persistent challenges. CNNs adept at local feature extraction comprom…
LocalMamba: Visual State Space Model with Windowed Selective Scan
Tao Huang, Xiaohuan Pei, Shan You +3
Recent advancements in state space models, notably Mamba, have demonstrated significant progress in modeling long sequences for tasks like language understanding. Yet, their applic…
Active Generation for Image Classification
Tao Huang, Jiaqi Liu, Shan You +1
Recently, the growing capabilities of deep generative models have underscored their potential in enhancing image classification accuracy. However, existing methods often demand the…
Adaptive Training Meets Progressive Scaling: Elevating Efficiency in Diffusion Models
Wenhao Li, Xiu Su, Yu Han +3
Diffusion models have demonstrated remarkable efficacy in various generative tasks with the predictive prowess of denoising model. Currently, diffusion models employ a uniform deno…