11 papers
Distorted or Fabricated? A Survey on Hallucination in Video LLMs
Yiyang Huang, Yitian Zhang, Yizhou Wang +4
Despite significant progress in video-language modeling, hallucinations remain a persistent challenge in Video Large Language Models (Vid-LLMs), referring to outputs that appear pl…
The Indra Representation Hypothesis for Multimodal Alignment
Jianglin Lu, Hailing Wang, Kuo Yang +3
Recent studies have uncovered an interesting phenomenon: unimodal foundation models tend to learn convergent representations, regardless of differences in architecture, training ob…
Fine-T2I: An Open, Large-Scale, and Diverse Dataset for High-Quality T2I Fine-Tuning
Xu Ma, Yitian Zhang, Qihua Dong +1
High-quality and open datasets remain a major bottleneck for text-to-image (T2I) fine-tuning. Despite rapid progress in model architectures and training pipelines, most publicly av…
Any Orthogonal Transform Will Do: Range Reduction Explains Rotation-Based Quantization for Super-Resolution
Dorsa Zeinali, Hailing Wang, Yitian Zhang +1
Rotation-based quantization -- multiplying weights and activations by a Hadamard matrix before quantizing -- is now standard, but why it works is explained informally, via incohere…
Outlier-Aware Post-Training Quantization for Image Super-Resolution
Hailing Wang, jianglin Lu, Yitian Zhang +1
Quantization techniques, including quantization-aware training (QAT) and post-training quantization (PTQ), have become essential for inference acceleration of image super-resolutio…
REGEN: Learning Compact Video Embedding with (Re-)Generative Decoder
Yitian Zhang, Long Mai, Aniruddha Mahapatra +5
We present a novel perspective on learning video embedders for generative modeling: rather than requiring an exact reproduction of an input video, an effective embedder should focu…
GmNet: Revisiting Gating Mechanisms From A Frequency View
Yifan Wang, Xu Ma, Yitian Zhang +5
Gating mechanisms have emerged as an effective strategy integrated into model designs beyond recurrent neural networks for addressing long-range dependency problems. In a broad und…
Scale-Free Graph-Language Models
Jianglin Lu, Yixuan Liu, Yitian Zhang +1
Graph-language models (GLMs) have demonstrated great potential in graph-based semi-supervised learning. A typical GLM consists of two key stages: graph generation and text embeddin…
Slicing Vision Transformer for Flexible Inference
Yitian Zhang, Huseyin Coskun, Xu Ma +6
Vision Transformers (ViT) is known for its scalability. In this work, we target to scale down a ViT to fit in an environment with dynamic-changing resource constraints. We observe…
SoupLM: Model Integration in Large Language and Multi-Modal Models
Yue Bai, Zichen Zhang, Jiasen Lu +1
Training large language models (LLMs) and multimodal LLMs necessitates significant computing resources, and existing publicly available LLMs are typically pre-trained on diverse, p…
Accessing Vision Foundation Models via ImageNet-1K
Yitian Zhang, Xu Ma, Yue Bai +2
Vision foundation models are renowned for the generalization ability due to massive training data. Nevertheless, they demand tremendous training resources, and the training data is…